Proxy failed - Intermittent issue

[shr-nielsen]

Hi all,

we are facing intermittent timeout issues on lyft gateway running on Amazon linux server. we are trying to move from CentOS to Amz linux but facing this issue.

DEBUG [2024-11-21 08:03:29,578] com.lyft.data.proxyserver.ProxyServletImpl.qaasRouter: 369397021 rewriting: https://qaas-test.svc.nlsn.media/viewgateway -> https://qaas-gateway-test.aws-mp-us-east-1-np.nlsn.media:8091/viewgateway
DEBUG [2024-11-21 08:03:29,578] com.lyft.data.proxyserver.ProxyServletImpl: adding pre-connection hook of proxy handler..
DEBUG [2024-11-21 08:03:29,578] com.lyft.data.proxyserver.ProxyHandler: Setting header [Accept-Encoding] with value [application/json]
DEBUG [2024-11-21 08:03:29,578] com.lyft.data.proxyserver.ProxyServletImpl.qaasRouter: 369397021 proxying to upstream:
GET /viewgateway HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15
Sec-Fetch-Site: none
Sec-Fetch-Dest: document
Host: qaas-test.svc.nlsn.media
Accept-Language: en-IN,en-GB;q=0.9,en;q=0.8
Accept-Encoding: gzip, deflate, br
Sec-Fetch-Mode: navigate

HttpRequest[GET /viewgateway HTTP/1.1]@549d3833

DEBUG [2024-11-21 08:03:59,586] com.lyft.data.proxyserver.ProxyServletImpl.qaasRouter: 369397021 proxying failed
! java.util.concurrent.TimeoutException: Idle timeout 30000 ms
! at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onIdleExpired(HttpConnectionOverHTTP.java:145)
! at org.eclipse.jetty.io.ssl.SslConnection.onIdleExpired(SslConnection.java:270)
! at org.eclipse.jetty.io.AbstractEndPoint.onIdleExpired(AbstractEndPoint.java:401)
! at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
! at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
! at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
! at java.util.concurrent.FutureTask.run(FutureTask.java:266)
! at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
! at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
! at java.lang.Thread.run(Thread.java:750)
DEBUG [2024-11-21 08:03:59,587] com.lyft.data.proxyserver.ProxyServletImpl.qaasRouter: 369397021 proxying complete

No issue on network and Infra side and it works fine in old centos images. And this timeout happens intermittently , not always.