Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Request parsing into buffer is taking time leading to p99 latency issues
First, upgrade your version of Jetty.

9.4.11 is very old and is missing countless bug fixes and updates.
And is also subject to several security issues. See: https://www.eclipse.org/jetty/security_reports.php

Use at least 9.4.46.

The output shows that you have a request being processed by a webapp, consisting of 26090 bytes, and it's reading until the the content is closed by the client.
The client doesn't appear to be sending your the content in a timely manner, the sending is slow.
That's as far as we can go with this limited debug logs on that very old Jetty version.

Upgrade, use DEBUG again, and we should see more details that might reveal something else.

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Tue, May 10, 2022 at 10:32 AM sanketh reddy <sanre6@xxxxxxxxx> wrote:
Hi Jetty Users,

I am facing a peculiar problem in jetty server 9.4.11.v20180605. There seems to be a delay in request parsing AFAIK from the debug logs affecting about 1% of the requests.
How do I solve this issue ? Do we have any timeout option to restrict this on the server side. Attaching the debug logs below of one of the request where you see the age getting bumped up till 15 seconds.

Any suggestions are welcome.


DEBUG [2022-05-10 11:30:43,805] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,7144 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=2/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,7144 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2}
DEBUG [2022-05-10 11:30:43,805] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,8192 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,8192 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2}
DEBUG [2022-05-10 11:30:43,805] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,8192 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,8192 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2}
DEBUG [2022-05-10 11:30:45,941] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,16384 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2138}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,16384 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2138}
DEBUG [2022-05-10 11:30:45,941] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,16384 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2138}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,16384 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=2138}
DEBUG [2022-05-10 11:30:59,012] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,24576 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=15209}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,24576 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=15209}
DEBUG [2022-05-10 11:30:59,012] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,24576 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=15209}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=CONTENT,24576 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=15209}
DEBUG [2022-05-10 11:30:59,251] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=END,26090 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=15448}<-SocketChannelEndPoint@60b4f649{/172.31.23.178:22996<->/172.31.16.150:8080,OPEN,fill=-,flush=-,to=1/30000}{io=0/0,kio=0,kro=1}->HttpConnection@608e8c9f[p=HttpParser{s=END,26090 of 26090},g=HttpGenerator@27cf3879{s=START}]=>HttpChannelOverHttp@399cc4bf{r=1,c=false,a=DISPATCHED,uri=https://global-api.mpl.live/gameplay/103619231/submitscore,age=15448}
DEBUG [2022-05-10 11:30:59,307] org.eclipse.jetty.server.HttpConnection: releaseRequestBuffer HttpConnection@608e8c9f[p=HttpParser{s=CLOSED,0 of -1}





--
Thanks,
sanre6
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jetty-users

Back to the top