Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Websocket handling issues
Joakime,

but why is jetty trying to parse a HTTP request after sending the 101?  It is as if the 101 has been committed, but somehow the upgrade itself is not done?? but it can't be an exception as that would also prevent another request from being parsed?


On Tue, 19 Mar 2019 at 22:03, Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
The HTTP/1.0 400 Illegal character is a proper HTTP response.to the second HTTP request.

Sequence is this ...

1. The upgrade request is sent by the client. (This is HTTP request 1 of 2)
2. The client sends the first websocket text frame before waiting for the upgrade response. (This is HTTP request 2 of 2)
3. The server is accepting the upgrade request sends back a 101 upgrade response. (This is HTTP response 1 of 2)
4. The server sends a 400 response due to an illegal character 0x81 which was the first byte  (This is HTTP response 2 of 2)
of the websocket TEXT frame. 
5. Connection is upgraded.

You can see this yourself in the Wireshark breakdown.
Even Wireshark sees this sequence as the HTTP exchange of 2 requests on a single connection.

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Tue, Mar 19, 2019 at 2:37 AM Lachlan Roberts <lachlan@xxxxxxxxxxx> wrote:
Hi,

Looking at "unsuccessful connection.pcapng" it seems that the sequence of events is like this.

1. The upgrade request is sent by the client.
2. The client sends the first websocket text frame before waiting for the upgrade response.
3. The server is accepting the upgrade request sends back a 101 upgrade response.
4. The server sends a 400 response due to an illegal character 0x81 which was the first byte 
of the websocket TEXT frame. 

So it looks like the server is trying to read the first WebSocket frame as a HTTP 
frame even though it has accepted the websocket upgrade. 

See packet 18:

Hypertext Transfer Protocol
    HTTP/1.1 400 Illegal character OTEXT=0x81\r\n
    Content-Type: text/html;charset=iso-8859-1\r\n
    Content-Length: 71\r\n
    Connection: close\r\n
    Server: Jetty(9.4.15.v20190215)\r\n
    \r\n


Also note that the server is not accepting the permessage-deflate extension even in the
"successful connection.pcapng" even though it was offered in the client upgrade request.
This may be related to the RSV bit problems.

On Tue, Mar 19, 2019 at 7:59 AM Meggyesi, Zalán <zmeggyesi@xxxxxxxx> wrote:
Hello Joakim,

Thanks for the insight. 
I'll try to start off with making the application running on Jetty "upgrade-aware" - it's a Ninja Framework app that should not be setting headers, so I'm guessing something is also off there.
I suspected the app itself before too, but since the success/failure seemed to be controlled more closely by the Jetty version, I discounted the framework being faulty itself. I'll also try to isolate a "test case" app, see if it needs to be reported to Ninja's developers.

Thank you again,

Zalan Meggyesi
Chief Support Engineer
Skawa Innovation Kft.

Mobile: +36205146666


On Mon, Mar 18, 2019 at 8:32 PM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
Another problem seen in unsuccessful connection.pcapng is that the WebSocket client you are using is violating the WebSocket protocol by sending WebSocket frames before the opening handshake is complete.

It sends the HTTP Upgrade Request (packet 4)
Then a few WebSocket Frames (packet 8)
Then the server responds with the HTTP Upgrade Response (to websocket, in packet 12)
Then your WebSocket client sends more frames (packet 15)

It should not send websocket frames until the handshake is completed.
This is especially true with permessage-deflate as that requires knowledge of prior websocket frames to operate properly.

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Mon, Mar 18, 2019 at 2:25 PM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
In unsuccessful connection.pcapng the response from the server is ...

HTTP/1.1 101 Switching Protocols
Date: Mon, 18 Mar 2019 17:04:47 GMT
Connection: Upgrade
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Upgrade: WebSocket
Date: Mon, 18 Mar 2019 17:04:47 GMT
Sec-WebSocket-Accept: 2ehI1RO+UMwBNHwQJy91/GWGaJA=
Content-Type: application/json;charset=utf-8

This is a very strange response.
The `Cache-Control`, `Expires`, and `Content-Type` headers are not produced by the Jetty WebSocket implementation.
Do you have some kind of Filter that adds those headers?
If so, you'll need to make sure that Filter is "upgrade aware" and not do anything to the request/response if it detects a Upgrade being attempted.
This includes setting headers, wrapping the request or response, or attempting to access/control the request.inputStream/reader or response.outputStream/writer.

You can make your Filter "upgrade aware" by testing the request headers for "Connection: upgrade" (the value is case insensitive)

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Mon, Mar 18, 2019 at 2:17 PM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
ISO-8859-1 is the default charset of HTTP/1.x (you can see that in the old, now obsolete, https://tools.ietf.org/html/rfc2616)

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Mon, Mar 18, 2019 at 12:54 PM Meggyesi, Zalán <zmeggyesi@xxxxxxxx> wrote:
Hello Joakim,

I did take a Wireshark to both the Java-Websocket server and the Jetty webserver (packet captures attached), but to be honest, it only got me even more confused, since I'm now seeing encoding differences between the two, and I'm not really sure where the ISO-8859-1 charset is coming from nor why it would cause a length mismatch...

Best Regards,

Zalan Meggyesi
Chief Support Engineer
Skawa Innovation Kft.

Mobile: +36205146666


On Fri, Mar 8, 2019 at 10:23 PM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
You can use a tool like Wireshark and see what's being sent back/forth with a breakdown of the WebSocket frames (headers+payload)
You should be able to see where that bad Frame with RSV1 set to true is coming from.

From what I can tell, the project org.java_websocket is throwing that exception because it thinks it sees a frame with RSV1 set to true.
That should only be true for the combination of using permessage-deflate extension and the frame being a non-continuation data frame (TEXT/BINARY).

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Fri, Mar 8, 2019 at 11:52 AM Meggyesi, Zalán <zmeggyesi@xxxxxxxx> wrote:
Hello Joakim,

I ran a build of Jetty 10.0.0 based on the latest source, but I'm still getting the exception: org.java_websocket.exceptions.InvalidFrameException: bad rsv RSV1: true RSV2: false RSV3: false
I saw your PR merged, so I'm probably doing something wrong, but I have no idea what...

Best,

Zalan Meggyesi
Chief Support Engineer
Skawa Innovation Kft.

Mobile: +36205146666


On Fri, Mar 1, 2019 at 11:08 PM Meggyesi, Zalán <zmeggyesi@xxxxxxxx> wrote:
Hello again,

I did try sending Sec-WebSocket-Extensions:[deflate-frame] this time, but it still resulted in an exception (org.java_websocket.exceptions.InvalidFrameException: bad rsv RSV1: true RSV2: false RSV3: false).

Just to let you know.

Best,
Zalan

On Fri, Mar 1, 2019 at 10:36 PM Meggyesi, Zalán <zmeggyesi@xxxxxxxx> wrote:
Hello Joakim,

Thanks for your quick reply, this is very helpful!
I added a line to log the request headers:
22:18:22.772 [qtp33233312-29] INFO  h.s.s.g.filters.LoggerFilter - Headers: 
Sec-WebSocket-Key:[SquKmRXFd5Bt4jUcPD0mkw==]
Connection:[Upgrade]
Sec-WebSocket-Version:[13]
Host:[localhost:8000]
Upgrade:[websocket]
while the response headers have
Connection: Upgrade; Upgrade
Date: Fri, 01 Mar 2019 21:35:24 GMT
Sec-WebSocket-Accept: LlhSOQ+9EI8rCJ2z9vb5wEhG2bM=
Server: Jetty(9.4.12.v20180830)
Upgrade: WebSocket
Let me know if this is helpful to you!

Best Regards,
Zalan

On Fri, Mar 1, 2019 at 10:16 PM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
Actually, I am making an assumption that https://github.com/TooTallNate/Java-WebSocket/ only supports permessage-deflate.
It could be that it also supports deflate-frame (an older deprecated extension that Jetty also supports).

Can you please double check what the HTTP Upgrade Request headers and HTTP Upgrade Response headers are?
That will tell us exactly what extensions were negotiated.

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Fri, Mar 1, 2019 at 4:11 PM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
Disable permessage-deflate and don't negotiate it between your client / server.

You've stumbled into a variant of issue https://github.com/eclipse/jetty.project/issues/3159

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Fri, Mar 1, 2019 at 2:29 PM Meggyesi, Zalán <zmeggyesi@xxxxxxxx> wrote:
Hi,

I've floated this problem a while ago, but I only just got around to actually getting some data on it.

It seems like websocket handling ... well, for lack of a better word, "broke" after v9.4.12.v20180830.
I'm using the Ninja Framework, which bundles this version of Jetty as its internal webserver, and if I launch a skeleton WS server using the bundled Jetty instance, connections go through just fine. On the other hand, if I use the latest Jetty plugin from Maven Central, and launch the same WS server, I get an error when I try to send a message, specifically, org.java_websocket.exceptions.InvalidFrameException: bad rsv RSV1: true RSV2: false RSV3: false. I'm using this project as my WS client.

Now, my go-to reaction would be to simply downgrade my standalone Jetty installation to 9.4.12, but unfortunately because my real application runs under Java 11, I cannot do that (due to an outdated ASM in the 9.4.12 version).

Can anyone give me some pointers as where this might be going wrong?

Best,
Zalan
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--
Greg Wilkins <gregw@xxxxxxxxxxx> CTO http://webtide.com

Back to the top