Re: [jetty-users] how to handle invalid UTF8 chars

[Mattia Merzi <mattia.merzi@xxxxxxxxx>]

right, a few more infos:

- requests that generate this exception do not specify the
content-type head or have a correct content-type (UTF8)
(we trash any request with a content-type != UTF8)
- requests *may* contain *invalid utf8 sequences*, but we would like
to *accept them anyway*, trashing the invalid characters or
replacing them with something else; please not to ask
why, it would be really complicated :)
- we do not have any kind of control on the clients, it is impossible
for us to change the requests or to ask the clients to change their
behaviour
- implementing a filtering on the data (something like a proxy that
deletes all invalid characters) would be very expensive: we have
sometimes something like tens of thousands of requests per minute,
and most of all, it would be one more software to install, configure,
and maintain :)
- requests pass through an apache web server with mod_proxy;
if interesting, I can post the configuration
- various types of post or get data generate this exception, maybe
very short (8~10 bytes payload), maybe very big (8~10 MB payload)
- jetty is used embedded in our application; if useful I can post the most
relevant code used to start the servlet container, let me know
- jetty 7.3.0 works perfectly,  jetty 8.1.0.RC2 generate this exception:
org.eclipse.jetty.util.Utf8Appendable$NotUtf8Exception: Not valid
UTF8! byte A1 in state 0
        at org.eclipse.jetty.util.Utf8Appendable.appendByte(Utf8Appendable.java:168)
        at org.eclipse.jetty.util.Utf8Appendable.append(Utf8Appendable.java:93)
        at org.eclipse.jetty.util.UrlEncoded.decodeUtf8To(UrlEncoded.java:482)
        at org.eclipse.jetty.util.UrlEncoded.decodeTo(UrlEncoded.java:533)
        at org.eclipse.jetty.server.Request.extractParameters(Request.java:277)
        at org.eclipse.jetty.server.Request.getParameterNames(Request.java:709)
[... our classes that extends servlet ]
- if needed, I can post the tcpdump of a few requests that generate the
exception, but sorry, I have not one available here

Thanks,

Greetings,

Mattia.



2012/1/24 Simone Bordet <sbordet@xxxxxxxxxxx>:
> Hi,
>
> On Tue, Jan 24, 2012 at 15:13, Mattia Merzi <mattia.merzi@xxxxxxxxx> wrote:
>> Hi everyone,
>>
>> I've recently updated jetty libs (I'm using jetty embedded) to the latest
>> version, and I'm having troubles with this exception:
>>
>> org.eclipse.jetty.util.Utf8Appendable$NotUtf8Exception: Not valid
>> UTF8! byte 20 in state 3
>>
>> is there a way to ask jetty to just trash invalid chars instead of
>> throwing an exception?
>>
>> If you need some more details, just ask.
>
> Is this a body of a request ? If so, the solution is to specify the
> right content-type and charset.
>
> If this a URL ? If so, it must be encoded as UTF8.
>
> Something else ?
>
> Stack trace ?
>
> Simon
> --
> http://cometd.org
> http://intalio.com
> http://bordet.blogspot.com
> ----
> Finally, no matter how good the architecture and design are,
> to deliver bug-free software with optimal performance and reliability,
> the implementation technique must be flawless.   Victoria Livschitz
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-users