[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [jetty-users] [eclipse/jetty.project] Lockup processing POST request body with Jetty 12.0.1 using http/2 (Issue #10513)
|
Hello Joakim,
You talk about the code in the alternative servlet TestServlet2 that
treats the POST body as a ZIP. As posted the code however uses
TestServlet1 that simply uses input.transferTo to read and discard
all bytes of the POST body without making any assumptions about the
content. This results in the same lockup for larger files.
So although I understand your point about reading the complete body
when parsing input in some format while reading the request that is
not the cause of the issue. I merely posted the second servlet
because it demonstrates that the first part of the request body is
read properly, printing out the name/size of several initial entries
in a posted ZIP file and then stalling.
Kind regards,
Silvio
On 14-09-2023 14:55, Joakim Erdfelt
wrote:
You have to finish reading from the HTTP stream.
The ZipInputStream does not read to EOF.
Just closing the request.getInputStream is insufficient in many
cases, as it can leave data on the connection that is unread
(breaking HTTP/1 connection persistence in the process).
What if the transfer-encoding was chunked? and the terminal
chunk hasn't been read from the network buffers yet?
What if the transfer-encoding final chunk hasn't even been sent
by the client yet?
What if the stream contained superflous data, unrelated to the
Zip stream after the zip stream?
All your code is doing is iterating through the Zip
directory/dictionary/index for entries, and reading the entry
content.
How this is done is by reading the entire stream until it finds
the bytes indicating the start of directory section at the end
of the stream, then reading each entry of the directory to know
where in the previous parts of the stream the ZipEntry contents
are located (and what size they are).
So you are basically reading ALL of the file contents first,
then slowly iterating through the dictionary, and then going
backwards in the zip stream to each entry contents, once you
reach an entry in the Zip directory section that cannot be
represented by java ZipEntry it stops. You haven't actually read
the whole stream yet if that last entry is something structural
that cannot be represented by a java ZipEntry.
How do you fix this?
You have to satisfy the basics of HTTP here.
Even libraries that read / parse JSON and XML have this same
issue.
Every modern JSON and XML library have options to continue
reading to EOF.
They can even be configured to behave differently depending on
what is found after the content.
Eg: in many XML and JSON parsers there's an option to fail the
document parse if any non-whitespace is discovered after the
document.
See TODO in the code snippet below.
val in = request.getInputStream
val zip = new ZipInputStream(in)
var entry = zip.getNextEntry
while (entry != null)
{
val out = new CountingOutputStream(new NullOutputStream)
zip.transferTo(out)
out.close
System.err.println(entry.getName + ":" + out.bytesWritten)
entry = zip.getNextEntry
}
zip.close
// TODO: you need to finish reading from `in` here, the HTTP stream is not yet complete, just the ZIP content is.
// You have to choose a behavior here,
// eg: if there is content, do you fail the upload? or ignore it the extra content? warn the user?
// small amounts of content ok? large amounts of extra content show a problem?
If you don't want to do this, you can send the Zip
stream in a multipart/form-data
,
which puts the HTTP stream behavior requirement on the Server,
not you.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message
ID: <eclipse/jetty.project/issues/10513/1719399968@github.com>