Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Communications Framework (ECF) » TimeoutInputStream suffered IO error
TimeoutInputStream suffered IO error [message #541901] Wed, 23 June 2010 01:49 Go to next message
Meng Xin Zhu is currently offline Meng Xin Zhu
Messages: 5
Registered: June 2010
Junior Member
My application is based on p2, so it indirectly consumes ecf.

Now the application tried to install something from repository, and the repository locates in a mount point that is mounted from remote server.

However ECF met below io errors while copying the file to cache folder, whose size is 800+ MB.
I guess it might be caused by unstable network traffic, how about using the 'retry' mechanism of httpclient to avoid it?
Is there anything we can do besides bless no network issue during transferring the huge file?

!SUBENTRY 1 org.eclipse.equinox.p2.repository 4 1002 2010-06-22 10:08:01.014
!MESSAGE Unable to read repository at file:/mnt/dvd/images/CDR-R138542.1-1-00/binary/wr-lx-ref-src -pkgs-glibc-std-3.0-src.wrlinux-pkgs.common.common.common.co mmon.common-all-all_3.0.0.0_200902261413.
!STACK 0
java.io.IOException: Input/output error
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:199)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:21 8)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:2 58)
at java.io.BufferedInputStream.read(BufferedInputStream.java:31 7)
at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea m.readUntilDone(TimeoutInputStream.java:322)
at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea m.runThread(TimeoutInputStream.java:266)
at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea m$1.run(TimeoutInputStream.java:77)
at java.lang.Thread.run(Thread.java:619)
Re: TimeoutInputStream suffered IO error [message #542200 is a reply to message #541901] Thu, 24 June 2010 04:34 Go to previous messageGo to next message
Markus Kuppe is currently offline Markus Kuppe
Messages: 177
Registered: July 2009
Senior Member
On 06/23/2010 07:49 AM, Meng Xin Zhu wrote:
> My application is based on p2, so it indirectly consumes ecf.
>
> Now the application tried to install something from repository, and the
> repository locates in a mount point that is mounted from remote server.
>
> However ECF met below io errors while copying the file to cache folder,
> whose size is 800+ MB.
> I guess it might be caused by unstable network traffic, how about using
> the 'retry' mechanism of httpclient to avoid it? Is there anything we
> can do besides bless no network issue during transferring the huge file?
> !SUBENTRY 1 org.eclipse.equinox.p2.repository 4 1002 2010-06-22
> 10:08:01.014
> !MESSAGE Unable to read repository at
> file:/mnt/dvd/images/CDR-R138542.1-1-00/binary/wr-lx-ref-src
> -pkgs-glibc-std-3.0-src.wrlinux-pkgs.common.common.common.co
> mmon.common-all-all_3.0.0.0_200902261413.
> !STACK 0
> java.io.IOException: Input/output error
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:199)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:21 8)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:2 58)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:31 7)
> at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea
> m.readUntilDone(TimeoutInputStream.java:322)
> at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea
> m.runThread(TimeoutInputStream.java:266)
> at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea
> m$1.run(TimeoutInputStream.java:77)
> at java.lang.Thread.run(Thread.java:619)

Hi,

you might want to file a bug at
https://bugs.eclipse.org/bugs/enter_bug.cgi?product=ECF under the
filetransfer component. Not sure how many of the filetransfer devs
follow this newsgroup.

Markus
Re: TimeoutInputStream suffered IO error [message #542867 is a reply to message #541901] Sat, 26 June 2010 17:25 Go to previous messageGo to next message
Scott Lewis is currently offline Scott Lewis
Messages: 966
Registered: July 2009
Senior Member
Hi Meng,

Meng Xin Zhu wrote:
> My application is based on p2, so it indirectly consumes ecf.
>
> Now the application tried to install something from repository, and the
> repository locates in a mount point that is mounted from remote server.
>
> However ECF met below io errors while copying the file to cache folder,
> whose size is 800+ MB.

Is this repeatable with this file on some network, or did it happen just
a single time?

> I guess it might be caused by unstable network traffic, how about using
> the 'retry' mechanism of httpclient to avoid it?

So, first of all...I think that you are probably using the
UrlConnection-based filetransfer provider...as the httpclient-based
filetransfer provider does not use TimeoutInputStream (and I see from
the stack trace that TimeoutInputStream is being used for this file
transfer).

I would suggest first that you use the httpclient-based provider instead
of the UrlConnection-based provider. The httpclient-based provider is
present in org.eclipse.ecf.provider.httpclient (and
org.eclipse.ecf.provider.httpclient.ssl).

Eclipse/p2 come with the httpclient provider (as well as the
UrlConnection-based provider)...so it should be as simple as adding the
o.e.e.provider.httpclient/ssl to your p2-based product. If both the
UrlConnection provider and the httpclient provider are present, the
httpclient provider will be preferred.

This might/probably will solve this issue completely, as the
httpclient-based provider has historically been more reliable than the
UrlConnection/jre-based provider...and that's why it's used in Eclipse/p2.

Now, there are also a number of timeout parameters that can be adjusted
for your particular app/use case.

http://wiki.eclipse.org/Equinox/p2/TransportDebugging

I believe there is also a retry capability built into the p2 usage of
ECF, but I don't think it's turned on by default. To find out for sure,
I suggest sending email to p2-dev at eclipse.org. The author of this
part of p2 was Henrick Lindberg I believe.


>Is there anything we
> can do besides bless no network issue during transferring the huge file?

Yes, see above.

As a further option you can/could introduce your own/new provider for
use with your particular app requirements. You could also base it upon
and/or extend the existing providers (e.g. httpclient-based provider),
if you like. If you decide to do this, it might be a good idea to
contribute this as an enhancement (to ECF/p2).

In general, we don't want to put an automatic retry in place as the
default for all use cases (i.e. Eclipse use of p2) because this will not
be the right strategy for many...perhaps most...failure situations.

Scott


> !SUBENTRY 1 org.eclipse.equinox.p2.repository 4 1002 2010-06-22
> 10:08:01.014
> !MESSAGE Unable to read repository at
> file:/mnt/dvd/images/CDR-R138542.1-1-00/binary/wr-lx-ref-src
> -pkgs-glibc-std-3.0-src.wrlinux-pkgs.common.common.common.co
> mmon.common-all-all_3.0.0.0_200902261413.
> !STACK 0
> java.io.IOException: Input/output error
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:199)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:21 8)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:2 58)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:31 7)
> at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea
> m.readUntilDone(TimeoutInputStream.java:322)
> at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea
> m.runThread(TimeoutInputStream.java:266)
> at org.eclipse.ecf.provider.filetransfer.util.TimeoutInputStrea
> m$1.run(TimeoutInputStream.java:77)
> at java.lang.Thread.run(Thread.java:619)
Re: TimeoutInputStream suffered IO error [message #543031 is a reply to message #542867] Mon, 28 June 2010 05:08 Go to previous messageGo to next message
Meng Xin Zhu is currently offline Meng Xin Zhu
Messages: 5
Registered: June 2010
Junior Member
Thanks for your reply.

There are several ways to install the product.
One is the install the product from online repository, so the URLConnection would be used when downloading the artifacts from repository.
Another is the media, such as DVD or CDs. The download speed also is fast from the local media, no timeout issue to be considered.
However the linux users would like to mount the remote server as a local point, so the scheme of url of repository is the 'file'. The internal implementation of p2 will use the file transfer API for those url protocol. So third party application(my installer) is not easy to change the transfer implementation to url way.

I think the property of retry number or timeout is a good suggestion. I'll try that.
Re: TimeoutInputStream suffered IO error [message #543284 is a reply to message #543031] Mon, 28 June 2010 19:46 Go to previous message
Scott Lewis is currently offline Scott Lewis
Messages: 966
Registered: July 2009
Senior Member
Hi Meng,

Meng Xin Zhu wrote:
> Thanks for your reply.
>
> There are several ways to install the product.
> One is the install the product from online repository, so the
> URLConnection would be used when downloading the artifacts from repository.

What, in this case, is the p2 agent that is doing the initial install?
I assume it isn't Eclipse, but what is it? If it's something that you
create/control, then it would be up to you, as the installer creator, to
decide what bundles are included in this agent...and it can/could
include the httpclient provider if you like (or it could just include
the URLConnection provider).

> Another is the media, such as DVD or CDs. The download speed also is
> fast from the local media, no timeout issue to be considered.
> However the linux users would like to mount the remote server as a local
> point, so the scheme of url of repository is the 'file'.

FWIW, in the case of the 'file' protocol, ECF simply uses the
JRE-provided access to local filesystem.

>The internal
> implementation of p2 will use the file transfer API for those url
> protocol.

Yes. Actually, p2 will use the file transfer API for any/all protocols.

I haven't made this clear before on this thread, but I should probably
say that the ECF filetransfer system allows new providers to be plugged
in on a protocol-specific basis...e.g. if you want, you can define your
own transport protocol...call it 'meng' and define URIs/URLs of the form:

meng://whatever you want

And the ECF filetransfer API will then use your provider implementation.

This is how, btw, the httpclient provider is setup to provide for http
protocol support rather than the JRE/URLConnection-based provider...it's
registered as the provider (via extensions for these extension points:

org.eclipse.ecf.provider.filetransfer.retrieveFileTransferPr otocolFactory
org.eclipse.ecf.provider.filetransfer.browseFileTransferProt ocolFactory
org.eclipse.ecf.provider.filetransfer.sendFileTransferProtoc olFactory

There is also a simple priority system, that allows one to define the
priority of protocol factories WRT handling requests for a specific
protocol (e.g. http).

>So third party application(my installer) is not easy to change
> the transfer implementation to url way.

I'm not sure I understand the difficulty here...the ECF provider lets
individual providers to be associated with specific protocols...so the
JRE provider handles 'file' URLs, httpclient handles 'http/https' (by
default in Eclipse), and any/all other providers can be installed to
either support new protocols, or even to override the other protocol
support (e.g. if you like, you could just insert your own provider for
'file' or 'http' or whatever protocol).

>
> I think the property of retry number or timeout is a good suggestion.
> I'll try that.

Yes, that might fix this particular issue...but just to be clear...with
usage in p2 and Eclipse, after quite a lot of usage...we have found the
httpclient-based provider to be more robust (which is why it's the one
that's used in Eclipse/p2 by default).

Scott
Re: TimeoutInputStream suffered IO error [message #625280 is a reply to message #542867] Mon, 28 June 2010 05:08 Go to previous message
Meng Xin Zhu is currently offline Meng Xin Zhu
Messages: 5
Registered: June 2010
Junior Member
Thanks for your reply.

There are several ways to install the product.
One is the install the product from online repository, so the URLConnection would be used when downloading the artifacts from repository.
Another is the media, such as DVD or CDs. The download speed also is fast from the local media, no timeout issue to be considered.
However the linux users would like to mount the remote server as a local point, so the scheme of url of repository is the 'file'. The internal implementation of p2 will use the file transfer API for those url protocol. So third party application(my installer) is not easy to change the transfer implementation to url way.

I think the property of retry number or timeout is a good suggestion. I'll try that.
Re: TimeoutInputStream suffered IO error [message #625282 is a reply to message #625280] Mon, 28 June 2010 19:46 Go to previous message
Scott Lewis is currently offline Scott Lewis
Messages: 966
Registered: July 2009
Senior Member
Hi Meng,

Meng Xin Zhu wrote:
> Thanks for your reply.
>
> There are several ways to install the product.
> One is the install the product from online repository, so the
> URLConnection would be used when downloading the artifacts from repository.

What, in this case, is the p2 agent that is doing the initial install?
I assume it isn't Eclipse, but what is it? If it's something that you
create/control, then it would be up to you, as the installer creator, to
decide what bundles are included in this agent...and it can/could
include the httpclient provider if you like (or it could just include
the URLConnection provider).

> Another is the media, such as DVD or CDs. The download speed also is
> fast from the local media, no timeout issue to be considered.
> However the linux users would like to mount the remote server as a local
> point, so the scheme of url of repository is the 'file'.

FWIW, in the case of the 'file' protocol, ECF simply uses the
JRE-provided access to local filesystem.

>The internal
> implementation of p2 will use the file transfer API for those url
> protocol.

Yes. Actually, p2 will use the file transfer API for any/all protocols.

I haven't made this clear before on this thread, but I should probably
say that the ECF filetransfer system allows new providers to be plugged
in on a protocol-specific basis...e.g. if you want, you can define your
own transport protocol...call it 'meng' and define URIs/URLs of the form:

meng://whatever you want

And the ECF filetransfer API will then use your provider implementation.

This is how, btw, the httpclient provider is setup to provide for http
protocol support rather than the JRE/URLConnection-based provider...it's
registered as the provider (via extensions for these extension points:

org.eclipse.ecf.provider.filetransfer.retrieveFileTransferPr otocolFactory
org.eclipse.ecf.provider.filetransfer.browseFileTransferProt ocolFactory
org.eclipse.ecf.provider.filetransfer.sendFileTransferProtoc olFactory

There is also a simple priority system, that allows one to define the
priority of protocol factories WRT handling requests for a specific
protocol (e.g. http).

>So third party application(my installer) is not easy to change
> the transfer implementation to url way.

I'm not sure I understand the difficulty here...the ECF provider lets
individual providers to be associated with specific protocols...so the
JRE provider handles 'file' URLs, httpclient handles 'http/https' (by
default in Eclipse), and any/all other providers can be installed to
either support new protocols, or even to override the other protocol
support (e.g. if you like, you could just insert your own provider for
'file' or 'http' or whatever protocol).

>
> I think the property of retry number or timeout is a good suggestion.
> I'll try that.

Yes, that might fix this particular issue...but just to be clear...with
usage in p2 and Eclipse, after quite a lot of usage...we have found the
httpclient-based provider to be more robust (which is why it's the one
that's used in Eclipse/p2 by default).

Scott
Previous Topic:TimeoutInputStream suffered IO error
Next Topic:Navigator UI for browsing file systems
Goto Forum:
  


Current Time: Mon Jul 28 08:40:54 EDT 2014

Powered by FUDForum. Page generated in 0.01788 seconds