Re: [ecf-dev] File sharing inquiries...

[Scott Lewis <slewis@xxxxxxxxxxxxx>]

Hi Rem,

Remy Suen wrote:
> Hi everyone,
>
> I'm Remy Suen (you can just call me Rem), as some of you may know if 
> you've been listening in on some of the past couple of ECF meeting 
> conference calls. I am one of the three students that are working on 
> ECF-related projects via Google's Summer of Code program. 
> I'm writing a clean room implementation of the BitTorrent protocol and then creating a provider plug-in for ECF's file sharing APIs. 
> I've got some issues I'd like to bring to the table in hopes that some 
> of you would be able to help with.
>
> 1. How does one actually limit upload/download rates?
>
> I've been poking away at a number of BitTorrent issues and features 
> but have yet to start tackling this since I'm not even really sure how 
> to start. Is it really just a matter of having a thread in the 
> connection looking at past uploads and doing some kind of rolling 
> average and then deciding how much data to write to the channel/socket 
> in one second? This sounds like it could work (or not), but then now I 
> have two threads per connection, which sounds like a lot, although I'm 
> not sure if it really is. So any tips or hints on this would be most 
> appreciated as I don't even know where to begin.

For the filesharing API, some specification of upload/download rates 
could be added to the API to allow providers (like Bittorrent) that 
support such features to expose them to clients on an individual file 
level (I assume BT supports setting rates at the level of individual 
files...either before or during file retrieval).

> 2. How will the org.eclipse.ecf.fileshare API fit in with BitTorrent, 
> a bi-directional piece-sharing protocol?
>
> The file sharing API is currently heavily biased towards unilateral 
> file transportation services (rightfully so), which really leaves 
> little API for me to use. I could certainly send 
> IFileShareRetrieveDataEvents per block that I receive from a peer, but 
> I can see that that getData() method isn't going to be really useful. 
> Similar arguments could be had for 
> IFileShareRetrieveStartEvent and its sending event counterparts. I guess what I'm trying to get at is that a BitTorrent provider for ECF is certainly doable 
> (and I think I'm pretty close to having a simple one done, but the 
> whole Namespace and ID concept seems awfully confusing to 
> me), but just what kind of useful 
> information can clients actually retrieve right now based on the current API? 

Not very much...basically just the contents of the file itself.  But my 
intention with the fileshare API was to expose other methods (on either 
the IFileShareContainer, IFileShareRetrieveConfig, or IFileShare) OR add 
adapter interfaces via IAdaptable for categories of API like rate 
control, statistics gathering, etc.  Also new event types can easily be 
created added for handling asynch responses to requests.

So what I would like to do is to have Remy propose a desired set of 
features (e.g. rate control) and then we can propose an 
enchancement/generalization of the file share API to support.

RE: the namespace/ID concept...the notion is have some sort of 
structured and extensible way to represent addressing information (other 
than just String or URI).  The ECF namespace extension point allows ECF 
providers to create their own types of IDs...with whatever structure 
they require to refer to a remote service, file, user, etc.  e.g.

ecf.provider.bt://slewis@localhost/path/to/filename.bt

or

http://www.eclipse.org/file.txt

or

ecftcp://ecf.eclipse.org:3282/server

There's a little more available here: 
http://www.eclipse.org/ecf/documentation.html#Identity

But as usual more is needed.

Scott