Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » Re: [cdo] streaming large data?
Re: [cdo] streaming large data? [message #559222] Thu, 16 September 2010 05:20 Go to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6446
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------000901080308040604080005
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Hi Andr


Re: [cdo] streaming large data? [message #559239 is a reply to message #559222] Thu, 16 September 2010 07:07 Go to previous message
Eclipse UserFriend
Originally posted by: andre.dietisheim.gmail.com

Hi Eike, sorry: new mac, new setup, bad newsgroup ;-

Hi guys!

I currently do a little POC for guys, I convinced that CDO's is the best
solution for their needs ;-)

My usecase is the following:

A client-server application:
The clients are RCPs, the server is running in an OSGI runtime.
The server holds files that get derlivered to it by various channels
(ftp, stick, etc.). The server maintains an index of those files and
processes/parses them. The server shall deliver the files (raw or in
parsed form) if clients request it. The files are huge (up to 1GB and
more) and shall not get into a db.
The clients shall be informed of new files on the server (read only) and
shall be able to request them.

I intend to implement the shared index with cdo.
I currently don't see a way to stream those larges files with cdo
(imagined a 2nd repo with a custom file-store), since cdo is no able to
stream (net4j on the other hand is) large data:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=284307

Do I miss something? Any progress on this? Any better idea?

> I currently think at doing a minimalistic alternative and
> serving those files with a web-server. An interesting one
> would also be to provide remote osgi services. But this gets
> slightly off-topic.

thinking twice, even simpler would be to setup an additional net4j
communication between both. Guess this is the best solution as the
filetransfer needs none of the cdo specific feature (notification etc.)

Thanks for your ideas and inputs!

Cheers
André

On 9/16/10 7:20 AM, Eike Stepper wrote:
> Hi André,
>
> Please use the emf newsgroup for [CDO] questions ;-)
>
> More comments below...
>
> Cheers
> /Eike
> Contact: http://www.esc-net.de Blogger
> <http://thegordian.blogspot.com>Twitter
> <http://twitter.com/eikestepper>Linkedin
> <http://de.linkedin.com/in/eikestepper>Xing
> <http://www.xing.com/profile/Eike_Stepper>
> Article: What exactly is inside that p2 repository?
> < http://thegordian.blogspot.com/2010/05/what-exactly-is-insid e-that-p2.html>
>
> I'm speaking at Eclipse Summit Europe 2010 <http://www.eclipsesummit.org/>
>
>
>
> Am 15.09.2010 22:48, schrieb Andre Dietisheim:
>> Hi guys!
>>
>> I currently do a little POC for guys, I convinced that CDO's is the
>> best solution for their needs ;-)
>>
>> My usecase is the following:
>>
>> A client-server application:
>> The clients are RCPs, the server is running in an OSGI runtime.
>> The server holds files that get derlivered to it by various channels
>> (ftp, stick, etc.). The server maintains an index of those files and
>> processes/parses them.
> How's the server informed about additions to (removals from?) the file
> system store?
>
>> The server shall deliver the files (raw or in parsed form) if clients
>> request it. The files are huge (up to 1GB and more) and shall not get
>> into a db.
>> The clients shall be informed of new files on the server (read only)
>> and shall be able to request them.
> You can do this via CDO:
>
> 1. normal commit notifications (commits of changes to the index)
> 2. CDORemoteSessionManager
>
> Or via Net4j:
>
> 1. Make your own file share protocol
> 2. Look at
> http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org .eclipse.emf.cdo.deprecated/?root=Modeling_Project
>
> org.eclipse.net4j.examples.fshare/
> < http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org .eclipse.emf.cdo.deprecated/org.eclipse.net4j.examples.fshar e/?root=Modeling_Project>
>
> org.eclipse.net4j.examples.fshare.common/
> < http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org .eclipse.emf.cdo.deprecated/org.eclipse.net4j.examples.fshar e.common/?root=Modeling_Project>
>
> org.eclipse.net4j.examples.fshare.server/
> < http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org .eclipse.emf.cdo.deprecated/org.eclipse.net4j.examples.fshar e.server/?root=Modeling_Project>
>
> org.eclipse.net4j.examples.fshare.ui/
> < http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org .eclipse.emf.cdo.deprecated/org.eclipse.net4j.examples.fshar e.ui/?root=Modeling_Project>
>
>
> This example is explained in depth in
> http://it-republik.de/jaxenter/eclipse-magazin-ausgaben/Ecli pse-Galileo-000312.html
> (German!)
>
>
>> I intend to implement the shared index with cdo.
>> I currently don't see a way to stream those larges files with cdo
>> (imagined a 2nd repo with a custom file-store), since cdo is no able
>> to stream (net4j on the other hand is) large data:
>>
>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=284307
>>
>> Do I miss something?
> Nope.
>
>> Any progress on this?
> Nope.
>
>> Any better idea?
> Nope :P
>
> I'd support you if you'd decide to work on
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=284307
>
> Cheers
> /Eike
>
>
>>
>> I currently think at doing a minimalistic alternative and serving
>> those files with a web-server. An interesting one would also be to
>> provide remote osgi services. But this gets slightly off-topic.
>>
>> Thanks for your ideas and inputs!
>>
>> Cheers
>> André
>>
Previous Topic:Howto implement own serializiation format?
Next Topic:Emf->Xsd->Java code to serialize deserialize xml
Goto Forum:
  


Current Time: Sat Oct 19 04:32:08 GMT 2019

Powered by FUDForum. Page generated in 0.02490 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top