Eclipse Community Forums: EMF "Technology" (Ecore Tools, EMFatic, etc)

Help

Home

Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » [CDO][0.8.0] Handling Large Collection

Show: Today's Messages :: Show Polls :: Message Navigator

[CDO][0.8.0] Handling Large Collection [message #94249]

Sun, 26 August 2007 17:22

Simon Mc Duff

Messages: 596
Registered: July 2009

Senior Member

Hi Eike,

Do you see any value to chunk collection from the server to the client ? It
is possible to do right now in your framework ?

The problem we have right now is the following.

We have objects that have many attributes. Many of them are ToMany
relationship. In each collection, you have between 100 000 and 500 000
items.

When I fetch the object that own these collections... I don`t want to fetch
all of them.

So if we could fetch let`s say by chunk of 1000 thousand elements, it will
looks like the following

- ObjectA first fetch
- fetch CDOID for collection ObjectA.coll1 [0...1000].
- fetch CDOID for collection ObjectA.coll2 [0...1000].

If I want the last objects of the collection ObjectA.coll1 . I don`t need to
fetch all of them. I can fetch
[499000 to 499 999] (in the case where I have 500 000 item in my
collection)

It will be nice to have something hat looks like EStore :

CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature feature,
int indexFrom, int indexTo);

This bring another problem... how we will write changes...if we have only
a part of it.

It will be nice to have only what changed instead of having the whole
objects

Right now we have the following:
public void writeRevision(CDORevisionImpl revision);

WIll be better to have the following :
public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );

Thank you.

Report message to a moderator

Re: [CDO][0.8.0] Handling Large Collection [message #94266 is a reply to message #94249]

Mon, 27 August 2007 14:52

Eclipse User

Originally posted by: stepper.sympedia.de

Hi Simon,

Yes, the idea is good. Please add a feature request for it.
These chunk'ed collections will probably only be available for
CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
implement them if the object is backed by an EStore.

Could you please add a separate feature request for the revision deltas.
It is not 100% sure that we need this feature to implement chunk'ed
collections.
But having it will be good anyway.

Cheers
/Eike

Simon McDuff schrieb:
> Hi Eike,
>
> Do you see any value to chunk collection from the server to the client ? It
> is possible to do right now in your framework ?
>
> The problem we have right now is the following.
>
> We have objects that have many attributes. Many of them are ToMany
> relationship. In each collection, you have between 100 000 and 500 000
> items.
>
> When I fetch the object that own these collections... I don`t want to fetch
> all of them.
>
> So if we could fetch let`s say by chunk of 1000 thousand elements, it will
> looks like the following
>
> - ObjectA first fetch
> - fetch CDOID for collection ObjectA.coll1 [0...1000].
> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>
> If I want the last objects of the collection ObjectA.coll1 . I don`t need to
> fetch all of them. I can fetch
> [499000 to 499 999] (in the case where I have 500 000 item in my
> collection)
>
> It will be nice to have something hat looks like EStore :
>
> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature feature,
> int indexFrom, int indexTo);
>
> This bring another problem... how we will write changes...if we have only
> a part of it.
>
> It will be nice to have only what changed instead of having the whole
> objects
>
>
> Right now we have the following:
> public void writeRevision(CDORevisionImpl revision);
>
> WIll be better to have the following :
> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>
> Thank you.
>
>
>
>
>
>
>
>

Report message to a moderator

Re: [CDO][0.8.0] Handling Large Collection [message #94268 is a reply to message #94266]

Mon, 27 August 2007 14:57

Simon Mc Duff

Messages: 596
Registered: July 2009

Senior Member

https://bugs.eclipse.org/bugs/show_bug.cgi?id=201265

"Eike Stepper" <stepper@sympedia.de> wrote in message
news:fauoee$frt$2@build.eclipse.org...
> Hi Simon,
>
> Yes, the idea is good. Please add a feature request for it.
> These chunk'ed collections will probably only be available for
> CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
> implement them if the object is backed by an EStore.
>
> Could you please add a separate feature request for the revision deltas.
> It is not 100% sure that we need this feature to implement chunk'ed
> collections.
> But having it will be good anyway.
>
> Cheers
> /Eike
>
>
> Simon McDuff schrieb:
>> Hi Eike,
>>
>> Do you see any value to chunk collection from the server to the client ?
>> It is possible to do right now in your framework ?
>>
>> The problem we have right now is the following.
>>
>> We have objects that have many attributes. Many of them are ToMany
>> relationship. In each collection, you have between 100 000 and 500 000
>> items.
>>
>> When I fetch the object that own these collections... I don`t want to
>> fetch all of them.
>>
>> So if we could fetch let`s say by chunk of 1000 thousand elements, it
>> will looks like the following
>>
>> - ObjectA first fetch
>> - fetch CDOID for collection ObjectA.coll1 [0...1000].
>> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>>
>> If I want the last objects of the collection ObjectA.coll1 . I don`t need
>> to fetch all of them. I can fetch
>> [499000 to 499 999] (in the case where I have 500 000 item in my
>> collection)
>>
>> It will be nice to have something hat looks like EStore :
>>
>> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature
>> feature, int indexFrom, int indexTo);
>>
>> This bring another problem... how we will write changes...if we have
>> only a part of it.
>>
>> It will be nice to have only what changed instead of having the whole
>> objects
>>
>>
>> Right now we have the following:
>> public void writeRevision(CDORevisionImpl revision);
>>
>> WIll be better to have the following :
>> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>>
>> Thank you.
>>
>>
>>
>>
>>
>>
>>
>>

Report message to a moderator

Re: [CDO][0.8.0] Handling Large Collection [message #94269 is a reply to message #94266]

Mon, 27 August 2007 14:58

Simon Mc Duff

Messages: 596
Registered: July 2009

Senior Member

https://bugs.eclipse.org/bugs/show_bug.cgi?id=201266

"Eike Stepper" <stepper@sympedia.de> wrote in message
news:fauoee$frt$2@build.eclipse.org...
> Hi Simon,
>
> Yes, the idea is good. Please add a feature request for it.
> These chunk'ed collections will probably only be available for
> CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
> implement them if the object is backed by an EStore.
>
> Could you please add a separate feature request for the revision deltas.
> It is not 100% sure that we need this feature to implement chunk'ed
> collections.
> But having it will be good anyway.
>
> Cheers
> /Eike
>
>
> Simon McDuff schrieb:
>> Hi Eike,
>>
>> Do you see any value to chunk collection from the server to the client ?
>> It is possible to do right now in your framework ?
>>
>> The problem we have right now is the following.
>>
>> We have objects that have many attributes. Many of them are ToMany
>> relationship. In each collection, you have between 100 000 and 500 000
>> items.
>>
>> When I fetch the object that own these collections... I don`t want to
>> fetch all of them.
>>
>> So if we could fetch let`s say by chunk of 1000 thousand elements, it
>> will looks like the following
>>
>> - ObjectA first fetch
>> - fetch CDOID for collection ObjectA.coll1 [0...1000].
>> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>>
>> If I want the last objects of the collection ObjectA.coll1 . I don`t need
>> to fetch all of them. I can fetch
>> [499000 to 499 999] (in the case where I have 500 000 item in my
>> collection)
>>
>> It will be nice to have something hat looks like EStore :
>>
>> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature
>> feature, int indexFrom, int indexTo);
>>
>> This bring another problem... how we will write changes...if we have
>> only a part of it.
>>
>> It will be nice to have only what changed instead of having the whole
>> objects
>>
>>
>> Right now we have the following:
>> public void writeRevision(CDORevisionImpl revision);
>>
>> WIll be better to have the following :
>> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>>
>> Thank you.
>>
>>
>>
>>
>>
>>
>>
>>

Report message to a moderator

Re: [CDO][0.8.0] Handling Large Collection [message #609535 is a reply to message #94249]

Mon, 27 August 2007 14:52

Eike Stepper

Messages: 6682
Registered: July 2009

Senior Member

Hi Simon,

Yes, the idea is good. Please add a feature request for it.
These chunk'ed collections will probably only be available for
CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
implement them if the object is backed by an EStore.

Could you please add a separate feature request for the revision deltas.
It is not 100% sure that we need this feature to implement chunk'ed
collections.
But having it will be good anyway.

Cheers
/Eike

Simon McDuff schrieb:
> Hi Eike,
>
> Do you see any value to chunk collection from the server to the client ? It
> is possible to do right now in your framework ?
>
> The problem we have right now is the following.
>
> We have objects that have many attributes. Many of them are ToMany
> relationship. In each collection, you have between 100 000 and 500 000
> items.
>
> When I fetch the object that own these collections... I don`t want to fetch
> all of them.
>
> So if we could fetch let`s say by chunk of 1000 thousand elements, it will
> looks like the following
>
> - ObjectA first fetch
> - fetch CDOID for collection ObjectA.coll1 [0...1000].
> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>
> If I want the last objects of the collection ObjectA.coll1 . I don`t need to
> fetch all of them. I can fetch
> [499000 to 499 999] (in the case where I have 500 000 item in my
> collection)
>
> It will be nice to have something hat looks like EStore :
>
> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature feature,
> int indexFrom, int indexTo);
>
> This bring another problem... how we will write changes...if we have only
> a part of it.
>
> It will be nice to have only what changed instead of having the whole
> objects
>
>
> Right now we have the following:
> public void writeRevision(CDORevisionImpl revision);
>
> WIll be better to have the following :
> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>
> Thank you.
>
>
>
>
>
>
>
>

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper

Report message to a moderator