Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc)  » [CDO][0.8.0] Handling Large Collection
[CDO][0.8.0] Handling Large Collection [message #94249] Sun, 26 August 2007 17:22 Go to next message
Simon Mc Duff is currently offline Simon Mc DuffFriend
Messages: 596
Registered: July 2009
Senior Member
Hi Eike,

Do you see any value to chunk collection from the server to the client ? It
is possible to do right now in your framework ?

The problem we have right now is the following.

We have objects that have many attributes. Many of them are ToMany
relationship. In each collection, you have between 100 000 and 500 000
items.

When I fetch the object that own these collections... I don`t want to fetch
all of them.

So if we could fetch let`s say by chunk of 1000 thousand elements, it will
looks like the following

- ObjectA first fetch
- fetch CDOID for collection ObjectA.coll1 [0...1000].
- fetch CDOID for collection ObjectA.coll2 [0...1000].

If I want the last objects of the collection ObjectA.coll1 . I don`t need to
fetch all of them. I can fetch
[499000 to 499 999] (in the case where I have 500 000 item in my
collection)

It will be nice to have something hat looks like EStore :

CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature feature,
int indexFrom, int indexTo);

This bring another problem... how we will write changes...if we have only
a part of it.

It will be nice to have only what changed instead of having the whole
objects


Right now we have the following:
public void writeRevision(CDORevisionImpl revision);

WIll be better to have the following :
public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );

Thank you.
Re: [CDO][0.8.0] Handling Large Collection [message #94266 is a reply to message #94249] Mon, 27 August 2007 14:52 Go to previous messageGo to next message
Eclipse UserFriend
Originally posted by: stepper.sympedia.de

Hi Simon,

Yes, the idea is good. Please add a feature request for it.
These chunk'ed collections will probably only be available for
CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
implement them if the object is backed by an EStore.

Could you please add a separate feature request for the revision deltas.
It is not 100% sure that we need this feature to implement chunk'ed
collections.
But having it will be good anyway.

Cheers
/Eike


Simon McDuff schrieb:
> Hi Eike,
>
> Do you see any value to chunk collection from the server to the client ? It
> is possible to do right now in your framework ?
>
> The problem we have right now is the following.
>
> We have objects that have many attributes. Many of them are ToMany
> relationship. In each collection, you have between 100 000 and 500 000
> items.
>
> When I fetch the object that own these collections... I don`t want to fetch
> all of them.
>
> So if we could fetch let`s say by chunk of 1000 thousand elements, it will
> looks like the following
>
> - ObjectA first fetch
> - fetch CDOID for collection ObjectA.coll1 [0...1000].
> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>
> If I want the last objects of the collection ObjectA.coll1 . I don`t need to
> fetch all of them. I can fetch
> [499000 to 499 999] (in the case where I have 500 000 item in my
> collection)
>
> It will be nice to have something hat looks like EStore :
>
> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature feature,
> int indexFrom, int indexTo);
>
> This bring another problem... how we will write changes...if we have only
> a part of it.
>
> It will be nice to have only what changed instead of having the whole
> objects
>
>
> Right now we have the following:
> public void writeRevision(CDORevisionImpl revision);
>
> WIll be better to have the following :
> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>
> Thank you.
>
>
>
>
>
>
>
>
Re: [CDO][0.8.0] Handling Large Collection [message #94268 is a reply to message #94266] Mon, 27 August 2007 14:57 Go to previous messageGo to next message
Simon Mc Duff is currently offline Simon Mc DuffFriend
Messages: 596
Registered: July 2009
Senior Member
https://bugs.eclipse.org/bugs/show_bug.cgi?id=201265


"Eike Stepper" <stepper@sympedia.de> wrote in message
news:fauoee$frt$2@build.eclipse.org...
> Hi Simon,
>
> Yes, the idea is good. Please add a feature request for it.
> These chunk'ed collections will probably only be available for
> CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
> implement them if the object is backed by an EStore.
>
> Could you please add a separate feature request for the revision deltas.
> It is not 100% sure that we need this feature to implement chunk'ed
> collections.
> But having it will be good anyway.
>
> Cheers
> /Eike
>
>
> Simon McDuff schrieb:
>> Hi Eike,
>>
>> Do you see any value to chunk collection from the server to the client ?
>> It is possible to do right now in your framework ?
>>
>> The problem we have right now is the following.
>>
>> We have objects that have many attributes. Many of them are ToMany
>> relationship. In each collection, you have between 100 000 and 500 000
>> items.
>>
>> When I fetch the object that own these collections... I don`t want to
>> fetch all of them.
>>
>> So if we could fetch let`s say by chunk of 1000 thousand elements, it
>> will looks like the following
>>
>> - ObjectA first fetch
>> - fetch CDOID for collection ObjectA.coll1 [0...1000].
>> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>>
>> If I want the last objects of the collection ObjectA.coll1 . I don`t need
>> to fetch all of them. I can fetch
>> [499000 to 499 999] (in the case where I have 500 000 item in my
>> collection)
>>
>> It will be nice to have something hat looks like EStore :
>>
>> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature
>> feature, int indexFrom, int indexTo);
>>
>> This bring another problem... how we will write changes...if we have
>> only a part of it.
>>
>> It will be nice to have only what changed instead of having the whole
>> objects
>>
>>
>> Right now we have the following:
>> public void writeRevision(CDORevisionImpl revision);
>>
>> WIll be better to have the following :
>> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>>
>> Thank you.
>>
>>
>>
>>
>>
>>
>>
>>
Re: [CDO][0.8.0] Handling Large Collection [message #94269 is a reply to message #94266] Mon, 27 August 2007 14:58 Go to previous message
Simon Mc Duff is currently offline Simon Mc DuffFriend
Messages: 596
Registered: July 2009
Senior Member
https://bugs.eclipse.org/bugs/show_bug.cgi?id=201266


"Eike Stepper" <stepper@sympedia.de> wrote in message
news:fauoee$frt$2@build.eclipse.org...
> Hi Simon,
>
> Yes, the idea is good. Please add a feature request for it.
> These chunk'ed collections will probably only be available for
> CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
> implement them if the object is backed by an EStore.
>
> Could you please add a separate feature request for the revision deltas.
> It is not 100% sure that we need this feature to implement chunk'ed
> collections.
> But having it will be good anyway.
>
> Cheers
> /Eike
>
>
> Simon McDuff schrieb:
>> Hi Eike,
>>
>> Do you see any value to chunk collection from the server to the client ?
>> It is possible to do right now in your framework ?
>>
>> The problem we have right now is the following.
>>
>> We have objects that have many attributes. Many of them are ToMany
>> relationship. In each collection, you have between 100 000 and 500 000
>> items.
>>
>> When I fetch the object that own these collections... I don`t want to
>> fetch all of them.
>>
>> So if we could fetch let`s say by chunk of 1000 thousand elements, it
>> will looks like the following
>>
>> - ObjectA first fetch
>> - fetch CDOID for collection ObjectA.coll1 [0...1000].
>> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>>
>> If I want the last objects of the collection ObjectA.coll1 . I don`t need
>> to fetch all of them. I can fetch
>> [499000 to 499 999] (in the case where I have 500 000 item in my
>> collection)
>>
>> It will be nice to have something hat looks like EStore :
>>
>> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature
>> feature, int indexFrom, int indexTo);
>>
>> This bring another problem... how we will write changes...if we have
>> only a part of it.
>>
>> It will be nice to have only what changed instead of having the whole
>> objects
>>
>>
>> Right now we have the following:
>> public void writeRevision(CDORevisionImpl revision);
>>
>> WIll be better to have the following :
>> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>>
>> Thank you.
>>
>>
>>
>>
>>
>>
>>
>>
Re: [CDO][0.8.0] Handling Large Collection [message #609535 is a reply to message #94249] Mon, 27 August 2007 14:52 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6629
Registered: July 2009
Senior Member
Hi Simon,

Yes, the idea is good. Please add a feature request for it.
These chunk'ed collections will probably only be available for
CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
implement them if the object is backed by an EStore.

Could you please add a separate feature request for the revision deltas.
It is not 100% sure that we need this feature to implement chunk'ed
collections.
But having it will be good anyway.

Cheers
/Eike


Simon McDuff schrieb:
> Hi Eike,
>
> Do you see any value to chunk collection from the server to the client ? It
> is possible to do right now in your framework ?
>
> The problem we have right now is the following.
>
> We have objects that have many attributes. Many of them are ToMany
> relationship. In each collection, you have between 100 000 and 500 000
> items.
>
> When I fetch the object that own these collections... I don`t want to fetch
> all of them.
>
> So if we could fetch let`s say by chunk of 1000 thousand elements, it will
> looks like the following
>
> - ObjectA first fetch
> - fetch CDOID for collection ObjectA.coll1 [0...1000].
> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>
> If I want the last objects of the collection ObjectA.coll1 . I don`t need to
> fetch all of them. I can fetch
> [499000 to 499 999] (in the case where I have 500 000 item in my
> collection)
>
> It will be nice to have something hat looks like EStore :
>
> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature feature,
> int indexFrom, int indexTo);
>
> This bring another problem... how we will write changes...if we have only
> a part of it.
>
> It will be nice to have only what changed instead of having the whole
> objects
>
>
> Right now we have the following:
> public void writeRevision(CDORevisionImpl revision);
>
> WIll be better to have the following :
> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>
> Thank you.
>
>
>
>
>
>
>
>


Re: [CDO][0.8.0] Handling Large Collection [message #609537 is a reply to message #94266] Mon, 27 August 2007 14:57 Go to previous message
Simon Mc Duff is currently offline Simon Mc DuffFriend
Messages: 596
Registered: July 2009
Senior Member
https://bugs.eclipse.org/bugs/show_bug.cgi?id=201265


"Eike Stepper" <stepper@sympedia.de> wrote in message
news:fauoee$frt$2@build.eclipse.org...
> Hi Simon,
>
> Yes, the idea is good. Please add a feature request for it.
> These chunk'ed collections will probably only be available for
> CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
> implement them if the object is backed by an EStore.
>
> Could you please add a separate feature request for the revision deltas.
> It is not 100% sure that we need this feature to implement chunk'ed
> collections.
> But having it will be good anyway.
>
> Cheers
> /Eike
>
>
> Simon McDuff schrieb:
>> Hi Eike,
>>
>> Do you see any value to chunk collection from the server to the client ?
>> It is possible to do right now in your framework ?
>>
>> The problem we have right now is the following.
>>
>> We have objects that have many attributes. Many of them are ToMany
>> relationship. In each collection, you have between 100 000 and 500 000
>> items.
>>
>> When I fetch the object that own these collections... I don`t want to
>> fetch all of them.
>>
>> So if we could fetch let`s say by chunk of 1000 thousand elements, it
>> will looks like the following
>>
>> - ObjectA first fetch
>> - fetch CDOID for collection ObjectA.coll1 [0...1000].
>> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>>
>> If I want the last objects of the collection ObjectA.coll1 . I don`t need
>> to fetch all of them. I can fetch
>> [499000 to 499 999] (in the case where I have 500 000 item in my
>> collection)
>>
>> It will be nice to have something hat looks like EStore :
>>
>> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature
>> feature, int indexFrom, int indexTo);
>>
>> This bring another problem... how we will write changes...if we have
>> only a part of it.
>>
>> It will be nice to have only what changed instead of having the whole
>> objects
>>
>>
>> Right now we have the following:
>> public void writeRevision(CDORevisionImpl revision);
>>
>> WIll be better to have the following :
>> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>>
>> Thank you.
>>
>>
>>
>>
>>
>>
>>
>>
Re: [CDO][0.8.0] Handling Large Collection [message #609538 is a reply to message #94266] Mon, 27 August 2007 14:58 Go to previous message
Simon Mc Duff is currently offline Simon Mc DuffFriend
Messages: 596
Registered: July 2009
Senior Member
https://bugs.eclipse.org/bugs/show_bug.cgi?id=201266


"Eike Stepper" <stepper@sympedia.de> wrote in message
news:fauoee$frt$2@build.eclipse.org...
> Hi Simon,
>
> Yes, the idea is good. Please add a feature request for it.
> These chunk'ed collections will probably only be available for
> CDOObjectImpls and not for CDOLegacyImpls because it is much easier to
> implement them if the object is backed by an EStore.
>
> Could you please add a separate feature request for the revision deltas.
> It is not 100% sure that we need this feature to implement chunk'ed
> collections.
> But having it will be good anyway.
>
> Cheers
> /Eike
>
>
> Simon McDuff schrieb:
>> Hi Eike,
>>
>> Do you see any value to chunk collection from the server to the client ?
>> It is possible to do right now in your framework ?
>>
>> The problem we have right now is the following.
>>
>> We have objects that have many attributes. Many of them are ToMany
>> relationship. In each collection, you have between 100 000 and 500 000
>> items.
>>
>> When I fetch the object that own these collections... I don`t want to
>> fetch all of them.
>>
>> So if we could fetch let`s say by chunk of 1000 thousand elements, it
>> will looks like the following
>>
>> - ObjectA first fetch
>> - fetch CDOID for collection ObjectA.coll1 [0...1000].
>> - fetch CDOID for collection ObjectA.coll2 [0...1000].
>>
>> If I want the last objects of the collection ObjectA.coll1 . I don`t need
>> to fetch all of them. I can fetch
>> [499000 to 499 999] (in the case where I have 500 000 item in my
>> collection)
>>
>> It will be nice to have something hat looks like EStore :
>>
>> CDORevisionImpl IStoreReader get(CDORevisionImpl object, CDOFeature
>> feature, int indexFrom, int indexTo);
>>
>> This bring another problem... how we will write changes...if we have
>> only a part of it.
>>
>> It will be nice to have only what changed instead of having the whole
>> objects
>>
>>
>> Right now we have the following:
>> public void writeRevision(CDORevisionImpl revision);
>>
>> WIll be better to have the following :
>> public void writeRevision(CDORevisionImpl revision, ListOfChanges .... );
>>
>> Thank you.
>>
>>
>>
>>
>>
>>
>>
>>
Previous Topic:[CDO/NET4J][0.8.0] Failover mechanism
Next Topic:[CDO] Hanging (client)
Goto Forum:
  


Current Time: Thu Aug 11 15:56:29 GMT 2022

Powered by FUDForum. Page generated in 0.02361 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top