Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » Attaching cdo adapters takes long time
Attaching cdo adapters takes long time [message #482753] Thu, 27 August 2009 19:25 Go to next message
Florian Pirchner is currently offline Florian PirchnerFriend
Messages: 94
Registered: July 2009
Member

Hi,

i am using CDOAdapterPolicy.CDO to become notified of model changes.

After loading the model i have to read all eObject contained in the model
tree. So i am using EcoreUtil.getAllProperContents(baseElement, true) and
iterate over the tree to get each eObject.

During iteration i am adding an adapter to each eObject. That is necessary
because i have to become notified of each model change.

The model has 256 eObject and a size of 24 KB.
The cdo server is located about 500 km (312 miles) far of my client.

The time to iterate over the tree and attach the adapter is extremly long.

I have triggered the times.
Setting up the session and transaction:
218 ms

Iterating and attaching the adapter:
15218 ms

(Case 2:
If i first iterate over all eObjects without attaching the adapters:
8047 ms
A second iterate over all and attaching the adapter:
7907 ms
)

Each next iterate over all eObjects:
0 - 15 ms.


I am not sure, but it seems that cdo transfers each attached adapter to
the repository and uses demand loading, what causes the 15 sec.


Is it possible
- to submitt the attached adapters at once to the repository?
- to fetch the whole model on load time without demand loading?

Maybe there are some settings to set?

This might save about 10 - 14 sec. I hope so... ;-)

Thanks in advance for your inputs, Flo
Re: Attaching cdo adapters takes long time [message #482777 is a reply to message #482753] Thu, 27 August 2009 22:19 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Flo,

Comments below...



Florian Pirchner schrieb:
> Hi,
>
> i am using CDOAdapterPolicy.CDO to become notified of model changes.
>
> After loading the model i have to read all eObject contained in the
> model tree. So i am using EcoreUtil.getAllProperContents(baseElement,
> true) and iterate over the tree to get each eObject.
>
> During iteration i am adding an adapter to each eObject. That is
> necessary because i have to become notified of each model change.
>
> The model has 256 eObject and a size of 24 KB.
> The cdo server is located about 500 km (312 miles) far of my client.
>
> The time to iterate over the tree and attach the adapter is extremly
> long.
>
> I have triggered the times.
> Setting up the session and transaction:
> 218 ms
That seems ok.

>
> Iterating and attaching the adapter:
> 15218 ms
You can tweak the demand loading. Please read
http://wiki.eclipse.org/Tweaking_CDO_Performance

>
> (Case 2:
> If i first iterate over all eObjects without attaching the adapters:
> 8047 ms
> A second iterate over all and attaching the adapter:
> 7907 ms
Given the 15ms for the iterations after the first (lazy) one, it's
almost 7900ms for the adapter registration. That seems consistent with
the network latency of the the demand loading of this model.


> )
>
> Each next iterate over all eObjects:
> 0 - 15 ms.
>
>
> I am not sure, but it seems that cdo transfers each attached adapter
> to the repository
That's right. While that seems odd in your case I have currently no clue
how to optimize that. What would you suggest?

> and uses demand loading, what causes the 15 sec.
Yep.

>
>
> Is it possible - to submitt the attached adapters at once to the
> repository?
I don't think so. A single thread (your client thread) can only attach
one adapter at a time to any EMF model. When, if not immediately, should
we send the change subscription to the server?

In your very particular case where you want all deltas of all objects at
all times, I could imagine a special mode,i.e. not the normal change
subscriptions per object, but something more general. If you file a
bugzilla and come up with a nice design, I'm willing to consider and
maintain it ;-)


> - to fetch the whole model on load time without demand loading?
I think so. With appropriate options in the session and the
view/transaction.

>
> Maybe there are some settings to set?
>
> This might save about 10 - 14 sec. I hope so... ;-)
>
> Thanks in advance for your inputs, Flo
You're welcome!

Cheers
/Eike

----
http://thegordian.blogspot.com


Re: Attaching cdo adapters takes long time [message #482778 is a reply to message #482777] Thu, 27 August 2009 22:21 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Flo,

I forgot one thing: did you try to add the change subscription policy
*after* you attached your adapters?

Cheers
/Eike

----
http://thegordian.blogspot.com



Eike Stepper schrieb:
> Flo,
>
> Comments below...
>
>
>
> Florian Pirchner schrieb:
>> Hi,
>>
>> i am using CDOAdapterPolicy.CDO to become notified of model changes.
>>
>> After loading the model i have to read all eObject contained in the
>> model tree. So i am using EcoreUtil.getAllProperContents(baseElement,
>> true) and iterate over the tree to get each eObject.
>>
>> During iteration i am adding an adapter to each eObject. That is
>> necessary because i have to become notified of each model change.
>>
>> The model has 256 eObject and a size of 24 KB.
>> The cdo server is located about 500 km (312 miles) far of my client.
>>
>> The time to iterate over the tree and attach the adapter is extremly
>> long.
>>
>> I have triggered the times.
>> Setting up the session and transaction:
>> 218 ms
> That seems ok.
>
>>
>> Iterating and attaching the adapter:
>> 15218 ms
> You can tweak the demand loading. Please read
> http://wiki.eclipse.org/Tweaking_CDO_Performance
>
>>
>> (Case 2:
>> If i first iterate over all eObjects without attaching the adapters:
>> 8047 ms
>> A second iterate over all and attaching the adapter:
>> 7907 ms
> Given the 15ms for the iterations after the first (lazy) one, it's
> almost 7900ms for the adapter registration. That seems consistent with
> the network latency of the the demand loading of this model.
>
>
>> )
>>
>> Each next iterate over all eObjects:
>> 0 - 15 ms.
>>
>>
>> I am not sure, but it seems that cdo transfers each attached adapter
>> to the repository
> That's right. While that seems odd in your case I have currently no
> clue how to optimize that. What would you suggest?
>
>> and uses demand loading, what causes the 15 sec.
> Yep.
>
>>
>>
>> Is it possible - to submitt the attached adapters at once to the
>> repository?
> I don't think so. A single thread (your client thread) can only attach
> one adapter at a time to any EMF model. When, if not immediately,
> should we send the change subscription to the server?
>
> In your very particular case where you want all deltas of all objects
> at all times, I could imagine a special mode,i.e. not the normal
> change subscriptions per object, but something more general. If you
> file a bugzilla and come up with a nice design, I'm willing to
> consider and maintain it ;-)
>
>
>> - to fetch the whole model on load time without demand loading?
> I think so. With appropriate options in the session and the
> view/transaction.
>
>>
>> Maybe there are some settings to set?
>>
>> This might save about 10 - 14 sec. I hope so... ;-)
>>
>> Thanks in advance for your inputs, Flo
> You're welcome!
>
> Cheers
> /Eike
>
> ----
> http://thegordian.blogspot.com
>


Re: Attaching cdo adapters takes long time [message #482808 is a reply to message #482778] Fri, 28 August 2009 06:36 Go to previous messageGo to next message
Florian Pirchner is currently offline Florian PirchnerFriend
Messages: 94
Registered: July 2009
Member

Hi Eike,

what a great idea! Adding the changeSubscription after attaching the
adapter...

Currently i did not try it. But i will do so! ;-)

On weekend i'll do some hacks, think about your comments and post my
results / suggests coming week.

A lot of thanks, Flo
Re: Attaching cdo adapters takes long time [message #484244 is a reply to message #482808] Fri, 04 September 2009 18:11 Go to previous messageGo to next message
Florian Pirchner is currently offline Florian PirchnerFriend
Messages: 94
Registered: July 2009
Member

Hi,

i tried to add the changeSubscription after attaching all the adapters.
This speads up the ui rendering by about 50% ;-)

Adding the changeSubscriptionPolicy after adding adapters:
The time to load and attach the adapters is: 11282 ms -> For a big model.

Adding the changeSubPolicy before adding the adapters:
Time to load and attach is: 21578 ms


Eike, thanks for this idea. Thats a good step forward!



Additionaly i tried to attach the adapters in a background job. So this
should speed up the creation too.

In the next step i will optimize the CDOCollectionLoadingPolicy.

If i have new results, i will post them.

Greetings, Flo
Re: Attaching cdo adapters takes long time [message #484250 is a reply to message #482808] Fri, 04 September 2009 18:47 Go to previous messageGo to next message
Florian Pirchner is currently offline Florian PirchnerFriend
Messages: 94
Registered: July 2009
Member

Hi again,

if i want to load the whole model tree at once, is it enough to increase
the RevisionPrefetchingPolicy to - maybe 1000?
Will this resolve all the CDOIDs on load time, if the model contains about
300 references?
My tree is really deep. About 10 levels or more. (This means a list has a
reference to a list which has a reference to a list and so on and so on.)

Will the whole model be transfered to client on load time? This means on
calling resourceSet.getResource(resourceURI, true);

Best thanks, Flo
Re: Attaching cdo adapters takes long time [message #484265 is a reply to message #484250] Sat, 05 September 2009 00:08 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Hi Flo,

I cc'ed Simon because he's the optimization expert ;-)

Cheers
/Eike

----
http://thegordian.blogspot.com



Florian Pirchner schrieb:
> Hi again,
>
> if i want to load the whole model tree at once, is it enough to
> increase the RevisionPrefetchingPolicy to - maybe 1000?
> Will this resolve all the CDOIDs on load time, if the model contains
> about 300 references?
> My tree is really deep. About 10 levels or more. (This means a list
> has a reference to a list which has a reference to a list and so on
> and so on.)
>
> Will the whole model be transfered to client on load time? This means
> on calling resourceSet.getResource(resourceURI, true);
>
> Best thanks, Flo
>


Re: Attaching cdo adapters takes long time [message #484588 is a reply to message #484250] Tue, 08 September 2009 11:42 Go to previous messageGo to next message
Simon Mc Duff is currently offline Simon Mc DuffFriend
Messages: 596
Registered: July 2009
Senior Member
Florian Pirchner wrote:

> Hi again,

> if i want to load the whole model tree at once, is it enough to increase
> the RevisionPrefetchingPolicy to - maybe 1000?
> Will this resolve all the CDOIDs on load time, if the model contains about
> 300 references?
[SIMON] CDO is all about lazy loading. Usually, when an object has a many
relationship it will load only the CDOID instead of the collection itself.
Each time you need an objects in the collection it will fetch only that
objects.

In some cases it could not be efficient. So we introduced the
RevisionPrefetchingPolicy. When it is activated it will load collection by
chunk of max(size of the collection, 1000) elements when accessing one
objects in that collection.

> My tree is really deep. About 10 levels or more. (This means a list has a
> reference to a list which has a reference to a list and so on and so on.)
It goes only for one level. You will need something else but the interface
isn't well defined yet. (Rules to decide what to fetch)

> Will the whole model be transfered to client on load time? This means on
> calling resourceSet.getResource(resourceURI, true);
[SIMON] No only when you fetch one object of a collection.

> Best thanks, Flo
Re: Attaching cdo adapters takes long time [message #484632 is a reply to message #484250] Tue, 08 September 2009 14:38 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Hi Simon,

Only now I start to be able to *send* SMTP mail from Ed's home. The CC below was waiting in my outbox for some time. It was from the newsgroup:



Hi Flo,

I cc'ed Simon because he's the optimization expert ;-)

Cheers
/Eike

----
http://thegordian.blogspot.com



Florian Pirchner schrieb:
> Hi again,
>
> if i want to load the whole model tree at once, is it enough to
> increase the RevisionPrefetchingPolicy to - maybe 1000?
> Will this resolve all the CDOIDs on load time, if the model contains
> about 300 references?
> My tree is really deep. About 10 levels or more. (This means a list
> has a reference to a list which has a reference to a list and so on
> and so on.)
>
> Will the whole model be transfered to client on load time? This means
> on calling resourceSet.getResource(resourceURI, true);
>
> Best thanks, Flo
>


Re: Attaching cdo adapters takes long time [message #485609 is a reply to message #484588] Mon, 14 September 2009 06:37 Go to previous messageGo to next message
Florian Pirchner is currently offline Florian PirchnerFriend
Messages: 94
Registered: July 2009
Member

Hi Simon,

it would be great if you can tell me where i can find the logic in the cdo
server which loads the model and decides what to load / transfer.

So i can write a patch and contribute a solution. Otherwise it is not
really possible to use cdo in our project, because we have to reduce the
network latency for our encapsulated models.

Thanks, Flo
Re: Attaching cdo adapters takes long time [message #485611 is a reply to message #485609] Mon, 14 September 2009 07:08 Go to previous messageGo to next message
ekkehard gentz is currently offline ekkehard gentzFriend
Messages: 118
Registered: July 2009
Location: rosenheim, Germany, bavar...
Senior Member

This is a multi-part message in MIME format.
--------------010805050503050809090000
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit

Florian Pirchner schrieb:
> Hi Simon,
>
> it would be great if you can tell me where i can find the logic in the
> cdo server which loads the model and decides what to load / transfer.
>
> So i can write a patch and contribute a solution. Otherwise it is not
> really possible to use cdo in our project, because we have to reduce
> the network latency for our encapsulated models.
>
> Thanks, Flo
>
some additional infos:

We're using EMF to store Business (SWT) Views.
This is done by redView (Riena EMF Dynamic Views for Business Applications)
redView will be open-sourced soon (EPL)

redView supports all SWT Widgets + some Nebula and uses Riena Ridgets
with (Eclipse) Databinding

redView contains a WYSIWYG editor to be used from inside IDE
and redView also works from RCP clients where complete business views
are rendered dynamically
based from EMF models

Our redView's are deep trees and of course if a user opens a Business
View this should be fast ;-)

Flo spent much time to make redView run using CDO and it works great:
some RCP clients have redViews open, then the
Business-View-Admin(Designer) does some design-changes
using redView's WYSIWYG editor from inside Eclipse IDE and after saving
them to CDO server - the changes
are immediately visible to the users :-)
before using CDO the redViews (XMI) are part of a plug-in and after
changing something, the admin
had to update one bundle from OSGI console and to refresh some others to
provide the new views -
using CDO this is much easier and an eye-catcher demonstrating this :-)

If all RCP clients are working in a local network then CDO works fast,
but many clients may be working from other cities using high-speed DSL
connection -
and in these cases CDO fails :-(
it takes a long long time to open a View. While opening the view we have
to iterate over all objects
and this causes much network-traffic and consumes too much time.

Its great that CDO scales with large models, does lazy loading etc. and
there are many use-cases where
this works well and perhaps is the main reason to use CDO.

But in our use-cases we have the need to iterate over all objects and we
know that our model fits
into memory, because its a single business (SWT) view ;-)

I can imagine that there are many uses-cases where EMF is used with
models where these models
have to be completely loaded at once over a network connection. loading
step-by-step and only 1-level deep
doesnt work in these cases.

We have to find a soulution to load a model at once from CDO server -
otherwise I have to cancel
the use of CDO, because of performance slowdown

so it would be great if Flo can get enough information to know where to
patch CDO -
I really like the concepts of CDO and think its not an un-common
use-case to need a whole
emf model from CDO server at once

thx for any help

--

ekke (ekkehard gentz)
independent software-architect
senior erp-consultant
eclipse | osgi | equinox | mdsd | oaw | emf | uml
max-josefs-platz 30, D-83022 rosenheim, germany
mailto:ekke@ekkes-corner.org
homepage (de): http://gentz-software.de
blog (en): http://ekkes-corner.org
twitter: @ekkescorner
skype: ekkes-corner
Steuer-Nr: 156/220/30931 FA Rosenheim, UST-ID: DE189929490


--------------010805050503050809090000
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Florian Pirchner schrieb:
<blockquote
cite="mid:14170e359aefa0d6ea2ea9c5391f29b6$1@www.eclipse.org"
type="cite">Hi Simon,
<br>
<br>
it would be great if you can tell me where i can find the logic in the
cdo server which loads the model and decides what to load / transfer.
<br>
<br>
So i can write a patch and contribute a solution. Otherwise it is not
really possible to use cdo in our project, because we have to reduce
the network latency for our encapsulated models.
<br>
<br>
Thanks, Flo
<br>
<br>
</blockquote>
some additional infos:<br>
<br>
We're using EMF to store Business (SWT) Views. <br>
This is done by redView (Riena EMF Dynamic Views for Business
Applications)<br>
redView will be open-sourced soon (EPL)<br>
<br>
redView supports all SWT Widgets + some Nebula and uses Riena Ridgets
with (Eclipse) Databinding<br>
<br>
redView contains a WYSIWYG editor to be used from inside IDE<br>
and redView also works from RCP clients where complete business views
are rendered dynamically<br>
based from EMF models<br>
<br>
Our redView's are deep trees and of course if a user opens a Business
View this should be fast ;-)<br>
<br>
Flo spent much time to make redView run using CDO and it works great:<br>
some RCP clients have redViews open, then the
Business-View-Admin(Designer) does some design-changes<br>
using redView's WYSIWYG editor from inside Eclipse IDE and after saving
them to CDO server - the changes<br>
are immediately visible to the users :-)<br>
before using CDO the redViews (XMI) are part of a plug-in and after
changing something, the admin<br>
had to update one bundle from OSGI console and to refresh some others
to provide the new views -<br>
using CDO this is much easier and an eye-catcher demonstrating this :-)<br>
<br>
If all RCP clients are working in a local network then CDO works fast,<br>
but many clients may be working from other cities using high-speed DSL
connection -<br>
and in these cases CDO fails :-(<br>
it takes a long long time to open a View. While opening the view we
have to iterate over all objects<br>
and this causes much network-traffic and consumes too much time.<br>
<br>
Its great that CDO scales with large models, does lazy loading etc. and
there are many use-cases where<br>
this works well and perhaps is the main reason to use CDO.<br>
<br>
But in our use-cases we have the need to iterate over all objects and
we know that our model fits<br>
into memory, because its a single business (SWT) view ;-)<br>
<br>
I can imagine that there are many uses-cases where EMF is used with
models where these models<br>
have to be completely loaded at once over a network connection. loading
step-by-step and only 1-level deep <br>
doesnt work in these cases.<br>
<br>
We have to find a soulution to load a model at once from CDO server -
otherwise I have to cancel<br>
the use of CDO, because of performance slowdown<br>
<br>
so it would be great if Flo can get enough information to know where to
patch CDO -<br>
I really like the concepts of CDO and think its not an un-common
use-case to need a whole<br>
emf model from CDO server at once<br>
<br>
thx for any help<br>
<br>
<div class="moz-signature">-- <br>
<p style="margin-bottom: 0cm;"> <font ,="" color="#808080"
face="Verdana, sans-serif"> <font size="4">ekke (ekkehard gentz)<br>
</font> <font size="3"> independent software-architect<br>
senior erp-consultant<br>
</font> <font size="2">eclipse | osgi | equinox | mdsd | oaw | emf |
uml<br>
max-josefs-platz 30, D-83022 rosenheim, germany<br>
<a class="moz-txt-link-freetext" href="mailto:ekke@ekkes-corner.org">mailto:ekke@ekkes-corner.org</a><br>
homepage (de): <a class="moz-txt-link-freetext" href="http://gentz-software.de">http://gentz-software.de</a><br>
blog (en): <a class="moz-txt-link-freetext" href="http://ekkes-corner.org">http://ekkes-corner.org</a><br>
twitter: @ekkescorner<br>
skype: ekkes-corner<br>
</font> <font size="1"> Steuer-Nr: 156/220/30931 FA Rosenheim, UST-ID:
DE189929490<br>
</font> </font> </p>
</div>
</body>
</html>

--------------010805050503050809090000--


ekke
independent software architect

blog: http://ekkes-corner.org
twitter: @ekkescorner
Re: Attaching cdo adapters takes long time [message #485714 is a reply to message #485611] Mon, 14 September 2009 15:14 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Hi Ekke and Flo,

I understand your need for such prefetching (I think this is the term
for what you seem to need). In your case you seem to want to load whole
resources in one chunk and that should be possible with comparingly
little effort. However, as the responsible for the prosper of an open
framework I wonder if the new requirements should be broadened a bit to,
e.g., enable the specification of prefetch rules per EClass or so. I'm
aware that this is not exactly what you need and maybe it's generally
inadequate. At this stage I'd just like to start a requirements
discussion ;-)

Cheers
/Eike

----
http://thegordian.blogspot.com



ekke schrieb:
> Florian Pirchner schrieb:
>> Hi Simon,
>>
>> it would be great if you can tell me where i can find the logic in
>> the cdo server which loads the model and decides what to load /
>> transfer.
>>
>> So i can write a patch and contribute a solution. Otherwise it is not
>> really possible to use cdo in our project, because we have to reduce
>> the network latency for our encapsulated models.
>>
>> Thanks, Flo
>>
> some additional infos:
>
> We're using EMF to store Business (SWT) Views.
> This is done by redView (Riena EMF Dynamic Views for Business
> Applications)
> redView will be open-sourced soon (EPL)
>
> redView supports all SWT Widgets + some Nebula and uses Riena Ridgets
> with (Eclipse) Databinding
>
> redView contains a WYSIWYG editor to be used from inside IDE
> and redView also works from RCP clients where complete business views
> are rendered dynamically
> based from EMF models
>
> Our redView's are deep trees and of course if a user opens a Business
> View this should be fast ;-)
>
> Flo spent much time to make redView run using CDO and it works great:
> some RCP clients have redViews open, then the
> Business-View-Admin(Designer) does some design-changes
> using redView's WYSIWYG editor from inside Eclipse IDE and after
> saving them to CDO server - the changes
> are immediately visible to the users :-)
> before using CDO the redViews (XMI) are part of a plug-in and after
> changing something, the admin
> had to update one bundle from OSGI console and to refresh some others
> to provide the new views -
> using CDO this is much easier and an eye-catcher demonstrating this :-)
>
> If all RCP clients are working in a local network then CDO works fast,
> but many clients may be working from other cities using high-speed DSL
> connection -
> and in these cases CDO fails :-(
> it takes a long long time to open a View. While opening the view we
> have to iterate over all objects
> and this causes much network-traffic and consumes too much time.
>
> Its great that CDO scales with large models, does lazy loading etc.
> and there are many use-cases where
> this works well and perhaps is the main reason to use CDO.
>
> But in our use-cases we have the need to iterate over all objects and
> we know that our model fits
> into memory, because its a single business (SWT) view ;-)
>
> I can imagine that there are many uses-cases where EMF is used with
> models where these models
> have to be completely loaded at once over a network connection.
> loading step-by-step and only 1-level deep
> doesnt work in these cases.
>
> We have to find a soulution to load a model at once from CDO server -
> otherwise I have to cancel
> the use of CDO, because of performance slowdown
>
> so it would be great if Flo can get enough information to know where
> to patch CDO -
> I really like the concepts of CDO and think its not an un-common
> use-case to need a whole
> emf model from CDO server at once
>
> thx for any help
>
> --
>
> ekke (ekkehard gentz)
> independent software-architect
> senior erp-consultant
> eclipse | osgi | equinox | mdsd | oaw | emf | uml
> max-josefs-platz 30, D-83022 rosenheim, germany
> mailto:ekke@ekkes-corner.org
> homepage (de): http://gentz-software.de
> blog (en): http://ekkes-corner.org
> twitter: @ekkescorner
> skype: ekkes-corner
> Steuer-Nr: 156/220/30931 FA Rosenheim, UST-ID: DE189929490
>


Re: Attaching cdo adapters takes long time [message #485732 is a reply to message #485714] Mon, 14 September 2009 17:02 Go to previous messageGo to next message
Victor Roldan Betancort is currently offline Victor Roldan BetancortFriend
Messages: 524
Registered: July 2009
Senior Member
Guys,

I've created the following bug:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=289371
Provide Dynamic content prefetching mechanism upon CDOResource load

to gather the requirements for this new prefetching mechanism.

From what I understand from Ekke's post, a big chunk of a CDOResource
should be gathered upon load to reduce server round-trips while
transferring Revision data. Is this right?

My doubt here is how different is that from the already provided
prefetching mechanisms. See:

org.eclipse.emf.cdo.view.CDORevisionPrefetchingPolicy
org.eclipse.emf.cdo.session.CDOCollectionLoadingPolicy
org.eclipse.emf.cdo.view.CDOView.Options.getRevisionPrefetch ingPolicy()

My initial idea is to provide some custom options that can be passed
through:

org.eclipse.emf.ecore.resource.Resource.load(Map<?, ?>)

Please, post your thoughts about this new feature in the bugzilla.

Cheers,
Víctor.


Eike Stepper escribió:
> Hi Ekke and Flo,
>
> I understand your need for such prefetching (I think this is the term
> for what you seem to need). In your case you seem to want to load whole
> resources in one chunk and that should be possible with comparingly
> little effort. However, as the responsible for the prosper of an open
> framework I wonder if the new requirements should be broadened a bit to,
> e.g., enable the specification of prefetch rules per EClass or so. I'm
> aware that this is not exactly what you need and maybe it's generally
> inadequate. At this stage I'd just like to start a requirements
> discussion ;-)
>
> Cheers
> /Eike
>
> ----
> http://thegordian.blogspot.com
>
>
>
> ekke schrieb:
>> Florian Pirchner schrieb:
>>> Hi Simon,
>>>
>>> it would be great if you can tell me where i can find the logic in
>>> the cdo server which loads the model and decides what to load /
>>> transfer.
>>>
>>> So i can write a patch and contribute a solution. Otherwise it is not
>>> really possible to use cdo in our project, because we have to reduce
>>> the network latency for our encapsulated models.
>>>
>>> Thanks, Flo
>>>
>> some additional infos:
>>
>> We're using EMF to store Business (SWT) Views.
>> This is done by redView (Riena EMF Dynamic Views for Business
>> Applications)
>> redView will be open-sourced soon (EPL)
>>
>> redView supports all SWT Widgets + some Nebula and uses Riena Ridgets
>> with (Eclipse) Databinding
>>
>> redView contains a WYSIWYG editor to be used from inside IDE
>> and redView also works from RCP clients where complete business views
>> are rendered dynamically
>> based from EMF models
>>
>> Our redView's are deep trees and of course if a user opens a Business
>> View this should be fast ;-)
>>
>> Flo spent much time to make redView run using CDO and it works great:
>> some RCP clients have redViews open, then the
>> Business-View-Admin(Designer) does some design-changes
>> using redView's WYSIWYG editor from inside Eclipse IDE and after
>> saving them to CDO server - the changes
>> are immediately visible to the users :-)
>> before using CDO the redViews (XMI) are part of a plug-in and after
>> changing something, the admin
>> had to update one bundle from OSGI console and to refresh some others
>> to provide the new views -
>> using CDO this is much easier and an eye-catcher demonstrating this :-)
>>
>> If all RCP clients are working in a local network then CDO works fast,
>> but many clients may be working from other cities using high-speed DSL
>> connection -
>> and in these cases CDO fails :-(
>> it takes a long long time to open a View. While opening the view we
>> have to iterate over all objects
>> and this causes much network-traffic and consumes too much time.
>>
>> Its great that CDO scales with large models, does lazy loading etc.
>> and there are many use-cases where
>> this works well and perhaps is the main reason to use CDO.
>>
>> But in our use-cases we have the need to iterate over all objects and
>> we know that our model fits
>> into memory, because its a single business (SWT) view ;-)
>>
>> I can imagine that there are many uses-cases where EMF is used with
>> models where these models
>> have to be completely loaded at once over a network connection.
>> loading step-by-step and only 1-level deep
>> doesnt work in these cases.
>>
>> We have to find a soulution to load a model at once from CDO server -
>> otherwise I have to cancel
>> the use of CDO, because of performance slowdown
>>
>> so it would be great if Flo can get enough information to know where
>> to patch CDO -
>> I really like the concepts of CDO and think its not an un-common
>> use-case to need a whole
>> emf model from CDO server at once
>>
>> thx for any help
>>
>> --
>>
>> ekke (ekkehard gentz)
>> independent software-architect
>> senior erp-consultant
>> eclipse | osgi | equinox | mdsd | oaw | emf | uml
>> max-josefs-platz 30, D-83022 rosenheim, germany
>> mailto:ekke@ekkes-corner.org
>> homepage (de): http://gentz-software.de
>> blog (en): http://ekkes-corner.org
>> twitter: @ekkescorner
>> skype: ekkes-corner
>> Steuer-Nr: 156/220/30931 FA Rosenheim, UST-ID: DE189929490
>>
Re: Attaching cdo adapters takes long time [message #485737 is a reply to message #485714] Mon, 14 September 2009 17:24 Go to previous messageGo to next message
Victor Roldan Betancort is currently offline Victor Roldan BetancortFriend
Messages: 524
Registered: July 2009
Senior Member
Hi everyone,

I thought about the initial problem Flo exposed: adding adapters to the
whole Resource content. You need to get changes over all the objects,
right? Could you tell us more about this? From ekke's comment, you are
visualizing data in a view, so I assume you guys are visualizing in a
Tree. Would it help if we had an intelligent EContentAdapter, which
installs itself in the tree objects a soon as the object gets loaded?
Why do you need to receive notification on data you aren't visualizing
in the Tree? (hope not be assuming too many things :P)

Cheers!

Víctor.
Re: Attaching cdo adapters takes long time [message #485744 is a reply to message #485737] Mon, 14 September 2009 18:33 Go to previous messageGo to next message
ekkehard gentz is currently offline ekkehard gentzFriend
Messages: 118
Registered: July 2009
Location: rosenheim, Germany, bavar...
Senior Member

This is a multi-part message in MIME format.
--------------030103000502040704060805
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

V


ekke
independent software architect

blog: http://ekkes-corner.org
twitter: @ekkescorner
Re: Attaching cdo adapters takes long time [message #485795 is a reply to message #485737] Tue, 15 September 2009 06:02 Go to previous messageGo to next message
Florian Pirchner is currently offline Florian PirchnerFriend
Messages: 94
Registered: July 2009
Member

Hi Victor,

we have created a model based swt-ui with a designer editor (wysiwyg)
which allows dnd / copy/paste and different actions acting on the swt ui.
We are using different metamodels (elements, styles, layout, binding,...)
to describe the ui.

All the elements in the model are required to render our swt ui with
swt.Composites, swt.Tables, swt.Texts, swt.Label, swt.Trees,....

Currently, each model change directly reflects the swt-ui. So we need one
adapter for each element. It is also necessary to attach all adapter on
load time, because the user can change each property somewhere in the
model tree.


An intelligent EContentAdapter would be great. I guess it would be
attached directly in the server and the adapter does not have to be
transfered from the client to server.


This evening i'll publish a screenshot, how redview looks like.

Greetings, Flo
Re: Attaching cdo adapters takes long time [message #485813 is a reply to message #485732] Tue, 15 September 2009 07:32 Go to previous message
Florian Pirchner is currently offline Florian PirchnerFriend
Messages: 94
Registered: July 2009
Member

Hi,

a short explanation. Further discussions will be added to the bugzilla.

Yes, a big chunk would be great.

View days ago i tried the different policies, but they did not match my
requirements.

- org.eclipse.emf.cdo.view.CDORevisionPrefetchingPolicy
Defines how many CDOIDs should be resolved. This policy operates at one
tree level.
- org.eclipse.emf.cdo.session.CDOCollectionLoadingPolicy
Defines how to load the elements in a collection. But does not allow to
specify when the referenced elements of the collections elements are
resolved. So it also operates at one tree level.

To get a good performance for redview, we need some kind of definition
which allows to define the resolving depth. This means the depth of the
tree which will be resolved and transfered to the client at load time.

For example a depth of -1 would allow to transfer the whole model (all
containment references) at once to the client. So no additionally round
trips for resolving the CDOID would be required.

Further this would allow a good performance for rendering the swt-ui.

I think your idea according the additionally option is really good. This
might force the server to resolve all the objects in the model at load
time.

Greetings, Flo
Previous Topic:IWrapperItemProvider & databinding
Next Topic:Teneo/Hibernate - key conflicts
Goto Forum:
  


Current Time: Sat Apr 20 02:28:59 GMT 2024

Powered by FUDForum. Page generated in 0.05669 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top