Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Proxies / Lazy Loading
[CDO] Proxies / Lazy Loading [message #425953] Sat, 13 December 2008 14:12 Go to next message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
Hello again.

If I understand it right, CDO takes care about the lazy loading. Is this
completely independent from the persistence store (e.g. DerbyStore or
Teneo) used in the background?
What exaclty can be lazy loaded?

For example, let's assume the simple model in pseudo code like this:

class Document {
String small;
String large;
}

And now let's assume that I have 100000 documents of that type and want
to load all those documents into one table, but only want to show the
"small" string. The "large" string will only be accessed, when clicking
directly on one document. And now let's also assume that the whole
100000 documents won't fit into memory cause of the huge data contained
inside the "large" fields.
Will "large" be lazy loaded? (With Hibernate normal attributes by
default won't be lazy loaded.)
If not, would the following design help to use lazy loading for "large"
also in CDO? (It would help within Hibernate).

class Document {
String small;
LargeObject largeObject;
}

class LargeObject {
String large;
}

Best regards,
Kai
Re: [CDO] Proxies / Lazy Loading [message #425954 is a reply to message #425953] Sat, 13 December 2008 14:19 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Kai,

Comments below...


Kai Schlamp schrieb:
> Hello again.
>
> If I understand it right, CDO takes care about the lazy loading. Is
> this completely independent from the persistence store (e.g.
> DerbyStore or Teneo) used in the background?
Yes, completely independent.

> What exaclty can be lazy loaded?
All references, i.e. single-valued, many-valued, cross references,
containments...

>
> For example, let's assume the simple model in pseudo code like this:
>
> class Document {
> String small;
> String large;
> }
>
> And now let's assume that I have 100000 documents of that type and
> want to load all those documents into one table, but only want to show
> the "small" string. The "large" string will only be accessed, when
> clicking directly on one document. And now let's also assume that the
> whole 100000 documents won't fit into memory cause of the huge data
> contained inside the "large" fields.
> Will "large" be lazy loaded? (With Hibernate normal attributes by
> default won't be lazy loaded.)
No, CDO does currently not support lazy loading of single *attributes*.
What about making "large" a derived feature that is computed from a
contained data object?

> If not, would the following design help to use lazy loading for
> "large" also in CDO? (It would help within Hibernate).
>
> class Document {
> String small;
> LargeObject largeObject;
> }
>
> class LargeObject {
> String large;
> }
Oh yes, this is what I had in mind above ;-)

Cheers
/Eike

----
http://thegordian.blogspot.com


Re: [CDO] Proxies / Lazy Loading [message #425955 is a reply to message #425953] Sat, 13 December 2008 14:28 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
This is a multi-part message in MIME format.
--------------090804070001060707050000
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit

Kai,

By the way, if all your 100000 Document objects are contained in one
list, it could also be good to set a collection loading policy into the
session options:

| *public **void *testOpenSession() *throws *Exception
{
CDOSession session = openModel1Session();
CDOCollectionLoadingPolicy policy = CDOUtil.createCollectionLoadingPolicy(0, 100);
session.options().setCollectionLoadingPolicy(policy);
}|


This one initially loads no target IDs into objects. Later, when a list
element is accessed, a maximum range of 100 elements *around* the
accessed list index are loaded if necessary.

Cheers
/Eike

----
http://thegordian.blogspot.com


Kai Schlamp schrieb:
> Hello again.
>
> If I understand it right, CDO takes care about the lazy loading. Is
> this completely independent from the persistence store (e.g.
> DerbyStore or Teneo) used in the background?
> What exaclty can be lazy loaded?
>
> For example, let's assume the simple model in pseudo code like this:
>
> class Document {
> String small;
> String large;
> }
>
> And now let's assume that I have 100000 documents of that type and
> want to load all those documents into one table, but only want to show
> the "small" string. The "large" string will only be accessed, when
> clicking directly on one document. And now let's also assume that the
> whole 100000 documents won't fit into memory cause of the huge data
> contained inside the "large" fields.
> Will "large" be lazy loaded? (With Hibernate normal attributes by
> default won't be lazy loaded.)
> If not, would the following design help to use lazy loading for
> "large" also in CDO? (It would help within Hibernate).
>
> class Document {
> String small;
> LargeObject largeObject;
> }
>
> class LargeObject {
> String large;
> }
>
> Best regards,
> Kai
>

--------------090804070001060707050000
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Kai,<br>
<br>
By the way, if all your 100000 Document objects are contained in one
list, it could also be good to set a collection loading policy into the
session options:<br>
<br>
<title></title>
<style type="text/css">
<!--code { font-family: Courier New, Courier; font-size: 10pt; margin: 0px; }-->
</style>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<!-- ======================================================== -->
<!-- = Java Sourcecode to HTML automatically converted code = --><!-- = Java2Html Converter 5.0 [2006-02-26] by Markus Gebhard markus@jave.de = -->
<!-- = Further information: http://www.java2html.de = -->
<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3" cellspacing="0">
<tbody>
<tr>
<!-- start source code --> <td align="left" nowrap="nowrap"
valign="top"> <code><font color="#ffffff">


Re: [CDO] Proxies / Lazy Loading [message #425956 is a reply to message #425954] Sat, 13 December 2008 14:48 Go to previous messageGo to next message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
Hy Eike,

thanks a lot for the lightning fast answer.
A last question about that topic.
Let's get back to the table scenario with the little example model at
the bottom.
When I want to only show documents in the table that contain a specific
search string in the "large" attribute of LargeObject, all LargeObject
objects must be fetched (as there is no query solution in CDO 1.0.x), to
test if those contain the search string. Are already tested LargeObjects
garbage collected again and the largeObject attribute of Document set
back to the appropriate proxy? Cause otherwise the whole model would not
fit in the available memory.
I read somewhere that CDO (in contrast to Hibernate) supports this. Is
that right? Automatically without having me to do something?

Best regards,
Kai


Eike Stepper schrieb:
> Kai,
>
> Comments below...
>
>
> Kai Schlamp schrieb:
>> Hello again.
>>
>> If I understand it right, CDO takes care about the lazy loading. Is
>> this completely independent from the persistence store (e.g.
>> DerbyStore or Teneo) used in the background?
> Yes, completely independent.
>
>> What exaclty can be lazy loaded?
> All references, i.e. single-valued, many-valued, cross references,
> containments...
>
>>
>> For example, let's assume the simple model in pseudo code like this:
>>
>> class Document {
>> String small;
>> String large;
>> }
>>
>> And now let's assume that I have 100000 documents of that type and
>> want to load all those documents into one table, but only want to show
>> the "small" string. The "large" string will only be accessed, when
>> clicking directly on one document. And now let's also assume that the
>> whole 100000 documents won't fit into memory cause of the huge data
>> contained inside the "large" fields.
>> Will "large" be lazy loaded? (With Hibernate normal attributes by
>> default won't be lazy loaded.)
> No, CDO does currently not support lazy loading of single *attributes*.
> What about making "large" a derived feature that is computed from a
> contained data object?
>
>> If not, would the following design help to use lazy loading for
>> "large" also in CDO? (It would help within Hibernate).
>>
>> class Document {
>> String small;
>> LargeObject largeObject;
>> }
>>
>> class LargeObject {
>> String large;
>> }
> Oh yes, this is what I had in mind above ;-)
>
> Cheers
> /Eike
>
> ----
> http://thegordian.blogspot.com
>
Re: [CDO] Proxies / Lazy Loading [message #425957 is a reply to message #425955] Sat, 13 December 2008 15:11 Go to previous messageGo to next message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
Thanks for the tip, Eike.
Is this policy also available in CDO 1.0.x?
As in the docs
(http://download.eclipse.org/modeling/emf/cdo/javadoc/1.0.0/) I don't
see such a create method of CDOUtil, nor a class called
CDOCollectionLoadingPolicy.
Perhaps I switch to CDO 2.0.0 soon. I tried this once, but it didn't
seem to be so easy as there were some conflicts with plugins of the
Eclipse 3.4.1 Ganymede release (is there somewhere a little installation
instruction for CDO 2.0.0 + Eclipse Ganymede?) . But I will retry this,
at the latest when there is a query implementation (hql, sql, whatever)
for CDO 2.0.0.
Thanks a lot for your help.

Best regards,
Kai



Eike Stepper schrieb:
> Kai,
>
> By the way, if all your 100000 Document objects are contained in one
> list, it could also be good to set a collection loading policy into the
> session options:
>
> | *public **void *testOpenSession() *throws *Exception
> {
> CDOSession session = openModel1Session();
> CDOCollectionLoadingPolicy policy = CDOUtil.createCollectionLoadingPolicy(0, 100);
> session.options().setCollectionLoadingPolicy(policy);
> }|
>
>
> This one initially loads no target IDs into objects. Later, when a list
> element is accessed, a maximum range of 100 elements *around* the
> accessed list index are loaded if necessary.
>
> Cheers
> /Eike
>
> ----
> http://thegordian.blogspot.com
>
>
> Kai Schlamp schrieb:
>> Hello again.
>>
>> If I understand it right, CDO takes care about the lazy loading. Is
>> this completely independent from the persistence store (e.g.
>> DerbyStore or Teneo) used in the background?
>> What exaclty can be lazy loaded?
>>
>> For example, let's assume the simple model in pseudo code like this:
>>
>> class Document {
>> String small;
>> String large;
>> }
>>
>> And now let's assume that I have 100000 documents of that type and
>> want to load all those documents into one table, but only want to show
>> the "small" string. The "large" string will only be accessed, when
>> clicking directly on one document. And now let's also assume that the
>> whole 100000 documents won't fit into memory cause of the huge data
>> contained inside the "large" fields.
>> Will "large" be lazy loaded? (With Hibernate normal attributes by
>> default won't be lazy loaded.)
>> If not, would the following design help to use lazy loading for
>> "large" also in CDO? (It would help within Hibernate).
>>
>> class Document {
>> String small;
>> LargeObject largeObject;
>> }
>>
>> class LargeObject {
>> String large;
>> }
>>
>> Best regards,
>> Kai
>>
Re: [CDO] Proxies / Lazy Loading [message #425958 is a reply to message #425956] Sat, 13 December 2008 17:51 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Kai,

Comments below...


Kai Schlamp schrieb:
> Hy Eike,
>
> thanks a lot for the lightning fast answer.
You're welcome ;-)

> A last question about that topic.
> Let's get back to the table scenario with the little example model at
> the bottom.
> When I want to only show documents in the table that contain a
> specific search string in the "large" attribute of LargeObject, all
> LargeObject objects must be fetched (as there is no query solution in
> CDO 1.0.x), to test if those contain the search string. Are already
> tested LargeObjects garbage collected again and the largeObject
> attribute of Document set back to the appropriate proxy?
Yes, that's the expected behaviour.

> Cause otherwise the whole model would not fit in the available memory.
> I read somewhere that CDO (in contrast to Hibernate) supports this. Is
> that right? Automatically without having me to do something?
Yes, it's incredible :P

Cheers
/Eike

----
http://thegordian.blogspot.com


>
> Best regards,
> Kai
>
>
> Eike Stepper schrieb:
>> Kai,
>>
>> Comments below...
>>
>>
>> Kai Schlamp schrieb:
>>> Hello again.
>>>
>>> If I understand it right, CDO takes care about the lazy loading. Is
>>> this completely independent from the persistence store (e.g.
>>> DerbyStore or Teneo) used in the background?
>> Yes, completely independent.
>>
>>> What exaclty can be lazy loaded?
>> All references, i.e. single-valued, many-valued, cross references,
>> containments...
>>
>>>
>>> For example, let's assume the simple model in pseudo code like this:
>>>
>>> class Document {
>>> String small;
>>> String large;
>>> }
>>>
>>> And now let's assume that I have 100000 documents of that type and
>>> want to load all those documents into one table, but only want to
>>> show the "small" string. The "large" string will only be accessed,
>>> when clicking directly on one document. And now let's also assume
>>> that the whole 100000 documents won't fit into memory cause of the
>>> huge data contained inside the "large" fields.
>>> Will "large" be lazy loaded? (With Hibernate normal attributes by
>>> default won't be lazy loaded.)
>> No, CDO does currently not support lazy loading of single *attributes*.
>> What about making "large" a derived feature that is computed from a
>> contained data object?
>>
>>> If not, would the following design help to use lazy loading for
>>> "large" also in CDO? (It would help within Hibernate).
>>>
>>> class Document {
>>> String small;
>>> LargeObject largeObject;
>>> }
>>>
>>> class LargeObject {
>>> String large;
>>> }
>> Oh yes, this is what I had in mind above ;-)
>>
>> Cheers
>> /Eike
>>
>> ----
>> http://thegordian.blogspot.com
>>


Re: [CDO] Proxies / Lazy Loading [message #425959 is a reply to message #425957] Sat, 13 December 2008 17:55 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6682
Registered: July 2009
Senior Member
Kai Schlamp schrieb:
> Thanks for the tip, Eike.
> Is this policy also available in CDO 1.0.x?
> As in the docs
> (http://download.eclipse.org/modeling/emf/cdo/javadoc/1.0.0/) I don't
> see such a create method of CDOUtil, nor a class called
> CDOCollectionLoadingPolicy.
Apologies!!! I forgot for a moment that you are not using our wonderful
new version ;-)

> Perhaps I switch to CDO 2.0.0 soon. I tried this once, but it didn't
> seem to be so easy as there were some conflicts with plugins of the
> Eclipse 3.4.1 Ganymede release (is there somewhere a little
> installation instruction for CDO 2.0.0 + Eclipse Ganymede?) .
No, because this setup is not officially supported. Users have reported
that it works, but it's on your own risk!

> But I will retry this, at the latest when there is a query
> implementation (hql, sql, whatever) for CDO 2.0.0.
It's moving up ;-)

Cheers
/Eike

----
http://thegordian.blogspot.com


Previous Topic:[CDO] Update from 1.0.0 to 1.0.x
Next Topic:ECore Editor not displaying as model tree
Goto Forum:
  


Current Time: Thu Mar 28 23:22:51 GMT 2024

Powered by FUDForum. Page generated in 0.02939 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top