| Re: EMF load resource behavior when the resource is a Database via Teneo [message #612561] |
Fri, 16 November 2007 11:37 |
 Ed Merks
Messages: 33142
Registered: July 2009 |
Senior Member |
|
|
Jason,
Comments below.
jason henriksen wrote:
>
> Hi Ed,
>
>
>> I don't recognize EcoreUtil.getSuppressFullResourceLoading() is that
>> org.eclipse.emf.ecore.util.EcoreUtil?
>
> Yes, I've got my own patched version that I use for my project that
> adds this method.
That explains it.
>
> However, running my own patched version of things is not a fun and
> makes my client a little nervous. I'd rather be running directly on
> your releases and not have to manage my own special sauce if I can
> reasonably avoid it.
Yep, it would make me nervous. I'm not sure why you can't just
specialize your own derived ResourceSetImpl.
>
>> I assume you are using ID's because if you are using fragment paths,
>> it would normally just walk down the branch that's used for that path.
>
> The url's end up looking something like this:
> hibernate://Consumer/12345 (pseudo code)
>
> Teneo has protocol handlers that get the URL above and build a SQL
> query out of it. I'm not fully clear on all the teneo aspects of this
> but what I found from looking at the database is that if you clear the
> fragment the query gets turned from
> "Select * from Consumer where id=12345" // return 1 or 0 records.
> into
> "Select * from Consumer" // return 57 million records.
That would be quite a few!
>
> Or to put my own comments into the source:
>
> public EObject getEObject(URI uri, boolean loadOnDemand)
> {
> //--- Load the entire resource (i.e. the whole table)
> Resource resource = getResource(uri.trimFragment(), loadOnDemand);
I guess because the objects are all a huge list in resource.getContents().
>
> if (resource != null){
> //--- Out of the resource, get the item I asked for.
> // (But never get here because I ran out of memory last line)
And Java only lets you have about 1.5G max for the heap. It's not very
scalable. :-P
> return resource.getEObject(uri.fragment());
> }
> else{
> return null;
> }
>
>> I imagine you could just provide your override in your own derived
>> ResourceSetImpl without changing the base implementation that
>> everyone uses.
>
> In fact, that is what I'm doing now. But Teneo can't really be used
> against very large databases if EMF behaves this way, which is a bummer.
It's not the size of the database I suppose, but rather the height of
the tables in it. Does Hibernate address these kinds of issues?
> I can publish this as something for people to download and use as a
> patch, but setting a flag to control the behavior in ECoreUtil seemed
> like the better plan. If you don't ask for the load suppression,
> everything acts as it does today. But if you run into the problem,
> you can just set the flag and be happy.
I would imagine that most applications have complete control over what
resource set is used, so I'm still not sure why providing an alternative
resource set implementation wouldn't suffice.
>
>> I can't say I really understand what getting a the resource with the
>> fragment still attached would accomplish. Is this more a question
>> for Teneo that Martin would normally answer on the EMFT newgroup?
>
> I hope that makes sense. If I'm mis-understanding the behavior of
> that code, I'd love to be brought up to speed.
It does, but it also seems to me that JPA/JDO/Hibernate would all suffer
from these same fundamental problems so I'm curious how they deal with
it. Do they somehow produce a query that returns a much smaller set of
the objects you're actually likely to use? I'll add the EMFT newsgroup
to the reply so Martin is more likely to see this and comment.
>
> Thanks,
>
> Jason
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
Powered by
FUDForum. Page generated in 0.02772 seconds
] 
Search
Help
Register
Login
Home
