Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc)  » [TEXO] Memory management(How to do memory management with EMF/TEXO instances)
[TEXO] Memory management [message #1428381] Sun, 21 September 2014 19:22 Go to next message
Mr Cur is currently offline Mr CurFriend
Messages: 35
Registered: September 2014
Member
I want to store logging data in a database and query the database
using a web browser.
My idea is to Texo for this, and more specifically the JSONTexoResource
component. The logging component, written in Java, updates the database
continuously using the JSON Texo resource class, which connects via
REST/JSON to the web services exposed by the Texo JSON solution.
The web browser will use the REST/JSON services as exposed by Texo.

The logging component is a continuously running software component
that creates EMF instances, representing the data to be logged,
which are stored in the database using the JSONTexoResource.
I create the JSONTexoResource on beforehand, and then create the
EMF instances, and save these via the resource regularly. Because the logging
component is a continuously running component many EMF instances
are created over time (which are stored in the database) and thus
increase in number.

My question is now how to do memory management for the logging component.
Once logged EMF instances are stored in the database, they are not needed
anymore by the logging component and therefore can be garbage collected.
Is it possible to release these instances somehow from the resource,
so that they do not consume memory anymore? I tried to use unload()
on the resource, but then I still see instances afterwards.
This happens specifically if I use the query method on the resource twice
or more on the same object.
I suppose it is also not a good idea to create a JSONTexoResource for every
log/EMF instance created, because creating the resource is on my
platform a rather time-consuming operation.
Re: [TEXO] Memory management [message #1428973 is a reply to message #1428381] Mon, 22 September 2014 16:19 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Mr. Cur,
For my understanding, are you using java in the browser? Or is your java code on the server side? If it is the latter
there is no reason to use the texojsonresource and you can also use direct JPA on the server.

If you run java in the browser, can you work with a small producer-consumer queue which every few minutes empties the
log queue and creates a jsontexouresource to persist on the server and then discards it?

Creating/discarding a texojsonresource should be quite lightweight, so not sure what takes a lot of time, can you
somehow see what takes the larger time in the resource creation?

gr. Martin

On 22-09-14 13:30, Mr Cur wrote:
> I want to store logging data in a database and query the database
> using a web browser.
> My idea is to Texo for this, and more specifically the JSONTexoResource
> component. The logging component, written in Java, updates the database
> continuously using the JSON Texo resource class, which connects via
> REST/JSON to the web services exposed by the Texo JSON solution.
> The web browser will use the REST/JSON services as exposed by Texo.
>
> The logging component is a continuously running software component
> that creates EMF instances, representing the data to be logged,
> which are stored in the database using the JSONTexoResource. I create the JSONTexoResource on beforehand, and then
> create the
> EMF instances, and save these via the resource regularly. Because the logging
> component is a continuously running component many EMF instances
> are created over time (which are stored in the database) and thus
> increase in number.
>
> My question is now how to do memory management for the logging component. Once logged EMF instances are stored in the
> database, they are not needed
> anymore by the logging component and therefore can be garbage collected.
> Is it possible to release these instances somehow from the resource,
> so that they do not consume memory anymore? I tried to use unload()
> on the resource, but then I still see instances afterwards.
> This happens specifically if I use the query method on the resource twice
> or more on the same object.
> I suppose it is also not a good idea to create a JSONTexoResource for every
> log/EMF instance created, because creating the resource is on my
> platform a rather time-consuming operation.


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@xxxxxxxx - mtaal@xxxxxxxx
Web: www.springsite.com - www.elver.org
Re: [TEXO] Memory management [message #1428991 is a reply to message #1428381] Mon, 22 September 2014 16:45 Go to previous messageGo to next message
Mr Cur is currently offline Mr CurFriend
Messages: 35
Registered: September 2014
Member
Hi Martin,

Thanks for your answers.
The Java code is running in a server process, whereas on the client,
I want to use Javascript to interact via JSON with Texo.

Trying to understand Texo, what I have done with library example is to generate
EMF model Java code, and to use this code to create library/book instances, and to
save this via a JSONTexoResource instance to the Texo server (where I have the server side of Texo running in Tomcat, with the Library examples classes as you illustrate
in the video).

It is perhaps a bit confusing what the client and server is:
My logging program is from my point of view of server but behaves as a client to
Texo REST services running in Tomcat.

So my thought was:
- use a browser with Javascript to query the JSON Texo Rest services on the Tomcat server (is there some javascript library for doing this?)
- use EMF generated classes in the Java-based logging service to connect to the Texo JSON Rest services via the JSONTexoResource.

Is there a better alternative?
Re: [TEXO] Memory management [message #1429033 is a reply to message #1428991] Mon, 22 September 2014 18:01 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hi Mr Cur,
In this case I would propose to use direct JPA on the server so not go through EMF objects. You can directly create Texo
objects and store them using standard JPA approaches with EntityManager etc. Texo also generates a DAO layer if you like
that:
http://martintaal.wordpress.com/2011/11/10/emft-texo-generating-a-dao-layer-from-ecorexsd-models/

I don't have a specific javascript library for you to use, but most/all client side javascript libraries can talk json
with requests to the server, I have experience with smartclient, and minor experience with extjs and other js frameworks.

gr. Martin


On 22-09-14 18:45, Mr Cur wrote:
> Hi Martin,
>
> Thanks for your answers.
> The Java code is running in a server process, whereas on the client,
> I want to use Javascript to interact via JSON with Texo.
>
> Trying to understand Texo, what I have done with library example is to generate Cur
> EMF model Java code, and to use this code to create library/book instances, and to
> save this via a JSONTexoResource instance to the Texo server (where I have the server side of Texo running in Tomcat,
> with the Library examples classes as you illustrate
> in the video).
>
> It is perhaps a bit confusing what the client and server is:
> My logging program is from my point of view of server but behaves as a client to Texo REST services running in Tomcat.
>
> So my thought was:
> - use a browser with Javascript to query the JSON Texo Rest services on the Tomcat server (is there some javascript
> library for doing this?)
> - use EMF generated classes in the Java-based logging service to connect to the Texo JSON Rest services via the
> JSONTexoResource.
>
> Is there a better alternative?


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@xxxxxxxx - mtaal@xxxxxxxx
Web: www.springsite.com - www.elver.org
Re: [TEXO] Memory management [message #1429077 is a reply to message #1428381] Mon, 22 September 2014 19:29 Go to previous messageGo to next message
Mr Cur is currently offline Mr CurFriend
Messages: 35
Registered: September 2014
Member
Thanks for your answer.
Re: [TEXO] Memory management [message #1430828 is a reply to message #1428381] Wed, 24 September 2014 22:00 Go to previous messageGo to next message
Mr Cur is currently offline Mr CurFriend
Messages: 35
Registered: September 2014
Member
I tried to implement a JPA client based on the library case, by creating the Java classes with Texo and the orm's for the identifiable and library constructs.
Quering works ok, but updating not.
If I create a writer, and add this to database with the persist operation, I get a constraint violation on the primary key, namely that the key is duplicate. I do not
set the Id explicitly in the identifiable class, but I do not so either for the JSONTexoresource example I developed (and then creation of the writer works correctly, without setting the Id). If I set the Id explicitly in my JPA code, then everything works. My assumption was that the Texo implementation sets this Id automatically. Is this the case?
Re: [TEXO] Memory management [message #1431259 is a reply to message #1430828] Thu, 25 September 2014 12:22 Go to previous messageGo to next message
Martin Taal is currently offline Martin TaalFriend
Messages: 5468
Registered: July 2009
Senior Member
Hey,
In the orm there should be tags in the <id> tag to tell hibernate to generate a id. Can you post/link the orm you have?

gr. Martin

On 25-09-14 00:00, Mr Cur wrote:
> I tried to implement a JPA client based on the library case, by creating the Java classes with Texo and the orm's for
> the identifiable and library constructs.
> Quering works ok, but updating not.
> If I create a writer, and add this to database with the persist operation, I get a constraint violation on the primary
> key, namely that the key is duplicate. I do not
> set the Id explicitly in the identifiable class, but I do not so either for the JSONTexoresource example I developed
> (and then creation of the writer works correctly, without setting the Id). If I set the Id explicitly in my JPA code,
> then everything works. My assumption was that the Texo implementation sets this Id automatically. Is this the case?


--

With Regards, Martin Taal

Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Cell: +31 (0)6 288 48 943
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@xxxxxxxx - mtaal@xxxxxxxx
Web: www.springsite.com - www.elver.org
Re: [TEXO] Memory management [message #1431282 is a reply to message #1430828] Thu, 25 September 2014 13:05 Go to previous messageGo to next message
Mr Cur is currently offline Mr CurFriend
Messages: 35
Registered: September 2014
Member
Hi Martin,

I just use the orm as part of the Texo web example:

...
<orm:mapped-superclass access="FIELD" class="org.eclipse.emf.texo.test.model.base.identifiable.Identifiable">
<orm:attributes>
<orm:id name="db_Id">
<orm:generated-value strategy="AUTO"/>
</orm:id>
<orm:version access="FIELD" name="db_version">
<orm:convert></orm:convert>
</orm:version>
</orm:attributes>
</orm:mapped-superclass>
...

I observed the following:
- If I start with an empty database, I can create instances using 'Generate Test Data' option in the example via the web interface. However, If I look into the mysql database directly, I see a table 'SEQUENCE' with the columns 'SEQ_NAME' and 'SEQ_COUNT'. I assume the 'SEQ_COUNT' is used to keep track of identifiers that have to be generated? *After* I generated instances via the 'Generate Test Data' option, I see however that the 'SEQ_COUNT' still contains '0'.
- If however, I start again with an empty database, and use my small test program to create instances myself (just via JPA), then these instances are created, and the 'SEQ_COUNT' is increased to 50.

Is there perhaps a problem with how identifiers are created?

Thanks,
Re: [TEXO] Memory management [message #1431379 is a reply to message #1428381] Thu, 25 September 2014 16:04 Go to previous message
Mr Cur is currently offline Mr CurFriend
Messages: 35
Registered: September 2014
Member
To add to the above, it seems that using the database via
the JSONTexoResource (and the use of the 'Generate Test Data' option in the web interface) interworks with using the database via JPA direclty, with respect to generating identifiers for the indentifiable.

Insertion of an object via the JSONTexoResource (and the 'Generate Test Data' option in the web interface) seems to have an own counter for identifiers and accessing the database via JPA has another counter. Is this possible?
Previous Topic:[EMFTransaction] EMF Transaction in a web app
Next Topic:ViewModelFileExtensionsManager singleton
Goto Forum:
  


Current Time: Tue Mar 19 09:06:10 GMT 2024

Powered by FUDForum. Page generated in 0.03071 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top