Eclipse Community Forums: EclipseLink » Eclipselink Performance test failed at 95 percentile

Home » Eclipse Projects » EclipseLink » Eclipselink Performance test failed at 95 percentile(Eclipselink is calling Millions of call to DB)

Eclipselink Performance test failed at 95 percentile [message #1753476]

Mon, 06 February 2017 17:06

Eclipse User

Hi All,

I have CXF Jax-WS base webservices that is querying Oracle DB using Eclipselink JPA.
DB structure is Little hybride like use of Anchor table to maintain the history. I have close to 30 table out of which few are reference table like country name, type and 10 Data table 10 anchor table for data table and rest are relationship tables.

Webservices is for CRUD and I have
roperty name="eclipselink.persistence-context.flush-mode" value="commit" />

Now when good load on the server say 10-20 Transaction running in parallel for minute or so I saw at end of test(5 Min test) JPA(eclipselink) is pulling hell lots of data from DB with no relationship, all that is happening at the end of transaction when flush is happening. and it fails for transaction timeout.

when I re-run same request without load it get processed in 100MS.

Re: Eclipselink Performance test failed at 95 percentile [message #1753572 is a reply to message #1753476]

Tue, 07 February 2017 13:49

Eclipse User

There isn't enough information given on what you are doing in your test(s) to comment on anything.

Re: Eclipselink Performance test failed at 95 percentile [message #1753661 is a reply to message #1753572]

Wed, 08 February 2017 10:33

Eclipse User

We are running a load test(500 Transaction in 5 min ) to insert data into 30 tables.
Each request is unique request for concurrent requests so no data overlap,
Each transaction can be either pure insert or update to existing data. the code process is read xml and store that into database using JPA, XML and DB are different model so translation classes are there.
For first few minutes test runs smooth, suddenly JPA started behaving weird with pulling all the records with no relation to transaction.
if I run same request again it process well with appropriate data behaviour.

In code all of my entity relation are cascade and I do persist at once that is end of transaction.
I have below property to avoid unnecessary db call when I am processing data.
<property name="eclipselink.persistence-context.flush-mode" value="commit" />
I also have
<shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>

the help I am looking is what could cause eclipselink to get all the data from table without any relationship.

Re: Eclipselink Performance test failed at 95 percentile [message #1753682 is a reply to message #1753661]

Wed, 08 February 2017 13:10

Eclipse User

Nothing you've provided gives any indication as to why EclipseLink would start 'pulling all records'. EclipseLink wouldn't do that by itself - something within the application forces it to query data.

Check the process causing the issue, and ensure you have logging on. Check the data you are trying to use - you mention you update, so could you be calling merge on data that is accumulating from your load tests?

Re: Eclipselink Performance test failed at 95 percentile [message #1753689 is a reply to message #1753682]

Wed, 08 February 2017 18:17

Eclipse User

At the end of transaction I am calling entitymanger.persist(rootEntity). EntityManger is managed by Spring Injection. all the data fetch is happening by persist operation only. we are not using merge anywhere. my understanding is persist should manage both new and update.
The strange part in whole test is if I run same data on without load it process flawlessly.

I am really not sure what detail I should be proving, I have all default configuration, whatever I had in persistnace.xml is shared.

Again thanks for help. it would be great if i get some pointer to go on.

Re: Eclipselink Performance test failed at 95 percentile [message #1753764 is a reply to message #1753689]

Thu, 09 February 2017 16:41

Eclipse User

Persist is for new entities and will only insert into the database -it doesn't do updates. If your code works and runs flawlessly, then it isn't something standard causing the issue - it'll be something buried, specific to your usage that isn't thread safe. If you can get an exception when it starts pulling data, you'd be able to see what part of the code is causing the query.

Re: Eclipselink Performance test failed at 95 percentile [message #1753982 is a reply to message #1753764]

Mon, 13 February 2017 11:29

Eclipse User

I am doing Persist because of performance and at the end of services I am calling it so there will be no modification to it. with merge, EM will create a duplicate copy to work on which I don't want.
About the performance issue.
-- I am sending of unique request to a webservices that store data into oracle DB. I have close to 30 tables.
-- when I send data at 5 threads in concurrent for 5 min. first couple of minutes I do not see any issue.
-- issue stats when CPU goes to 100% and some how few request pulls whole table data for no reason when I do call Persist/Merge
-- if we take those Request out and re-run, it get processed in few millisec.
-- if it was programming error then re-run could have catched the issue.
Not sure why when I call persist/merge it

Re: Eclipselink Performance test failed at 95 percentile [message #1754117 is a reply to message #1753982]

Tue, 14 February 2017 10:30

Eclipse User

I am still sceptical it has anything to do with persist - persist cannot be used on existing ,unmanaged entities. So it is likely these entities are already managed and it is a no-op (other than to cascade the persist which happens on synchronization anyway), in which case Merge too would be a no-op.

Chances are good you have some shared objects or synchronization slowing things down. Without seeing what the application is doing in those threads, there isn't much we can help with. Grab threaddumps during the slow down to see what the threads might be getting stuck on, or causing a bottleneck. A few thread dumps might help show where the problem is in the code, or if some database access is locking, such as if you are using select for update.

Re: Eclipselink Performance test failed at 95 percentile [message #1756068 is a reply to message #1754117]

Fri, 10 March 2017 09:53

Eclipse User

Hi Chris,

Thanks for all the help, I have new finding to the problem, we are able to isolate issue and right now if I do only create operation on those table then there is no issue I get consistant 400 to 500 MS response. but if I start update in between then suddenly issue occurs.
Below is sudo logic we have

check root entity is present in the DB
if No, create new, all data will be new
If yes, get the root entity, traverse through each entity and check if any data has changed with incoming data and update the root entity tree graph.

to avoid enitimanager DB sync call we have marked
<property name="eclipselink.persistence-context.flush-mode" value="commit" />

is this a issue as all entities are cascade and I am having close to 1M record on root table.

Re: Eclipselink Performance test failed at 95 percentile [message #1757925 is a reply to message #1756068]

Tue, 21 March 2017 17:26

Eclipse User

Just update:

After having Static weaving we have reduced 200M DB calls to 10M DB call. The question still existing why dynamic weaving not working for 15-20 table classes.

Previous Topic:	SOAP IllegalArgumentException for array parameter
Next Topic:	Adding dependency to EclipseLink (ant+maven+Tycho)

Goto Forum:

-=] Back to Top [=-

Current Time: Wed Jul 09 22:41:13 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter