Hi Chris,
Thanks a lot, indeed you are correct, the caching doesn’t seem to be even remotely involved in my issue. My biggest misunderstanding there is that until now I believed that unit
of works are the same as transaction. In the failing log I can see a new “begin transaction” just before the issue start to manifest itself, and that without seeing a “commit transaction” of the previous one. That clearly both explains the foreign key violation
and the missing join entity, as in the new transaction those entities are not there. That new transaction doesn’t happen in the case where the test succeeds.
We are using Spring transaction management, so I guess my next step is to turn logging on at that place, and have spring explain me why the new transaction is necessary.
Thanks again for your eyes opener!
Regards,
Ben.
Mit freundlichen Grüßen / Kind regards
/ С уважением
Benoit Allard
Software Engineer
SOFTWARE. APPLICATIONS. SERVICES.
SLA
Software Logistik Artland GmbH
Support
DE +49 5431 9480 -
77
AT +43 720 115300
Geschäftsführer Jörg Brezl (CEO) |
Hermann Grevemeyer (COO)
Handelsregister Amtsgericht Osnabrück - HRB 20381
Follow us:
  
Von: eclipselink-users-bounces@xxxxxxxxxxx <eclipselink-users-bounces@xxxxxxxxxxx>
Im Auftrag von Christopher Delahunt
Gesendet: Thursday, 8 November, 2018 16:53
An: EclipseLink User Discussions <eclipselink-users@xxxxxxxxxxx>
Betreff: Re: [eclipselink-users] Multi-thread cache corruption
Why you think this is a cache corruption within Eclipselink? The database is stating the entry isn't there (not a cache), and you are stating it was added in the same transaction as the insert statement referring to it that is failing.
One of the two statements must be wrong, or you have an issue at the database level.
The only explanation other I can come up with would be that the transaction is not the same, so that the first insert is no longer visible to your existing UnitOfWork/EntityManager. You mention you are doing work asynchronously - check
that the datasource isn't returning a new connection/transaction to your thread processing work. From the EclipseLink logging you've shown, I don't see a reason for the exception, but you are using an external connection pool with JTA, so it all depends on
how the transaction is demarcated and being handled underneath EclipseLink.
I believe we have some case of cache corruption. We can reproduce it within a test-case of our application with ~70% reliability.
How the issues manifest itself: within a transaction, a flush of the EntityManager triggers a Foreign key violation about another entry that has been added (and successfully updated)
within the same transaction.
What we did to trigger the issue: we started passing id to the asynchronous workers (and let them fetch the entity themselves) instead of passing them the entity directly. This
does put a lot of more work onto the entity manager, causing it to corrupt its cache (I believe). When we remove that modification, that issue does not happens. Which leads us to believe that using eclipselink asynchronously more than before is what triggers
the issue.
Enclosed is an example of logs that get generated while the issue occurs. That log is trimmed to the instant where the erroneous unit-of-work (1298690816) is active. Some other
are visible, those are the mentioned asynchronous workers. Let me go briefly through it:
An entity of type slaughter_party is first inserted (id: 640). That entity is queried, updated a few times. Later another entity of the same type is created with a (supposedly)
reference to the first one (old_party_id). The flush of that second entity claims that the first one does not exist.
Sometimes, that same issue (a foreign key violation) happens with different objects, always related to a missing entity of type slaughter_party.
There is another cache error happening in this log (probably related): an entity of type join_partner_business (partner 635, business 607) is inserted with id 639 after checking
that it doesn’t exists (select count …). That same procedure happens a few moments later and decides to insert a new join_partner_business (id 647) about the same relation. In case where it works, that second join entity is (rightfully) never inserted.
I tried with various DB backend (HSQL in-memory, H2 in-memory and H2 file), as those are quite easy to setup, they all trigger the same issue.
I hope you can shed some light on our issue and point me to my mistake as I don’t really want to believe that multi-threaded eclipselink is fundamentally broken.
<red-trimmed.txt>_______________________________________________
eclipselink-users mailing list
eclipselink-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eclipse.org_mailman_listinfo_eclipselink-2Dusers&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=aeYmvFwRsCWfafng1rj-AcoD-cr8mkJQBEoVD5OszU0&m=fv5NJR7lxB27iwHQEuJg02XbNN2yukmOASh3qx1Qri8&s=vdyYBycMdPSzSfN87Jj1BjJ2WafXHNJiK-tDEptxkLA&e=
|
Friedrichstr. 30, D-49610 Quakenbrück
+49 5431 9480 - 379 
+49 5431 9480 - 979
Keithstraße 14, D-10787 Berlin
Dinglingsweg 1a, D-49565
Bramsche