| [CDO] Caching and explicit garbage collection [message #1799463] |
Fri, 07 December 2018 03:59  |
Eclipse User |
|
|
|
Our situation:
* Very large CDO archive which does not fit into RAM
* We want to recursively access every object
We access the objects recursively via the method eobject.eContents().
Even though we do not keep references to any objects and also tried using separate cdoviews which are closed from time to time, the RAM builds up.
What did work for us is opening new cdoSessions for larger chunks of data and then close the session again. But this does not seem a nice way to do it.
Is there any way to explicitly clear the cache of a session?
Please note: we are still using CDO 4.5.
|
|
|
|
|
| Re: [CDO] Caching and explicit garbage collection [message #1800853 is a reply to message #1800146] |
Wed, 09 January 2019 01:42  |
| Eclipse User |
|
|
|
I doubt that you could do better than a garbage collector. Perhaps you could try a different one? I have absolutely no experience with that, though.
There's also code at the beginning of CDOSessionImpl.refresh(Provider), which you could try to mimic:
Map<CDOBranch, List<InternalCDOView>> views = new HashMap<CDOBranch, List<InternalCDOView>>();
Map<CDOBranch, Map<CDOID, InternalCDORevision>> viewedRevisions = new HashMap<CDOBranch, Map<CDOID, InternalCDORevision>>();
collectViewedRevisions(views, viewedRevisions);
cleanupRevisionCache(viewedRevisions);
And:
private void collectViewedRevisions(Map<CDOBranch, List<InternalCDOView>> views, Map<CDOBranch, Map<CDOID, InternalCDORevision>> viewedRevisions)
{
for (InternalCDOView view : getViews())
{
if (view.getTimeStamp() == CDOView.UNSPECIFIED_DATE)
{
CDOBranch branch = view.getBranch();
Map<CDOID, InternalCDORevision> revisions = viewedRevisions.get(branch);
boolean needNewMap = revisions == null;
if (needNewMap)
{
revisions = CDOIDUtil.createMap();
}
view.collectViewedRevisions(revisions);
if (!revisions.isEmpty())
{
List<InternalCDOView> list = views.get(branch);
if (list == null)
{
list = new ArrayList<InternalCDOView>();
views.put(branch, list);
}
list.add(view);
if (needNewMap)
{
viewedRevisions.put(branch, revisions);
}
}
}
}
}
private void cleanupRevisionCache(Map<CDOBranch, Map<CDOID, InternalCDORevision>> viewedRevisions)
{
Set<InternalCDORevision> set = new HashSet<InternalCDORevision>();
for (Map<CDOID, InternalCDORevision> revisions : viewedRevisions.values())
{
for (InternalCDORevision revision : revisions.values())
{
set.add(revision);
}
}
InternalCDORevisionCache cache = getRevisionManager().getCache();
List<CDORevision> currentRevisions = cache.getCurrentRevisions();
for (CDORevision revision : currentRevisions)
{
if (!set.contains(revision))
{
cache.removeRevision(revision.getID(), revision);
}
}
}
It's important to NOT remove a revision from the cache if it is currently being referenced from at least one EObject in at least one CDOView. This restriction may not apply if you can guarantee that no user is committing changes to the repository. In that case you could even try to configure a dummy revision cache in your session:
IConnector connector = Net4jUtil.getConnector(container, connectorType, connectorDescription);
CDONet4jSessionConfiguration config = CDONet4jUtil.createNet4jSessionConfiguration();
config.setConnector(connector);
config.setRepositoryName(repositoryName);
config.setRevisionManager(CDORevisionUtil.createRevisionManager(CDORevisionCache.NOOP));
|
|
|
Powered by
FUDForum. Page generated in 0.04355 seconds
Home