Home » Eclipse Projects » Eclipse Platform » PropertyManager2 iterates the tree for infinite depth for deleting/moving an IFile(Platform Resources)
| PropertyManager2 iterates the tree for infinite depth for deleting/moving an IFile [message #1836701] |
Tue, 12 January 2021 14:07  |
 Palraj Jayaraj
Messages: 15
Registered: June 2015 |
Junior Member |
|
|
I have a scenario, where I have around 10K files inside a folder named '_out'. and '_out' in-turn contains other folders and inner files and sub folders.
I'm deleting around 6000 files , which are directly present under the _out folder.
I delete the files by using the following code snippets:
for(IFile iFile:files){
iFile.delete(true, new NullProgressMonitor());
}
During the delete of each file, Property of each resource is being deleted, internally in the API Resource.deleteResource(boolean, MultiStatus), using method call to IPropertyManager.deleteResource(IResource). This API in-turn calls PropertyManager2.deleteProperties(IResource, int depth) with INFINITE depth and hence, all the folder and sub-folders inside the location "\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\projectName\.indexes\8f" are visited and 'Properties.index' files are loaded in each folder and then, checked whether the given IFile has a property in them. Where '8f' folder represents the Bucket for the '_out' folder.
for all the 6000 files, this operation is being repeated and hence, it takes around 15 minutes to delete all the 6000 files.
Ideally, should we just visit the properties.index file under the '8f' and delete the property for the given file and return?
My assumption is that the Properties for any files present directly under the '_out' folder will be saved under the 'properties.index' file in the Bucket corresponding to '_out' folder. (in my scenario 8f folder).
Apologies, if I have misunderstood it. Kindly, clarify.
I have raised a bug for the same at link.
Thanks,
Palraj
|
|
| |
| Re: PropertyManager2 iterates the tree for infinite depth for deleting/moving an IFile [message #1837059 is a reply to message #1837002] |
Thu, 21 January 2021 13:55  |
Palraj Jayaraj
Messages: 15
Registered: June 2015 |
Junior Member |
|
|
Whenever an IResource is deleted, the properties of the specific resource are also deleted internally in the IResource.delete(boolean, IProgressMonitor) API.
PropertyManager2.deleteResource(IResource target) API is used to delete the properties associated with the given IResource.
The PropertyManager2.deleteProperties(IResource target, int depth) is called with DEPTH_INFINITE as depth, for all the resources.
For an IFile, should we visit for infinite depth for looking for properties for that IFile? The properties for the IFile should be present in the level zero.
Lets say, I have the below folder structure:
Project
->FolderA
-->FolderB
--->FileB
-->FolderC
--->FileC
-->FileD
-->FileE
->FolderAA
-->FolderBB
--->FileBB
-->FolderCC
--->FileCC
-->FileDD
->FolderXZY
->FileX
->FileY
->FileZ
For testing purpose I created properties for all the above IResources, using the API IResource.setPersistentProperty(QualifiedName, String).
The data are being stored inside the workspace folder. workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\projectName\.indexes.
The properties of each resources are stored in below folder structure by eclipse:
projectName -> \projectName\.indexes\properties.index
projectName/FolderA -> \projectName\.indexes\properties.index
projectName/FolderAA -> \projectName\.indexes\properties.index
projectName/FolderXZY -> \projectName\.indexes\properties.index
projectName/FileX -> \projectName\.indexes\properties.index
projectName/FileY -> \projectName\.indexes\properties.index
projectName/FileZ -> \projectName\.indexes\properties.index
projectName/FolderA/FolderB -> \projectName\.indexes\63\properties.index
projectName/FolderA/FolderC -> \projectName\.indexes\63\properties.index
projectName/FolderA/FileD -> \projectName\.indexes\63\properties.index
projectName/FolderA/FileE -> \projectName\.indexes\63\properties.index
projectName/FolderA/FolderB/FileB -> \projectName\.indexes\63\2e\properties.index
projectName/FolderA/FolderC/FileC -> \projectName\.indexes\63\2f\properties.index
projectName/FolderAA/FolderBB -> \projectName\.indexes\64\properties.index
projectName/FolderAA/FolderCC -> \projectName\.indexes\64\properties.index
projectName/FolderAA/FileDD -> \projectName\.indexes\64\properties.index
projectName/FolderAA/FolderBB/FileBB -> \projectName\.indexes\64\4d\properties.index
projectName/FolderAA/FolderCC/FileCC -> \projectName\.indexes\64\4e\properties.index
Lets consider the scenario, where the file 'FileZ' is being deleted. Ideally FileZ should be deleted and its respective property should be deleted, here the property is saved in \projectName\.indexes\properties.index, so this file should be loaded and the property of FileZ should be deleted.
But what happens actually is to delete the property of FileZ, all the properties.index files listed above are visited which is a performance overhead, the code flow is explained below.
When a FileZ is deleted PropertyManager2.deleteProperties(IResource target, int depth) is called with DEPTH_INFINITE as depth.
This API calls the BucketTree.accept(Visitor visitor, IPath base, int depth) with infinite depth. This API inturn calls BucketTree.internalAccept(Visitor, IPath, File, int, int).
The below 2 steps are performed, while deleting an IFile:
1. As we are deleting the file right under the project, '\projectName\.indexes\properties.index' file is loaded and the properties for that specific resource are removed.
2. And then again all the directories under the \projectName\.indexes\ are visited (63 and 64) aand the 'properties.index' file under these 2 folders are again loaded and the property of file FileZ is tried to be removed here as well. Then,even the folder inside the '63'(2e and 2f) are again visited, this goes on for all the level of directories found inside the \projectName\.indexes\ directory.
From my analysis, As the properties for the given file 'FileZ' is present in the '\projectName\.indexes\properties.index' file, this file alone could be loaded and deleted. The above step 2 is not required completely.
This recursive visiting for all the directories and loading the 'properties.index' file and checking for the resource for a given single IFile object, while deleting it, creates a performance issue.
We have a scenario where we are deleting around 6,000 files, for each delete the above 2 steps are repeated each time. So, the performance issue aggregates and this leads to a total time of 17 to 18 minutes for deleting the 6,000 files. This is the issue.
|
|
|
Goto Forum:
Current Time: Fri Apr 26 07:07:35 GMT 2024
Powered by FUDForum. Page generated in 0.03126 seconds |
] 
Search
Help
Register
Login
Home
