Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Platform » Asynchronuously write into workspace file(How to correctly write content into a workspace located file using background processes)
icon5.gif  Asynchronuously write into workspace file [message #1059989] Wed, 22 May 2013 09:06 Go to next message
Hendrik Renken is currently offline Hendrik Renken
Messages: 18
Registered: July 2009
Junior Member
Hi,

i'm (more than a little bit) confused about how to correctly write content into a file in the workspace using background processes.

At first, i used a simple thread which called IResource#setContent (and the like), however, that seemed to block the workspace and come with all sorts of downsizes.
Therefore, i've switched to the WorkspaceJob from the ResourcePlugin.


The content is streamed over the network into an instance of java.io.InputStream and i want to save this content into a specific file (defined by the given URI).
This process can take up to several minutes. Therefore, the write process should run asynchronuously in the background.

Here is my current code:
protected void writeToURI(final URI uri, final InputStream stream) throws IOException
{
  final IFile res = (IFile) WorkspaceTools.getResource(uri); //determine the IFile from the given URI
  final int hint = ... //determine correct write hint based on network request and actual resource state

  WorkspaceJob writeJob = new WorkspaceJob("Write content to " + uri)
  {

    public IStatus runInWorkspace(IProgressMonitor monitor)
    {
      try
      {
        if (!res.exists())
          ensureExists(res.getParent());
        switch (hint)
        {
           case CREATE:
             res.create(stream, true, monitor);
             break;
           case APPEND:
             res.appendContents(stream, true, false, monitor);
             break;
           case SET_CONTENT:
             res.setContents(stream, true, true, monitor);
        }
      } catch (CoreException e)
      {
        Close.close(stream);
        return new Status(IStatus.ERROR, "de.d3fact.server.net.interfaces", "Could not write to file " + uri, e);
      }
      return Status.OK_STATUS;
    }
  };

  writeJob.setRule(null);
  writeJob.schedule();
}


The problem with this code is, that it, regardless of the ISchedulingRule, it seems to block the workspace.
In my scenario, i'm writing to a simple text file "Test.txt" in a simple folder called "debug" in a java project. The content consists of simple lines of text.
Now, when i try to modify java files in the same project - while i receive data over the network and write it with said code into said file, everything locks up and i have to kill the eclipse instance.
I have tried different ISchedulingRules (the IFile, IResourceRuleFactory#createRule() and IResourceRuleFactory#modifyRule respectively and currently i'm using null to have maximum concurrency).

I'm running eclipse 4.3.0 RC1 on a Mac OSX system.


My last guess is, to use the java.io.file API and when streaming has finished to run an IResource#refreshLocal(). But before i implement that hack, i wanted to ask you how to correctly write content into a file without locking up the workspace.
Re: Asynchronuously write into workspace file [message #1060041 is a reply to message #1059989] Wed, 22 May 2013 11:34 Go to previous messageGo to next message
Devi Vara Prasad Bandaru is currently offline Devi Vara Prasad Bandaru
Messages: 98
Registered: March 2010
Location: Hyderabad
Member

You can try setting the resource as rule to worskpace job
writeJob.setRule(res);

Since you are modifying a single file ideally you don't even need to use WorkspaceJob as the purpose of it is for batching multiple workspace operations into single atomic operation to avoid unnecessary notifications for each operation (Check the documentation).

So you can create a simple job and set the resource as its rule. This is required if the writeToURI is called multiple times and you want one job to wait for the other job. I think it might work even without setting the rule for the job because org.eclipse.core.internal.resources.File.setContents() internally waits until no other thread is modifiying the file.

If you use the file API there might be chance that multiple threads might be writing to file simultaneously resulting in errors. So to make sure only one thread writes at a time its better to use eclipse API's.


Re: Asynchronuously write into workspace file [message #1060056 is a reply to message #1060041] Wed, 22 May 2013 13:57 Go to previous messageGo to next message
Hendrik Renken is currently offline Hendrik Renken
Messages: 18
Registered: July 2009
Junior Member
Setting the rule to res doesn't solve the problem that the whole Eclipse IDE becomes unresponsive when simulatinously editing another file (wheel of death on Mac OS) until i close the InputStream.

This is my main problem. I would even accept errors in the file content when this helps me to avoid a completely hanging eclipse UI.
Re: Asynchronuously write into workspace file [message #1060059 is a reply to message #1060056] Wed, 22 May 2013 14:22 Go to previous messageGo to next message
Devi Vara Prasad Bandaru is currently offline Devi Vara Prasad Bandaru
Messages: 98
Registered: March 2010
Location: Hyderabad
Member

Ok that's little strange.. ideally editing one file should not affect the other.

If you are using file API then may be you can create a single class level job and use the same for updating content. This way you could make sure no two threads(jobs) are writing to the same file.



Re: Asynchronuously write into workspace file [message #1060129 is a reply to message #1060059] Thu, 23 May 2013 05:01 Go to previous messageGo to next message
Hendrik Renken is currently offline Hendrik Renken
Messages: 18
Registered: July 2009
Junior Member
I will try that, maybe that helps me to understand the problem better. Thanks for the advice. I will report back.
Re: Asynchronuously write into workspace file [message #1060135 is a reply to message #1060129] Thu, 23 May 2013 05:24 Go to previous messageGo to next message
Hendrik Renken is currently offline Hendrik Renken
Messages: 18
Registered: July 2009
Junior Member
I have thought this problem over. I think, i know where the problem lies.

The problem is the WorkspaceJob. Lets say, the streaming takes ~10 minutes or so.
Now, according to my code, i create the WorkspaceJob, schedule it and then it blocks the whole workspace for said 10 minutes.

So instead, i will need to
* call the IFile#create, IFile#appendContents, IFile#setContents method from a temporary thread
* and then have a WorkspaceJob (or what ever Job) pumping incoming network data into the inputstream, connected to the file

My uninformed guess is, that this is somewhat akward and there really should be some sort of mechanism for non-blocking file transfer, but i couldn't find any information about that in the eclipse articles or on the web.
Re: Asynchronuously write into workspace file [message #1060241 is a reply to message #1060135] Thu, 23 May 2013 12:24 Go to previous message
Hendrik Renken is currently offline Hendrik Renken
Messages: 18
Registered: July 2009
Junior Member
For the records: My specific problem is, that the eclipse API does not offer any (good) mechanism to handle long taking file operations.
The suggested workaround from the IRC was:
* Write the incoming data stream into a file in the private area of your plugin.
* When finished write the content, using a WorkspaceJob (or something similar) into the final location in the workspace.

This will prevent problems regarding the locking of the workspace (and the eclipse UI). Another idea would be to write the data in chunks into the file when the data becomes available. However, this will clutter the local history of the file and create an unusual high amount of load due to delta changes.

Advanced idea:
* Create a new filesystem provider (EFS) and with that a virtual file that can be plugged into the workspace to allow the user (while the streaming is running in the background) to mess with the virtual file and to show the user that there is something going on. With the virtual file, the user e.g. can delete the resource and i can, based on this feedback, stop the background streaming.
Previous Topic:p2 repository produces outdated jar/plugin
Next Topic:missing icon decorators in custom navigator
Goto Forum:
  


Current Time: Wed Aug 27 07:12:46 EDT 2014

Powered by FUDForum. Page generated in 0.01944 seconds