Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » scout » RemoteFileService (RemoteFileService with large directories)
RemoteFileService [message #766498] Thu, 15 December 2011 22:18 Go to next message
Bertin Kiekebosch is currently offline Bertin KiekeboschFriend
Messages: 309
Registered: August 2011
Senior Member
I am using RemoteFileService. In some directories we have over a 1000 mp3 files. This does not work (Out of memory) because contents of the file is loaded as well.

What I need is that I can only load the headers of the files and that I can specify the maximum number of headers returned.

Is there already some other way available in Scout to do this?


Re: RemoteFileService [message #767906 is a reply to message #766498] Mon, 19 December 2011 07:31 Go to previous messageGo to next message
Jeremie Bresson is currently offline Jeremie BressonFriend
Messages: 118
Registered: November 2010
Senior Member
I never used RemoteFiles but as I understood it, they can be resolved (with content) or just holders (without content). In the second case, they are just references on some content and they are resolve when they are consumed.

org.eclipse.scout.rt.server.services.common.file.RemoteFileService.getRemoteFiles(String, FilenameFilter, RemoteFile[], String, long)

the last long parameter is called maxBlockSize.

The default value is defined in RemoteFile:
  public static final long DEFAULT_MAX_BLOCK_SIZE = 20000000; // 20MB


Maybe you can try it with 0.

Have a look at the other getRemoteFiles(..) in the service in order to have an idea on the default values for the other parameters.
Re: RemoteFileService [message #767988 is a reply to message #767906] Mon, 19 December 2011 10:22 Go to previous messageGo to next message
Bertin Kiekebosch is currently offline Bertin KiekeboschFriend
Messages: 309
Registered: August 2011
Senior Member
Just tested with
org.eclipse.scout.rt.server.services.common.file.RemoteFileService.getRemoteFiles(String, FilenameFilter, RemoteFile[], String, long) 

and that works.

This method is not available in the interface so to use it I extended the interface like this:
public interface IExtRemoteFileService extends IRemoteFileService {

  RemoteFile[] getRemoteFiles(String folderPath, FilenameFilter filter, RemoteFile[] existingFileInfoOnClient, String charsetName, long maxBlockSize) throws ProcessingException;

  RemoteFile[] getRemoteFileHeaders(String folderPath, FilenameFilter filter, RemoteFile[] existingFileInfoOnClient, int maxCount) throws ProcessingException;

}


The first method in the interface makes the call available to the client. The second method I added to test another way to do the same. I noticed that in RemoteFileService internally there is a boolean "includeContent" but it is not available in any public method.

So I also extended RemoteFileServices and implemented the second method on it that makes use of this internal boolean.

There is a difference in performance if I run the following test code


System.out.println("get sounds with getRemoteFileHeaders");
Calendar cal = Calendar.getInstance();
Date startTime = cal.getTime();
files = SERVICES.getService(IExtRemoteFileService.class).getRemoteFileHeaders("gramma6/sounds", new NoFilenameFilter(), null, -1);
cal = Calendar.getInstance();
Date endTime = cal.getTime();
System.out.println("time = " + (endTime.getTime() - startTime.getTime()) + " milleseconds");

System.out.println("get sounds with getRemoteFiles , blocksize = 0");
cal = Calendar.getInstance();
startTime = cal.getTime();
files = SERVICES.getService(IExtRemoteFileService.class).getRemoteFiles("gramma6/sounds", new NoFilenameFilter(), null, "UTF-8", 0);
System.out.println("Total " + files.length + " files");
cal = Calendar.getInstance();
endTime = cal.getTime();
System.out.println("time = " + (endTime.getTime() - startTime.getTime()) + " milleseconds");


Output:
get sounds with getRemoteFileHeaders
Total 4057 files
time = 582 milleseconds
get sounds with getRemoteFiles , blocksize = 0
Total 4057 files
time = 2162 milleseconds




Re: RemoteFileService [message #768051 is a reply to message #767988] Mon, 19 December 2011 13:44 Go to previous message
Jeremie Bresson is currently offline Jeremie BressonFriend
Messages: 118
Registered: November 2010
Senior Member
The time difference probably comes from this block: (RemoteFileService:114 - RemoteFileService:118)
      if (maxBlockSize > -1 && partLength > maxBlockSize) {
        partLength = partLength - startPosition;
        if (partLength > maxBlockSize) partLength = maxBlockSize;
        if (partLength <= 0) partLength = 0;
      }
From the internal function: org.eclipse.scout.rt.server.services.common.file.RemoteFileService.getRemoteFileInternal(RemoteFile, Boolean, long, long)

In the second case, the block is skipped, because maxBlockSize = -1.


PS: I do not know why all methods are not available on the interface. This could probably be changed...

PS2: You could copy or extend this service to a ExtRemoteFileService. If you do so, do not forget to register your service in plugin.xml (client and server).

[Updated on: Mon, 19 December 2011 13:48]

Report message to a moderator

Previous Topic:External Component
Next Topic:AbstractCalendarField
Goto Forum:
  


Current Time: Thu Nov 27 05:51:28 GMT 2014

Powered by FUDForum. Page generated in 0.01915 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software