RE: [geclipse-dev] Replicable File Systems - proposal tothe corechanges

["Harald Kornmayer" <harald.kornmayer@xxxxxxxxxxxx>]

Hi Mateusz

First of all, maybe we concentrate a few more minutes on the use cases for
the LFC. 

The LFC EFS implementatin should be used by the user. The
LogicalFileCatalogue was introduced to hide the details from the data grid
from the user. LFC provides a "file system" which spans over many
distributed Storage resources. So the LFC file system is "virtual". The
replication mechanism is just one of the features of the LFC. The main
feature is to hide complexity and to enable a user to find his distributed
data much easier. 

But the LFC looks like a real file system. And at the end of each directory
tree is a "file" with a name. But LFC provides here only the link to the
original endpoint of the file (cardinality [1, inf.)). As a user I just want
to get the file open by clicking on it. It should not matter where the file
resides. From a performance view point, the closest file should be opened. 

So from my point of view, the file system in the project view resp. Grid
connection view looks like: 

- LFC
  - directory 1
  - directory 2
    - subdirectory 3
      - file1.txt
      - file2.txt 
    - subdirectory 4
    ...

The user should not see anything about the locations of the file's replicas
in these views.
But I propose to show some details in the propoerties view like:

filename	file1.txt
checksum    1234567
closest rep augustin.de
replicas
  - URL     gsiftp://poznan/xx/xx/xx/file1.txt
  - URL     s3://xx/xx/xx/file1.txt
  - URL     gsiftp://cyprus/zz//xx/xx/file1.txt

So the user can get more information from the properties view.


So now coming to actions: 

1. Drag'n'drop: 
The user must be able to drag a file from the LFC connection to any other
connection. Before the real transfer starts, 
our framework looks for the "closest SE" and uses that to make the transfer.


It work for the "import drag'n'drop" correspondingly. Look for the closest
SE and transfer the data there and put the information in the Cataloque. 

2. Actions: 
If the users wants to manage some aspects of LFC, he gots the corresponding
entries in the context menues: 
Here we need a much better understanding of the actions a Grid user wants to
perform.
We must provide the support for the most common use cases. And not for every
use case, that is possible. 

If the file is already in the LFC, actions might be: 
- replicate...
- delete all replicas
- delete replicas ...
- manage replicas ...

IF the file is in another file system type (srm, gsiftp,..) and there is a
LFC connection open, the following actions might
appear in the context menu: 
- register in LFC
- register in LFC ...
- unregister from LFC
- unregister from LFC ...
- delete from LFC (deleting all replicas)

You see that there are already different possibilities, but we need to find
the most useful once, and that ones we implement. 


So for me the LFC filesystem ends with the "virtual" leaves. If I understood
you right, your LFC ends with the "real replica" leaves.
I don't want to see them in in the filesystem/connection. That is complexity
we want to hide!!!


Maybe it is worth to define the useful processes and the GUI interactions,
before we start to implement the modell. 

 
Harald 


==========================================
Dr. Harald Kornmayer
Senior researcher

NEC Laboratories Europe 
IT Division
Rathausallee 10 
D-53757 Sankt Augustin
Germany

Phone: +49 2241 92 52 57 
Fax:   +49 2241 92 52 99
Email: harald.kornmayer@xxxxxxxxxxxx

NEC Europe Ltd., Registered Office: NEC House,
1 Victoria Road, London W3 6bL,
Registered in England 2832014
==========================================



>>>>> Subject: [geclipse-dev] Replicable File Systems - proposal to 
>>>>> the corechanges
>>> 
>>>>> I'm about to create GUI for the replicable file system (the 
>>>>> only one we 
>>>>> support right now is LFC), but it would be nice to let the 
>>>>> third-party 
>>>>> plugins also take advantage of the work already done.
>>>>>
>>>>> The abstraction would consist of:
>>>>> - IReplicableStore,
>>>>>   - public boolean isReplicable();
>>>>>   - public void addReplica( URI replicaURI ) throws CoreException;
>>>>>   - public void delReplica( URI replicaURI ) throws CoreException;
>>>>>   - public URI[] listReplicas() throws CoreException;
>>>>>   - public String[] listReplicasHosts() throws CoreException;
>>>>>
>>>>> - IReplicableFileSystem:
>>>>>   - boolean isReplicable(); // does file system support 
>>replication?
>>>>>   - URI getDefaultReplicaURI; // get default location 
>>where replicas 
>>>>> will be created
>>>>>
>>>>> - GUI components:
>>>>>   - replicas view
>>>>>   - preferences page
>>>>>   - context menu actions
>>> 
>>> Why do we need "yet another view" for this? 
>>> What is the rationale for that? 
>>> Can't we use the propoerty view here to present the 
>>propoerties of the
>>> replica entry (i.e. list the URI for all replicas, ...)? 
>>> And the actions are "bound" to context menus? 
>>
>>The problem is:
>>Where then we put properties for the replicas?
>>How do we maintain drag'n'drop func. for replicas?
>>How do we introduce replica context menus, like computeCheckSum?
>>
>>Showing replicas in context menu is cumbersome:
>>  - what if there is ~100 replicas for the given file?
>>  - what if user want to change sorting order, filters etc.?
>>
>>But you're right, we need the rationale for this.
>>My point is written above.
>>
>>It's still an open question, that's why I put it into the discussion.
>>
>>-- 
>>Mateusz Pabis
>>_______________________________________________
>>geclipse-dev mailing list
>>geclipse-dev@xxxxxxxxxxx
>>https://dev.eclipse.org/mailman/listinfo/geclipse-dev