The default open source implementation
in Orion will likely be a very simple direct file system storage. Including
and configuring a DB adds a lot of complexity, as well as build/package/IP
issues. Having said that my goal is that the API could easily be implemented
to use a different back-end, and time permitting I would like to play around
with different back ends to see how they scale and perform. I think simple
noSQL databases like Redis or MongoDB would be interesting candidates.
Even Solr might work. My gut feeling is that to really scale out to many
orion instances, would need one of these storage systems for their concurrency
and caching characteristics. But really my main goal right now is to make
that piece pluggable so changing back ends in the future doesn't require
another massive rewrite.
It is not currently possible to do this. The current metadata storage uses
OSGi preference API and doesn't offer this level of customization of storage
layout. As part of our server scalability work I am currently designing
and implementing a new metadata storage API. This will enable metadata
storage to be completely pluggable on the Orion server, such as allowing
database storage or arbitrary file system storage formats. For default
implementation of that new storage API I am looking at a file system layout
that is segmented by user, which looks close to your picture below. For
more details see this bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=406607
The guid-like generation then will be as a feature, which can be switched
on/off. This will give great flexibility for those who wants to integrate
with Orion at lowest level (file system) reliably. Also the remapping at
every request as well as the mapping management (bottleneck) itself can
be disabled, if it is not needed.
What do you think?