|Re: [smila-dev] Smila - scaling capabilities|
welcome to SMILA!
The official documentation can be found in our wiki: http://wiki.eclipse.org/SMILA
There you’ll find anything about the most interesting things in SMILA:
Scaling Jobs/Asynchronous worfklows, scaling Import/Crawling, integration of your own components (Workers, Pipelets), etc.
Feel free to ask about more details that may not be mentioned there...
You can also try our “Installing and Running SMILA (5 Minutes to Success)” to get an entry point in SMILA.
On your question about zookeeper: Among other things, we use it for the cluster node coordination during processing of jobs in the cluster. It is also used as a kind of temporary storage for runtime data and state information that must be shared between the cluster nodes. So, if you have a look at the documentation, a lot of components concerning processing of jobs/asynch.workflows (JobManager, TaskManager, ...) are based on a zookeeper layer.
Hope this helps.