Home » Modeling » EMF » [CDO] Server Choices and Deployment Options
[CDO] Server Choices and Deployment Options [message #1696205] |
Sat, 23 May 2015 00:40 |
Eclipse User |
|
|
|
We're done with our initial CDO client work and now are turning
attention to the CDO Server.
Performance - Store and DB Choices:
Its cool that there are many choices for the server store and adapter,
but are there any performance comparisons or benchmarks to help making
the choice easier?
We will be supporting large models on the server with connected and
disconnected clients, but will not need auditing or branching. Any
recommendations as to the store - db vs hibernate and the adapter?
We have experience with MongoDB but if that has scaling issues with
large models or cannot work with disconnected clients than a SQL would
be acceptable. We are doing a lot of GeoLocation work so perhaps
Postgresql? Any suggestions/recommendations?
Deployment - OSGi & Docker:
Had planned to use standard OSGi frameworks to run the server
application, but have seen mail threads in the regarding major blocks to
that approach. That's unfortunate. Is this considered an issue of
importance - ie is it worth opening a bug on this?
Nevertheless, it looks like the standard approach is to create an
eclipse product, export it to a binary and just execute it. This does
not fit so well with our OSGi strategy for deployment but since we are
dockerizing our database we may just as well wrap the server in a docker
container too. Has anyone done any work with docker 'izing' a CDO
server?
thanks for any insights,
John
|
|
|
Re: [CDO] Server Choices and Deployment Options [message #1696211 is a reply to message #1696205] |
Sat, 23 May 2015 05:35 |
|
Hi John,
Comments below...
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Am 23.05.2015 um 02:40 schrieb John E. Conlon:
> We're done with our initial CDO client work and now are turning attention to the CDO Server.
>
> Performance - Store and DB Choices:
> Its cool that there are many choices for the server store and adapter, but are there any performance comparisons or
> benchmarks to help making the choice easier?
The DBStore is certainly the most used and the best tested store. I would start with the DBStore and come back in case
of specific problems.
We have no benchmarks, but there are some feature matrices:
http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/StoreFeatures.html
http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/MappingStrategies.html
>
> We will be supporting large models on the server with connected and disconnected clients, but will not need auditing
> or branching. Any recommendations as to the store - db vs hibernate and the adapter?
See above.
>
> We have experience with MongoDB but if that has scaling issues with large models or cannot work with disconnected
> clients than a SQL would be acceptable. We are doing a lot of GeoLocation work so perhaps Postgresql? Any
> suggestions/recommendations?
My customer who funded the development of the MongoDBStore stopped using it and it hasn't been evolved/maintained since.
It's our only "delta store", i.e., it has reversed write/read characteristics. Writes are atomic and fast, reads not.
>
> Deployment - OSGi & Docker:
> Had planned to use standard OSGi frameworks to run the server application, but have seen mail threads in the regarding
> major blocks to that approach. That's unfortunate. Is this considered an issue of importance - ie is it worth opening
> a bug on this?
I have no idea what you're referring to or asking for ;-)
>
> Nevertheless, it looks like the standard approach is to create an eclipse product, export it to a binary and just
> execute it.
That is one way that should work. From a CDO point of view a distinct CDO server product isn't strictly needed. The
CDOServerApplication is a convenient entry point but if you have a look at it you see that it'd be trivial to do the
same thing somewhere else.
> This does not fit so well with our OSGi strategy for deployment
Why not?
> but since we are dockerizing our database we may just as well wrap the server in a docker container too. Has anyone
> done any work with docker 'izing' a CDO server?
I know almost nothing about Docker.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Re: [CDO] Server Choices and Deployment Options [message #1696228 is a reply to message #1696211] |
Sat, 23 May 2015 22:01 |
Eclipse User |
|
|
|
Hi Eike,
Comments inline...
best regards
John
On 05/23/2015 12:35 AM, Eike Stepper wrote:
> Hi John,
>
> Comments below...
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
>
> Am 23.05.2015 um 02:40 schrieb John E. Conlon:
>> We're done with our initial CDO client work and now are turning
>> attention to the CDO Server.
>>
>> Performance - Store and DB Choices:
>> Its cool that there are many choices for the server store and adapter,
>> but are there any performance comparisons or benchmarks to help making
>> the choice easier?
> The DBStore is certainly the most used and the best tested store. I
> would start with the DBStore and come back in case of specific problems.
DBStore was my first thought as well. And as for the DBStore databases?
I like H2 as it is Java and OSGi, but am also considering PostgreSQL.
Any thoughts + or - on these?
>
> We have no benchmarks, but there are some feature matrices:
These are nice graphics - I have looked through the docs but never
noticed these. I see they can be found in the help docs, but the links
should be more visible in the web site docs too.
>
> http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/StoreFeatures.html
DBStore ++
>
> http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/MappingStrategies.html
>
>
>>
>> We will be supporting large models on the server with connected and
>> disconnected clients, but will not need auditing or branching. Any
>> recommendations as to the store - db vs hibernate and the adapter?
> See above.
>
>>
>> We have experience with MongoDB but if that has scaling issues with
>> large models or cannot work with disconnected clients than a SQL would
>> be acceptable. We are doing a lot of GeoLocation work so perhaps
>> Postgresql? Any suggestions/recommendations?
> My customer who funded the development of the MongoDBStore stopped using
> it and it hasn't been evolved/maintained since. It's our only "delta
> store", i.e., it has reversed write/read characteristics. Writes are
> atomic and fast, reads not.
Nuff said. Mongo is out.
>
>>
>> Deployment - OSGi & Docker:
>> Had planned to use standard OSGi frameworks to run the server
>> application, but have seen mail threads in the regarding major blocks
>> to that approach. That's unfortunate. Is this considered an issue of
>> importance - ie is it worth opening a bug on this?
> I have no idea what you're referring to or asking for ;-)
My clients use Eclipse but my server infrastructure is OSGi and
dynamically deployed via Apache ACE (https://ace.apache.org/).
Would be sweet to have both CDOServer and its database like H2
deployable as OSGi services and configurable with OSGi ConfigAdmin.
I think this is similar to what Cristiano avi, described in the 26 Oct
2011 thread: '[CDO] I need help to start server from a OSGi Service'.
I got the impression from Anders Jonsson 16 Feb 2015 Newsgroup Thread
'[CDO] Server in a non-Equinox OSGi container'.
That one can NOT run the server in a non-Eclipse OSGi framework.
His last entry is not so clear as to whether you can or cannot. WDYT?
>
>>
>> Nevertheless, it looks like the standard approach is to create an
>> eclipse product, export it to a binary and just execute it.
> That is one way that should work. From a CDO point of view a distinct
> CDO server product isn't strictly needed. The CDOServerApplication is a
> convenient entry point but if you have a look at it you see that it'd be
> trivial to do the same thing somewhere else.
Yes, I see this and it is encouraging me to dig into the OSGi potentials
myself.
>
>> This does not fit so well with our OSGi strategy for deployment
> Why not?
ACE deployment is very efficient way to deploy things. Like I said, I
have to dig into to see if any of the problems Anders encountered can be
overcome.
>> but since we are dockerizing our database we may just as well wrap the
>> server in a docker container too. Has anyone done any work with
>> docker 'izing' a CDO server?
> I know almost nothing about Docker.
Docker is another containerization technology that is gaining a lot of
traction. A docker file basically specifies how to create a runtime
image. Creating a docker file describing a CDOServer would allow a
rapid automated deployment (< 5 min for server and db) and portability
across machines and cloud providers. Docker is new, but there is a
tremendous momentum already. There are docker files for most of the
databases supported by CDO: H2, PostgreSQL, MySQL.
|
|
| |
Re: [CDO] Server Choices and Deployment Options [message #1696277 is a reply to message #1696228] |
Mon, 25 May 2015 08:15 |
|
Am 24.05.2015 um 00:01 schrieb John E. Conlon:
> Hi Eike,
> Comments inline...
>
> best regards
> John
>
> On 05/23/2015 12:35 AM, Eike Stepper wrote:
>> Hi John,
>>
>> Comments below...
>>
>> Cheers
>> /Eike
>>
>> ----
>> http://www.esc-net.de
>> http://thegordian.blogspot.com
>> http://twitter.com/eikestepper
>>
>>
>>
>> Am 23.05.2015 um 02:40 schrieb John E. Conlon:
>>> We're done with our initial CDO client work and now are turning
>>> attention to the CDO Server.
>>>
>>> Performance - Store and DB Choices:
>>> Its cool that there are many choices for the server store and adapter,
>>> but are there any performance comparisons or benchmarks to help making
>>> the choice easier?
>> The DBStore is certainly the most used and the best tested store. I
>> would start with the DBStore and come back in case of specific problems.
>
> DBStore was my first thought as well. And as for the DBStore databases? I like H2 as it is Java and OSGi, but am also
> considering PostgreSQL. Any thoughts + or - on these?
H2 is the only DB we use in our CI tests. It's small, fast (at least up to the capacity we use in our tests) and easy to
configure. I have customers that use it in production. I've never used PostgreSQL myself.
>
>
>>
>> We have no benchmarks, but there are some feature matrices:
> These are nice graphics - I have looked through the docs but never noticed these. I see they can be found in the help
> docs, but the links should be more visible in the web site docs too.
The wiki is publicly maintained. You can add such links wherever you find them appropriate ;-)
>
>>
>>>
>>> Deployment - OSGi & Docker:
>>> Had planned to use standard OSGi frameworks to run the server
>>> application, but have seen mail threads in the regarding major blocks
>>> to that approach. That's unfortunate. Is this considered an issue of
>>> importance - ie is it worth opening a bug on this?
>> I have no idea what you're referring to or asking for ;-)
> My clients use Eclipse but my server infrastructure is OSGi and dynamically deployed via Apache ACE
> (https://ace.apache.org/).
>
> Would be sweet to have both CDOServer and its database like H2 deployable as OSGi services and configurable with OSGi
> ConfigAdmin.
>
> I think this is similar to what Cristiano avi, described in the 26 Oct 2011 thread: '[CDO] I need help to start server
> from a OSGi Service'.
There are so many runtime platforms and deployment technologies out there and each year some new come and some old go
away. It's impossible for us to stay up to date and support them all. This is an area where contributions would make sense.
>
> I got the impression from Anders Jonsson 16 Feb 2015 Newsgroup Thread
> '[CDO] Server in a non-Equinox OSGi container'.
> That one can NOT run the server in a non-Eclipse OSGi framework.
> His last entry is not so clear as to whether you can or cannot. WDYT?
I suspect this is related to bundles requiring org.eclipse.core.runtime or other Equinox-specific things. It would be
nice to get rid of those requirements but it would likely break API compatibility and is a no no for us.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Re: [CDO] Server Choices and Deployment Options [message #1696279 is a reply to message #1696260] |
Mon, 25 May 2015 08:34 |
|
Am 24.05.2015 um 18:47 schrieb Erdal Karaca:
> I am not sure if the offline clone uses the same backend system as the remote system. AFAIK, CDO always uses H2 for
> the offline mode. Or am I wrong: is the clone configurable, Eike?
CDO supports two totally different offline scenarios:
1) Offline clones replicate the entire remote history and can serve one or more normal CDO clients. They require
branching in all participating servers and clients. There are two replication modes available:
1a) Raw replication is fast and streams data directly from a master DB into a clone DB. The master and clone
IStores need to be the same. With DBStores the DBAdapter types can be different, though.
1b) Logical replication replays master commits in local clones on a logical CDO level, which has often proven to be
too slow (especially after longer offline periods of a clone). Master and clones can use different IStore types.
2) A CDOWorkspace is more like a checkout from a single remote branch point, i.e., the entire remote history is not
copied to local. It supports operations such as checkout, update, merge and checkin. The checked out CDOWorkspace acts a
little bit like a CDOSession for your application; it provides CDOViews and CDOTransactions in addition to the
aforementioned operations. You can see them in action with the new CDO UI (look for offline checkouts). A CDOWorkspace
locally uses a DBStore with H2 (in theory that could be made configurable, too, but I thought the effort doesn't justify
the benefits) and has no restrictions on the remote store.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Re: [CDO] Server Choices and Deployment Options [message #1696330 is a reply to message #1696260] |
Mon, 25 May 2015 18:06 |
Eclipse User |
|
|
|
On 05/24/2015 11:47 AM, Erdal Karaca wrote:
> If you intend to use spatial queries, then you would need support for
> spatial indexing and the like.
Yes, which at first would suggest PostgreSQL + PostGIS but have recently
found out about:
H2GIS
http://www.h2gis.org/docs/dev/home/
> I am not sure if the offline clone uses the same backend system as the
> remote system. AFAIK, CDO always uses H2 for the offline mode. Or am I
> wrong: is the clone configurable, Eike?
>
> Eclipse User wrote on Sat, 23 May 2015 02:40
>> We have experience with MongoDB but if that has scaling issues with
>> large models or cannot work with disconnected clients than a SQL would
>> be acceptable. We are doing a lot of GeoLocation work so perhaps
>> Postgresql? Any suggestions/recommendations?
>>
>>
>> John
>
>
|
|
|
Re: [CDO] Server Choices and Deployment Options [message #1696463 is a reply to message #1696277] |
Tue, 26 May 2015 14:55 |
Eclipse User |
|
|
|
On 05/25/2015 03:15 AM, Eike Stepper wrote:
> Am 24.05.2015 um 00:01 schrieb John E. Conlon:
>> Hi Eike,
>> Comments inline...
>>
>> best regards
>> John
>>
>> On 05/23/2015 12:35 AM, Eike Stepper wrote:
>>> Hi John,
>>>
>>> Comments below...
>>>
>>> Cheers
>>> /Eike
>>>
>>> ----
>>> http://www.esc-net.de
>>> http://thegordian.blogspot.com
>>> http://twitter.com/eikestepper
>>>
>>>
>>>
>>> Am 23.05.2015 um 02:40 schrieb John E. Conlon:
>>>> We're done with our initial CDO client work and now are turning
>>>> attention to the CDO Server.
>>>>
>>>> Performance - Store and DB Choices:
>>>> Its cool that there are many choices for the server store and adapter,
>>>> but are there any performance comparisons or benchmarks to help making
>>>> the choice easier?
>>> The DBStore is certainly the most used and the best tested store. I
>>> would start with the DBStore and come back in case of specific problems.
>>
>> DBStore was my first thought as well. And as for the DBStore
>> databases? I like H2 as it is Java and OSGi, but am also considering
>> PostgreSQL. Any thoughts + or - on these?
> H2 is the only DB we use in our CI tests. It's small, fast (at least up
> to the capacity we use in our tests) and easy to configure. I have
> customers that use it in production. I've never used PostgreSQL myself.
>
>>
>>
>>>
>>> We have no benchmarks, but there are some feature matrices:
>> These are nice graphics - I have looked through the docs but never
>> noticed these. I see they can be found in the help docs, but the
>> links should be more visible in the web site docs too.
> The wiki is publicly maintained. You can add such links wherever you
> find them appropriate ;-)
Did so:
https://wiki.eclipse.org/CDO#DB_Store_Features
https://wiki.eclipse.org/CDO/DB_Store
>>
>>>
>>>>
>>>> Deployment - OSGi & Docker:
>>>> Had planned to use standard OSGi frameworks to run the server
>>>> application, but have seen mail threads in the regarding major blocks
>>>> to that approach. That's unfortunate. Is this considered an issue of
>>>> importance - ie is it worth opening a bug on this?
>>> I have no idea what you're referring to or asking for ;-)
>> My clients use Eclipse but my server infrastructure is OSGi and
>> dynamically deployed via Apache ACE (https://ace.apache.org/).
>>
>> Would be sweet to have both CDOServer and its database like H2
>> deployable as OSGi services and configurable with OSGi ConfigAdmin.
>>
>> I think this is similar to what Cristiano avi, described in the 26 Oct
>> 2011 thread: '[CDO] I need help to start server from a OSGi Service'.
> There are so many runtime platforms and deployment technologies out
> there and each year some new come and some old go away. It's impossible
> for us to stay up to date and support them all. This is an area where
> contributions would make sense.
Will look into this and give some feedback.
>
>>
>> I got the impression from Anders Jonsson 16 Feb 2015 Newsgroup Thread
>> '[CDO] Server in a non-Equinox OSGi container'.
>> That one can NOT run the server in a non-Eclipse OSGi framework.
>> His last entry is not so clear as to whether you can or cannot. WDYT?
> I suspect this is related to bundles requiring org.eclipse.core.runtime
> or other Equinox-specific things. It would be nice to get rid of those
> requirements but it would likely break API compatibility and is a no no
> for us.
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
>
|
|
| | | | |
Goto Forum:
Current Time: Thu Sep 26 17:17:18 GMT 2024
Powered by FUDForum. Page generated in 0.05851 seconds
|