Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] Server Choices and Deployment Options
[CDO] Server Choices and Deployment Options [message #1696205] Sat, 23 May 2015 00:40 Go to next message
Eclipse UserFriend
We're done with our initial CDO client work and now are turning
attention to the CDO Server.

Performance - Store and DB Choices:
Its cool that there are many choices for the server store and adapter,
but are there any performance comparisons or benchmarks to help making
the choice easier?

We will be supporting large models on the server with connected and
disconnected clients, but will not need auditing or branching. Any
recommendations as to the store - db vs hibernate and the adapter?

We have experience with MongoDB but if that has scaling issues with
large models or cannot work with disconnected clients than a SQL would
be acceptable. We are doing a lot of GeoLocation work so perhaps
Postgresql? Any suggestions/recommendations?



Deployment - OSGi & Docker:
Had planned to use standard OSGi frameworks to run the server
application, but have seen mail threads in the regarding major blocks to
that approach. That's unfortunate. Is this considered an issue of
importance - ie is it worth opening a bug on this?

Nevertheless, it looks like the standard approach is to create an
eclipse product, export it to a binary and just execute it. This does
not fit so well with our OSGi strategy for deployment but since we are
dockerizing our database we may just as well wrap the server in a docker
container too. Has anyone done any work with docker 'izing' a CDO
server?


thanks for any insights,

John
Re: [CDO] Server Choices and Deployment Options [message #1696211 is a reply to message #1696205] Sat, 23 May 2015 05:35 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6382
Registered: July 2009
Senior Member
Hi John,

Comments below...

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper



Am 23.05.2015 um 02:40 schrieb John E. Conlon:
> We're done with our initial CDO client work and now are turning attention to the CDO Server.
>
> Performance - Store and DB Choices:
> Its cool that there are many choices for the server store and adapter, but are there any performance comparisons or
> benchmarks to help making the choice easier?
The DBStore is certainly the most used and the best tested store. I would start with the DBStore and come back in case
of specific problems.

We have no benchmarks, but there are some feature matrices:

http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/StoreFeatures.html
http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/MappingStrategies.html

>
> We will be supporting large models on the server with connected and disconnected clients, but will not need auditing
> or branching. Any recommendations as to the store - db vs hibernate and the adapter?
See above.

>
> We have experience with MongoDB but if that has scaling issues with large models or cannot work with disconnected
> clients than a SQL would be acceptable. We are doing a lot of GeoLocation work so perhaps Postgresql? Any
> suggestions/recommendations?
My customer who funded the development of the MongoDBStore stopped using it and it hasn't been evolved/maintained since.
It's our only "delta store", i.e., it has reversed write/read characteristics. Writes are atomic and fast, reads not.

>
> Deployment - OSGi & Docker:
> Had planned to use standard OSGi frameworks to run the server application, but have seen mail threads in the regarding
> major blocks to that approach. That's unfortunate. Is this considered an issue of importance - ie is it worth opening
> a bug on this?
I have no idea what you're referring to or asking for ;-)

>
> Nevertheless, it looks like the standard approach is to create an eclipse product, export it to a binary and just
> execute it.
That is one way that should work. From a CDO point of view a distinct CDO server product isn't strictly needed. The
CDOServerApplication is a convenient entry point but if you have a look at it you see that it'd be trivial to do the
same thing somewhere else.

> This does not fit so well with our OSGi strategy for deployment
Why not?

> but since we are dockerizing our database we may just as well wrap the server in a docker container too. Has anyone
> done any work with docker 'izing' a CDO server?
I know almost nothing about Docker.
Re: [CDO] Server Choices and Deployment Options [message #1696228 is a reply to message #1696211] Sat, 23 May 2015 22:01 Go to previous messageGo to next message
Eclipse UserFriend
Hi Eike,
Comments inline...

best regards
John

On 05/23/2015 12:35 AM, Eike Stepper wrote:
> Hi John,
>
> Comments below...
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
>
> Am 23.05.2015 um 02:40 schrieb John E. Conlon:
>> We're done with our initial CDO client work and now are turning
>> attention to the CDO Server.
>>
>> Performance - Store and DB Choices:
>> Its cool that there are many choices for the server store and adapter,
>> but are there any performance comparisons or benchmarks to help making
>> the choice easier?
> The DBStore is certainly the most used and the best tested store. I
> would start with the DBStore and come back in case of specific problems.

DBStore was my first thought as well. And as for the DBStore databases?
I like H2 as it is Java and OSGi, but am also considering PostgreSQL.
Any thoughts + or - on these?


>
> We have no benchmarks, but there are some feature matrices:
These are nice graphics - I have looked through the docs but never
noticed these. I see they can be found in the help docs, but the links
should be more visible in the web site docs too.

>
> http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/StoreFeatures.html
DBStore ++

>
> http://download.eclipse.org/modeling/emf/cdo/drops/R20140610-0212/help/org.eclipse.emf.cdo.doc/html/reference/MappingStrategies.html


>
>
>>
>> We will be supporting large models on the server with connected and
>> disconnected clients, but will not need auditing or branching. Any
>> recommendations as to the store - db vs hibernate and the adapter?
> See above.
>
>>
>> We have experience with MongoDB but if that has scaling issues with
>> large models or cannot work with disconnected clients than a SQL would
>> be acceptable. We are doing a lot of GeoLocation work so perhaps
>> Postgresql? Any suggestions/recommendations?
> My customer who funded the development of the MongoDBStore stopped using
> it and it hasn't been evolved/maintained since. It's our only "delta
> store", i.e., it has reversed write/read characteristics. Writes are
> atomic and fast, reads not.
Nuff said. Mongo is out.

>
>>
>> Deployment - OSGi & Docker:
>> Had planned to use standard OSGi frameworks to run the server
>> application, but have seen mail threads in the regarding major blocks
>> to that approach. That's unfortunate. Is this considered an issue of
>> importance - ie is it worth opening a bug on this?
> I have no idea what you're referring to or asking for ;-)
My clients use Eclipse but my server infrastructure is OSGi and
dynamically deployed via Apache ACE (https://ace.apache.org/).

Would be sweet to have both CDOServer and its database like H2
deployable as OSGi services and configurable with OSGi ConfigAdmin.

I think this is similar to what Cristiano avi, described in the 26 Oct
2011 thread: '[CDO] I need help to start server from a OSGi Service'.

I got the impression from Anders Jonsson 16 Feb 2015 Newsgroup Thread
'[CDO] Server in a non-Equinox OSGi container'.
That one can NOT run the server in a non-Eclipse OSGi framework.
His last entry is not so clear as to whether you can or cannot. WDYT?



>
>>
>> Nevertheless, it looks like the standard approach is to create an
>> eclipse product, export it to a binary and just execute it.
> That is one way that should work. From a CDO point of view a distinct
> CDO server product isn't strictly needed. The CDOServerApplication is a
> convenient entry point but if you have a look at it you see that it'd be
> trivial to do the same thing somewhere else.
Yes, I see this and it is encouraging me to dig into the OSGi potentials
myself.

>
>> This does not fit so well with our OSGi strategy for deployment
> Why not?

ACE deployment is very efficient way to deploy things. Like I said, I
have to dig into to see if any of the problems Anders encountered can be
overcome.

>> but since we are dockerizing our database we may just as well wrap the
>> server in a docker container too. Has anyone done any work with
>> docker 'izing' a CDO server?
> I know almost nothing about Docker.
Docker is another containerization technology that is gaining a lot of
traction. A docker file basically specifies how to create a runtime
image. Creating a docker file describing a CDOServer would allow a
rapid automated deployment (< 5 min for server and db) and portability
across machines and cloud providers. Docker is new, but there is a
tremendous momentum already. There are docker files for most of the
databases supported by CDO: H2, PostgreSQL, MySQL.
Re: [CDO] Server Choices and Deployment Options [message #1696260 is a reply to message #1696205] Sun, 24 May 2015 16:47 Go to previous messageGo to next message
Erdal Karaca is currently offline Erdal KaracaFriend
Messages: 840
Registered: July 2009
Senior Member
If you intend to use spatial queries, then you would need support for spatial indexing and the like.
I am not sure if the offline clone uses the same backend system as the remote system. AFAIK, CDO always uses H2 for the offline mode. Or am I wrong: is the clone configurable, Eike?

Eclipse User wrote on Sat, 23 May 2015 02:40

We have experience with MongoDB but if that has scaling issues with
large models or cannot work with disconnected clients than a SQL would
be acceptable. We are doing a lot of GeoLocation work so perhaps
Postgresql? Any suggestions/recommendations?


John

Re: [CDO] Server Choices and Deployment Options [message #1696277 is a reply to message #1696228] Mon, 25 May 2015 08:15 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6382
Registered: July 2009
Senior Member
Am 24.05.2015 um 00:01 schrieb John E. Conlon:
> Hi Eike,
> Comments inline...
>
> best regards
> John
>
> On 05/23/2015 12:35 AM, Eike Stepper wrote:
>> Hi John,
>>
>> Comments below...
>>
>> Cheers
>> /Eike
>>
>> ----
>> http://www.esc-net.de
>> http://thegordian.blogspot.com
>> http://twitter.com/eikestepper
>>
>>
>>
>> Am 23.05.2015 um 02:40 schrieb John E. Conlon:
>>> We're done with our initial CDO client work and now are turning
>>> attention to the CDO Server.
>>>
>>> Performance - Store and DB Choices:
>>> Its cool that there are many choices for the server store and adapter,
>>> but are there any performance comparisons or benchmarks to help making
>>> the choice easier?
>> The DBStore is certainly the most used and the best tested store. I
>> would start with the DBStore and come back in case of specific problems.
>
> DBStore was my first thought as well. And as for the DBStore databases? I like H2 as it is Java and OSGi, but am also
> considering PostgreSQL. Any thoughts + or - on these?
H2 is the only DB we use in our CI tests. It's small, fast (at least up to the capacity we use in our tests) and easy to
configure. I have customers that use it in production. I've never used PostgreSQL myself.

>
>
>>
>> We have no benchmarks, but there are some feature matrices:
> These are nice graphics - I have looked through the docs but never noticed these. I see they can be found in the help
> docs, but the links should be more visible in the web site docs too.
The wiki is publicly maintained. You can add such links wherever you find them appropriate ;-)

>
>>
>>>
>>> Deployment - OSGi & Docker:
>>> Had planned to use standard OSGi frameworks to run the server
>>> application, but have seen mail threads in the regarding major blocks
>>> to that approach. That's unfortunate. Is this considered an issue of
>>> importance - ie is it worth opening a bug on this?
>> I have no idea what you're referring to or asking for ;-)
> My clients use Eclipse but my server infrastructure is OSGi and dynamically deployed via Apache ACE
> (https://ace.apache.org/).
>
> Would be sweet to have both CDOServer and its database like H2 deployable as OSGi services and configurable with OSGi
> ConfigAdmin.
>
> I think this is similar to what Cristiano avi, described in the 26 Oct 2011 thread: '[CDO] I need help to start server
> from a OSGi Service'.
There are so many runtime platforms and deployment technologies out there and each year some new come and some old go
away. It's impossible for us to stay up to date and support them all. This is an area where contributions would make sense.

>
> I got the impression from Anders Jonsson 16 Feb 2015 Newsgroup Thread
> '[CDO] Server in a non-Equinox OSGi container'.
> That one can NOT run the server in a non-Eclipse OSGi framework.
> His last entry is not so clear as to whether you can or cannot. WDYT?
I suspect this is related to bundles requiring org.eclipse.core.runtime or other Equinox-specific things. It would be
nice to get rid of those requirements but it would likely break API compatibility and is a no no for us.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] Server Choices and Deployment Options [message #1696279 is a reply to message #1696260] Mon, 25 May 2015 08:34 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6382
Registered: July 2009
Senior Member
Am 24.05.2015 um 18:47 schrieb Erdal Karaca:
> I am not sure if the offline clone uses the same backend system as the remote system. AFAIK, CDO always uses H2 for
> the offline mode. Or am I wrong: is the clone configurable, Eike?
CDO supports two totally different offline scenarios:

1) Offline clones replicate the entire remote history and can serve one or more normal CDO clients. They require
branching in all participating servers and clients. There are two replication modes available:
1a) Raw replication is fast and streams data directly from a master DB into a clone DB. The master and clone
IStores need to be the same. With DBStores the DBAdapter types can be different, though.
1b) Logical replication replays master commits in local clones on a logical CDO level, which has often proven to be
too slow (especially after longer offline periods of a clone). Master and clones can use different IStore types.

2) A CDOWorkspace is more like a checkout from a single remote branch point, i.e., the entire remote history is not
copied to local. It supports operations such as checkout, update, merge and checkin. The checked out CDOWorkspace acts a
little bit like a CDOSession for your application; it provides CDOViews and CDOTransactions in addition to the
aforementioned operations. You can see them in action with the new CDO UI (look for offline checkouts). A CDOWorkspace
locally uses a DBStore with H2 (in theory that could be made configurable, too, but I thought the effort doesn't justify
the benefits) and has no restrictions on the remote store.

Cheers
/Eike

----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Re: [CDO] Server Choices and Deployment Options [message #1696330 is a reply to message #1696260] Mon, 25 May 2015 18:06 Go to previous messageGo to next message
Eclipse UserFriend
On 05/24/2015 11:47 AM, Erdal Karaca wrote:
> If you intend to use spatial queries, then you would need support for
> spatial indexing and the like.
Yes, which at first would suggest PostgreSQL + PostGIS but have recently
found out about:
H2GIS
http://www.h2gis.org/docs/dev/home/




> I am not sure if the offline clone uses the same backend system as the
> remote system. AFAIK, CDO always uses H2 for the offline mode. Or am I
> wrong: is the clone configurable, Eike?
>
> Eclipse User wrote on Sat, 23 May 2015 02:40
>> We have experience with MongoDB but if that has scaling issues with
>> large models or cannot work with disconnected clients than a SQL would
>> be acceptable. We are doing a lot of GeoLocation work so perhaps
>> Postgresql? Any suggestions/recommendations?
>>
>>
>> John
>
>
Re: [CDO] Server Choices and Deployment Options [message #1696463 is a reply to message #1696277] Tue, 26 May 2015 14:55 Go to previous messageGo to next message
Eclipse UserFriend
On 05/25/2015 03:15 AM, Eike Stepper wrote:
> Am 24.05.2015 um 00:01 schrieb John E. Conlon:
>> Hi Eike,
>> Comments inline...
>>
>> best regards
>> John
>>
>> On 05/23/2015 12:35 AM, Eike Stepper wrote:
>>> Hi John,
>>>
>>> Comments below...
>>>
>>> Cheers
>>> /Eike
>>>
>>> ----
>>> http://www.esc-net.de
>>> http://thegordian.blogspot.com
>>> http://twitter.com/eikestepper
>>>
>>>
>>>
>>> Am 23.05.2015 um 02:40 schrieb John E. Conlon:
>>>> We're done with our initial CDO client work and now are turning
>>>> attention to the CDO Server.
>>>>
>>>> Performance - Store and DB Choices:
>>>> Its cool that there are many choices for the server store and adapter,
>>>> but are there any performance comparisons or benchmarks to help making
>>>> the choice easier?
>>> The DBStore is certainly the most used and the best tested store. I
>>> would start with the DBStore and come back in case of specific problems.
>>
>> DBStore was my first thought as well. And as for the DBStore
>> databases? I like H2 as it is Java and OSGi, but am also considering
>> PostgreSQL. Any thoughts + or - on these?
> H2 is the only DB we use in our CI tests. It's small, fast (at least up
> to the capacity we use in our tests) and easy to configure. I have
> customers that use it in production. I've never used PostgreSQL myself.
>
>>
>>
>>>
>>> We have no benchmarks, but there are some feature matrices:
>> These are nice graphics - I have looked through the docs but never
>> noticed these. I see they can be found in the help docs, but the
>> links should be more visible in the web site docs too.
> The wiki is publicly maintained. You can add such links wherever you
> find them appropriate ;-)

Did so:
https://wiki.eclipse.org/CDO#DB_Store_Features
https://wiki.eclipse.org/CDO/DB_Store
>>
>>>
>>>>
>>>> Deployment - OSGi & Docker:
>>>> Had planned to use standard OSGi frameworks to run the server
>>>> application, but have seen mail threads in the regarding major blocks
>>>> to that approach. That's unfortunate. Is this considered an issue of
>>>> importance - ie is it worth opening a bug on this?
>>> I have no idea what you're referring to or asking for ;-)
>> My clients use Eclipse but my server infrastructure is OSGi and
>> dynamically deployed via Apache ACE (https://ace.apache.org/).
>>
>> Would be sweet to have both CDOServer and its database like H2
>> deployable as OSGi services and configurable with OSGi ConfigAdmin.
>>
>> I think this is similar to what Cristiano avi, described in the 26 Oct
>> 2011 thread: '[CDO] I need help to start server from a OSGi Service'.
> There are so many runtime platforms and deployment technologies out
> there and each year some new come and some old go away. It's impossible
> for us to stay up to date and support them all. This is an area where
> contributions would make sense.
Will look into this and give some feedback.
>
>>
>> I got the impression from Anders Jonsson 16 Feb 2015 Newsgroup Thread
>> '[CDO] Server in a non-Equinox OSGi container'.
>> That one can NOT run the server in a non-Eclipse OSGi framework.
>> His last entry is not so clear as to whether you can or cannot. WDYT?
> I suspect this is related to bundles requiring org.eclipse.core.runtime
> or other Equinox-specific things. It would be nice to get rid of those
> requirements but it would likely break API compatibility and is a no no
> for us.
>
> Cheers
> /Eike
>
> ----
> http://www.esc-net.de
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
>
Re: [CDO] Server Choices and Deployment Options [message #1764737 is a reply to message #1696205] Thu, 01 June 2017 23:54 Go to previous messageGo to next message
Missing name Mising name is currently offline Missing name Mising nameFriend
Messages: 9
Registered: April 2011
Junior Member
Hi John,

I'm not sure if you are still interested in CDO dockerized... anyway, here is my version to run CDO on Docker

https://github.com/adalrsjr1/docker/tree/master/cdo

enjoy!

--
Adalberto
Re: [CDO] Server Choices and Deployment Options [message #1764746 is a reply to message #1764737] Fri, 02 June 2017 05:28 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6382
Registered: July 2009
Senior Member
Hi Adalberto,

That sounds interesting. Please feel free to add a link to https://wiki.eclipse.org/CDO/User_Contributed_Documentation ;-)

Cheers
Eike
Re: [CDO] Server Choices and Deployment Options [message #1766210 is a reply to message #1764746] Sat, 17 June 2017 18:54 Go to previous messageGo to next message
Missing name Mising name is currently offline Missing name Mising nameFriend
Messages: 9
Registered: April 2011
Junior Member
Hi Eiki,

I couldn't update CDO's Wiki Section. Following a Snippet about how to create CDO's Docker Images.

Running CDO into Docker Container

There is a little trick to run CDO on Docker containers:

1. Download CDO Server from Oomph [https://wiki.eclipse.org/Eclipse_Installer]
2. Create a Dockerfile into the same directory where CDO has been installed.
\home\user\cdo\
\eclipse\
\Dockerfile

Dockerfile content:
FROM openjdk:8-jre
COPY /eclipse /root/eclipse
RUN chmod +x /root/eclipse/eclipse & \
mkdir -p /root/eclipse /root/workspace && \
cp -r /root/eclipse.p2 /root/. && \
rm -rf /root/eclipse/.p2 && \
export PATH=$PATH:/root/eclipse
EXPOSE 2036
WORKDIR /root/eclipse
ENTRYPOINT ["./eclipse"]

3.In config.ini and all bundle files, it's necessary to substitute the path of files and *.jars for that will be used by the container.

e.g., in bundle.info, susbstitute file:/this/is/your/folder/.p2/pool/plugins/org.eclipse.** with
file:/root/.p2/pool/plugins/org.eclipse.**

the same idea must be applied in config.ini

4. Building an image just run docker build -t eclipse/cdo .
5. Running docker run -d eclipse/cdo
Re: [CDO] Server Choices and Deployment Options [message #1766561 is a reply to message #1766210] Fri, 23 June 2017 05:26 Go to previous message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6382
Registered: July 2009
Senior Member
Thanks for writing this tutorial! What was the problem with the wiki? If you want me to add it for you, please provide at least your real name.
Previous Topic:[CDO] Saving 1st model element to new repo crashes Eclipse
Next Topic:[CDO] debugging schema derivation?
Goto Forum:
  


Current Time: Mon Jun 25 16:09:33 GMT 2018

Powered by FUDForum. Page generated in 0.02295 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top