| [CDO] experience with huge data amount [message #915984] |
Tue, 18 September 2012 06:55  |
Eclipse User |
|
|
|
Hi,
currently i am evaluating different technologies for a new product.
The application domain is "energy producing plants".
On key feature of the product is, that different customers have different businessmodels. So the idea is to model the businessmodel with EMF and persist it via a CDO Server in a relational database.
Despite the pure businessmodel for the plants, there are huge amount of timeseries data which are measured during the runtime of the product.
For example:
there is a businessobject "transformer" which received voltage is measured every minute (-> timeseries measurement-timestamp/value). Both, the transformer and the timeseries data should persisted via a CDO-Server.
So my question is:
Is CDO-Server the right persistenz technologie for appliactions with real big data amounts?
As far as i see for each datarecord persisted via CDO-Server one record in the table CDOObject is inserted. Will this scale for applications with hundrets of million datarecords?
with kind regards
Peter
|
|
|
|
| Re: [CDO] experience with huge data amount [message #916240 is a reply to message #915984] |
Tue, 18 September 2012 15:42  |
| Eclipse User |
|
|
|
I have used CDO with "big data sets" (300k+ rows in a single table with 10+ numeric columns)...
I used Oracle DB as backend which had good performance for SELECTs (using sql query handler) out of the box without any tweaking...
Deleting an object took really long when the ensure referential integrity was set to true (querying for cross references).
So, it depends on the backend you will be using.
As Eike has written, the only problem could be the central cdo_objects table which has an entry of each object that is managed by CDO.
Maybe, it is possible to split your time series data by plants, e.g. for each plant provide a repository which manages the objects in a different db/schema?
BTW if you continuously produce time series data at an interval of 1 minute by just one "transformer", then you will have reached 100M+ objects just after 190 years 
peter meyer wrote on Tue, 18 September 2012 06:55Hi,
currently i am evaluating different technologies for a new product.
The application domain is "energy producing plants".
On key feature of the product is, that different customers have different businessmodels. So the idea is to model the businessmodel with EMF and persist it via a CDO Server in a relational database.
Despite the pure businessmodel for the plants, there are huge amount of timeseries data which are measured during the runtime of the product.
For example:
there is a businessobject "transformer" which received voltage is measured every minute (-> timeseries measurement-timestamp/value). Both, the transformer and the timeseries data should persisted via a CDO-Server.
So my question is:
Is CDO-Server the right persistenz technologie for appliactions with real big data amounts?
As far as i see for each datarecord persisted via CDO-Server one record in the table CDOObject is inserted. Will this scale for applications with hundrets of million datarecords?
with kind regards
Peter
|
|
|
Powered by
FUDForum. Page generated in 0.43441 seconds
Home