|
| Re: [CDO] experience with huge data amount [message #916085 is a reply to message #915984] |
Tue, 18 September 2012 13:54   |
|
Am 18.09.2012 12:55, schrieb peter meyer:
> Hi,
>
> currently i am evaluating different technologies for a new product.
>
> The application domain is "energy producing plants".
>
> On key feature of the product is, that different customers have different businessmodels. So the idea is to model the
> businessmodel with EMF and persist it via a CDO Server in a relational database.
> Despite the pure businessmodel for the plants, there are huge amount of timeseries data which are measured during the
> runtime of the product. For example:
> there is a businessobject "transformer" which received voltage is measured every minute (-> timeseries
> measurement-timestamp/value). Both, the transformer and the timeseries data should persisted via a CDO-Server.
>
> So my question is:
> Is CDO-Server the right persistenz technologie for appliactions with real big data amounts?
> As far as i see for each datarecord persisted via CDO-Server one record in the table CDOObject is inserted. Will this
> scale for applications with hundrets of million datarecords?
The records themselves map to many different tables?
If the central cdo_objects table is a blocker for you you could try if the HibernateStore is smarter than the DBStore in
this regard. I vaguely remember it embeds the class/table names into the object IDs so that it may not need this central
table to map object IDs to tables.
In general I know of no other model repository for huge models than CDO ;-)
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] experience with huge data amount [message #916240 is a reply to message #915984] |
Tue, 18 September 2012 19:42 |
 Erdal Karaca
Messages: 854
Registered: July 2009 |
Senior Member |
|
|
I have used CDO with "big data sets" (300k+ rows in a single table with 10+ numeric columns)...
I used Oracle DB as backend which had good performance for SELECTs (using sql query handler) out of the box without any tweaking...
Deleting an object took really long when the ensure referential integrity was set to true (querying for cross references).
So, it depends on the backend you will be using.
As Eike has written, the only problem could be the central cdo_objects table which has an entry of each object that is managed by CDO.
Maybe, it is possible to split your time series data by plants, e.g. for each plant provide a repository which manages the objects in a different db/schema?
BTW if you continuously produce time series data at an interval of 1 minute by just one "transformer", then you will have reached 100M+ objects just after 190 years 
peter meyer wrote on Tue, 18 September 2012 06:55Hi,
currently i am evaluating different technologies for a new product.
The application domain is "energy producing plants".
On key feature of the product is, that different customers have different businessmodels. So the idea is to model the businessmodel with EMF and persist it via a CDO Server in a relational database.
Despite the pure businessmodel for the plants, there are huge amount of timeseries data which are measured during the runtime of the product.
For example:
there is a businessobject "transformer" which received voltage is measured every minute (-> timeseries measurement-timestamp/value). Both, the transformer and the timeseries data should persisted via a CDO-Server.
So my question is:
Is CDO-Server the right persistenz technologie for appliactions with real big data amounts?
As far as i see for each datarecord persisted via CDO-Server one record in the table CDOObject is inserted. Will this scale for applications with hundrets of million datarecords?
with kind regards
Peter
|
|
|
Powered by
FUDForum. Page generated in 0.03149 seconds
] 
Search
Help
Register
Login
Home
