Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Modeling » EMF » [CDO] data type precisions by annotations
[CDO] data type precisions by annotations [message #431424] Thu, 09 July 2009 07:50 Go to next message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
How about a way to alter the limits for some data types.
For example, a varchar without any precision declaration of the H2 database uses by default the
maximum Integer length of the system. But CDO always restricts this by automatically setting a
precision of 32672. Simply not enough for some of my attributes.
I see two possible solutions:
1) Add a global option to turn precisions off. I don't like that idea as it would then affect any
varchar column, even in the CDO management tables.
2) Introduce an EMF annotation to set the precision or to simply turn it off (maybe in the
TENEO/Hibernate style) for a specific string attribute.
Also a definition to use CLOB would be usefull. The only problem I see is that HSQLDB does not
support CLOB. But I could maybe query the db adapter name before setting a data type.
As I am more or less voluntarily involved in the CDO development ;-), I could implement it. Isn't a
big deal, I think (but I always think that when I start to program something ;-)).

I would suggest the following annotation:

<eAnnotations source="cdo">
<details key="appinfo" value="
@Lob/>
</eAnnotations>

@Lob leads to using CLOB instead of VARCHAR (if the DB supports this) when the EMF data type is a
string.
@Column(length=0) leads to a suppressed precision of VARCHAR. Or maybe we should simply use @Lengh=0.

How about supporting EByteArray's? Why are those not supported yet? Cause of using streams for
BLOB's and so on?
Couldn't we translate the ByteArray into an hex string and then inject it directly by using SQL?

Do we already have bugzillas for the above tasks?

Regards,
Kai
Re: [CDO] data type precisions by annotations [message #431425 is a reply to message #431424] Thu, 09 July 2009 08:12 Go to previous messageGo to next message
Eike Stepper is currently offline Eike StepperFriend
Messages: 6463
Registered: July 2009
Senior Member
Kai,

Comments below...


Kai Schlamp schrieb:
> How about a way to alter the limits for some data types.
> For example, a varchar without any precision declaration of the H2
> database uses by default the
> maximum Integer length of the system. But CDO always restricts this by
> automatically setting a
> precision of 32672. Simply not enough for some of my attributes.
> I see two possible solutions:
> 1) Add a global option to turn precisions off. I don't like that idea
> as it would then affect any
> varchar column, even in the CDO management tables.
> 2) Introduce an EMF annotation to set the precision or to simply turn
> it off (maybe in the
> TENEO/Hibernate style) for a specific string attribute.
> Also a definition to use CLOB would be usefull. The only problem I see
> is that HSQLDB does not
> support CLOB. But I could maybe query the db adapter name before
> setting a data type.
> As I am more or less voluntarily involved in the CDO development ;-),
> I could implement it. Isn't a
> big deal, I think (but I always think that when I start to program
> something ;-)).
>
> I would suggest the following annotation:
>
> <eAnnotations source="cdo">
> <details key="appinfo" value="
> @Lob/>
> </eAnnotations>
>
> @Lob leads to using CLOB instead of VARCHAR (if the DB supports this)
> when the EMF data type is a string.
> @Column(length=0) leads to a suppressed precision of VARCHAR. Or maybe
> we should simply use @Lengh=0.
>
I like the idea of having annotations that influence the mapping. We
already talked about that but have no bugzilla so far. Please enter one
so that we can discuss the details there.

> How about supporting EByteArray's? Why are those not supported yet?
> Cause of using streams for BLOB's and so on?
Maybe Stefan knows this?
My guess is that we just don't have a matching ITypeMapping
implementation...

Cheers
/Eike

----
http://thegordian.blogspot.com
http://twitter.com/eikestepper



> Couldn't we translate the ByteArray into an hex string and then inject
> it directly by using SQL?
>
> Do we already have bugzillas for the above tasks?
>
> Regards,
> Kai
>


Re: [CDO] data type precisions by annotations [message #431426 is a reply to message #431425] Thu, 09 July 2009 08:34 Go to previous messageGo to next message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
Eike Stepper wrote:
> Kai,
>
> Comments below...
>
>
> Kai Schlamp schrieb:
>> How about a way to alter the limits for some data types.
>> For example, a varchar without any precision declaration of the H2
>> database uses by default the
>> maximum Integer length of the system. But CDO always restricts this by
>> automatically setting a
>> precision of 32672. Simply not enough for some of my attributes.
>> I see two possible solutions:
>> 1) Add a global option to turn precisions off. I don't like that idea
>> as it would then affect any
>> varchar column, even in the CDO management tables.
>> 2) Introduce an EMF annotation to set the precision or to simply turn
>> it off (maybe in the
>> TENEO/Hibernate style) for a specific string attribute.
>> Also a definition to use CLOB would be usefull. The only problem I see
>> is that HSQLDB does not
>> support CLOB. But I could maybe query the db adapter name before
>> setting a data type.
>> As I am more or less voluntarily involved in the CDO development ;-),
>> I could implement it. Isn't a
>> big deal, I think (but I always think that when I start to program
>> something ;-)).
>>
>> I would suggest the following annotation:
>>
>> <eAnnotations source="cdo">
>> <details key="appinfo" value="
>> @Lob/>
>> </eAnnotations>
>>
>> @Lob leads to using CLOB instead of VARCHAR (if the DB supports this)
>> when the EMF data type is a string.
>> @Column(length=0) leads to a suppressed precision of VARCHAR. Or maybe
>> we should simply use @Lengh=0.
>>
> I like the idea of having annotations that influence the mapping. We
> already talked about that but have no bugzilla so far. Please enter one
> so that we can discuss the details there.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=282976

>
>> How about supporting EByteArray's? Why are those not supported yet?
>> Cause of using streams for BLOB's and so on?
> Maybe Stefan knows this?
> My guess is that we just don't have a matching ITypeMapping
> implementation...
>
> Cheers
> /Eike
>
> ----
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
>
>> Couldn't we translate the ByteArray into an hex string and then inject
>> it directly by using SQL?
>>
>> Do we already have bugzillas for the above tasks?
>>
>> Regards,
>> Kai
>>
Re: [CDO] data type precisions by annotations [message #431428 is a reply to message #431425] Thu, 09 July 2009 09:07 Go to previous messageGo to next message
Stefan Winkler is currently offline Stefan WinklerFriend
Messages: 301
Registered: July 2009
Location: Germany
Senior Member
Kai, Eike,

comment below

Eike Stepper schrieb:
> Kai,
>
> Comments below...
>
>
> Kai Schlamp schrieb:
>
>> How about a way to alter the limits for some data types.
>> For example, a varchar without any precision declaration of the H2
>> database uses by default the
>> maximum Integer length of the system. But CDO always restricts this by
>> automatically setting a
>> precision of 32672. Simply not enough for some of my attributes.
>> I see two possible solutions:
>> 1) Add a global option to turn precisions off. I don't like that idea
>> as it would then affect any
>> varchar column, even in the CDO management tables.
>> 2) Introduce an EMF annotation to set the precision or to simply turn
>> it off (maybe in the
>> TENEO/Hibernate style) for a specific string attribute.
>> Also a definition to use CLOB would be usefull. The only problem I see
>> is that HSQLDB does not
>> support CLOB. But I could maybe query the db adapter name before
>> setting a data type.
>> As I am more or less voluntarily involved in the CDO development ;-),
>> I could implement it. Isn't a
>> big deal, I think (but I always think that when I start to program
>> something ;-)).
>>
>> I would suggest the following annotation:
>>
>> <eAnnotations source="cdo">
>> <details key="appinfo" value="
>> @Lob/>
>> </eAnnotations>
>>
>> @Lob leads to using CLOB instead of VARCHAR (if the DB supports this)
>> when the EMF data type is a string.
>> @Column(length=0) leads to a suppressed precision of VARCHAR. Or maybe
>> we should simply use @Lengh=0.
>>
>>
> I like the idea of having annotations that influence the mapping. We
> already talked about that but have no bugzilla so far. Please enter one
> so that we can discuss the details there.
>
Yes, we do have a bugzilla:
The existing annotation discussion is kept in Bug 256635
<https://bugs.eclipse.org/bugs/show_bug.cgi?id=256635>
Can you please try to keep the discussion focused in one of these bugs
so that we get a common API for all of the configuration by annotation
issues?
>
>> How about supporting EByteArray's? Why are those not supported yet?
>> Cause of using streams for BLOB's and so on?
>>
> Maybe Stefan knows this?
> My guess is that we just don't have a matching ITypeMapping
> implementation...
>
There is a byte array type mapping. Eike has implemented it. It uses the
JDBC setBytes() call. I currently don't know how well and up to which
length this is supported.

Cheers,
Stefan


> Cheers
> /Eike
>
> ----
> http://thegordian.blogspot.com
> http://twitter.com/eikestepper
>
>
>
>
>> Couldn't we translate the ByteArray into an hex string and then inject
>> it directly by using SQL?
>>
>> Do we already have bugzillas for the above tasks?
>>
>> Regards,
>> Kai
>>
>>
Re: [CDO] data type precisions by annotations [message #431435 is a reply to message #431428] Thu, 09 July 2009 12:26 Go to previous messageGo to next message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
> There is a byte array type mapping. Eike has implemented it. It uses the
> JDBC setBytes() call. I currently don't know how well and up to which
> length this is supported.

Where does this stuff happen? DBRevisionCache.setParameters method?
I wonder if there is any chance to influence such things by the DBAdapter implementations.
I guess not. It would be nice for example, if I have a feature of DBType.BLOB, to store it with the
use of streams when using H2, but using VARCHAR for HSQLDB (as it doesn't support BLOB's). Such
things have to be somehow implemented in the DBAdapter.
At the moment DBAdapter just helps to build the SQL string, but doesn't have any influence on the
statement itself, as far as I understand the DB implementation.
Is that right?
Re: [CDO] data type precisions by annotations [message #431436 is a reply to message #431435] Thu, 09 July 2009 13:24 Go to previous messageGo to next message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
I looked a bit at the JDBC streaming things.

So I would suggest...
field of DBType.CLOB:
use for insert and update: PreparedStatement.setCharacterStream()
use for selects: ResultSet.getCharacterStream()
field of DBType.BLOB:
use for insert and update: PreparedStatement.setBinaryStream()
use for selects: ResultSet.getBinaryStream()

The INSERT and UPDATE happens in DBUtil.update(). Should be no problem.
Where does the SELECT happen that assembles the appropriate object?


Kai Schlamp wrote:
>> There is a byte array type mapping. Eike has implemented it. It uses the
>> JDBC setBytes() call. I currently don't know how well and up to which
>> length this is supported.
>
> Where does this stuff happen? DBRevisionCache.setParameters method?
> I wonder if there is any chance to influence such things by the
> DBAdapter implementations.
> I guess not. It would be nice for example, if I have a feature of
> DBType.BLOB, to store it with the use of streams when using H2, but
> using VARCHAR for HSQLDB (as it doesn't support BLOB's). Such things
> have to be somehow implemented in the DBAdapter.
> At the moment DBAdapter just helps to build the SQL string, but doesn't
> have any influence on the statement itself, as far as I understand the
> DB implementation.
> Is that right?
Re: [CDO] data type precisions by annotations [message #431438 is a reply to message #431435] Thu, 09 July 2009 13:54 Go to previous messageGo to next message
Stefan Winkler is currently offline Stefan WinklerFriend
Messages: 301
Registered: July 2009
Location: Germany
Senior Member
Kai,

comments below.

Kai Schlamp schrieb:
>> There is a byte array type mapping. Eike has implemented it. It uses the
>> JDBC setBytes() call. I currently don't know how well and up to which
>> length this is supported.
>
> Where does this stuff happen? DBRevisionCache.setParameters method?
There is a type mapping for that: TypeMapping.TMBytes

> I wonder if there is any chance to influence such things by the
> DBAdapter implementations.
> I guess not. It would be nice for example, if I have a feature of
> DBType.BLOB, to store it with the use of streams when using H2, but
> using VARCHAR for HSQLDB (as it doesn't support BLOB's). Such things
> have to be somehow implemented in the DBAdapter.
> At the moment DBAdapter just helps to build the SQL string, but
> doesn't have any influence on the statement itself, as far as I
> understand the DB implementation.
> Is that right?
Not really anymore. The appendValue framework of the DBAdapter should
never (or at least usually not) be called as we are using prepared
statements since a few months.

For the genal decision of a string should be put als as varchar or as
CLOB, my thoughs for redesigning the DB store had been that the mapping
factory

org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappi ngStrategy.createValueMapping(EStructuralFeature)

could in the long term understand annotations (such as the ones for the
field lengths you implemented). Maybe that would even make
user-contributed type mappings possible. Anyway, I understand the
problem that in particular LOB handling is DB dependent. Is there a
JDBC-method for querying for LOB support?
If so, we could use that to ask the driver if LOBs are supported. Else,
we should maybe implement a method to query the DB adapter if the
adapted DB supports LOBs.

Combining annotation-based configuration and DB-specific handling, I
would like to see different TypeMapping strategies for the affected
types and
org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappi ngStrategy.createValueMapping(EStructuralFeature)
implementing the decision which of the strategies to choose based on
annotation and DB capabilities.

On the long term, if the amount of available type mappings grows, we
should think about introducing an extension point for type mappings and
factor out the type mapping decision and factory code.

Cheers,
Stefan
Re: [CDO] data type precisions by annotations [message #431440 is a reply to message #431436] Thu, 09 July 2009 15:01 Go to previous messageGo to next message
Stefan Winkler is currently offline Stefan WinklerFriend
Messages: 301
Registered: July 2009
Location: Germany
Senior Member
Kai,

comments below:

Kai Schlamp schrieb:
> I looked a bit at the JDBC streaming things.
>
> So I would suggest...
> field of DBType.CLOB:
> use for insert and update: PreparedStatement.setCharacterStream()
> use for selects: ResultSet.getCharacterStream()
> field of DBType.BLOB:
> use for insert and update: PreparedStatement.setBinaryStream()
> use for selects: ResultSet.getBinaryStream()
>
> The INSERT and UPDATE happens in DBUtil.update(). Should be no problem.
DBUtil is only used for CDO meta data.
For implementation of CDO data have a look at the ITypeMapping and
IMappingStrategy implementations.
> Where does the SELECT happen that assembles the appropriate object?
See DBStoreAccessor.readRevision() for the data stuff. This is what you
are really looking for.

As I said in my other post. DBAdapter appendValue stuff is kind of
deprecated. (well, not really, but it's no longer used in favor of
prepared statement operations in ITypeMapping implementors).

BTW, for CDOType - DBType mappings have a look into the MetaDataManager.

Cheers
Stefan




>
>
> Kai Schlamp wrote:
>>> There is a byte array type mapping. Eike has implemented it. It uses
>>> the
>>> JDBC setBytes() call. I currently don't know how well and up to which
>>> length this is supported.
>>
>> Where does this stuff happen? DBRevisionCache.setParameters method?
>> I wonder if there is any chance to influence such things by the
>> DBAdapter implementations.
>> I guess not. It would be nice for example, if I have a feature of
>> DBType.BLOB, to store it with the use of streams when using H2, but
>> using VARCHAR for HSQLDB (as it doesn't support BLOB's). Such things
>> have to be somehow implemented in the DBAdapter.
>> At the moment DBAdapter just helps to build the SQL string, but
>> doesn't have any influence on the statement itself, as far as I
>> understand the DB implementation.
>> Is that right?
Re: [CDO] data type precisions by annotations [message #431449 is a reply to message #431438] Thu, 09 July 2009 19:43 Go to previous message
Kai Schlamp is currently offline Kai SchlampFriend
Messages: 344
Registered: July 2009
Senior Member
Stefan Winkler wrote:
> Kai,
>
> comments below.
>
> Kai Schlamp schrieb:
>>> There is a byte array type mapping. Eike has implemented it. It uses the
>>> JDBC setBytes() call. I currently don't know how well and up to which
>>> length this is supported.
>> Where does this stuff happen? DBRevisionCache.setParameters method?
> There is a type mapping for that: TypeMapping.TMBytes
>
>> I wonder if there is any chance to influence such things by the
>> DBAdapter implementations.
>> I guess not. It would be nice for example, if I have a feature of
>> DBType.BLOB, to store it with the use of streams when using H2, but
>> using VARCHAR for HSQLDB (as it doesn't support BLOB's). Such things
>> have to be somehow implemented in the DBAdapter.
>> At the moment DBAdapter just helps to build the SQL string, but
>> doesn't have any influence on the statement itself, as far as I
>> understand the DB implementation.
>> Is that right?
> Not really anymore. The appendValue framework of the DBAdapter should
> never (or at least usually not) be called as we are using prepared
> statements since a few months.
>
> For the genal decision of a string should be put als as varchar or as
> CLOB, my thoughs for redesigning the DB store had been that the mapping
> factory
>
> org.eclipse.emf.cdo.server.internal.db.mapping.AbstractMappi ngStrategy.createValueMapping(EStructuralFeature)
>
> could in the long term understand annotations (such as the ones for the
> field lengths you implemented). Maybe that would even make
> user-contributed type mappings possible. Anyway, I understand the
> problem that in particular LOB handling is DB dependent. Is there a
> JDBC-method for querying for LOB support?
> If so, we could use that to ask the driver if LOBs are supported. Else,
> we should maybe implement a method to query the DB adapter if the
> adapted DB supports LOBs.

I implemented it now in that way that the user annotate the feature with the data type he wants (by
directly using an annotation with value "VARCHAR", "CLOB", whatever).
But only types of DBType are supported and allowed.
We could now fetch again the annotation for example in TMString.getResultSetValue, check if the user
annotated the feature with CLOB and then use result.getCharacterStream(), otherwise fetch with
getString (like it is done now).
This won't allow custom types and the user has to think about what database he uses and what data
types that database supports (if he uses annotations).
But at least we have access to all those streaming stuff and more data types, without doing a big
refactoring.
In the long term, I agree with you, a bigger refactoring would be nice.

On the other hand, to rebuild something like Hibernate with all it's features and annotations, is
that really what DBStore is intended for?
If the user wants all those customizations, wouldn't it better for him to simply take the
HibernateStore?
What I like about DBStore is that it is fast and easy to configure (nearly no configuration at all).
For me some rudimentary configurations (like the now implemented option to choose the precision or
to choose CLOB instead of VARCHAR) is absolutely enough.
But I don't know what the long term plans for the DBStore are.

Regards,
Kai

P.S. By the way, I am at the Fernuni Hagen sometime in the upcoming weeks. Maybe we can get together
for a coffee or beer :-)
Previous Topic:[Teneo] No longer need EMFTupelizer override?
Next Topic:Re: EMF Migration - EDataType error
Goto Forum:
  


Current Time: Mon Dec 16 02:57:09 GMT 2019

Powered by FUDForum. Page generated in 0.03039 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top