Home » Eclipse Projects » Eclipse Titan » Support for JSON binary encodings in Titan: BSON
| Support for JSON binary encodings in Titan: BSON [message #1769061] |
Wed, 26 July 2017 06:31 |
|
Dear all,
BSON is a binary serialization of JSON-like documents. BSON is also the default format to store documents in MongoDB NoSQL databases.
Unlike CBOR, the primary goal of BSON is to enable traversability, that is fast processing, and not compactness. BSON adds extra information to documents such as length prefixes, that makes them faster to process. In terms of storage footprint BSON is not necessarily more efficient than JSON.
BSON is supported in Titan in form of two conversion functions json2bson and bson2json ,
see par. 4.13.14, 4.13.15 of the reference guide:
4.13.14 json2bson
Syntax: json2bson(in universal charstring us) return octetstring
The function json2bson(in universal charstring us) return octetstring converts a TITAN encoded json document into the binary representation of that json document using a binary coding called BSON.
Only top level json objects and arrays can be encoded. (Note that an encoded top level json array will be decoded as a json object).
The encoding follows the rules written in the BSON standard
([23] BSON specification version 1.1: http://bsonspec.org/spec.html ).
The encoding handles the extension rules written in the MongoDB Extended JSON document (https://docs.mongodb.com/manual/reference/mongodb-extended-json/#bson-data-types-and-associated-representations).
The encoding of 128-bit float values is not supported.
EXAMPLE: json2bson("{"a":1,"b":2}") --> '13000000106100010000001062000200000000'O
4.13.15 bson2json
Syntax: bson2json(in octetstring os) return universal charstring
The function bson2json(in octetstring os) return universal charstring converts a BSON encoded bytestream into a json document which can be decoded using the built in JSON decoder. The decoding follows
the extension rules written in the BSON standard [23]. The decoding handles the rules written in the MongoDB Extended JSON document (https://docs.mongodb.com/manual/reference/mongodb-extended-json/#bson-data-types-and-associated-representations).
The decoding of 128-bit float values is not supported.
EXAMPLE: bson2json('13000000106100010000001062000200000000'O) --> "{"a":1,"b":2}"
The attached file demoes the json-bson (and back) conversions and it consists of three parts:
-encode/decode JSON_RPC messages
-encode/decode random JSON and BSON samples
-summary performance measurements
Note1:
The BSON serialization is not necessarily symmetrical; for instance , JSON arrays such as this one:
[
{"jsonrpc": "2.0", "method": "sum", "params": [1,2,4], "id": "1"},
{"jsonrpc": "2.0", "method": "notify_hello", "params": [7]},
{"jsonrpc": "2.0", "params": [42,23], "method": "subtract", "id": "2"},
{"jsonrpc": "2.0", "method": "foo.get", "params": [42,23], "id": "5"},
{"jsonrpc": "2.0", "method": "get_data", "id": "9"}
]
after serialization/deserialization will be returned as JSON objects, in this case as below:
{
"0":{"jsonrpc":"2.0","method":"sum","params":[1,2,4],"id":"1"},
"1":{"jsonrpc":"2.0","method":"notify_hello","params":[7]},
"2":{"jsonrpc":"2.0","params":[42,23],"method":"subtract","id":"2"},
"3":{"jsonrpc":"2.0","method":"foo.get","params":[42,23],"id":"5"},
"4":{"jsonrpc":"2.0","method":"get_data","id":"9"}
}
Note2:
BSON is based on an extended JSON which was prepared for binary data storage;
{ "$binary": "<bindata>", "$type": "<t>" }
where:
bindata> is the base64 representation of a binary string.
<t> is a representation of a single byte indicating the data type (a hexadecimal string)
For instance:
'C0AAAAAAAAAAAAAABBBBBBBBBBBBFFFFABBBBBBBBBBBBAAAAAAAAAAAAAAC0C0AAAAAAAAAAAAAABBBBBBBBBBBBFFFFABBBBBBBBBBBBAAAAAAAAAAAAAAC0'O
is encoded in base64 as
"wKqqqqqqqqq7u7u7u7v//6u7u7u7u7qqqqqqqqqsDAqqqqqqqqqru7u7u7u///q7u7u7u7uqqqqqqqqqwA=="
Subsequently, the below JSON:
{
"_id":{
"$oid":"546651E74BF6E4CB017C5312"
},
"a":1.000000,
"b":"I am a string",
"c":{
"$timestamp":{
"t":1415991783,
"i":1
}
},
"d":{
"$binary":"wKqqqqqqqqq7u7u7u7v//6u7u7u7u7qqqqqqqqqsDAqqqqqqqqqru7u7u7u///q7u7u7u7uqqqqqqqqqwA==",
"$type":"00"
}
}
will be BSON encoded as:
'7D000000075F696400546651E74BF6E4CB017C5312016100000000000000F03F0262000E0000004920616D206120737472696E670011630001000000E75166540564003D00000000C0AAAAAAAAAAAAAABBBBBBBBBBBBFFFFABBBBBBBBBBBBAAAAAAAAAAAAAAC0C0AAAAAAAAAAAAAABBBBBBBBBBBBFFFFABBBBBBBBBBBBAAAAAAAAAAAAAAC000'O
It's easy to see (towards the end of the string) that the base64 representation was converted back to a binary representation, which is also less space consuming.
In the above case the JSON/BSON ratio was measured to be 1.746269, so for storage of binary data BSON is probbaly more efficient than JSON.
Note3:
The average JSON/BSON ratio was found to be 1.055607 , based on samples in the test, so in general BSON is not necessarily more compact.
An encode cycle (nbased on 100.000 runs was measure to be 38 microseconds, and a decode cycle 30 microseconds.
These are of course ballpark figures and will vary based on message content, execution environment etc.
Using BSON serialization becomes interesting when one intends to talk to a MongoDB database.
We'll come back to that later.
P.S. This feature is available in Titan post 6.2.0 only; If someone needs it now, will have to compile a fresh Titan from source code.
Best regards
Elemer
[Updated on: Wed, 26 July 2017 06:34] Report message to a moderator |
|
|
Goto Forum:
Current Time: Sat Apr 20 01:53:28 GMT 2024
Powered by FUDForum. Page generated in 0.02642 seconds |
] 
Search
Help
Register
Login
Home
