Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » Eclipse Titan and the dilemma of PER-encoding(ASN.1 PER-codec for Titan )
Eclipse Titan and the dilemma of PER-encoding [message #1708432] Wed, 16 September 2015 13:02 Go to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 850
Registered: January 2015
Senior Member

I think this subject needs a bit of an introduction, so let me go through some ASN.1 basics first:

ASN.1 is a notation to describe structures , typically used as messages in communication systems. ASN.1 offers an abstract syntax , that is, a detailed structural description, with no reference to technology or implementation details; and a transfer syntax, a description of how the abstract structures are sent over the wire; this is more commonly known as encoding or serialization.

ASN.1 standardizes several of such encodings: XER (XML Encoding Rules) for XML encoding , BER(Basic Encoding Rules) for a binary, byte-oriented encoding, and PER (Packed Encoding Rules), for a bit stream-oriented, compact encoding. The latter two will be of interest for us here.

BER is a simple TLV(Type-Length-Value) encoding, meaning that types are assigned predefined codes, and structures are encoded as a succession consisting of a type code, a field signifying the length of the data encoded, and the data itself in a binary form. BER-encoded messages can be easily
decoded even without having access to the ASN.1 (minus the names assigned to structures of course, which are not transmitted over the wire).

PER is another piece of cake: it uses the minimum amount of bits needed to represent the structural information (for instance if an integer field is constrained to 16 possible values, say from 0 to 15, PER will represent it on 4 bits, while BER will use one full byte ) and it comes in two flavors,
aligned and unaligned. For aligned PER, certain types are aligned to byte boundaries using padding bits; unaligned PER is the most economical, at the cost of some extra processing requirements. Unaligned PER stands the comparison to a BER-encoded message compressed with an efficient compression algorithm. PER is typically used for protocols that justify such savings, radio protocols where channel bandwidth is a scarce resource being good examples. Where bandwidth is not an issue, the less resource-hungry BER can be used.


ASN.1 is still intensively used today when publishing telecom standards; the message structures used are annexed to the standard in form of ASN.1 code. Any implementer can pluck it directly and compile it; there is a
lot less chance of misinterpreting the standard, no need to transfer long tables of messages to code etc; overall it's a brilliant way of describing a protocol, far superior to the practice in other industry segments.

TTCN-3 owes a lot to ASN.1, into which is heavily rooted ; not incidentally, TTCN-3's way of describing structures is very similar to the one used in ASN.1; the structures can be directly mapped and the TTCN-3 standard itself describes the translation between ASN.1 and TTCN-3 structures;
thus, it becomes possible to write TTCN-3 message templates directly against ASN.1 type declarations.

As a consequence most TTCN-3 tools will understand ASN.1 to a degree and so does Titan. Titan has a rather evolved ASN.1 compiler included, with one deficiency: it does not support PER encoding. The reasons are manifold and historical: for PER-encoding we have found early a solution using a 3rd party tool, and after that we did not see the business justification of spending hundreds of man-hours worth, especially that only a relatively small amount of protocols use it.

So what is this solution involving a 3rd party tool consists of? Within Ericsson, we had (and still have ) access to an exceptionally performant and also expensive commercial ASN.1 compiler which was capable of all standard encodings, so we have came up with the following , a little bit convoluted, but perfectly operational workflow:

-first and foremost , we compile the ASN.1 code with this 3rd party compiler, and generate BER and PER codecs
-in the encoding/sending direction we first encode with Titan's BER encoder , then decode it with the external BER decoder and recode it with PER
-in the decoding/receiving direction, first we use the external PER decoder to extract the information into an internal structure and then we recode in BER and send it to Titan; Titan decodes the BER and regenerates the structure sent from the other end

Anyone willing to use this workflow had to have both Titan and the external ASN.1 compiler and had to be able to compile the ASN.1 code with both. Due to the commercial nature of the ASN.1 compiler , the same workflow cannot be used in open source projects. To replicate it in open source, we'd need an open source ASN.1 compiler that supports PER encoding.

Luckily, there is such an open source compiler, namely asn1c by Lev Walkin (http://lionet.info/asn1c/blog/ - not to be confused with asn1c from Objective Systems). Problem is, it comes with its' own restrictions, it does not support Information Object Classes for instance, so it cannot be used for all protocols. Nonetheless, if it can compile your ASN.1 file, it can be used to generate the PER-codec to be used with Titan.

If you are now sufficiently confused by the fact that three compilers are needed to run a protocol test, let's quickly recap:
Titan adds the TTCN-3 compiler and an ASN.1 compiler that have in common the same internal representation, so they can talk to each other directly. Any external compiler will use its' own internal representation, incompatible with Titan. This external compiler will talk to Titan using the API represented by the BER-codec and it's only role is to deal with PER.

In conclusion, asn1c will work for a category of ASN.1 files and it will not work for others. To find out to which category your ASN.1 code belongs,
install asn1c ( version 0.9.27 being the latest to date) and try to compile your code with it. If it compiles, you're in.

The exact implementation details can be followed using the attached example of CAM protocol which is part of the ITS communication stack.
(see ETSI source repository http://forge.etsi.org/websvn/).
ITS (Intelligent Transport System) is a good example where PER is justifiably used: the cars are connected over wireless (a wired connection between moving cars has yet to be invented Neutral)


CAM_EncDec.cc is the central part of the code where the BER and PER codecs are invoked according to the logic described above.
The bunch of .c and .h files have been generated by asn1c ( which creates separate files for each type it encounters)
with the following command:

asn1c gen-PER CAM.asn ITS_Container.asn


where -gen-PER triggers asn1c to generate PER support code.

(Note that we renamed ITS-Container.asn to ITS_Container.asn. This is an internal Titan convention, all ASN.1 file names containing hyphen will have to be changed accordingly.)


Mind that asn1c generates code not entirely compatible with g++. For a full compatibility, some changes are necessarry.
This might be different from case to case, but for reference here are the changes we have made:



-Some enum uses have to be prefixed with its' namespace as shown below (APC_CONSTRAINED, APC_UNCONSTRAINED):

static asn_per_constraints_t asn_PER_type_AccelerationConfidence_constr_1 GCC_NOTUSED = {
                { asn_per_constraint_t::APC_CONSTRAINED,     7,  7,  0,  102 }    /* (0..102) */,
                { asn_per_constraint_t::APC_UNCONSTRAINED,              -1, -1,  0,  0 },
                0, 0         /* No PER value map */
};


-Bitwise operators with enums have to be casted to int, and then back to the enumtype:

For example:

Original code:
static asn_per_constraints_t asn_PER_type_PositioningSolutionType_constr_1 GCC_NOTUSED = {
                { APC_CONSTRAINED | APC_EXTENSIBLE,  3,  3,  0,  5 }    /* (0..5,...) */,
                { APC_UNCONSTRAINED,             -1, -1,  0,  0 },
                0, 0         /* No PER value map */
};


Casted code:
static asn_per_constraints_t asn_PER_type_PositioningSolutionType_constr_1 GCC_NOTUSED = {
                { (asn_per_constraint_t::asn_per_constraint_flags)((int)(asn_per_constraint_t::APC_CONSTRAINED) | (int)(asn_per_constraint_t::APC_EXTENSIBLE)),  3,  3,  0,  5 }        /* (0..5,...) */,
                { asn_per_constraint_t::APC_UNCONSTRAINED,              -1, -1,  0,  0 },
                0, 0         /* No PER value map */
};


-Specifically cast pointers into the correct type:

enc_dyn_arg *arg = (enc_dyn_arg*)key;
uper_ugot_key *arg = (uper_ugot_key*)pd->refill_key;


-Cast char* variables to const char *:

Original code:
static char *_bit_pattern[16] = {
                "0000", "0001", "0010", "0011", "0100", "0101", "0110", "0111",
                "1000", "1001", "1010", "1011", "1100", "1101", "1110", "1111"
};

Casted code:
static const char *_bit_pattern[16] = {
                "0000", "0001", "0010", "0011", "0100", "0101", "0110", "0111",
                "1000", "1001", "1010", "1011", "1100", "1101", "1110", "1111"
};





To install the example, unpack it , then:

cd bin
../install.script
make
./CAM

The execution is a loop encoding-decoding of a CAM message, not very spectacular in itself , but demonstrating the usage of the PER codec.
As you can see from the log, the three compilers are happily interacting and crunching the unaligned bits of PER-encoding.

In summary, using the example above, anyone can hack a PER codec for his/her favorite protocol ; the only condition is that asn1c should support it.



Regards

Re: Eclipse Titan and the dilemma of PER-encoding [message #1763482 is a reply to message #1708432] Tue, 16 May 2017 15:06 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 850
Registered: January 2015
Senior Member
Dear all,

an update to this subject:

as Coco Wang noticed ( see make error in LTE test of coding by asn1c
https://www.eclipse.org/forums/index.php/t/1086125/)

the Makefile.am.sample generated by asn1c while compiling the ASN.1 file can be used to build the generated
source and header files creating the corresponding object files without the complex modifications above, which is a great time-saver.

However , using the object files might be inconvenient as a "make clean' will erase them so they will have to be copied again and again.

So I propose to create a library instead of object files.

This is how to create a static library from the asn1c generated sources:

Edit the generated Makefile.am.sample, add the following lines:


LIBNAME = librrc.a
lib: ${OBJS}
      ar -rcs ${LIBNAME} $^


The

make -f Makefile.am.sample lib 


command will create the librrc.a static library.

now create two directories /include and /lib:
the /lib will contain the librrc.a static archive
and the /include all the header files generated by asn1c

Now edit the Titan Makefile and remove all asn1c generated source files from the USER_SOURCES and USER_HEADERS variable.

Next, modify the following rule:


$(EXECUTABLE): $(OBJECTS)
                if $(CXX) $(LDFLAGS) -o $@ $(OBJECTS) \
                -L$(TTCN3_DIR)/lib -l$(TTCN3_LIB) \
                -L$(OPENSSL_DIR)/lib -lcrypto \
                -L$(XMLDIR)/lib $($(PLATFORM)_LIBS) \
                -lrrc -L../src; \
                then : ; else $(TTCN3_DIR)/bin/titanver $(OBJECTS); exit 1; fi 



and add the /include directory to CPPFLAGS:

# Flags for the C++ preprocessor (and makedepend as well):
CPPFLAGS = -D$(PLATFORM) -I$(TTCN3_DIR)/include -I../include 


See the attached compressed file. After unpacking,
you need first to edit file2 to adapt it to your directory structure, then

cd bin
#create symlinks
../file2 
#compile
make 
#execute
./testcodec testcodec.cfg



should work flawlessly, demonstrating the PER encoding/decoding.

Best regards

Elemer
Re: Eclipse Titan and the dilemma of PER-encoding [message #1769213 is a reply to message #1763482] Thu, 27 July 2017 20:02 Go to previous messageGo to next message
taqi raza is currently offline taqi razaFriend
Messages: 3
Registered: July 2017
Junior Member
Dear Elemer,

I have followed your instructions and modified the code as follow:

original code, that I have commented:
/*
static asn_per_constraints_t asn_PER_type_AccelerationConfidence_constr_1 GCC_NOTUSED = {
{ APC_CONSTRAINED, 7, 7, 0, 102 } // (0..102) ,
{ APC_UNCONSTRAINED, -1, -1, 0, 0 },
0, 0 // No PER value map
};*/

new code that I have added:

static asn_per_constraints_t asn_PER_type_AccelerationConfidence_constr_1 GCC_NOTUSED = {
{ asn_per_constraint_t::APC_CONSTRAINED, 7, 7, 0, 102 } /* (0..102) */,
{ asn_per_constraint_t::APC_UNCONSTRAINED, -1, -1, 0, 0 },
0, 0 /* No PER value map */
};


But I get following Error, can you please let me know how can I fix this.

AccelerationControl.c:123:4: error: 'APC_UNCONSTRAINED' was not declared in this scope
{ APC_UNCONSTRAINED, -1, -1, 0, 0 },
^
AccelerationControl.c:124:4: error: 'APC_CONSTRAINED' was not declared in this scope
{ APC_CONSTRAINED, 0, 0, 7, 7 } /* (SIZE(7..7)) */,
^
make: *** [AccelerationControl.o] Error 1

thank you
taqi
Re: Eclipse Titan and the dilemma of PER-encoding [message #1769229 is a reply to message #1769213] Fri, 28 July 2017 05:47 Go to previous message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 850
Registered: January 2015
Senior Member
Hi taqi,

could you please explain in detail what you are trying to do: what source files are using and what modifications you have done?
Also, please attach the files you are trying to compile.


BR

Elemer
Previous Topic:xsd2ttcn tool: FATAL ERROR while processing XMLSchema.dtd
Next Topic:There is no local or imported definition with name
Goto Forum:
  


Current Time: Tue Nov 13 04:56:02 GMT 2018

Powered by FUDForum. Page generated in 0.02237 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top