Eclipse Community Forums: Eclipse Titan » Encoding text-based protocols in Titan

Help

Home

Home » Eclipse Projects » Eclipse Titan » Encoding text-based protocols in Titan

Show: Today's Messages :: Show Polls :: Message Navigator

Encoding text-based protocols in Titan [message #1724487]

Wed, 24 February 2016 08:43

Elemer Lelik

Messages: 1120
Registered: January 2015

Senior Member

The content of text-based protocols is represented in a human-readable form, as opposed to binary protocols which are intended for machine consumption.
Although with higher overhead, these protocols are simple to read and debug; some of them are originated in the dawn of internet as Telnet or SMTP.
XML or JSON protocol representations, textual in appearance , do not fall in this category, at least according to my ad-hoc classification I have invented on-the-fly.
Typically text-based protocols appear as character strings with internal separators specifying start and end of an information element, or delimiters between information elements.

( Readers more scientifically minded than I am might object that everything transmitted "on the wire" appears in a binary form, so how text based protocols distinguish themselves?
The simple answer is that text-based protocol messages can be represented using only ASCII characters.
An interesting problem is transmitting (unrestricted) binary data over such an ASCII channel.
The frequently deployed solution is to use base 64 encoding which can represent binary data
with 64 symbols (the ASCII character set) at the expense of some overhead.)

Usually the products delivered with Titan( test ports or protocol modules ) which are text-based, use one of the following two approaches:
where speed is potentially an issue , a flex/bison based parser is written (for flex/bison basics, see http://aquamentus.com/flex_bison.html);
a good example for this is the SIP protocol specification part of the SIPMmsg test port(https://github.com/eclipse/titan.TestPorts.SIPmsg).
Flex and Bison are obvious choices as parsing of TTCN-3 code in Titan is based on them (and on ANTLR for the Java-based plug-ins).

Otherwise a controllable TEXT-encoder is used , which is a lot simpler to handle but not as performant as the previous solution.
The TEXT-encoder is described in detail in ch. 4.24 TEXT Encoder and Decoder of the referenceguide.
An example of protocol module that uses the TEXT encoder is the SMTP protocol(https://github.com/eclipse/titan.ProtocolModules.SMTP) still acknowledged by a wide variety of mail-servers worldwide.

When dealing with a text-based protocol the first problem we have to solve is how to delimit information elements present in the stream;
this can be done either by using start-stop characters or groups of characters- keywords, as in "START my first message is here STOP START my second message is here STOP" , or delimiters between information elements, as in "my first message;my second message" , where ";" plays the role of delimiter (which of course cannot be used directly in info elements as this would create confusion; same applies to start-stop keywords).
The next problem is how we encode the information elements themselves: for instance, an integer can be encoded
with or without leading zeroes, with fixed or variable length etcaetera.

Accordingly , the TEXT encoder acknowledges the following attributes: BEGIN, END, SEPARATOR, TEXT_CODING.

As an example, a template for the following structure declared in SMTP

type record SMTP_Rcpt_cmd {
  SMTP_Path           forward_path
} with {
  variant "BEGIN('RCPT TO:',,case_insensitive)"
  variant "END('\r\n')"
}

could look like this:

template SMTP_Rcpt_cmd t_SMTP_Rcpt_cmd:={
forward_path:="name@domain.com"  
}

and will be TEXT-encoded:

RCPT TO:name@domain.com\r\n

A variable v_rec of type

type record Rec_1{
   charstring field_1,
   charstring field_2
}
  with {
    variant "BEGIN('Header: ')"
    variant "SEPARATOR(';')"
}

var Rec_1 v_rec:={field1:="value_field1",
                  field2:="value_field2"}

will result in the encoded string:

Header: value_field1; value_field2

Example for encoding a boolean value:

type boolean My_bool with {
  variant "TEXT_CODING(true:'good';false:'bad')"
}

var my_bool v_bbb=false;

The encoded value:

bad

Obviously , the principles of the controllable codec are similar to the binary RAW codecs'.
As in the case of RAW, the codec functions are autogenerated, pending on the existence of:

-a final 'with { encode "TEXT" }' instruction for the module
-at least an empty 'with {variant "TEXT_CODING()"}' instruction for the type we want to encode-decode ( in lack of other instructions, defaults will apply)

External functions will also have to be declared to connect to the auto-generated functions:

 external function enc_SMTP_PDU(in SMTP_PDU pdu) return charstring
 with { extension "prototype(convert) encode(TEXT)" };
 external function dec_SMTP_PDU(in charstring stream) return SMTP_PDU
 with { extension "prototype(convert) decode(TEXT)" };

Error behaviour can be adjusted with an extension "errorbehavior" , default errorbehavior being set to ERROR:

  external function dec_SMTP_PDU(in charstring stream) return SMTP_PDU
 with { extension "prototype(convert) decode(TEXT)" 
        extension "errorbehavior(ALL:WARNING)"
 };

Besides the above two possibilities of course the use of custom-written codecs is always available.

Best regards

Elemer

[Updated on: Wed, 24 February 2016 10:30]

Report message to a moderator