Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » Support for HTTP/2 in Eclipse Titan- with a twist( @decoded and decmatch)
Support for HTTP/2 in Eclipse Titan- with a twist [message #1773897] Fri, 06 October 2017 09:54 Go to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Dear all,

I will continue with the HTTP2 example presented previously; however, this post is not focused on HTTP2 , but on some new language elements related to matching and decoding that I will try to introduce under the pretext of HTTP2.


First , let me summarize the problem we are facing: we are using a generic IPL4 port in TCP mode, which communicates with TCP ASPs (Abstract Service Primitives); let's just call them TCP structures for simplicity.
However as users we re not interested to see TCP structures, we'd rather deal with HTTP2 structures instead; but the problem is , these structures are encoded in the payload part of TCP structures
hence they are not directly readable to us, at least not without running them through decode first.

There are several possible answers to this dilemma:

-we could rewrite the generic TCP port and create an HTTP2 test port instead; although this seems convenient, in the long term it will prove a maintenance nightmare: protocol specific test port will star to proliferate uncontrollably and you will probably end up with several test ports for different protocols, test ports sharing the largest part of their code ; so any correction in the common part will have to be propagated through all akin code. We've been through this phase and I don't recommend this to anyone.
-we could use a dual faced /translation port ; this technical possibility has been thoroughly treated previously and I believe this is the best option
-we could decode the TCP structures after receiving them and match them using an if-else or similar structure ; this is what we have done in the previous HTTP2 example:

   alt 
  { 
    [] p.receive(ASP_RecvFrom:?) -> value v_ASP_RecvFrom { 
    
     f_HTTP2_decode_frame(v_ASP_RecvFrom.msg, v_HTTP2_Frame, v_err) 

     log("**********received frame:  ***********",v_HTTP2_Frame)
     
      if (ischosen(v_HTTP2_Frame.settings_frame))
      {
     if (match(v_HTTP2_Frame,t_HTTP2_init_settings_frame_ack ) ) 
     {  
     log("**********received init_settings_frame_ack   ") 
     repeat;
     } 
     else if  (match(v_HTTP2_Frame, t_HTTP2_init_settings_frame_r))
     {  
     log("**********received init_settings_frame    ") ;
     p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_init_settings_frame_ack))));
     repeat;
     
     }
     
      }

    else  if (ischosen(v_HTTP2_Frame.header_frame))
      {
      log("**********received header frame    ") ;
        var HTTP2_header_block vl_HTTP2_header_block
    
    HTTP2_comp_context_decode(v_HTTP2_comp_context,vl_HTTP2_header_block, v_HTTP2_Frame.header_frame.header_block_fragment )
    
     log("************received header block:  ",vl_HTTP2_header_block)
     repeat;
      }

 else  if (ischosen(v_HTTP2_Frame.data_frame))
      {
           log("*************received data_frame    ") ;
      }

 else  if (ischosen(v_HTTP2_Frame.ping_frame))
      {
           log("*************received ping_frame    ") ;
      }

    };
    [] t.timeout{log("Bye")}

  }//endalt 
  



-finally we could use decmatch and @decoded, new language elements standardized recently, that try to provide an alternative to this dilemma

The usage of decmatch and @decoded can be exemplified as below:
 type MyType2 record {
 Header        header,
 octetstring   payload
 } 

 MyPort.receive(MyType2:{header := ?, payload := decmatch mw_myTemplate})  -> value (v_myVar := @decoded payload);
  // The encoded payload field of the received message is decoded and matched with
 // mw_myTemplate; if the matching is successful the decoded payload is stored in v_myVar
 



However , usage of decmatch and @decoded in Titan has a few caveats:

-first and foremost, this feature is only supported in the function ttst runtime RT2 , for reasons I hope are fairly obvious: although this mechanism is convenient, it's not the best approach for an architecture optimized for speed.

To introduce the second limitation, let's quickly recap how codecs can be invoked from within TTCN-3:
-the first option is to call the declared codec functions explicitly;
Say in the module HTTP_Types.ttcn (part of the protocol module HTTP2.0-which is an HTTP1.0/HTTP1.1 implementation !!!) the codecs are declared as:
  external function ef_HTTP_Encode(in HTTP_Message pl_pdu) return octetstring
  external function ef_HTTP_Decode(in octetstring pl_stream, in boolean pl_assemble_chunked:=true) return HTTP_Message
  

and invoked e.g. as:
  template  ASP_Send t_data1(in integer p_id ) :={
  connId:=p_id,
  proto:=omit,
  msg:=ef_HTTP_Encode(valueof(t_message("")))
}
 


-the second option is to call them implicitly, by using encvalue or decvalue; the TTCN-3 tool will infer and invoke the applicable codec ;
this assumes that the codec for the type in question is unique (else , for multiple encoding that has been standardized in 4.9.1,
the encvalue/decvalue will have to be extended with a parameter that points to the desired codec)
Note that encvalue and decvalue are using bitstring as output/input.
decmatch and @decoded use this latter logic, meaning that they invoke the codecs implicitly (and they also use bitstrings , not octetstrings)

Codec declarations are part of the specific protocol modules (and not directly part of Titan) and typically are using octetstrings as input/output.
This means that the existing protocol modules are to be complemented with additional code to make them encvalue/decvalue (and implicitly decmatch/@decoded) compatible.
Updating the already published dozens of protocol modules will take some time, but the example below will explain what needs to be done so if someone is in a hurry to deploy a protocol module with encvalue/decvalue can simply transpose the code.


Let's start with the runtime issue: as you might now, Titan contains two runtimes: one rich in features and one optimized for speed.
This practically means that during the first phase of compilation (TTCN-3 ---> C++) two different sets of code will be generated depending on the
presence/absence of the compiler flag -R. Load test runtime is the default , while function test runtime has to be triggered by using an appropriate Makefile.

So here are the steps to be taken when migrating a project from Runtime1 (load test runtime, default) to Runtime2(function test runtime):

1. add -DTITAN_RUNTIME_2 to CPPFLAGS:
# Flags for the C++ preprocessor (and makedepend as well):
CPPFLAGS = -D$(PLATFORM) -I$(TTCN3_DIR)/include -DIPL4_USE_SSL -I$(OPENSSL_DIR)/include   -DTITAN_RUNTIME_2
 

2. add -R to compiler flags:
# Flags for the TTCN-3 and ASN.1 compiler:
COMPILER_FLAGS = -L -R
 

3.change TTCN3_LIB to ttcn3-rt2 or ttcn3-rt2-parallel (from ttcn3 or ttcn3-parallel)
# Execution mode: (either ttcn3 or ttcn3-parallel)
TTCN3_LIB = ttcn3-rt2-parallel
  



Next, let's look into what's needed for encvalue/decvalue friendliness.
At this moment the HTTP2 code has not yet been updated for that in github; to make the codecs encvalue/decvalue friendly , the following additions have been made in the attached code:


in HTTP2_Types.ttcn:
//  encvalue/decvalue friendly codecs
//******************************************************************************

external function f_HTTP2_encode(in HTTP2_Frame pl_frame) return bitstring
with { extension "prototype(convert) encode(HTTP2)" }

external function f_HTTP2_decode(inout bitstring buff, out HTTP2_Frame pl_frame) return integer
with { extension "prototype(sliding) decode(HTTP2)" }
//******************************************************************************
  


and

// HTTP/2 Frame definition
// The length field handled automatically
type union HTTP2_Frame {
  HTTP2_Data_frame          data_frame,
  HTTP2_Header_frame        header_frame,
  HTTP2_Priority_frame      priority_frame,
  HTTP2_RST_Stream_frame    rst_frame,
  HTTP2_Settings_frame      settings_frame,
  HTTP2_Push_Promise_frame  push_promise_frame,
  HTTP2_Ping_frame          ping_frame,
  HTTP2_Goaway_frame        goaway_frame,
  HTTP2_Window_Update_frame window_update_frame,
  HTTP2_Continuation_frame  continuation_frame,
  HTTP2_Generic_frame       generic_frame
}
with {
 encode "HTTP2"
}
 



in HTTP2_EncDec.cc:

namespace HTTP2__Types {
// encvalue/decvalue friendly codecs

BITSTRING f__HTTP2__encode(const HTTP2__Frame& pl__frame)
{
	return oct2bit(f__HTTP2__encode__frame(pl__frame));
}



INTEGER f__HTTP2__decode(BITSTRING &buff, HTTP2__Frame &pl__frame)
{
	OCTETSTRING x(bit2oct(buff));
	HTTP2__decoder__error__descr dummy;
	INTEGER ret_val = f__HTTP2__decode__frame(x, pl__frame, dummy);
	buff = BITSTRING(0, NULL);
	return ret_val;
}

}
 



With these additions an encvalue/decvalue invocation referring to a value of type HTTP2_Frame will work properly; else the compiler will complain.
Besides the decmatch/@decoded inspired changes I have also extracted part of the code into a preamble/postamble.
So now the test case looks like this:



 //************************************************************************* 	
testcase TC_http2() runs on GeneralComp system SystemComp {
  //*************************************************************************


  f_preamble();


  //init compression context
  v_HTTP2_comp_context:=HTTP2_comp_context_init();
  // send connection preface "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"
  p.send(t_data2(v_cid,HTTP2_connection_preface));
  // send settings
  p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_init_settings_frame))));
  // send window update
  p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_window_update_frame))));

  if(not(tsp_start==upgrade_conn))
  {
    // send HEADER frame
    HTTP2_comp_context_encode(v_HTTP2_comp_context,valueof(t_HTTP2_header_block), v_data )
    //log("v_HTTP2_header_block",v_HTTP2_header_block)
    p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_header_frame(v_data))))); 
  }

  t.start(1.0) 

  alt {

    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_init_settings_frame_ack
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received init_settings_frame_ack   :",v_HTTP2_Frame)  ;repeat}

    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_init_settings_frame_r
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received init_settings_frame_r   :",v_HTTP2_Frame)  
      log("**********received init_settings_frame r   ") ;
      p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_init_settings_frame_ack))));
      repeat}

    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_header_frame0
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received header_frame   :",v_HTTP2_Frame) 

      log("**********received header frame    ") ;
      var HTTP2_header_block vl_HTTP2_header_block

      HTTP2_comp_context_decode(v_HTTP2_comp_context,vl_HTTP2_header_block, v_HTTP2_Frame.header_frame.header_block_fragment )

      log("************received header block:  ",vl_HTTP2_header_block) ;
      repeat
    }

    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_data_frame0
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received data_frame   :",v_HTTP2_Frame)  ;repeat}

    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_ping_frame
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received ping_frame   :",v_HTTP2_Frame)  ;repeat}



    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= ?
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received something   :", v_HTTP2_Frame)  ;repeat}


    [] t.timeout{log("Bye")}

  }


  HTTP2_comp_context_free(v_HTTP2_comp_context);

  //----------------------------------------------------------------------------


  f_postamble();

  setverdict(pass);

}//endtestcase
 



major difference compared to the previosu HTTP2 example being the usage of decmatch/@decoded:
 :
 alt {

    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_init_settings_frame_ack
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received init_settings_frame_ack   :",v_HTTP2_Frame)  ;repeat}
 
 :
 
  

The above snippet should be read like this:

"If the TCP ASP queued in the port contains a payload that when decode matches t_HTTP2_init_settings_frame_ack, then the ASP should be extracted from the queue, and the decoded payload deposited in
the variable v_HTTP2_Frame"


The same nghttp2.org has been used as before , with a request towards /httpbin/ip.
A browser request to this URL (https://nghttp2.org/httpbin/ip ) returns:
{
  "origin": "91.82.100.59"
}
 

The same as seen by Titan:

05:26:48.162027 HTTP2.ttcn:458 received something   :{
    data_frame := {
        stream_id := 1,
        end_stream_flag := true,
        data := '7B0A2020226F726967696E223A202239312E38322E3130302E3539220A7D0A'O ("{
  \"origin\": \"91.82.100.59\"
}
"),
        padding := omit
    }
} 
 


Code and logs attached.



I hope this will be useful if someone wishes to use decmatch/@decoded. However I'd personally prefer the translation port-based solution.


Best regards

Elemer

[Updated on: Mon, 09 October 2017 06:51]

Report message to a moderator

Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1774008 is a reply to message #1773897] Mon, 09 October 2017 07:01 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hey guys,

I have just realized that I have made a mistake in this post , namely data frames are not received where they are supposed to be received , namely here:


   [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_data_frame0
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received data_frame   :",v_HTTP2_Frame)  ;repeat}



but here:
 [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= ?
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received something   :", v_HTTP2_Frame)  ;repeat}


which reduces from the exemplifying impact of the post.

By changing

template  HTTP2_Frame t_HTTP2_data_frame0:= {
  data_frame :={
    stream_id:=1,
    end_stream_flag :=false,
    data:=?,
    padding:=omit
  }
}



to
template  HTTP2_Frame t_HTTP2_data_frame0:= {
  data_frame :={
    stream_id:=1,
    end_stream_flag :=?,
    data:=?,
    padding:=omit
  }
}



things will fall into place and "received something"


05:26:48.162027 HTTP2.ttcn:458 received something   :{
    data_frame := {
        stream_id := 1,
        end_stream_flag := true,
        data := '7B0A2020226F726967696E223A202239312E38322E3130302E3539220A7D0A'O ("{
  \"origin\": \"91.82.100.59\"
}
"),
        padding := omit
    }
} 
 


will change to "received data_frame":

23:40:20.123542 HTTP2.ttcn:371 received data_frame   :{
    data_frame := {
        stream_id := 1,
        end_stream_flag := true,
        data := '7B0A2020226F726967696E223A202239312E38322E3130302E3539220A7D0A'O ("{
  \"origin\": \"91.82.100.59\"
}
"),
        padding := omit
    }
} 



I have updated the formatted log files accordingly, not the code though, so you will have to correct yourselves if interested.

Sorry and best regards
Elemer

[Updated on: Mon, 09 October 2017 07:04]

Report message to a moderator

Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814297 is a reply to message #1774008] Mon, 09 September 2019 06:23 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

I can see the difference between the if else structure and alt step in previous post and this post.
If I have changed t_HTTP2_data_frame0.end_stream_flag :=? from false,
and I received a encoded namf JSON data,
assumed it looks just like this 7B0A20......0A7D0A'O
would that data stored in v_HTTP2_Frame by alt step
    [] p.receive(ASP_RecvFrom:{
        connId:=v_cid,
        remName:=tsp_hostname,
        remPort:=?,
        locName:=?,
        locPort:=?,
        proto:=?,
        userData:=?,
        msg:= decmatch t_HTTP2_data_frame0
      }) -> value (v_HTTP2_Frame := @decoded msg) {     log("received data_frame   :",v_HTTP2_Frame)  ;repeat}

?

And if I use corresponding json decode function (like below), could I get the json value inside "7B0A20......0A7D0A'O "?
var integer v_decodedPDU := f_dec_AmfStatusChangeNotification(v_HTTP2_Frame, t_jsonTemplate);


---update1---

Oh! I suddenly noticed that there is a t_HTTP2_data_frame(in octetstring pdu)!
So, if I want to send encoded json data, could I put this in dataframe like this?
p.send(t_data2(v_cid,f_HTTP2_encode_frame( valueof(t_HTTP2_data_frame(v_encodedPDU )))));

while the encodedPDU is :
var octetstring v_encodedPDU := f_enc_UeContextTransferReqData(valueof(t_jsonTemplate));


---update2---

You said "This data frame will have to be preceded by init frame etc." in https://www.eclipse.org/forums/index.php?t=msg&th=1089118&#msg_1814259,
I don't quite understand why,
but I came up with an idea, this is http2, then where is the method?
so I searched "GET", and I found out it was in the t_HTTP2_header_block.
Does that means the whole sending process should be
  p.send(t_data2(v_cid,HTTP2_connection_preface));
  p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_init_settings_frame))));
  p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_window_update_frame))));
  if(not(tsp_start==upgrade_conn))
  {
    HTTP2_comp_context_encode(v_HTTP2_comp_context,valueof(t_HTTP2_header_block_POST), v_data )
    p.send(t_data2(v_cid,f_HTTP2_encode_frame(valueof(t_HTTP2_header_frame(v_data))))); 
  }
  p.send(t_data2(v_cid,f_HTTP2_encode_frame( valueof(t_HTTP2_data_frame(v_encodedPDU )))));


Supposed that t_HTTP2_header_block_POST is like below:
template  HTTP2_header_block t_HTTP2_header_block:= {
  pseudo_headers :={
    method:="POST",
    scheme:="http",
    authority:=tsp_hostname,
    path:=tsp_url,
    status:=omit
  },
  headers :={{
      header_name:="accept",
      header_value:="*/*"
    },
    {
      header_name:="user-agent",
      header_value:="Titan"
    }
  }
}

while the encodedPDU is :
var octetstring v_encodedPDU := f_enc_UeContextTransferReqData(valueof(t_jsonTemplate));


Is this correct?
BTW, I searched soo long but I couldn't found any resource that I can learn how http dataframe should preceded by init frame, so if you could point out a way that would be really really amazing.
P.S. should I open a new thread when next time I have so many questions?
Would it be ok if I ask questions right under the thread about the topic (like this time)?

Best regards

Sean

[Updated on: Mon, 09 September 2019 08:24]

Report message to a moderator

Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814315 is a reply to message #1814297] Mon, 09 September 2019 10:41 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi, Sean,

Please look up and read

HTTP/2
High performance browser networking
By Ilya Grigorik(O'Reilly)

Which explains quite well messages, streams and frames in http/2

I believe an excerpt might be available for free download.

A good source of info is also if you acquire a wireshark trace.

Also I suggest you execute the attached code and look in the logs; your questions are answered there.
Also try to modify the code and see what happens.

BR
Elemer

P.S. I think it's better if you ask questions directly in the related topic.
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814317 is a reply to message #1814315] Mon, 09 September 2019 11:28 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi Sean, the online version of the book can be found at
Https://hpbn.co
See ch 12
See also http2.github.io
Http2.github.io/faq

BR
Elemer
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814362 is a reply to message #1814317] Tue, 10 September 2019 09:06 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

Thank you very much for the resource you provided, which really helps a lot for me to acquire knowledge of http2!

I have do some test by trying to modify the code like your advise.
In the result of today's experiments, I add a couple lines like below:
index.php/fa/36250/0/

and I got that packet in wireshark like this:
index.php/fa/36251/0/

I was wondering isn't that text should be encoded in to binary in http2?
But I downloaded the example http2 packets provided by wireshark,
index.php/fa/36253/0/
that means its correct to see the text I sent directly in wireshark, isn't it?

Even more, I noticed that the test server sent a json dataframe to me!
index.php/fa/36252/0/
Does that means if I import some protocol modules which encode JSON data to octestring, and sent it as data frame, I can see its been recognized as json in wireshark like this?

Thanks for your patience and time,

Best regards

Sean
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814371 is a reply to message #1814362] Tue, 10 September 2019 11:02 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi Sean, you are absolutely right:

When a TTCN-3 JSON template/variable is encoded, first the corresponding characterstring is assembled and then this is converted to octetstring , same as char2oct. So it's perfectly normal that the JSON string appears and is readable in wireshark.

Also, yes , all you need is to import one or more or all 5G JSON protocol modules and you can start talking to the other node.

Congratulations, you are a quick learner, and this subject is not the simplest one.

Best regards
Elemer
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814379 is a reply to message #1814371] Tue, 10 September 2019 13:35 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

Thanks a lot for your helps, reviews and encouragements!
I will be eternally grateful ( ͡° ͜ʖ ͡°)
Its was such a relieve when I know the http2 part is done.
Hopes that I can meet up product manager's expectation of working 5g testing suite as soon as possible.

Best regards

Sean
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814426 is a reply to message #1814379] Wed, 11 September 2019 09:49 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

What if I want to test our AMF test server on my 127.0.0.2:29518,
where can I change the port number?
I have tried many places like below:
    vl_result :=f_IPL4_connect( p, tsp_hostname, 29518,"",0, -1, {tcp := {} }, {} )

but it doesn't work.
I even tried the HTTP_Headers set in HTTP_Types , still don't have a clue.

Best regards

Sean
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814429 is a reply to message #1814426] Wed, 11 September 2019 11:00 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi Sean,

Again, the what log files are saying?
Why do you think it's the port number?
What are the other settings?

BR

Elemer
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814441 is a reply to message #1814429] Wed, 11 September 2019 13:32 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

I saw this line in mtc log file,
f__IPL4__PROVIDER__connect: :0 -> 127.0.0.2:443 / SSL

so I tried to traced it back,
there is some result in IPL4_PT.hh& .cc
and in the IPL4asp_PortType.ttcn
external function f_IPL4_connect(

looks the most similar to it.
looks like here is nothing relative in hc log file...

Best regards

Sean
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814446 is a reply to message #1814441] Wed, 11 September 2019 14:23 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi Sean,

Yes, this is an attempt to connect to port 443 over SSL/TLS. Look into wireshark to confirm.

From the log you should see the line nr. of the code where this happens and change the port nr. there.

Also, have in mind that the http2 code contains three
distinct scenarios, separated by if branches, for three different ways to establish the http2 connection.

You should chose one of them(probably alpn , which is default) and remove the other two, to simplify your code.


BR
Elemer
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814450 is a reply to message #1814446] Wed, 11 September 2019 14:43 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

Thanks for your reply,
turns out I have been too silly to forget remaking it, maybe need some redbull to stimulate my brain lol.
Port has been changed successfully by adjusting f_IPL4_connect, thanks again.
However, things would never been easy, I suddenly founds out that my colleges' AMF server only takes http"s" connection.
Gotta working on that "s" for tomorrow's progression report, they don't believe that it is not enough time to build a namf test in one week, no sleeping tonight I guess.
P.S. is this the right direction for https ?https://www.eclipse.org/forums/index.php/m/1755290/#msg_1755290

P.S.2(<- is that legit usage of P.S. ? Whatever)
Does all http2 server support all three kinds of connection establishment?
if yes, would upgrade connection be easier to make it https than http, since it's start with http/1.1 and alpn is http2?

---update---

hmm, I see, alpn is for https in http2., from here https://www.eclipse.org/forums/index.php/m/1783014/?srch=SSL#msg_1783014

Best regards

Sean

[Updated on: Wed, 11 September 2019 15:41]

Report message to a moderator

Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814455 is a reply to message #1814450] Wed, 11 September 2019 16:13 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi Sean,
Alpn is the de facto standard, but AMF guys should confirm this.

Https is already there if you chose alpn .
Then a TLS connection will automatically be initiated.

BR
Elemer
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814457 is a reply to message #1814455] Wed, 11 September 2019 16:43 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

Thanks for your reply, it is nice to hear that change to alpn would make https in default.
It would definitely saved a lot of time, maybe I could go to sleep soon.

However new problem appear and I'm working on this connection error.
09:34:54.227645 HTTP2.ttcn:260 entering f__IPL4__PROVIDER__connect: :0 -> nghttp2.org:80 / SSL
09:34:55.122063 HTTP2.ttcn:265 connect result{ errorCode := ERROR_SOCKET (4), connId := omit, os_error_code := 0, os_error_text := "Success" }

Gotta trace this connect result.

Best regards

Sean
Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814459 is a reply to message #1814457] Wed, 11 September 2019 16:58 Go to previous messageGo to next message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi Sean, you are trying to connect to port 80 and there is no reply.
You sure 80 is OK?

[Updated on: Wed, 11 September 2019 16:59]

Report message to a moderator

Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814461 is a reply to message #1814459] Wed, 11 September 2019 17:06 Go to previous messageGo to next message
Sean Kuo is currently offline Sean KuoFriend
Messages: 28
Registered: August 2019
Junior Member
Hi Elemer,

Yeah, I added this HTTP2.tsp_port :=80 in cfg and in HTTP2.ttcn, because the port was 80 in upgrade conn...
oh wait, maybe I should try 443?

---update---

Yeah, 443 works.

---update 2---

Seems like https would encrypted everythings that even wire shark could not know what it is?
index.php/fa/36271/0/
However I can know that frame have been successfully sent and receive in log files

---update 3---

Its really weird that when I change port to 29513, connection successfully datafrme too,
but my AMF server shows that he diddn't receive any thing.
But if I used POSTMAN to send message, it could react properly,
what ever this problem is, I gotta go to sleep first.
Too damn tired.

---update 4---

I think I found the problem,
it is POSTMAN on the left and ttcn on the right,
index.php/fa/36272/0/
when in ttcn, my port should be TCPPort := 8035 in cfg.
but obviously it is not.
Even more, it changes every time.
index.php/fa/36273/0/

---update 5---

No I don't think port is the problem now.
Just came up with one Idea that I should disable ssl certification in postman to make our server response,
so I change ssl to tcp:
index.php/fa/36274/0/
wow server finally showing something.
now, how to make that looks like tls handshake?

---update 6---

That is not tcp, it should be ssl, but I should set *.p.ssl_verify_certificate := "No" in cfg, now the handshake looks right.

Now the problem is, if I sent request to different URI, the server can response a 400 to ttcn, however, nothing happened when /namf-comm/v1/ue-contexts/456/transfer.
Turns out it is because I didn't set content type to server.
How to set content type in http2?

---update 7---

Look what I found here!

https://http2.github.io/http2-spec/

An HTTP POST request that includes request header fields and payload data is transmitted as one HEADERS frame, followed by zero or more CONTINUATION frames containing the request header fields, followed by one or more DATA frames, with the last CONTINUATION (or HEADERS) frame having the END_HEADERS flag set and the final DATA frame having the END_STREAM flag set:

  POST /resource HTTP/1.1          HEADERS
  Host: example.org          ==>     - END_STREAM
  Content-Type: image/jpeg           - END_HEADERS
  Content-Length: 123                  :method = POST
                                       :path = /resource
  {binary data}                        :scheme = https

                                   CONTINUATION
                                     + END_HEADERS
                                       content-type = image/jpeg
                                       host = example.org
                                       content-length = 123

                                   DATA
                                     + END_STREAM
                                   {binary data}


And, there is a really beautiful continuation frame in http2_types!
Does that means I have to send a continuation frame to tell the server what the content type is?

---update 8---

Fortunately, by tracing back to http2_types, content-type was successfully sent after I programmed a template and a continuation header block for continuation frame, and I could see that beautiful application/json on the server.
but I'm still curious about why port works in upgrade connection,
but refused in alpn? and why 443 working?
is that the setting of server made this?

---udate 9---

What a silly question this is lol
Found out these ports are reserved for some purpose.
Seriously lack of these basic network knowledge.

Best regards

Sean

[Updated on: Thu, 12 September 2019 12:12]

Report message to a moderator

Re: Support for HTTP/2 in Eclipse Titan- with a twist [message #1814547 is a reply to message #1814461] Fri, 13 September 2019 04:27 Go to previous message
Elemer Lelik is currently offline Elemer LelikFriend
Messages: 1120
Registered: January 2015
Senior Member
Hi Sean,

I could not answer your questions any better than you do;
The answers require knowledge of server config specifics.

BR
Elemer
Previous Topic:How to reference union value in another module?
Next Topic:import error while module exists
Goto Forum:
  


Current Time: Sat Jun 19 16:53:59 GMT 2021

Powered by FUDForum. Page generated in 0.02160 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top