Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cf-dev] Plans regarding CoAP over TCP?
No, I ended up using a later version of Netty, which simplified some things (it supports connection pools).


On Thu, Jun 23, 2016 at 2:45 PM Simon Lemay <simon.lemay@xxxxxxxxx> wrote:
Awesome, ill try and find some time to look the PR over, its a big one!!

did you end up using my branch?

Cheers

Simon



On Tue, Jun 14, 2016 at 10:09 AM Joe Magerramov <joe.magerramov@xxxxxxxxx> wrote:

On Wed, Jun 22, 2016 at 7:44 AM Simon Lemay <simon.lemay@xxxxxxxxx> wrote:
Awesome, yes of course


let me know if you have any question.  You might need to redo the length framing to reflect the new draft, but i guess that can also wait since it wont matter for a quick end to end test.

On Mon, Jun 20, 2016 at 7:09 PM Joe Magerramov <joe.magerramov@xxxxxxxxx> wrote:
I have been experimenting with changes to CoapStack and Matcher to make this work. I can clean those up and send out a PR.

Simon, if you send me the link to your branch I can pull your connector and run some end-to-end tests against the whole thing.

-Joe

On Sun, Jun 19, 2016 at 4:00 AM Kai <sophokles.kh@xxxxxxxxx> wrote:
Sounds good to me, Joe. Thanks for taking such an interest :-)


On Sat, Jun 18, 2016 at 9:57 PM Joe Magerramov <joe.magerramov@xxxxxxxxx> wrote:
I started looking at what changes to CoapEndpoint would need to happen. Here's what I see:

* Use different Matcher that ignores Mids and only matches on tokens.
* Use different message Serializer
* Use Stack that skips reliability layer. 
* Somehow allow Connector to cancel all exchanges for specified remote endpoint when TCP connection drops.

Yes, the Matcher will probably the part that will differ most, in fact  I guess it will become much simpler (which is good).
 
We could create a different endpoint implementation, but it'll likely have 90% in common with the original CoapEndpoint (could introduce abstract superclass to avoid duplication). An alternative approach would be to introduce "USE_RELIABLE_TRANSPORT" NetworkConfig property. If set:

Your're probably right about CoapEndpoint itself whose main purpose is to assemble the CoapStack. In case of TCP the stack looks differently as you figured out ...
 
* CoapEndpoint initializes TCP/TLS connector instead of UDP.
* Coap endpoint initializes MID-less matcher (we'll need to extract Matcher interface and have 2 implementations)
* Coap endpoint initializes different serializer (we'll need to introduce Serializer interface and have 2 implementations)
* Coap endpoitn passes Matcher to TcpConnector, so latter can invoke Matcher.cancelAllForRemoteEndpoint(ip/port pair) method when connection drops.
 
In this approach, the CoapConnector stays exactly the same as today (there are really no changes to it needed), but we change its behavior via different components (Matcher / Serliazer).

My preference is towards the latter method, it keeps CoapEndpoint clean, but for a single if in the constructor to inject different implementation of key interfaces (Matcher and Serializer). And that pattern is already used in other places within Californium. But I don't feel super strongly about that.

Thoughts?

Looks good to me as a first approach. Would you like to give it a shot and create a PR based on master? I think it would be helpful to try to use Simon's TCP Connector implementation ...

@Simon: what do you think, could it work the way Joe proposed?

Kai

On Wed, Jun 15, 2016 at 12:51 PM Simon Lemay <simon.lemay@xxxxxxxxx> wrote:
haha, yes sadly this means I don't get as much time on it but my interest is still there :D

Yeah, I am up to try it like that, I guess in the mean time I was going to create a different matcher and mod the new header semantic there.  I guess it was making it easier as well because of how Netty works, and how easy is it to add any length shim to the decoding and encoding and then pass the payload to the Coap decoder.

In the end a builder could have bound everything together for an easy setup.

The question is what is the intent with element connector.  If we want to limit the dev there, then Netty component could be moved to Californium and have its logic in a CoAPTCPEndpoint.

is that what you meant? 



On Tue, Jun 14, 2016 at 10:09 AM Hudalla Kai (INST/ESY1) <Kai.Hudalla@xxxxxxxxxxxx> wrote:
Hi Simon,

it's great to hear from you again :-) We already worried that you lost
interest in Californium altogether now that Zebra has moved to mbed ...

We are currently heavily struggling with supporting horizontal scale-
out and fail-over of observations with CoAP/UDP. This seems to be very
hard to do in NATed environments so I guess having CoAP/TCP would help
a lot in such scenarios.

I have taken a look at your TCP branch of Californium. I was a little
surprised to see that you implemented TCP at the Connector level. Given
that the encoding of the CoAP messages on the wire is different for UDP
and TCP, I was expecting to see a separate Endpoint implementation for
TCP because 
a) the Connector API uses raw byte arrays containing the
(already/still) encoded CoAP messages sent to/received from the network
in contrast to the Endpoint's Request/Response based API and
b) some of the layers in CoAPEndpoint are not necessary for TCP

So I thought that it is probably not worth the effort to add code for
distinguishing UDP and TCP to CoapEndpoint but instead create a clean &
lean CoapTcpEndpoint instead.

Any thoughts?

--
Mit freundlichen Grüßen / Best regards

Kai Hudalla
Chief Software Architect

Bosch Software Innovations GmbH
Schöneberger Ufer 89-91
10785 Berlin
GERMANY
www.bosch-si.com

Registered office: Berlin, Register court: Amtsgericht Charlottenburg,
HRB 148411 B;
Executives: Dr.-Ing. Rainer Kallenbach, Michael Hahn


On Do, 2016-06-09 at 18:58 +0000, Simon Lemay wrote:
> I had started to reimplement TCP under my GitHub account (Areontar),
> I have a branch that should have the simplistic (just a basic TCP
> connector with CoAP/UDP sematic) version working, but there still a
> lot to be done.  I am still locked in on another project for at least
> 1 week and half, but I have been poking at it when I have time.  I
> use Netty for the connector at the moment and still use the same
> interface.  I should get back on it in 2 weeks or so as we are
> progressing on the draft
>
> We had a lot of discussion in the pass around what needs to be done
> to make it work.  As Kai mentioned, I think the consensus was to have
> this done in CF2.0 since it would most likely break backward
> compatibility. 
>
> I would be more than happy to reboot this in about 2 weeks, we can
> get on a hangout with whoever is interested and plan out what we
> need.  What do you guys think?  might be a good time if we can get
> hands on deck and might help resolve some issue on the draft
>
> cheers,
> Simon
>
>
> On Wed, Jun 8, 2016 at 9:17 AM Hudalla Kai (INST/ESY1) <Kai.Hudalla@b
> osch-si.com> wrote:
> > From my point of view it doesn't really make sense to make the
> > CoAPEndpoint distinguish between different transports but instead I
> > think we should create a separate Endpoint implementation, e.g.
> > CoapTcpEndpoint, that then simply does what needs to be done when
> > using
> > TCP (which is a lot less as for running CoAP over UDP as you
> > already
> > figured out).
> >
> > AFAIK the most important thing will be to get the Endpoint
> > interface
> > right so that it can support exchanging CoAP messages over
> > arbitrary
> > transports. This might include adding the possibility to add
> > callbacks
> > for getting notified when a connection is established or terminated
> > etc).
> >
> > I am currently more concerned with the Endpoint API and how to make
> > it
> > better support message exchanges in a more "generic" way so that it
> > can
> > work with generic handlers for incoming responses instead of
> > requiring
> > individual handlers for every single request being sent. This will
> > be
> > important for supporting fail-over of observations between nodes.
> > For
> > this scenario, CoAPClient is not really relevant because it will
> > always
> > run in the same process as the Endpoint it uses to interact with
> > the
> > CoAP server, i.e. when the node it runs on crashes there is no need
> > to
> > fail over anything.
> >   
> > --
> > Mit freundlichen Grüßen / Best regards
> >
> > Kai Hudalla
> > Chief Software Architect
> >
> > Bosch Software Innovations GmbH
> > Schöneberger Ufer 89-91
> > 10785 Berlin
> > GERMANY
> > www.bosch-si.com
> >
> > Registered office: Berlin, Register court: Amtsgericht
> > Charlottenburg,
> > HRB 148411 B;
> > Executives: Dr.-Ing. Rainer Kallenbach, Michael Hahn
> >
> >
> > On Mi, 2016-06-08 at 00:12 +0000, Joe Magerramov wrote:
> > > Looking at the spec[1], I'm trying to understand changes to
> > > Californium. On the surface looks like we'll need:
> > >
> > > * A new TCP/TLS connector (it'll need to do non-blocking IO in
> > > receive thread)
> > > * Chagnes to CoAPEndpiont, that when using TCP connector will:
> > > *** Use new data serializer
> > > *** Instruct matcher to skip using MID hashmap puts/gets
> > > *** Skip reliability layer
> > > *** Have a callback from the Connector to cancel all open
> > exchanges
> > > when a TCP connection is dropped
> > >
> > > There's also a question of what to do with Type (CON vs NON) on
> > the
> > > CoapClient APIs. Trivial option just keep them as is, and ignore
> > when
> > > using TCP connector. Less trivial option is to change APIs
> > (how?).
> > >
> > > I'm sure there's something else I'm missing. Thoughts?
> > >
> > > [1] https://datatracker.ietf.org/doc/draft-ietf-core-coap-tcp-tls
> > /?in
> > > clude_text=1
> > >
> > >
> > > On Mon, Jun 6, 2016 at 11:57 PM Kai <sophokles.kh@xxxxxxxxx>
> > wrote:
> > > > Not that I am aware of... As Matthias mentioned, Simon Lemay
> > had
> > > > plans for this but it seems priorities have been changed. Maybe
> > you
> > > > want to spark the discussion and get involved? There is
> > definitely
> > > > some interest in it (at least from Bosch), but we are currently
> > > > more focused on adding support for horizontal scalability to
> > > > Californium. TCP support would be something for Cf 2.0, I guess
> > > > since it would probably require some changes to public APIs...
> > > > Kai
> > > >
> > > > Joe Magerramov <joe.magerramov@xxxxxxxxx> schrieb am Di., 7.
> > Juni
> > > > 2016, 04:23:
> > > > > Did not see any e-mail threads on TCP/TLS support since
> > April.
> > > > > Did anybody ever start working on this?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Joe
> > > > >
> > > > > On Fri, Apr 22, 2016 at 3:28 AM Kovatsch Matthias <kovatsch@i
> > nf.e
> > > > > thz.ch> wrote:
> > > > > > Simon (Lemay) had to push this work over to his free time.
> > > > > > Myself not having much time lately does not help either :/
> > > > > > Still, we will need something running for the IETF, so I
> > have
> > > > > > not lost all hope yet :)
> > > > > >  
> > > > > > Ciao
> > > > > > Matthias
> > > > > >  
> > > > > >  
> > > > > > From: cf-dev-bounces@xxxxxxxxxxx [mailto:cf-dev-bounces@ecl
> > ipse
> > > > > > .org] On Behalf Of Benjamin Cabé
> > > > > > Sent: Montag, 4. April 2016 15:54
> > > > > > To: Californium (Cf) developer discussions <cf-dev@eclipse.
> > org>
> > > > > > Subject: Re: [cf-dev] Plans regarding CoAP over TCP?
> > > > > >  
> > > > > > Ping? :-)
> > > > > > I'm just curious whether it's in the roadmap to provide a
> > TCP
> > > > > > binding in Californium?
> > > > > >  
> > > > > > Thanks!
> > > > > > Benjamin –
> > > > > >  
> > > > > >  
> > > > > > De : Benjamin Cabé <benjamin@xxxxxxxxxxx>
> > > > > > Date : jeudi 4 février 2016 14:37
> > > > > > À : "Californium (Cf) developer discussions" <cf-dev@eclips
> > e.or
> > > > > > g>
> > > > > > Objet : Plans regarding CoAP over TCP?
> > > > > >  
> > > > > > Hi, Californium gang!
> > > > > >  
> > > > > > It looks like the Internet-Draft about CoAP over TCP/TLS
> > [1] is
> > > > > > making good progress, and I was wondering about your
> > views/plan
> > > > > > on making this available in Californium? 
> > > > > > I believe the Zatar guys have worked on a fork of
> > Californium
> > > > > > [2] that does just that since IIRC they are experimenting
> > with
> > > > > > LWM2M over CoAP-TCP.
> > > > > >  
> > > > > > FWIW I get asked very often about CoAP over TCP, as many
> > people
> > > > > > do see lots of value in CoAP itself, but would really like
> > to
> > > > > > see it be more IT/cloud friendly.
> > > > > >  
> > > > > > Looking forward to getting your feedback,
> > > > > > Thanks!
> > > > > > Benjamin –
> > > > > >  
> > > > > > [1] https://datatracker.ietf.org/doc/draft-ietf-core-coap-t
> > cp-t
> > > > > > ls/
> > > > > > [2] https://github.com/zatar-iot/californium/tree/tcp_tls_i
> > mpl
> > > > > >  
> > > > > > Benjamin Cabé – IoT Evangelist
> > > > > >
> > > > > >
> > > > > >
> > > > > > Eclipse Foundation
> > > > > > +33 (0) 619196101
> > > > > > @kartben
> > > > > > _______________________________________________
> > > > > > cf-dev mailing list
> > > > > > cf-dev@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/cf-dev
> > > > > _______________________________________________
> > > > > cf-dev mailing list
> > > > > cf-dev@xxxxxxxxxxx
> > > > > To change your delivery options, retrieve your password, or
> > > > > unsubscribe from this list, visit
> > > > > https://dev.eclipse.org/mailman/listinfo/cf-dev
> > > > _______________________________________________
> > > > cf-dev mailing list
> > > > cf-dev@xxxxxxxxxxx
> > > > To change your delivery options, retrieve your password, or
> > > > unsubscribe from this list, visit
> > > > https://dev.eclipse.org/mailman/listinfo/cf-dev
> > > _______________________________________________
> > > cf-dev mailing list
> > > cf-dev@xxxxxxxxxxx
> > > To change your delivery options, retrieve your password, or
> > > unsubscribe from this list, visit
> > > https://dev.eclipse.org/mailman/listinfo/cf-dev
> > _______________________________________________
> > cf-dev mailing list
> > cf-dev@xxxxxxxxxxx
> > To change your delivery options, retrieve your password, or
> > unsubscribe from this list, visit
> > https://dev.eclipse.org/mailman/listinfo/cf-dev
> _______________________________________________
> cf-dev mailing list
> cf-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cf-dev

Back to the top