Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » Log FileMask / ConsoleMask filter by component type?
Log FileMask / ConsoleMask filter by component type? [message #1837653] Wed, 03 February 2021 16:41 Go to next message
Harald Welte is currently offline Harald WelteFriend
Messages: 128
Registered: July 2017
Location: Berlin, Germany
Senior Member

I understand it is possible to configure the log verbosity ("log filtering") based on the name of a component (at time of the create), as explained in the Reference Guide under "Component and Plugin Based Filtering"

However, there doesn't seem to be any documented option to do the same based on the component type, or possibly based on wildcard/regex matching of the name given in crate().

I wonder how other people deal with this. Let's say you start one hundred components of a certain type, and you want to make sure to enable or disable certain log flags for all of those components?

Adding hundreds of different individual component names to the config file doesn't seem like a very elegant or scalable approach.

As always, any help is appreciated.
Re: Log FileMask / ConsoleMask filter by component type? [message #1837658 is a reply to message #1837653] Wed, 03 February 2021 17:56 Go to previous messageGo to next message
roland gecse is currently offline roland gecseFriend
Messages: 20
Registered: December 2015
Junior Member
I am afraid create() does not offer the wanted functionality. I hope current TITAN designers can correct me if I am wrong.

Here's a workaround. Configure logging so that each component has its own log file. Ensure that the filename includes the component type identifier (%t in LogFile). Use ttcn3_logfilter to postprocess logfiles along with ttcn3_logmerge if necessary. You may need to do some scripting but I believe this could do the trick.
Alternatively, you can request this new feature or develop and contribute it yourself.
Writing your own logger plugin may also work but its probably an overkill.

How do other people deal with this?

I don't believe that creating hundreds of components is a common use case. It can be done--we went up to 32k. We saw it was not so effective mainly due to UNIX kernel performance deficiencies related to handling of so many processes (which may have been partly fixed ever since). In our experience, for such performance testing use cases, it turned out to be more efficient to realize parallelism by interleaving multiple components' behavior into a single component. Doing so we can execute thousands of state machines within a single component. My colleagues implemented a complete framework around the concept. Parts of it are available in Core Load Library (CLL)!
Re: Log FileMask / ConsoleMask filter by component type? [message #1837690 is a reply to message #1837658] Thu, 04 February 2021 09:10 Go to previous messageGo to next message
Harald Welte is currently offline Harald WelteFriend
Messages: 128
Registered: July 2017
Location: Berlin, Germany
Senior Member

Hi Roland,

thanks for your response.

"create()" of course just allows you to specify the identity of the component, and that should of course be unique, otherwise it doesn't serve much purpose. Sure, it is not mandatory for the name to be unique (If I remember correctly, but what point does a name have if it doesn't uniquely identify a component.

The feature I'm talking about is more one in terms of the config file parsing, specifically the LogFilter / ConsoleFilter syntax. If one could specify the Component Type there, it would be very easy to configure the desired scenario.

We already have per-component log files. The point is to not even generate tons of verbose logs unless it is needed. So, e.g. by default I would want to see TIMEROP_START logging of all my components, but I want to explicitly disable that for all components of one type, as that component is constantly starting very short timers of 5ms, and that creates lots of useless log (filling disk, costing CPU resources to print, ...).

Running hundreds of components works generally well in my experience.. This is not a performance use case, but relatively simple "functional" testing. We (at osmocom.org) typically have one component for every simulated UE, and then some other components to multiplex.

For example on the Gb interface in GPRS, the NS stack already has one component per NS-VC, and then one component as NS_Emulation for each NSE. On top of that, there is the BSSGP_Emulation with one shared + one component per BVC, and then you have one component for each of the UEs on top of each PTP-BVC component. Modern CPUs, particularly those on servers usd for CI or build purposes have many cores and even more threads, so parallel execution of many processes (= components) is not a problem. Furthermore, TITAN can even scale out and control components on other nodes/hosts, so scaling that way is not a problem.

Sure, things can always be done differently, but the strength of TTCN3 is in its programming model resembling an "actor" mdoel with incoming and outgoing ports, related queues, etc. - If I wanted to program in asynchronous state machines within a single component/thread/..., I would write it in C, not in TTCN3 ;) - this is how all of our actual IUT are developed. Also, it would be a rather strange motivation to let your software architecture be guided by how the logging system can or cannot be configured.

Re: Log FileMask / ConsoleMask filter by component type? [message #1837698 is a reply to message #1837690] Thu, 04 February 2021 10:57 Go to previous messageGo to next message
roland gecse is currently offline roland gecseFriend
Messages: 20
Registered: December 2015
Junior Member
Perhaps a bit off-topic, but I cannot resist to answer:

> Running hundreds of components works generally well in my experience.
I am happy that you are satisfied with our design/implementation.

> Modern CPUs, particularly those on servers usd for CI or build purposes have many cores and even more threads, so parallel execution of many processes (= components) is not a problem.
Lucky you! When we run some traffic tests with TITAN we need to simulate millions of UEs. 1 PTC/UE will not do. We need a rack of high end multi processor, multi core servers to do it.

> Furthermore, TITAN can even scale out and control components on other nodes/hosts, so scaling that way is not a problem.
I know. Distributed execution over heterogeneous environment feature was kind of my idea :)

> Sure, things can always be done differently, but the strength of TTCN3 is in its programming model resembling an "actor" mdoel with incoming and outgoing ports, related queues, etc.
I am well aware.

> If I wanted to program in asynchronous state machines within a single component/thread/..., I would write it in C, not in TTCN3 ;)
Sure you could. The nice thing is that you can combine both languages with TITAN. You could use e.g. abstract communication primitives, type-value-template system of TTCN-3 along with more performance C/C++ behavior.

End of off-topic.

W.r.t. the original question I am sorry but we didn't prepare for your UC when doing the original design. But improvement ideas are always welcome. I really encourage you to submit a new feature request or write and contribute the necessary code changes yourself!
Re: Log FileMask / ConsoleMask filter by component type? [message #1837713 is a reply to message #1837698] Thu, 04 February 2021 14:20 Go to previous message
Gábor Szalai is currently offline Gábor SzalaiFriend
Messages: 86
Registered: December 2015
Member
Some of the logging option can be configured during run time:
See the 7.13 chapter of the Reference guide.

Currently the wildcard/regexp of the component specification is not supported, but we're discussing what we can do.
Previous Topic:Strategies for shutdding down complex component hierarchy
Next Topic:use of select() and file-descriptor limit
Goto Forum:
  


Current Time: Tue Jun 15 15:23:49 GMT 2021

Powered by FUDForum. Page generated in 0.01979 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top