Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Titan » Titan Architecture Internals: On where things are located in the source code(Some background information that is important to know when developing Titan)
Titan Architecture Internals: On where things are located in the source code [message #1748477] Wed, 23 November 2016 13:29
Kristof Szabados is currently offline Kristof SzabadosFriend
Messages: 79
Registered: July 2009
Member
People finding the Titan source codes for the first time usually have the same question: Where is what?

Let me give you a high level overview, on where different (bigger parts of the architecture/system) are located in the file/package system.
You will see, that once you have the general idea ... the system will be very simple.
/* In a follow up posts I will also write about the static structure of classes, how information is flowing and special interest items ... but right now I just wish to concentrate on this aspect */

I will show both the command line (titan.core) and eclipse (titan.EclipsePlug-ins) architecture.
As there are several operations that both of them do (but in different ways) it is very important to keep them in sync.


The command line side (titan.core)

The compiler2 folder holds the titan compiler, that checks and translates TTCN-3 and ASN.1 codes to C/C++.
- the asn1 subfolder holds the ASN.1 specific elements and lexer/parser files (asn1la.l, asn1p.y)
- the ttcn3 subfolder holds the ttcn-3 specific elements and lexer/parser files (compiler.l, compiler.y)
- The machinery that is shared/used by both the ASN.1 and TTCN-3 sides are found directly in the compiler2 folder.
These include: the generic AST (semantic tree) nodes, code generation, type and value handling, etc..
Please note here the main.hh/cc files which serve as the entry point for the compiler.

The core and core2 folders hold most of the content delivered in the Titan libraries.
Core was the first and holds files for the load test runtime.
The name of the files describes nicely what they contain, for example: Boolean.{hh/cc} is how the boolean type is represented in the runtime, BitString.{hh/cc} is how bitstrings are represented in the runtime.
Core also holds the Single_main.cc and the Parallel_main.cc files. Single_main.cc is the entry point for the generated binary when compiled in single mode, Parallel_main.cc holds the entry point for the binary when generated in parallel mode (and will connect to the main controller).
Core2 was added later to hold files for the function test runtime.
But please note, that most of the classes for built-in types are actually located in the Core folder, and only their function test runtime specific parts are generated into the core2 folder when Titan is compiled.

The regression_test and function_test folders hold most of our tests.
Both are designed to contain subfolders that test one specific language element or concept. For example: the regression_test/intOper tests all kinds of operations possible with integers in TTCN-3, function_test/RAW tests that different types and values are encoded/decoded correctly by the RAW encoder.
There are 2 important points to notice here:
1)
It is possible to run these tests in bulk using the make run command in their base folder ... and it is also possible to run the test separately using the make run command in each test's own folder.
This allows for us to easily run the complete test suites, and when investigating something, to iterate quickly on the important targets only.

2)
By the ISTQB naming conventions both the regression_test and the function_test folders actually hold regression tests.
Originally these folders were separated based on the time needed to run the tests: usually the regression_tests finished quickly, while each test in the function_test folder could run for 1+ hour.

The hello folder holds the hello world example code.
It might not be obvious at first, but this is one of the most important tests, so it is done at the end of build process.
Although it sounds very simple, if the hello world example runs, that shows: the compiler can read, check and convert TTCN-3 files + the generated code can be compiled into binary (has no big C/C++ issues), the binary can execute (the libraries were properly linked to the generated code) + testcases, components, altsteps, waiting for input works + the testcase verdicts can be collected.
Or in short there is no very big issue with the code.

The mctr2 folder holds the code of the command line main controller, to which the generated binaries connect in parallel build mode.
The JNI folder holds the code of the JNI interface of the main controller, through which the Executor plug-in's JNI execution mode can directly connect to titan's main controller engine.

The common folder holds several items that are used from multiple places in the system.
For example:
- version.h is used to set the version number of the binaries.
- memory.{h/c} is our debug mode memory management, that can help us find memory leaks, memory overlappings, etc..
- license handling, hostid collection, etc...

usrguide holds the user guides.
help holds the files for a web page based help system.
xsdconvert holds the xsd converter.


The eclipse side (titan.EclipsePlug-ins)

To understand why we have several eclipse plug-ins it is good to understand how they can be used and how they came to be.

The 5 starting plug-ins:
- The Designer is meant for developing TTCN-3 and ASN.1 code in an integrated development environment.
As such internally it is usually used in development teams, on machines with enough memory, to help efficient test creation/maintenance.

- The executor is meant for directing/controlling the execution of tests.
Internally these test are usually run in networks that are separated from the usual work environments.
If it would escape, the load generated for load testing, could make the usage of the normal work networks impossible.
During negative testing we intentionally generate incorrect messages ... that could reconfigure networks or even crash unprepared devices.
In these cases the machine doing the coordination does not need to be very strong, have much memory or disc space ... but the ones it controls might be.

- The LogViewer is meant for processing large log files after the test have executed.
As this can mean hundreds of gigabytes of data, plenty of disc space is advised

- The regressiontests plug-in contains regression tests for the semantic analysis.
- The help plug-in provides the documentation and help pages in eclipse.
Please note they are not stored here directly, but generated to here during build.

This was later extended with a few more plug-ins:
- the Common plug-in holds code, that is used by 2 or more other plug-ins.
For example configuration file processing needs to be done by the Designer and the Executor too, using the exact same grammar.

- The Titanium plug-in is offering static code quality analysis, metrics measurement and architecture visualization, based on the semantic database built up by the Designer plug-in.

In more detail:

General rules for all plug-ins:

- Please note that none of the eclipse plug-ins has a main function.
All functionalities are defined using the plugin.xml files in the plug-ins, where we register which class extends which eclipse provided interface. This file is always located in the root folder of all plug-ins.

- the preference settings are declared and initialized in each package in the org.eclipse.titan.<pluginname>.preferences package.
The preference pages to change these setting are located in the pages subfolder.

- If the plug-in defines any actions that can be invoked by menu, or toolbar button they are located in the org.eclipse.titan.<pluginname>.actions package of the plugin.

- if the plug-in offers headless mode commands they can be found in the org.eclipse.titan.<pluginname>.application packages

The Common plug-in (inside the org.eclipse.titan.common package):

- the parsers package (and actually it's cfg subpackage) contains the configuration file parser and processing
- the path package contains path conversion functionality we use to convert between windows-linux, IPath and URI objects
- the log package contains the logmerger and log formatter utilities (in their sub packages)
- the usagestats package contains the code we use to get some information on how our tools are used.
This is only used internally within Ericsson and only to check if we still need to support certain old Java/eclipse versions. Just like our internal license checking the code is available in open source, but is not active in the default open source builds.
- The utils package contains some nice functions that we use to simplify the code.

The Designer plug-in (inside the org.eclipse.titan.designer package):

- The parsers package holds all parsers (.g4 files) for all supported formats, in their own sub-packages: ttcn3parser for ttcn3 files, asn1parser for ASN.1 files, extensionattributeparser for the parsing of extension attributes and pre-process for pre-processing ttcnpp files.
The base folder also holds the:
- GlobalParser.java: which can be used to gain access of all semantic information parsed from projects
- ProjectSourceParser.java: is the class responsible for handling the general parsing of a single project.
- ProjectSourceSyntacticAnalyzer.java: is responsible for directing of the syntactic analysis part of a project's parsing process.
- ProjectSourceSemanticAnalyzer.java: is responsible for the directing of the semantic analysis part of a project (and referenced project's) semantic process.
In each semantic check, we first do a syntactic check of all new/changed files, and when this is ready do the semantic checking of all the known information.

- AST stands for Abstract Syntax Tree and this package holds all classes that are used to describe semantic meaning parsed from the source codes.
- The TTCN3 subpackage holds TTCN-3 and ASN.1 related classes separated into definitions, statements, templates, types, values and expressions.
Expressions are found within the value package as in this representation expressions are also values.
- The ASN1 package holds ASN.1 specific entities separated into definitions, types, values, objects.
Please note that types/values that are not different from their TTCN-3 counterpart are not replicated here. In these cases we use their TTCN-3 counterpart to represent ASN.1 entities as well.

- The core folder holds support for driving the build process and redirecting command line generated error markers to the eclipse problems view.
- The makefile subpackage holds the code for the makefile generation.

- The editors package holds the supporting codes for all provided editors, in their sub packages: asn1editor for ASN.1, ttcn3editor, for TTCN-3, configeditor for configuration files, ttcnppeditor for pre-processable files.
- within each of these packages the actions subpackage holds support for all menu and keyboard actions we extend eclipse with.

- The properties package holds support for all project/folder/file related settings (properties in eclipse).
- The wizards package folds the classes for the provided wizards (new project, ASN.1/TTCN-3/config file) and project importation support.
- The decorators package holds the classes the decorate the eclipse navigators with information on project build setting and exclusion information.
- The consoles package holds support for the TITAN console and TITAN debug console ... which we use to show build or debug related information.

- The extension package holds the description of the extension point the Designer defines.
Through this it is possible for external plug-ins to build upon the Designer and be called after each semantic checking is run to provide additional checks/features.

The Executor plug-in (inside the org.eclipse.titan.executor package):

- The views package describes all three (executor Monitor, Notification, Test execution) views that can be used to drive and observe execution.

- The tabpages package defines all of the tab pages seen on the LaunchConfiguration wizards provided by the Executor.
/* they are used from the specific executors packages */

- The executors package holds support for provided execution modes.
- In the general package:
- the BaseExecutor.java file holds the common operations of all execution modes.
- the LaunchShortcut.java and LaunchShortcutConfig.java files are bases of how to handle when the user uses the "run" menu while selecting a project or a configuration file in a eclipse navigator view.
- the jni,mctr.cli and single packages hold the specialized versions of the general package's classes for the needs of the different execution modes.
The mctr.cli mode communicates with an externally executed mctr_cli main controller via command line.
While the JNI mode is able to connect to a main controller directly, offering more options ... but being limited to a single instance at any time.
The single mode drives binaries compiled in single mode, that do not need to be connected to parallel components.

- The jni and jni.adapter packages hold the necessary files for the JNI mode executor to handle the direct connection to a binary main controller.

The Log Viewer plug-in (inside the org.eclipse.titan.log.viewer package):

- The parsers package holds the functionalities for processing the content of single log record in the log file.
- The readers package holds support for reading all the records from a log file, using the parsers package.
- The extractors package extracts indexing information from log files.
For example what testcases were executed and at which log record they start.
- The models package holds the classes that are used to describe log files and log records.

- If you are interested in the graphical elements you should look at the views package.
All of the navigator extensions, the text table, the msc (Message Sequence Chart) and detail view is located here (in packages of similar names).

The Titanium plug-in (inside org.eclipse.titanium package):

Titanium is a fairly new addition to our toolset.
- All code smell detection related code is in the markers package
- spotters and spotters.implementation holds all of the available code smell spotters that Titanium offers currently
- the export package handles all code smell result exporting functionality to the different file formats (+ merge in case of xls files)
- the utils package holds the ExecuteAnalyer.java file. This is the class that is actually extending the designer provided extension point. So when it is enabled, every time the semantic analyser finishes analysing the code, this will be used to update the list of detected code smells.

- All metric measurement related functionality is located in the metrics package.
The visitors that provide the framework for measurement, the actual implementations, the view and topview displays ... all in packages of the same name.

- All architecture visualization related code is contained in the graph package.
Here gui holds all the user interface related menus, layouts, dialogs, windows (the main and satellite views) in packages of the same name.
The generators package holds the code for extracting architectural information from the Designer semantic data and generating the graph.
And the clustering package is responsible for all the code that we use to cluster nodes together on the user interface (thus generating new graphs from them)

- The last package is the organize package, which holds the code for the organize imports feature.


====

This was a long post and I hope by now the logic between separating code into packages and naming those packages can be seen.

Please note, that this is only a depiction of the current state.
As the tool/toolset will evolve, we will add more features and hopefully find even simpler ways of sorting code.
For example: the organize imports feature was developed to experiment with the power of automatically correcting code, before we started working with refactoring. Now as we are adding a new experimental plug-in dedicated to refactoring, this feature might also be refactored to the new framework and later moved to that package.
Previous Topic:Open source contributions to the Eclipse Titan
Next Topic:Experimental Titan build for Raspberry Pi
Goto Forum:
  


Current Time: Tue Jun 19 01:19:23 GMT 2018

Powered by FUDForum. Page generated in 0.02089 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top