.. ******************************************************************************* Copyright (c) 2021-2023 Bayerische Motoren Werke Aktiengesellschaft (BMW AG) 2021 in-tech GmbH This program and the accompanying materials are made available under the terms of the Eclipse Public License 2.0 which is available at http://www.eclipse.org/legal/epl-2.0. SPDX-License-Identifier: EPL-2.0 ******************************************************************************* .. |win_only| image:: /_static/win_only.png .. _prerequisites: Installing Prerequisites ======================== Before being able to compile and run |op|, make sure to have all dependencies installed. The third party software dependency of |op| can be divided into: - **Build environment**, which manages third party software and is used to compile and install |op| - **Binary packages**, which can be easily installed via a package manager of the respective build environment - **Source packages**, which need to be built from source code This section gives detailed information about the prerequisites and tested version. .. _building_under_windows: Installing the Build Environment -------------------------------- .. tabs:: .. tab:: Windows #. MSYS2 On Windows, the build environment of choice are :term:`MSYS2` programming tools. :term:`MSYS2` is used to install some third-party software on which |op| depends. Also, the unix-like shell simplifies c++ compilation on Windows. For details, see `MSYS2 website `_. .. _msys2: #. Download MSYS2 The latest 64-bit packages are located at https://repo.msys2.org/distrib/x86_64/. Download a non-base package, i.e. `msys2-x86_64-20200903.exe `_ .. _msys2_installation: #. Install MSYS2 Run the downloaded executable and adjust suggested settings to your needs (defaults are fine). In the following, it is assumed that MSYS2 is installed under ``C:\msys64``. #. Understand the Build Environment MSYS2 provides three different environments, located in the MSYS2 installation directory: .. _fig_msys2_environments: .. figure:: _static/images/msys2.png :align: center :alt: MSYS2 Apps MSYS2 Environments - MSYS2 MSYS: Common environment, i.e. for package management - MSYS2 MinGW 32-bit: A MinGW 32-bit environment - MSYS2 MinGW 64-bit: A MinGW 64-bit environment .. warning:: | MSYS2 MinGW 64-bit is **the** |op| development environment and will be referred to as |mingw_shell|. .. _building_under_linux: .. tab:: Linux On Linux, no special build environment is needed. |Op| is developed under Debian 64-Bit, which means that developing under a recent Ubuntu distribution will also work. Debian Bookworm or Ubuntu 22.04 LTS is recommended. Debian uses ``apt`` (or ``apt-get``) as package managing system. Details will be given in :ref:`binary_packages` and :ref:`building_prerequisites`. .. _binary_packages: Installing the Binary Packages ------------------------------ The first set of dependencies we need to install in order to successfully compile |op| are the binary packages. These can be installed via appropiate package manager. .. tabs:: .. tab:: Windows (MSYS2) #. Open ``MSYS2 MinGW 64-bit`` and execute the following package manager ``pacman`` commands to update the package repository and upgrade system packages: .. code-block:: pacman -Syuu If the upgrade requires a restart of MSYS2, resume the upgrade by re-opening the shell and call: .. code-block:: pacman -Suu #. Required packages (can be specified in single command line if desired): .. literalinclude:: _static/msys2_packages.txt :language: bash .. admonition:: Versions | MSYS2 provides rolling release versions, so some packages might be too "up-to-date". | Tested packages - at time of writing - have been listed above as comment. | If in doubt, download the package in the right version from the `MSYS2 package repository `_. | Install with ``pacman -U `` | If there is no old enough MSYS2 Package Version available in the package Repository you can download specific versions with python-pip | Install with ``pip3 install ==`` #. Optional Packages .. code-block:: pacman -S git pacman -S diffutils pacman -S patch pacman -S mingw-w64-x86_64-ag pacman -S mingw-w64-x86_64-qt5-debug pacman -S zlib-devel # for api documentation (optional) pip3 install breathe exhale .. admonition:: GIT/SSH The |mingw_shell| does not access an already existing git installation or available SSH keys. Make sure, to update/copy your configuration and credentials within the |mingw_shell| before working with git. .. tab:: Linux (Debian Bookworm) #. Update the package database on the system .. code-block:: apt update #. Upgrade existing software to latest version .. code-block:: apt upgrade #. Install required binary packages .. code-block:: # for simulator apt -y install ccache apt -y install cmake apt -y install doxygen apt -y install googletest apt -y install gcc apt -y install g++ apt -y install graphviz apt -y install libboost-dev apt -y install libqt5xmlpatterns5-dev apt -y install qtbase5-dev apt -y install qtchooser apt -y install zlib1g-dev # for documentation sudo apt install python3 python3-pip libenchant-2-2 dvipng pip3 install sphinx sphinx-rtd-theme sphinx-tabs breathe exhale sphinxcontrib-spelling myst-parser # for testing pip3 install pytest pandas Under Linux, it is deliberate that the googletest package only installs the header files to the system, but not the static and dynamic libraries. The missing libraries can be build and installed to ``/usr/lib`` via .. code-block:: cd /usr/src/googletest cmake . make make install **EndToEnd Test Framework** If end to end tests shall be executed, additional requirements have to be considered. Please refer to :ref:`testing_endtoend` for more details on installation of the prerequisites and usage of the framework. .. _building_prerequisites: Installing the Source Packages ------------------------------ This section describes how to compile prerequisites of |op| using source packages. .. note:: If you are unfamiliar to ``CMake`` or working within a |mingw_shell|, Section :ref:`cmake` and :ref:`MSYS2 ` might give you a short introduction on these topics in the scope of building |op| itself. Location Of Installed Source Packages ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The goal of this section is to download necessary source packages and install them into a suitable directory. This directory will later on be copied into the |op| repository in order to resolve third party dependency. The following directory tree shows the folder structure, which will be created by following the recommendations of this guide. .. note:: The |op| repository isn't needed yet. .. tabs:: .. tab:: Windows :: C:\openpass\thirdParty ├── FMILibrary │ ├── include │ └── lib ├── osi │ ├── include │ └── lib ├── protobuf │ ├── bin │ ├── include │ └── lib ├── protobuf-shared │ ├── bin │ ├── include │ └── lib └── zlib-1.2.12 └── contrib └── minizip In the folder structure above: - ``C:\openpass\thirdParty`` refers to a temporary directory used to built the prerequisites from source, **not** the ``openpass`` repository - ``FMILibrary`` is the install directory of the :term:`Functional Mock-up Interface (FMI) ` when build from source - ``osi`` is the install directory of the :term:`Open Simulation Interface (OSI) ` when build from source. - ``protobuf`` and ``protobuf-shared`` are the install directories of ``Google Protocol Buffers`` for shared and static builds, respectively. .. tab:: Linux :: ~/openpass/thirdParty ├── FMILibrary │ ├── include │ └── lib ├── osi │ ├── include │ └── lib ├── protobuf │ ├── bin │ ├── include │ └── lib ├── protobuf-shared │ ├── bin │ ├── include │ └── lib └── zlib-1.2.12 └── contrib └── minizip In the folder structure above: - ``~/openpass/thirdParty`` refers to a temporary directory used to built the prerequisites from source, **not** the ``openpass`` repository - ``FMILibrary`` is the install directory of the :term:`Functional Mock-up Interface (FMI) ` when build from source - ``osi`` is the install directory of the :term:`Open Simulation Interface (OSI) ` when build from source. - ``protobuf`` and ``protobuf-shared`` are the install directories of ``Google Protocol Buffers`` for shared and static builds, respectively. On the basis of this structure, we will explain further steps. .. _building_protobuf: Build and Install Protobuf ~~~~~~~~~~~~~~~~~~~~~~~~~~ Google Protocol Buffers provide the foundation of :term:`OSI` (see also :ref:`building_osi`). Due to the usage of OSI different situations (|Op| executables, libraries, tests, :term:`FMUs `, etc.) static and shared libraries of protobuf have to be provided. This section gives instructions, how to compile version 3.20.0. and hook it into the |op| build. This currently applies to the |op| build, as static and shared libraries of Protobuf are required. #. Download release 3.20.0 from https://github.com/protocolbuffers/protobuf/releases #. Extract - for Windows to ``C:\openpass\thirdParty\sources\protobuf-cpp-3.20.0`` - for Linux to ``~/openpass/thirdParty/sources/protobuf-cpp-3.20.0`` #. Navigate to the extracted folder .. tabs:: .. tab:: Windows Start |mingw_shell| .. code-block:: cd /C/openpass/thirdParty/sources/protobuf-cpp-3.20.0 .. tab:: Linux Start ``Bash`` shell .. code-block:: cd ~/openpass/thirdParty/sources/protobuf-cpp-3.20.0 #. Create build directory .. code-block:: cd cmake mkdir build cd build #. Run CMake .. tabs:: .. tab:: Windows .. code-block:: :emphasize-lines: 6 cmake -G "MSYS Makefiles" \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=C:/openpass/thirdParty/protobuf \ -Dprotobuf_BUILD_SHARED_LIBS=OFF \ -Dprotobuf_BUILD_TESTS=OFF \ -DCMAKE_CXX_FLAGS=-fPIC \ .. .. tab:: Linux .. code-block:: :emphasize-lines: 5 cmake -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=$HOME/openpass/thirdParty/protobuf \ -Dprotobuf_BUILD_SHARED_LIBS=OFF \ -Dprotobuf_BUILD_TESTS=OFF \ -DCMAKE_CXX_FLAGS=-fPIC \ .. .. note:: Tests are disabled due to compiler warnings treated as errors (may vary with compiler version). #. Compile .. code-block:: make -j3 #. Install .. code-block:: make install .. note:: After completion, **repeat the steps described above** with the following changes applied to the CMake call: #. Set ``protobuf_BUILD_SHARED_LIBS`` to ``ON`` and remove -DCMAKE_CXX_FLAGS=-fPIC flag. #. Use ``$HOME/openpass/thirdParty/protobuf-shared`` as ``CMAKE_INSTALL_PREFIX`` This creates and installs the Protobuf libraries, as the Protobuf dynamic and static libs are required for the openPASS build. .. _building_osi: Build and Install OSI ~~~~~~~~~~~~~~~~~~~~~ As can be looked up in :ref:`Prerequisites_osi`, the core component ``World_OSI`` uses :term:`OSI` as backend storage. OSI itself uses ``protobuf`` to describe data structures in a platform independent way by means of `*.proto` files. When building OSI, these files are converted into C++ headers and sources, using the protobuf compiler ``protoc``. Finally, the sources are then compiled into a library. |Op| finally uses the library and the generated headers to interface the library. #. Open and create directory structure .. tabs:: .. tab:: Windows Start |mingw_shell| .. code-block:: cd /C/ mkdir -p openpass/thirdParty/sources .. tab:: Linux Start ``Bash`` shell .. code-block:: cd ~ mkdir -p openpass/thirdParty/sources #. Download release 3.5.0 from https://github.com/OpenSimulationInterface/open-simulation-interface #. Extract - for Windows to ``C:\openpass\thirdParty\sources\open-simulation-interface-3.5.0`` - for Linux to ``~/openpass/thirdParty/sources/open-simulation-interface-3.5.0`` #. Navigate to the extracted folder .. tabs:: .. tab:: Windows .. code-block:: cd /C/openpass/thirdParty/sources/open-simulation-interface-3.5.0 .. tab:: Linux .. code-block:: cd ~/openpass/thirdParty/sources/open-simulation-interface-3.5.0 #. Optional: Enable Arenas For better performance, |op| supports protobuf Arenas allocation (https://developers.google.com/protocol-buffers/docs/reference/arenas). To use this feature, OSI and |op| needs to be compiled with Arenas support. See :ref:`cmake_protobuf_arenas` how this feature is enabled in |op|. To enable Arenas support for OSI, the line ``option cc_enable_arenas = true;`` needs to be added **manually** to all OSI proto files before compilation. This can be achieved in two ways. Either the line ``option cc_enable_arenas = true;`` gets added manually **after the second line** of each PROTO file - for Windows in ``C:\openpass\thirdParty\sources\open-simulation-interface`` - for Linux in ``~/openpass/thirdParty/sources/open-simulation-interface`` by using a text editor or one makes use of the stream editor in the shell: .. code-block:: find . -maxdepth 1 -name '*.proto' -exec sed -i '2i option cc_enable_arenas = true;' {} \; .. warning:: The first line of each OSI proto file specifies the protobuf syntax used. If Arenas support is added before the syntax specification, errors occur. #. Create build directory .. code-block:: mkdir build cd build #. Run Cmake and hook Protobuf into OSI .. tabs:: .. tab:: Windows .. code-block:: cmake -G "MSYS Makefiles" \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=C:/openpass/thirdParty/osi \ -DCMAKE_PREFIX_PATH=C:/openpass/thirdParty/protobuf-shared/ .. .. tab:: Linux .. code-block:: cmake -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=$HOME/openpass/thirdParty/osi \ -DCMAKE_PREFIX_PATH=$HOME/openpass/thirdParty/protobuf-shared/ .. #. Compile .. code-block:: make -j3 .. warning:: If ``protobuf`` compiler complains, do .. code-block:: protoc --version and check if correct protobuf version is used. .. note:: If no protobuf is found the path to protoc needs to be first set manually in the terminal. ``export PATH=$HOME/openpass/thirdParty/protobuf-shared/bin:$PATH`` #. Install .. code-block:: make install .. admonition:: Documentation The OSI class documentation is part of the source code and can be compiled using Doxygen. Instructions are located in the OSI ``Readme.md``. A pre-compiled version is located `here `_. So far, the documentation does not include the extensions from the openpass-trafficAgents branch. .. _building_fmil: Build and Install FMIL ~~~~~~~~~~~~~~~~~~~~~~ #. Download release 2.0.3 from https://github.com/modelon-community/fmi-library .. note:: If the required version isn't listed as proper release, it can be downloaded via Github-Tags #. Extract - for Windows to ``C:\openpass\thirdParty\sources\fmi-library-2.0.3`` - for Linux to ``~/openpass/thirdParty/sources/fmi-library-2.0.3`` #. Navigate to the extracted folder .. tabs:: .. tab:: Windows Start |mingw_shell| .. code-block:: cd /C/openpass/thirdParty/sources/fmi-library-2.0.3 .. tab:: Linux Start ``Bash`` shell .. code-block:: cd ~/openpass/thirdParty/sources/fmi-library-2.0.3 #. Create build directory .. code-block:: mkdir build cd build #. Run Cmake .. tabs:: .. tab:: Windows .. code-block:: cmake -G "MSYS Makefiles" \ -DFMILIB_INSTALL_PREFIX=C:/openpass/thirdParty/FMILibrary \ -DCMAKE_BUILD_TYPE=Release \ -DFMILIB_BUILD_STATIC_LIB=OFF \ -DFMILIB_BUILD_SHARED_LIB=ON \ .. .. tab:: Linux .. code-block:: cmake -DFMILIB_INSTALL_PREFIX=$HOME/openpass/thirdParty/FMILibrary \ -DCMAKE_BUILD_TYPE=Release \ -DFMILIB_BUILD_STATIC_LIB=OFF \ -DFMILIB_BUILD_SHARED_LIB=ON \ .. #. Leave build directory .. code-block:: cd .. #. Apply Patch As FMIL and the internally used `FMU Compliance Checker `_ has issues with loading and private entry points, the following patch needs to be applied: - :download:`Windows/Linux Patch ` .. code-block:: git apply --ignore-whitespace "/fmi-library-2.0.3-fixes.patch" #. Enter build directory .. code-block:: cd build #. Compile .. code-block:: make -j3 #. Install .. code-block:: make install .. _appending_minizip: Append Minizip Through Installing zlib Library ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Minizip is a contributed dependency that is not available through the package installed zlib library. To add minizip to the project it is not required to build the whole repository. #. Open and create directory structure .. tabs:: .. tab:: Windows Start |mingw_shell| .. code-block:: cd /C/ mkdir -p openpass/thirdParty/ .. tab:: Linux Start ``Bash`` shell .. code-block:: cd ~ mkdir -p openpass/thirdParty/ #. Download release v1.2.12 from https://github.com/madler/zlib #. Extract - for Windows to ``C:\openpass\thirdParty\zlib`` - for Linux to ``~/openpass/thirdParty/zlib``