Hello,
I would like to introduce you to a new open source project for scanning hardware and completing scientific experiments. It has been donated
to the Eclipse Foundation as an incubation project and has number of interesting features. The original project proposal can be read here:
https://projects.eclipse.org/proposals/scanning
However, I would like to point out some of the things it can do to you, as a subscriber to this list or person who has expressed an interest
in the past with the project.
Scan Paths
The start of a scan is the scan path. It is a series of n-Dimensional motor/scalar positions provided by a generator pattern - we call them
scan point generators. The possible paths are created using a factory, with each path requiring a model. The generators may be nested, limited only by their ability to be addressed. The paths scale to hundreds of millions of points because not all points are
created in memory at the start of the scan, so really large scans are possible. The paths are defined in Python to make it easy for scientists to create custom generators on the fly, in the unlikely event that they cannot achieve the type of scan required.
Unlimited regions of interest are allowed so the user may for instance create a grid and then draw various shapes to filter the scan points. A simple example of a scan path is a snake scan running over a grid of 2D positions of a stage motors for x and y.
Runnable Devices and Scannable Devices
When it comes time to run the scan, the motors and detectors are represented in code by the simple interfaces IRunnableDevice and IScannable.
These classes are each managed by an OSGi service which allows devices to be added by a Spring layer or by creating and registering them with the server in Python. Once they are added, they provide the link to the hardware such as Area Detector or EPICS PVs,
you can then start to run scans.
Scanning Algorithms
The scanning algorithm is also part of a factory pattern and allows users to override or create their own. The default algorithm uses the
power of Java Executor Service. It splits up the devices by a concept known as level and uses a fully multi-threaded design to move motors in the most efficient way. Motors are moved while detectors are read out and the file is written. For those motors in
a 2D scan which have a fast and slow axis on the same EPICS IOC, the power of the scan path design means that the code can look forwards to figure out each fast path segment and run lines close to the theoretical maximum.
User Interface
The project comes with a complete user interface for creating and running scans written in Eclipse RCP. The bundles which contribute the
UI are modular and may be removed for those people without an RCP front end. The user interface interacts seamlessly with the python layer giving the user either the ability to submit without scripting or, for advanced users, the ability to copy the scan command
into Python. It is easy to create Java-Swing, Python-QT or any front end because the scanning system is driven externally by JSON.
Python and Malcolm
The Scanning project is partly written in Python and partly Java. Most of the C-python device layer is contributed in a separate project
called pymalcolm
and the project interacts with Malcolm devices (which are IRunnableDevice) using the EPICSv4 protocol,
http://epics-pvdata.sourceforge.net/.
Eclipse January, NeXus HDF5 -SWMR
Scanning includes the implementation of the January concept “ILazyWriteableDataset”. This means that if you are creating a device, you only
need depend on January and your control layer (EPICS for instance). The device layer is modular and multi-threaded so it allows the full speed of HDF5-SWMR to be utilised without the need to mix up HDF5 and device code.
Online Analysis
For the first time, online analysis has been considered and designed into a scanning system from the start. This means that online analysis
codes running on the cluster can run processing SWMR files on the data during the live data collection. The user interface allows the choose between different analysis or even create your own graphically using DAWN and then deploy the analysis with the scan.
This is done efficiently using the new Single Write Multiple Read (SWMR) HDF5 library.
Builds and Tests
Scanning does not skimp on tests, it has hundreds of tests executing different scanning scenarios and different device options. It checks
operation of the algorithms with machine topup and beam dump scenarios, it writes HDF5 SWMR files and checks the supported scan point generators. It gives the user interface composites a workover using SWTBot. For the first time this layer has be abstracted
into a separately built (travisCi+maven) and tested(travisCI+junit+sonarqube) project which enables anyone to participate and contribute.
Getting Started
If you want to check out ‘Scanning’ and run it, providing you are experienced with targets and products it is easy to run by following these
instructions:
https://github.com/eclipse/scanning/blob/master/GETTINGSTARTED.pdf
Sincerely,
Matthew Gerring
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
