As Matt mentioned, the Java-Python RPC-based integration developed for DAWN has been migrated/duplicated in Triquetrum.
This was done in collaboration with Peter & Matt together with/by Jonah (who also developed the original package for DAWN).
Test code for analysis.rpc is also available & some example code on how to use the PythonService class (this last part is in the examples folder).
The main dependencies are on Apache XML-RPC that have been added in orbit during the integration in Triquetrum.
From your request it's not yet clear to me if it would fit your needs in its current state.
In particular, the version in Triquetrum does not contain anything (yet?) related to January IDataSet.
It currently has flatteners for simple Java types, and maps them to "input" arguments of your Python scripts that should have a "run()" function as a starting point.
Integration with January is planned for Triquetrum, so I would be in favor of adding IDataSet support in the Python service as well.
But we should discuss planning and approach between the science projects for this.
(also taking into account Matt's remark about defining what the best place is for the long-term maintenance of such an integration layer)
If your project would have an interest in getting this to move forward (and even would maybe be able to contribute/fund some of this?) that would of course be great!
Hi Scott,
It is exciting that you are using it for that! This is a great new usage, exactly the kind of thing we were hoping for in making the work open source, thanks for picking it up.
I introduced py4j to dawnsci and in that project we send around ndarry<->IDataset from a py4j connection to do things like plotting; so you are not alone in doing this kind of thing. There is something called AnalysisRCP which flattens ndarray and unflattens in Java as IDataset. This was commissioned by Diamond Light Source as a work package with Kichwa Coders and Jonah from Kichwa is one of the project leads of January...
To expand on what Peter said, the flattening service, which is the heart of integrating ndarry with IDataset is currently here:
https://github.com/DawnScience/scisoft-core/tree/master/uk.ac.diamond.scisoft.analysis.xmlrpc
We have spoken often and done little (AFAIK) about making a separately compiling/building/testing github project for this work. However my current understanding is that this code needs to be reworked to make it usable by any project. I did some work to put it into its current low dependency form but it would need more I think to be useful to you. There are still a few dependencies there:
https://github.com/DawnScience/scisoft-core/blob/master/uk.ac.diamond.scisoft.analysis.xmlrpc/META-INF/MANIFEST.MF
Eclipse Triquetrum also looked at some of this stuff and may have another version. I have copied in Erwin and Christopher just in case.
In terms of releases, the plan is to follow eclipse release trains but as Peter indicates there may be need for more. This is because Diamond Light Source funds changes in it that sometimes require something at short notice (well it could be anyone doing the funding but DLS tends to so far...)
Cheerio,
Matt
-----Original Message-----
From: january-dev-bounces@xxxxxxxxxxx [mailto:january-dev-bounces@xxxxxxxxxxx] On Behalf Of Peter.Chang@xxxxxxxxxxxxx
Sent: 30 January 2017 09:28
To: january-dev@xxxxxxxxxxx
Subject: Re: [january-dev] IDataset <-> python
Hi Scott,
We do have and use two mechanisms for communicating with Python: xml-rpc and Py4J. At the moment, they are part of Dawn Science (https://github.com/DawnScience/scisoft-core) and are based on saving the data to a filesystem that is accessible to both sides.
The next version of January will be a major one though there has not been any discussion I am looking at a March release.
Best regards,
Peter
-----Original Message-----
From: january-dev-bounces@xxxxxxxxxxx [mailto:january-dev-bounces@xxxxxxxxxxx] On Behalf Of Scott Lewis
Sent: 27 January 2017 19:46
To: january-dev@xxxxxxxxxxx
Subject: [january-dev] IDataset <-> python
Hi Folks,
Is there any existing work on going between an IDataset instance (in
java) and a Dataset instance in Python using Py4j? We are using
January to build an IoT data analytics framework that is built on
java/OSGi/kura/our stuff. It includes a way to dynamically
create/add/remove osgi services that pass IDataset instances between components to perform various analytics.
One thing that we might be interested in doing is having these osgi services implemented via Python...which would necessitate exchanging IDataset <-> Python dataset instances.
Another question: When is the next expected release of January?
Will it be a major, minor, maintenance?
Thanksinadvance for any information.
Scott
_______________________________________________
january-dev mailing list
january-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/january-dev
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
_______________________________________________
january-dev mailing list
january-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/january-dev
