Eclipse Community Forums: Science WG » Triquetrum Components

Home » Eclipse Working Groups » Science WG » Triquetrum Components(Description of initial components for the Triquetrum workflow project)

Show: Today's Messages :: Show Polls :: Message Navigator

Triquetrum Components [message #1697925]

Tue, 09 June 2015 15:36

Jay Billings

Messages: 54
Registered: July 2011

Member

This thread describes the individual components of the initial contribution for Triquetrum, a new workflow project that is being proposed to the Eclipse Foundation.

Jay Jay Billings
Oak Ridge National Laboratory
Twitter: @jayjaybillings

Report message to a moderator

Re: Triquetrum Components [message #1697926 is a reply to message #1697925]

Tue, 09 June 2015 15:36

Jay Billings

Messages: 54
Registered: July 2011

Member

From Christopher Brooks on the contribution from Ptolemy:

In Ptolemy II, we have the notion of configurations. The Ptiny configuration is a small, useful configuration with a minimum of third party licenses. Ptiny includes the kernel, common polymorphic actors, common models of computation and Vergil, the UI. Vergil is AWT and is what we are looking at replacing with something that is more based on Eclipse.

The Ptolemy II source code is primarily BSD, so contributing it should not be that difficult. We've done a reasonably good job managing the licenses, the dependencies in the core are fairly straightforward. A tricky part could be tracking down certain contributors and getting an assignment of copyright. Fortunately, the core is fairly stable and not that many people have worked on it.

Hallvard Trætteberg created some OSGi modules that use the core Ptolemy II classes. One issue that we ran in to is that Ptolemy II uses the class path to add actors, thus we don't always know in advance the class names of all the actors that are to be used in a system. I believe that this is a bit contrary to how OSGi works, where OSGi expects to know all the dependencies in advance. There are workarounds to this though.

Jay Jay Billings
Oak Ridge National Laboratory
Twitter: @jayjaybillings

Report message to a moderator

Re: Triquetrum Components [message #1697934 is a reply to message #1697926]

Tue, 09 June 2015 16:21

Erwin De Ley

Messages: 52
Registered: August 2013

Member

Hi Jay, Christopher,

It would indeed be great if core parts of ptolemy could be(come) part of the triquetrum code-base! My initial idea was to have ptolemy core modules packaged as OSGi bundles and as dependencies for triquetrum to start with. I've already been "playing" a bit with this and have a minimal POC of a graphiti-based RCP editor working on that basis.

Christopher, you should also be aware that the code contribution should follow org.eclipse.triquetrum.... naming conventions. So I think it's a non-trivial step to move or duplicate parts of the current Ptolemy code, as it breaks all existing package names.

About the dynamic extensions with extra actor bundles : that problem is covered/solved using OSGi services, as done in Passerelle. It's already working in that POC mentioned above as well.

cheers
erwin

Report message to a moderator

Re: Triquetrum Components [message #1697946 is a reply to message #1697934]

Tue, 09 June 2015 17:37

Christopher Brooks

Messages: 6
Registered: July 2009

Junior Member

We don't necessarily have to reuse the ptII code core code, though if we don't use it, then we end up reimplementing the wheel. There is quite a bit of engineering there, and some particularly subtle issues.

Looking at the ptII code and using it as the basis for triquetrum would probably be a win. One big issue would be splitting things in to interface and implementation. It could be that in the new code, the package names, class names and method names might change from the names in ptII, but the bodies of a number of the methods would be similar.

It is ok if we lose backward compatibility between ptII and triquetrum. It would be nice if we had either a conversion script or had only a few types of changes to start with. Delaying the split for awhile could be a good thing.

In this message, I had a bunch of text with links to the triquetrum wiki that would help get the conversation rolling, but the forum says:

"You can only use links to eclipse.org sites while you have fewer than 5 messages."

Feh.

Anyway, triquetrum.org links to a wiki that has some info.

Also, try searching for

Ptolemy Package Dependencies

the first link is a page on the kepler site that discusses the packages that are there.

BTW - I don't find forums that easy to work unless I get email about updates to the forum. I checked the 'Post Notification' button for this post, so we will see. If I'm not responding to a future post then it is because I probably because I have not seen it, so feel free to email me directly and I'll reply within the forum. Does this forum have an email gateway that would allow me to read and post?

Is there a wiki that we can use? As an alternative, I can update the package dependency work on the triq wiki and give out accounts to anyone who wants one.

Report message to a moderator

Re: Triquetrum Components [message #1698229 is a reply to message #1697946]

Fri, 12 June 2015 11:15

Erwin De Ley

Messages: 52
Registered: August 2013

Member

As described in the other msg about the project proposal (don't know how to have a clean link to that one in here...) there are 3 lines of work in triquetrum (isn't that nice Wink

).

I would propose following components in an initial code-drop :

For the Ptolemy RCP editor :
1. A minimal editor capable of drawing simple toplevel models and running them, built on Graphiti and EMF.
2. An EMF-based model as a binding layer towards the underlying Ptolemy model elements.
3. Essential ptolemy bundles as binary dependencies. These will at first be built from the Ptolemy II repository.
4. A minimal set of task-based actors as a show-case of the contents of the other lines of work.

For the Task-based processing :
1. A domain API for processing arbitrary sequences of Tasks, with their parameters, lifecycle tracing and final results.
2. A basic implementation for in-memory processing.

Supporting services :
1. A ProcessingService API with brokers and handlers etc for synchronous and asynchronous/buffered task processing
2. An Adapter API to execute Tasks that require external services or applications
3. An integration of that API with DAWNsci's python analysis RPC
4. Some trivial implementations to connect to SOAP web-services

Most of the proposed elements in the 2 last topics are based on what's available in Passerelle, but will need to be refactored & extracted from there.
This could then be a basis for discussions with the triquetrum participants about requirements&evolution.

For the first line, the principles have already been tried out, but the current Passerelle code-base is not appropriate as an initial code drop.
So that will take the most effort...

Would that be a good starting-point?

cheers
erwin

Report message to a moderator

Re: Triquetrum Components [message #1698259 is a reply to message #1698229]

Fri, 12 June 2015 15:36

Christopher Brooks

Messages: 6
Registered: July 2009

Junior Member

That seems reasonable to me. Do you have the ptII OSGI packages built? If not, we could start with Hallvard's work in ptII/pt-modules, see ptolemy.org/src/pt-conf (I can't use URLs yet because I have not posted enough)

pt-akore pt-de pt-kore pt-lib pt-moml pt-ptera pt-sr
pt-ddf pt-junit pt-kore-test pt-modal pt-momlapp pt-sdf pt-test

kore is a Ptolemy word for the core, akore is the actor core. The names could be anything, what is in them is what matters. I think a minimal set is: pt-akore pt-kore pt-moml pt-momlapp

The models of computation are pt-de, pt-ptera, pt-sr, pt-ddf, pt-modal and pt-sdf. A useful system wants at least one model of computation

pt-junit pt-kore-test and pt-test are testing frameworks.

_Christopher

Report message to a moderator

Re: Triquetrum Components [message #1698266 is a reply to message #1698259]

Fri, 12 June 2015 16:17

Erwin De Ley

Messages: 52
Registered: August 2013

Member

Christopher,

I've continued on the work of Hallvard. You can find the current status of "bundled" ptolemy on https://repo.eecs.berkeley.edu/svn/projects/eal/ptII/branches/osgi-2-0 for :
- diva
- ptolemy.actor.lib
- ptolemy.actor.lib.gui
- ptolemy.core
- ptolemy.domains.process
- ptolemy.domains.sdf
- ptolemy.gui
- ptolemy.moml

I have added 2 new bundles, with code taken from Passerelle :
- org.ptolemy.commons : defines a VersionSpecification for model elements + a FutureValue implementation
- org.ptolemy.testsupport : some simple tools to write unit tests on models

- org.ptolemy.core.test.actor : illustrates how to add new actors in a dynamic way to Ptolemy in an OSGi application
- org.ptolemy.core.test : some unit tests using the testsupport utils and that extra actor

- org.ptolemy.modulebuilder is a small java tool that uses the pt-jar.files content lists (as done by Hallvard) in the module projects to create linked sources/resources.

Remark that as I wrote to you before, this all works fine for testing from inside your eclipse, but the PDE export doesn't seem to include linked sources.
So this approach seems to be a bit of a dead end for "real" builds I think.

If you are willing to "fork"/replicate ptolemy core stuff into triquetrum, this problem will be gone, as we will have "real" sources then from the start,
and no longer linked sources from the ptII source tree.

regards
erwin

[Updated on: Fri, 12 June 2015 16:32]

Report message to a moderator

Re: Triquetrum Components [message #1698588 is a reply to message #1698266]

Tue, 16 June 2015 14:28

Jay Billings

Messages: 54
Registered: July 2011

Member

I was reviewing the list of supported services above. ICE currently has something similar in spirit to number 1 and it will be good to adopt something a little more robust and community supported.

Something that I think is missing from that list is an I/O service. One of the things that we do in ICE, which has been wonderful, is register I/O parsers and emitters as OSGi services. We have a robust Intermediate Object Representation (IOR), so all of the services marshal and unmarshal to that IOR via the OSGi components. We can do something like

form = ioService.getReader("csv",csvFile).getForm();
ioService.getWriter("otherFormat",form,otherFile).writeForm();

to translate between formats. The first argument in each is completely declarative, so in theory this could be used in arbitrary workflows, and we use it as such for data export.

I'll continue to review all of this before our meeting; very good looking stuff at the moment!

Jay Jay Billings
Oak Ridge National Laboratory
Twitter: @jayjaybillings

Report message to a moderator

Re: Triquetrum Components [message #1698920 is a reply to message #1698588]

Thu, 18 June 2015 20:56

Christopher Brooks

Messages: 6
Registered: July 2009

Junior Member

I took a look at the licensing of the Java files that are included in the Ptolemy II branch that Erwin has been working on to set up OSGi bundles.

The summary is that the branch uses the following licenses:

* Ptolemy II - BSD Copyright, held by the Regents of the Univ. of Calif
* AElfred - Freely available. Used by MoMLParser. It would be difficult to not use this package, though it is, in theory, possible.
* Graph - BSD Copyright, held by a former student. Used by the type system. It would be difficult to execute a model in Ptolemy without this license.
* Extension Filename Filter - Fairly open copyright, though limits use in nuclear facilities or aircraft. License is held by Sun. This dependency could be removed.

The more complete analysis is at

chess.eecs.berkeley.edu/triq/wiki/Main/Osgi2-0Licensing

(I'm not permitted to post links yet)

My reading of the above licenses is that if we remove the dependency on Extension Filename Filter, then the code has BSD or better licenses.

Using an open source license scanning tool on the code would be useful.

Report message to a moderator

Re: Triquetrum Components [message #1699035 is a reply to message #1698920]

Fri, 19 June 2015 16:23

Christopher Brooks

Messages: 6
Registered: July 2009

Junior Member

I ran Fossology and found an additional license:

ptolemy.util.DoubleUtilities is used by ptolemy.actor.util.Time to improve performance.

In theory this dependency could be removed, though performance would suffer.

DoubleUtilities is based on code.google.com/p/guava-libraries/source/browse/guava/src/com/google/common/math/DoubleUtils.java which is a part of Guava and has the following license:

/*
* Copyright (C) 2011 The Guava Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* [elided] www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

Fossology was run

identified the license in DoubleUtilities
false positives for GenerateCopyrights.java
Files in com/microstar/xml were identified as not having a license, though they link to the README
Files in ptolemy/data/expr were identified as not having a license. These files are generated by JavaCC

Open Source License Checker V.3 was run

The installer was from 2009, so the licenses could be out of date
The tool identified some minor typos in files where the line that lists the copyright holder differed slightly
The tool states that there is a conflict between Apache-2.0 and the "All Rights Reserved" statement in the BSD copyright that appears in most of the files.

_Christopher

Report message to a moderator

Re: Triquetrum Components [message #1707469 is a reply to message #1699035]

Fri, 04 September 2015 23:29

Christopher Brooks

Messages: 6
Registered: July 2009

Junior Member

I checked and the Triquetrum third party package dependencies are unchanged.

My notes are at https://chess.eecs.berkeley.edu/triq/wiki/Main/Osgi2-0Licensing

_Christopher

Report message to a moderator

Previous Topic:

Triquetrum project proposal

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Tue Apr 23 16:00:39 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter