Hi,
I just pushed the first part to github [1]. With these two projects
is possible to launch an EASE Jupyter kernel (that does not do
anything yet). Jupyter is registering the kernel correctly and the
pass-along of all the data works fine. Please refer to the README
for the launcher to see the usage.
I am still working on the ZMQ part but will hopefully get the
HeartBeat socket running tomorrow. With HeartBeat working we already
have a good skeleton for future work. Getting the socket running
means that we are correctly parsing Jupyter launch files and
correctly handle both incoming and outgoing ZMQ messages. After that
we will hopefully be able to continue with the different Jupyter
sockets very fast. I will let you know once the HeartBeat is
running.
Best regards,
Martin
[1] https://github.com/kmhsonnenkind/ease_jupyter_kernels
Am 2016-06-10 um 13:45 schrieb
Christian Pontesegger:
Hi Martin,
if you prefer, you may stick on github for the time being.
Regarding license we may add a requirement that jupyter needs to
be installed correctly. So no need to bundle it with EASE. We are
having the same situation with Py4J as we also require a local
python implementation there, which we do not bundle. Later we
might try to put the jupyter code into a plugin and host it
outside of eclipse. Therefore we need to work together with the
eclipse legal team.
For your github code, please provide a readme clearly stating how
to install components and make things run. It is perfectly ok if
this might be a complicated at the beginning, we just need to be
able to run this on our machines, too.
Christian
On 06/06/2016 11:28 PM, Martin
Kloesch wrote:
Hi Christian,
you are of course right regarding the launch order.
I have been busy with another project and did not get as much
work done over the weekend as I had hoped.
As mentioned I created a dispatching tool that passes the
connection file over a socket. This is working fine and I can
parse the JSON content on the server side without problems. The
next step will be to set up the ZMQ sockets for the different
tasks and implement the heartbeat socket. This socket - as the
name suggests - is only used for heartbeat messages to check if
the kernel is still alive. As it also uses ZMQ it is a good
first start to test the protocol implementation and see if
everything is working correctly already.
At the moment I am using a plain old Java application as I do
not need any eclipse functionality yet and it gives me less
overhead during development phase. As it is not really ecipse
code yet, I am not sure if it would make more sense to host it
somewhere on github for now or if I should use the eclipse
repository already? What do you think would be better? Further I
am not sure if we need anything special regarding licensing. The
code itself should not be a problem but at one point or another
we will also need the Jupyter executable (could wget that from
Jenkins though).
I will try to get my code into a commitable state and will push
it the very latest end of this week. I'll let you all know once
it is available somewhere.
Best regards,
Martin
Am 2016-06-06 um 08:57 schrieb
Christian Pontesegger:
Hi Martin,
not sure if we talk about the right launch order here. From my
point of view eclipse should never publish its launch to
jupyter, because eclipse is there first. Each Eclipse instance
launches its own jupyter core (if necessary) and then launches
jupyter kernels for each opened notebook. As we do open
notebooks from eclipse we know when its time to launch the
core and kernels. Eclipse could provide calls, batch files or
whatever is needed to launch jupyter stuff. As a first step it
is perfectly ok if this relies on some locally installed,
external python engine with jupyer extensions installed.
does this answer your question?
Christian
On 05/31/2016 10:55 PM, Martin
Klösch wrote:
Hi,
Jupyter basically consists of 3 parts. The
core, clients, and kernels.
In the typical scenario users want to create a
notebook. This is done using a call to the Jupyter core
(the Jupyter executable). Clients can then connect and
display the notebook page (typically via a webbrowser).
To execute code from the notebook, the core needs to
launch a kernel. You can think of a kernel as an
interpreter doing REPL over sockets. The kernel opens a
list of sockets chosen by the core for different types of
commands. The information is given to the kernel using a
launch file (my term) as a parameter to the kernel
executable. We cannot influence this behavior without
modifying Jupyter (or at least I haven't found a way yet).
Tobias already has some code to launch the
Jupyter core and connect with clients, so this is not the
problem.
My current issue is the launch of EASE as a
kernel. As described, the typical scenario is to launch an
interpreter in a new process. Since we actually want to
connect to a running eclipse instance rather than
launching a new one, we need to find a solution. My
current strategy is to start a simple command line tool
that passes the kernel launch file to a running eclipse.
The command line tool can be freely configured and will
receive the launch file as a parameter.
I might have been unclear on my plans in the
last mail. I implemented a command line tool that receives
the launch file and the socket information for the running
eclipse instance as parameters. It simply tries to read
the file, connect to eclipse and send the configuration
over the socket.
In the future eclipse should try to publish its startup to
Jupyter and choose a port for each instance. We then can
have a kernel for each running instance and can choose
which one we want to use from the client. For this I will
have a look at your code and hopefully can reuse some (the
port choosing mechanism).
For now, to be able to progress with the actual
kernel implementation, I chose a port to be used and
skipped the launch registration part. The basic concept
stays the same but I can focus on the rest of the code.
Hope this sheds some light,
Best regards,
Martin
Am 31.05.2016 14:32 schrieb "Jonah
Graham" < jonah@xxxxxxxxxxxxxxxx>:
>
Does anyone of you know of a better solution to this
problem?
As Christian mentioned I think some more context may
help to address this.
You can solve the issue of allocating ports though.
Depending on who
is doing which part of the launching may affect this.
For example when
launching Python from EASE I need unique ports in each
direction. I
allocate first a port in Java, then launch Python, have
Python
allocate the return direction port number and then
Python calls back
into Java to notify it of the port.
Have a look of the current draft implementation of the
Python engine
(using Py4J for comms):
https://git.eclipse.org/r/#/c/73149/7/plugins/org.eclipse.ease.lang.python.py4j/src/org/eclipse/ease/lang/python/py4j/internal/Py4jScriptEngine.java
In that you should be able to see that:
1. setupEngine starts the Java server side (setting java
and python
ports to 0, meaning automatically allocate free port),
2. launches the Python process, passing the actually
allocated port
3. waits for Python to notify it of the port in the
other direction
(Python does this by calling back into Java method
pythonStartupComplete)
To allocate an unused port, use 0 when creating your
socket, see
http://stackoverflow.com/questions/2675362/how-to-find-an-available-port
for a good example.
HTH,
Jonah
~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com
On 29 May 2016 at 22:55, Martin Klösch <martin.kloesch@xxxxxxxxx>
wrote:
> Hi,
>
> I've been working on launcher strategies for the
EASE Jupyter kernels this
> week.
>
> The flow for launching Jupyter kernels is basically
the opposite of what we
> want. To launch a new kernel, an executable is
called with command line
> parameters, the most important of which specify the
ports that will be used
> for communication. We would like to tell Jupyter
which ports we are using as
> to have more control and be more customizable from
within eclipse.
>
> I currently do not see an easy solution to this
problem, so I decided to
> simply create a command line tool that will be
called by Jupyter and
> dispatches the parameters over a socket to a
running eclipse instance. This
> strategy has some downsides, the most obvious being
that with this simple
> solution we can only have one eclipse instance
running at a time.
>
> I would still like to keep this method for now as
it allows me to continue
> with the actual implementation of the kernels. In
the future it would be
> possible to dynamically add running eclipse
instance to Jupyter using the
> same method but with different dispatching ports
for each. This code would
> not be platform independent as Jupyter has
different locations for its
> kernel information on each OS.
>
> Does anyone of you know of a better solution to
this problem?
>
> Besides working on this topic, Tobias from the
Science working group has
> given me his code regarding Jupyter clients in
eclipse. I have been checking
> the code and it looks as if I could reuse some of
it for the kernels as well
> (mostly zmq messaging). You can check out his code
at:
> https://github.com/openanalytics/japyter
> https://github.com/openanalytics/jupyter-console
>
> Thanks again to him and the guys from the Science
working group for the
> great collaboration. I think this project can help
both teams and get EASE
> and eclipse even further...
>
> Best regards,
> Martin
>
>
> _______________________________________________
> ease-dev mailing list
> ease-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your
password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/ease-dev
_______________________________________________
ease-dev mailing list
ease-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password,
or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ease-dev
_______________________________________________
ease-dev mailing list
ease-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ease-dev
_______________________________________________
ease-dev mailing list
ease-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ease-dev
_______________________________________________
ease-dev mailing list
ease-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ease-dev
Dieser Nachrichteninhalt wird auf Anfrage komplett heruntergeladen.
|
