Eclipse Community Forums: Sirius » Eclipse Sirius Web's Diagram manupulation by chatbot

Home » Eclipse Projects » Sirius » Eclipse Sirius Web's Diagram manupulation by chatbot(I am expecting a chatbot taking user input to handle multiple built-in properties of the diagram. How to attach chatbot with a Sirius Web?)

Show: Today's Messages :: Show Polls :: Message Navigator

Eclipse Sirius Web's Diagram manupulation by chatbot [message #1860588]

Fri, 18 August 2023 14:42

Suhas Devmane

Messages: 2
Registered: August 2023

Junior Member

Hello,

I'm Suhas. I am a PhD student; I am involved with the Eclipse Sirius Web and chatbot projects. My plan is to handle eclipse Sirius components in a project by receiving natural language input from the end user using a chatbot interface. For instance, renaming components (contextual palette), introducing parent-child relationships (Edges), colours, size present on a nested container. Instead of introducing new attributes/properties, I anticipate my chatbot to modify existing ones that already exist in the drawings.

I wanted to utilise a chatbot to take user input and manipulate Eclipse Sirius Web's built-in operations on the diagrams present on container to change various features/properties of the diagrams as user expected using Chatbot and the backend's API.

Could you kindly advise me on how to use the Eclipse Sirius Web and a chatbot (like Rasa) in conjunction? I'm having trouble figuring out how to connect a chatbot to it to perform diagram manipulation. Your ideas will greatly help me.

Sincerely,
Suhas

Report message to a moderator

Re: Eclipse Sirius Web's Diagram manupulation by chatbot [message #1860589 is a reply to message #1860588]

Fri, 18 August 2023 14:58

Pierre-Charles David

Messages: 705
Registered: July 2009

Senior Member

Hi,

Interesting subject!

I don't know anything about chat bots, but if you want to control a Sirius Web application programatically, you "simply" need to use the exposed GraphQL APIs.
Everything the frontend code does to query or manipulate the representations (including asking which tools are available on a given diagram element, i.e. in its contextual palette, or invoking such a tool) is done using GraphQL queries or mutations.

Depending on your familiarity with JavaScript/TypeScript/React, your best bet would be either to:
* look at the frontend's source code (for example https://github.com/eclipse-sirius/sirius-web/blob/master/packages/diagrams/frontend/sirius-components-diagrams/src/palette/ContextualPalette.tsx), or
* open your browser's dev tools in the "Network" tab, invoke some tool manually and look at the HTTP request sent by the frontend to invoke the tool in question (and then generalize to pass your own arguments you would have determined from the message in the chat).

For diagrams specifically, you'll get a good overview of the queries and mutations (edition operations, e.g. invoke a tool) by looking at the GraphQL Schema at https://github.com/eclipse-sirius/sirius-web/blob/master/packages/diagrams/backend/sirius-components-collaborative-diagrams/src/main/resources/schema/diagram.graphqls
For more generic operations, see https://github.com/eclipse-sirius/sirius-web/blob/master/packages/core/backend/sirius-components-collaborative/src/main/resources/schema/core.graphqls and https://github.com/eclipse-sirius/sirius-web/blob/master/packages/sirius-web/backend/sirius-web-graphql/src/main/resources/schema/siriusweb.graphqls.

Good luck, and if you get intersting results do not hesitate to post some screenshots or videos here to show your work to the community.

Regards,
Pierre-Charles

Pierre-Charles David - Obeo

Need training or professional services for Sirius?
http://www.obeodesigner.com/sirius

Report message to a moderator

Re: Eclipse Sirius Web's Diagram manupulation by chatbot [message #1860591 is a reply to message #1860589]

Fri, 18 August 2023 19:52

Ed Willink

Messages: 7677
Registered: July 2009

Senior Member

Indeed a very intersting topic but...

You have identified the need to establish a mapping from Natural Language to Sirius API. But how?

To allow AI techniques to prosper you need a large true corpus of examples. Unfortunately your sources are limited. The Javadoc is perhaps 50% missing and 10% inaccurate. Tutorials and documentation may have 50% coverage but be 10% stale. Forum discussions often involve poorly expressed misunderstandings so automated extraction may be unhelpful. With so little available or reliable you may have to develop the corpus manually.

You might do slightly better if you can generalize to many CASE tools and identify/re-use an underlying normalized drawing API so that you can start with a larger generic corpus and just manually handcraft a mapping from generic API to Sirius API.

Possibly relevant to you is the "Combining OCL and Natural Language: a Call for a Community Effort" paper from OCL 2022 [1]. This paper proposes building a corpus for NLtoOCL by providing an online 'game' in which users provide the OCL for example UML diagram challenges. You could try to do the same thing, but I'm not sure how many players you will get to give you competitive solutions to 'double the size of the property containers for all classes inheriting from ragbag'. Getting users to populate your corpus for 'free' requires a fun game and many capable users. Few users are familiar with the Sirius API.

(I doubt that you can make progress until you have become proficient in the use of the Sirius API.)

Regards

Ed Willink

[1] https://modeling-languages.com/combining-ocl-and-natural-language-a-call-for-a-community-effort/

Report message to a moderator

Re: Eclipse Sirius Web's Diagram manupulation by chatbot [message #1860693 is a reply to message #1860589]

Fri, 25 August 2023 13:07

Suhas Devmane

Messages: 2
Registered: August 2023

Junior Member

Thank you for your valuable suggestions. I am now reading documentation and Sirius Videos to find out how exactly I can implement this idea. I found a useful video by @Stephane Begaudeau, Sirius Web Architect @Obeo.fr, to Customize and Extend the platform. At first, it is hard to understand this customization. I would really appreciate it if I get some guidance/steps involved from Stephane or other Architects @Obeo.

Report message to a moderator

Previous Topic:	Get the label/value of an item selected from a Sirius List
Next Topic:	Multiple auto-arrange

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sun Oct 13 05:27:33 GMT 2024

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter