Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Service Oriented Architecture Tools Platform (STP) » Is BPMN and BPEL the right choice for my problem?
Is BPMN and BPEL the right choice for my problem? [message #375077] Tue, 09 June 2009 14:07 Go to next message
David Wynter is currently offline David Wynter
Messages: 4578
Registered: July 2009
Senior Member
Hi,

I have a set of components that I use in workflows to basically do value
added ETL work with reported outcomes etc. It currently has a Web
interface and uses a FSM (blissed.codehaus.org) interleaved with the
Quartz scheduler. All data about the ETL reports, scheduling etc is
stored in the database, no files involved.

I want to now extend this to incorporate tasks for humans to handle
through dedicated screens in amongst the workflow. So time for a richer
graphical representation of the workflow. The BPMN modeller and
transformation to IM and thus to BPEL look like a good starting point. I
would replace the Quartz and FSM workflow with a BPEL execution engine.

I have read a lot of the material covering STP over the last couple of
days. But a few question have formed probably because I have not read
everything yet.

Apologies if my language is not precisely aligned with all the STP
buzzwords.

1. Since I have many instances of an ETL involving a particular pair of
input and output I need to have a clear display of which ETL we are
dealing with. Is the annotation extension to the task icon the correct
way of presenting these?

2. Can the annotation be transformed through to the BPEL?

3. Do the BPEL engines allow for execution of a component using the
annotation to indicate to the component being run?

4. Since this is not a general business workflow is it acceptable to add
specific task icons for the specialized tasks we carry out (e.g. an ETL
task?)

thx.

David
Re: Is BPMN and BPEL the right choice for my problem? [message #375078 is a reply to message #375077] Tue, 09 June 2009 14:37 Go to previous messageGo to next message
Eclipse User
Originally posted by: atoulme.intalio.com

Hi David, see comments below:

David Wynter wrote:
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
Please clarify what you mean by ETL exactly, I'm not familiar with this
acronym.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
I suppose it would be correct to show your special tasks with an annotation.
>
> 2. Can the annotation be transformed through to the BPEL?
That would be a question for the STP-IM maintainers. My blind guess is
that you would need to hack something to achieve it.
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
I don't understand the question. Do you mean that you'd like the BPEL
engine to execute your component because the task would have an annotation ?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
You can do that in BPMN.


OK, it looks like you are trying to customize BPMN and BPEL to carry on
some ETL thingy. Neither BPMN nor BPEL are extensible when it comes to
executing, however you could probably represent an ETL task in BPEL and
model it as pattern in BPMN.
>
> thx.
>
> David
Re: Is BPMN and BPEL the right choice for my problem? [message #375079 is a reply to message #375077] Tue, 09 June 2009 14:47 Go to previous messageGo to next message
Oisin Hurley is currently offline Oisin Hurley
Messages: 204
Registered: July 2009
Senior Member
Hi David,

David Wynter wrote:
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.

I'm trying to picture this... you have a number of workflows, that are
performing ETL tasks. The workflows are implemented using a FSM
and a Quartz scheduler. There are reports generated as the result of
a workflow happening. I'm simplifying here a bit, I know, but I'm
trying to cut it down to some core capabilities...

> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.

The BPMN modeller would be a good choice for presenting your flows,
I think. I'm not so sure, however, about whether BPEL is really where you
want to go. BPEL has a number of specific technology requirements (in
the general case) that you might find puts more pressure on your existing
architecture than you want.

For example, is your architecture based on web services? If not, BPEL means
that you will need to express the endpoints that you are coordinating as
web services. This may mean you need to use adapters to the real sources.
This can be a lot of overhead. From the details you mention of your system
I think that maybe something like Apache Camel could help you out in a
lighterweight way : http://camel.apache.org

Antoine will no doubt answer the BPMN modeller related questions
you have :) One thing to point out is that STP doesn't provide a way to
turn BPMN to BPEL, so that will be an issue.

best regards
Oisin
Re: Is BPMN and BPEL the right choice for my problem? [message #375080 is a reply to message #375079] Tue, 09 June 2009 15:00 Go to previous messageGo to next message
David Wynter is currently offline David Wynter
Messages: 4578
Registered: July 2009
Senior Member
Hi,

For Antione, ETL is "Extract Transform Load" but this simplifies what we
do. Currently the FSM definition ( in XML ) just passes in the name of
the instance of the ETL task and ETL component then knows which one to use.

What you say about BPEL makes sense, and when I think about it
decorating the various components with the stuff needed to make them web
services adds complexity. It sounds like it might be easier to create my
own IM->FSM transformation and use the existing workflow engine I have.

So long as the annotation about which particular ETL task is to be run
in the workflow is passed through the BPMN->IM transformation then I can
take it from there.

Thx.

David

Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
Re: [stp-newsgroup] Re: Is BPMN and BPEL the right choice for my problem? [message #375081 is a reply to message #375080] Tue, 09 June 2009 15:05 Go to previous messageGo to next message
Antoine Toulmé is currently offline Antoine Toulmé
Messages: 31
Registered: July 2009
Member
--0015174c0fa85c3e14046bebb033
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Indeed. If you have questions when extending the modeler, please ask them on
the BPMN newsgroup, we'll help. We already provide a dozen of samples on how
to extend it, as well as our talk of EclipseCon 2008.
Thanks,

Antoine

On Tue, Jun 9, 2009 at 17:00, David Wynter <david@roamware.co.uk> wrote:

> Hi,
>
> For Antione, ETL is "Extract Transform Load" but this simplifies what we
> do. Currently the FSM definition ( in XML ) just passes in the name of the
> instance of the ETL task and ETL component then knows which one to use.
>
> What you say about BPEL makes sense, and when I think about it decorating
> the various components with the stuff needed to make them web services adds
> complexity. It sounds like it might be easier to create my own IM->FSM
> transformation and use the existing workflow engine I have.
>
> So long as the annotation about which particular ETL task is to be run in
> the workflow is passed through the BPMN->IM transformation then I can take
> it from there.
>
> Thx.
>
> David
>
>
> Oisin Hurley wrote:
>
>>
>> Hi David,
>>
>> David Wynter wrote:
>>
>>> I have a set of components that I use in workflows to basically do value
>>> added ETL work with reported outcomes etc. It currently has a Web interface
>>> and uses a FSM (blissed.codehaus.org) interleaved with the Quartz
>>> scheduler. All data about the ETL reports, scheduling etc is stored in the
>>> database, no files involved.
>>>
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a richer
>>> graphical representation of the workflow. The BPMN modeller and
>>> transformation to IM and thus to BPEL look like a good starting point. I
>>> would replace the Quartz and FSM workflow with a BPEL execution engine.
>>>
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
>> _______________________________________________
> stp-newsgroup mailing list
> stp-newsgroup@eclipse.org
> https://dev.eclipse.org/mailman/listinfo/stp-newsgroup
>

--0015174c0fa85c3e14046bebb033
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Indeed. If you have questions when extending the modeler, please ask them o=
n the BPMN newsgroup, we&#39;ll help. We already provide a dozen of samples=
on how to extend it, as well as our talk of EclipseCon 2008.<div><br></div=
>

<div>Thanks,</div><div><br></div><div>Antoine</div><div ><br><div class=3D"g=
mail_quote">On Tue, Jun 9, 2009 at 17:00, David Wynter <span dir=3D"ltr">&l=
t;<a href=3D"mailto:david@roamware.co.uk">david@roamware.co.uk</a>&gt;</spa=
n> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">Hi,<br>
<br>
For Antione, ETL is &quot;Extract Transform Load&quot; but this simplifies =
what we do. Currently the FSM definition ( in XML ) just passes in the name=
of the instance of the ETL task and ETL component then knows which one to =
use.<br>


<br>
What you say about BPEL makes sense, and when I think about it decorating t=
he various components with the stuff needed to make them web services adds =
complexity. It sounds like it might be easier to create my own IM-&gt;FSM t=
ransformation and use the existing workflow engine I have.<br>


<br>
So long as the annotation about which particular ETL task is to be run in t=
he workflow is passed through the BPMN-&gt;IM transformation then I can tak=
e it from there.<br>
<br>
Thx.<br><font color=3D"#888888">
<br>
David</font><div><div></div><div class=3D"h5"><br>
<br>
Oisin Hurley wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<br>
Hi David,<br>
<br>
David Wynter wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I have a set of components that I use in workflows to basically do value ad=
ded ETL work with reported outcomes etc. It currently has a Web interface a=
nd uses a FSM (<a href=3D"http://blissed.codehaus.org" target=3D"_blank">bl=
issed.codehaus.org</a>) interleaved with the Quartz scheduler. All data abo=
ut the ETL reports, scheduling etc is stored in the database, no files invo=
lved.<br>


</blockquote>
<br>
I&#39;m trying to picture this... you have a number of workflows, that are<=
br>
performing ETL tasks. The workflows are implemented using a FSM<br>
and a Quartz scheduler. There are reports generated as the result of<br>
a workflow happening. I&#39;m simplifying here a bit, I know, but I&#39;m<b=
r>
trying to cut it down to some core capabilities...<br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I want to now extend this to incorporate tasks for humans to handle through=
dedicated screens in amongst the workflow. So time for a richer graphical =
representation of the workflow. The BPMN modeller and transformation to IM =
and thus to BPEL look like a good starting point. I would replace the Quart=
z and FSM workflow with a BPEL execution engine.<br>


</blockquote>
<br>
The BPMN modeller would be a good choice for presenting your flows,<br>
I think. I&#39;m not so sure, however, about whether BPEL is really where y=
ou<br>
want to go. BPEL has a number of specific technology requirements (in<br>
the general case) that you might find puts more pressure on your existing<b=
r>
architecture than you want.<br>
<br>
For example, is your architecture based on web services? If not, BPEL means=
<br>
that you will need to express the endpoints that you are coordinating as<br=
>
web services. This may mean you need to use adapters to the real sources.<b=
r>
This can be a lot of overhead. From the details you mention of your system<=
br>
I think that maybe something like Apache Camel could help you out in a<br>
lighterweight way : =A0<a href=3D"http://camel.apache.org" target=3D"_blank=
">http://camel.apache.org</a><br>
<br>
Antoine will no doubt answer the BPMN modeller related questions<br>
you have :) =A0One thing to point out is that STP doesn&#39;t provide a way=
to<br>
turn BPMN to BPEL, so that will be an issue.<br>
<br>
best regards<br>
=A0Oisin<br>
<br>
</blockquote>
_______________________________________________<br>
stp-newsgroup mailing list<br>
<a href=3D"mailto:stp-newsgroup@eclipse.org" target=3D"_blank">stp-newsgrou=
p@eclipse.org</a><br>
<a href=3D"https://dev.eclipse.org/mailman/listinfo/stp-newsgroup" target=
=3D"_blank">https://dev.eclipse.org/mailman/listinfo/stp-newsgroup</a><br>
</div></div></blockquote></div><br></div>

--0015174c0fa85c3e14046bebb033--
Re: Is BPMN and BPEL the right choice for my problem? [message #375082 is a reply to message #375079] Tue, 09 June 2009 15:08 Go to previous messageGo to next message
David Wynter is currently offline David Wynter
Messages: 4578
Registered: July 2009
Senior Member
Hi again,

Camel looks like a good choice, except for two things.

1. Can the BPMN be transformed to suit two coordinated execution
environments, one for the ESB stuff in Camel and one for the human
interactions with screens that depend on certain outcomes from the ETL
tasks executing in the ESB?

2. Is there any transformation from the BPMN to one of Camels DSLs? (
Scala, Spring XML config or Fluent API) I could find nothing. I suppose
I could do a IM->Scala transformation...

thx.

David

Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
Re: Is BPMN and BPEL the right choice for my problem? [message #375083 is a reply to message #375077] Tue, 09 June 2009 18:58 Go to previous messageGo to next message
Marc Dutoo is currently offline Marc Dutoo
Messages: 71
Registered: July 2009
Member
Hi David

If you don't need web service integration nor composition (which seems
so, since you previously used a non distributed java FSM), rather than
using BPEL, I'd suggest to stay on the workflow side and take a look at
Scarbo ( http://scarbo.ow2.org ). It integrates the multipurpose (and
STP BPMN, IM compatible) Workflow Editor of the Eclipse Java Workflow
Tooling project ( http://eclipse.org/jwt ) to the XPDL-based Bonita 4
workflow engine and console ( http://bonita.ow2.org ).

I admit it is a shameless plug since I head JWT :) But I think it makes
sense, since it would provide full task extensibility (model and UI, see
what we've done on e.g. Groovy tasks), a true executable workflow format
(XPDL), workflow execution management and monitoring (in Bonita
console), still import / export with STP BPMN...

Regards,
Marc

David Wynter a écrit :
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
>
> 2. Can the annotation be transformed through to the BPEL?
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
>
> thx.
>
> David
Re: Is BPMN and BPEL the right choice for my problem? [message #375084 is a reply to message #375082] Tue, 09 June 2009 19:06 Go to previous messageGo to next message
Marc Dutoo is currently offline Marc Dutoo
Messages: 71
Registered: July 2009
Member
Hi again

(From my use of STP IM)

On this order of things (starting from BPMN and going in different,
complementary directions + code generation), have a look at what Spagic
( http://www.spagoworld.org/ecm/faces/public/guest/home/soluti ons/spagic
) does thanks to the STP BPMN editor, STP IM and some JET templating.
Either it'll give you ideas, or you'll want to write your own JET code
generation extension... Though I don't know exactly what it does for
human tasks, Andrea maybe ?

You could also use JWT as I wrote above and write a (simple or
composite) JWT transformation that exports parts of your model to Camel
DSLs, using the provided basis and samples (e.g. XSL transformation, ATL
model-to-model transformation)

Regards,
Marc

David Wynter a écrit :
> Hi again,
>
> Camel looks like a good choice, except for two things.
>
> 1. Can the BPMN be transformed to suit two coordinated execution
> environments, one for the ESB stuff in Camel and one for the human
> interactions with screens that depend on certain outcomes from the ETL
> tasks executing in the ESB?
>
> 2. Is there any transformation from the BPMN to one of Camels DSLs? (
> Scala, Spring XML config or Fluent API) I could find nothing. I suppose
> I could do a IM->Scala transformation...
>
> thx.
>
> David
>
> Oisin Hurley wrote:
>>
>> Hi David,
>>
>> David Wynter wrote:
>>> I have a set of components that I use in workflows to basically do
>>> value added ETL work with reported outcomes etc. It currently has a
>>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>>> is stored in the database, no files involved.
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a
>>> richer graphical representation of the workflow. The BPMN modeller
>>> and transformation to IM and thus to BPEL look like a good starting
>>> point. I would replace the Quartz and FSM workflow with a BPEL
>>> execution engine.
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your
>> system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
Re: Is BPMN and BPEL the right choice for my problem? [message #375085 is a reply to message #375077] Mon, 15 June 2009 07:37 Go to previous messageGo to next message
Andrea Zoppello is currently offline Andrea Zoppello
Messages: 48
Registered: July 2009
Member
Hi David,

If you are interested you could look at spagic platform here:

http://forge.ow2.org/projects/spagic

It's licensed under the LGPL paltform.

Actually we are using STP->IM stuff and then from the IM we are able to
provide the following two transformations

IM-> JBI Service Assemby for servicemix
IM-> BPEL ( In alpha stage )

Regards your questions:

1) In spagic we provide a JBI components that is able to integrate ETL
jobs designed with Talend, basically you must design your etl processes
with talend and then you could use the code generated by talend by our jbi
component

2) Regards workflow you could look at spagic workflow in spagic-2.5.x
where
workflow is simply another type of runtime you could use to annotate your
bpmn
editor..

Actually we are able to generate code for JBPM 3.2.x and in future we plan
to support JBPM 4.X.

We are working a lot on this and we are planning to integrate form support
for human based activities

for more information about this: Look at spagic-workflow documents.

3) Regarding BPEL things are a little more complex, this is because is
very difficult to produce a valid bpel file without haveing restrictions
on bpmn model ( this is basicallu because bpel is a structured language
instead bpmn is not ) and then bpel require partner links definitions and
so on, so the spagic approach is that the IM -> BPEL transformation
produce a bpel file that you must complete with bpel editor to complete.

For all the above you could have monitoring capabilities provided by
spagic console to keep your process monitored.

Andrea Zoppello

David Wynter wrote:

> Hi,

> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.

> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.

> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.

> Apologies if my language is not precisely aligned with all the STP
> buzzwords.

> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?

> 2. Can the annotation be transformed through to the BPEL?

> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?

> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)

> thx.

> David
Hi
Re: Is BPMN and BPEL the right choice for my problem? [message #375090 is a reply to message #375082] Thu, 18 June 2009 13:06 Go to previous message
Oisin Hurley is currently offline Oisin Hurley
Messages: 204
Registered: July 2009
Senior Member
Late getting back to this, my apologies.

David Wynter wrote:

> Camel looks like a good choice, except for two things.

> 1. Can the BPMN be transformed to suit two coordinated execution
> environments, one for the ESB stuff in Camel and one for the human
> interactions with screens that depend on certain outcomes from the ETL
> tasks executing in the ESB?

I think that you would be breaking new ground here in
bringing the two together.

> 2. Is there any transformation from the BPMN to one of Camels DSLs? (
> Scala, Spring XML config or Fluent API) I could find nothing. I suppose
> I could do a IM->Scala transformation...

Nothing like that exists at the moment, I'm afraid. I don't know
enough about BPMN to even give a reasonable guess as to how
they would work together.

--oh
Re: Is BPMN and BPEL the right choice for my problem? [message #621209 is a reply to message #375077] Tue, 09 June 2009 14:37 Go to previous message
Eclipse User
Originally posted by: atoulme.intalio.com

Hi David, see comments below:

David Wynter wrote:
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
Please clarify what you mean by ETL exactly, I'm not familiar with this
acronym.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
I suppose it would be correct to show your special tasks with an annotation.
>
> 2. Can the annotation be transformed through to the BPEL?
That would be a question for the STP-IM maintainers. My blind guess is
that you would need to hack something to achieve it.
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
I don't understand the question. Do you mean that you'd like the BPEL
engine to execute your component because the task would have an annotation ?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
You can do that in BPMN.


OK, it looks like you are trying to customize BPMN and BPEL to carry on
some ETL thingy. Neither BPMN nor BPEL are extensible when it comes to
executing, however you could probably represent an ETL task in BPEL and
model it as pattern in BPMN.
>
> thx.
>
> David
Re: Is BPMN and BPEL the right choice for my problem? [message #621210 is a reply to message #375077] Tue, 09 June 2009 14:47 Go to previous message
Oisin Hurley is currently offline Oisin Hurley
Messages: 204
Registered: July 2009
Senior Member
Hi David,

David Wynter wrote:
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.

I'm trying to picture this... you have a number of workflows, that are
performing ETL tasks. The workflows are implemented using a FSM
and a Quartz scheduler. There are reports generated as the result of
a workflow happening. I'm simplifying here a bit, I know, but I'm
trying to cut it down to some core capabilities...

> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.

The BPMN modeller would be a good choice for presenting your flows,
I think. I'm not so sure, however, about whether BPEL is really where you
want to go. BPEL has a number of specific technology requirements (in
the general case) that you might find puts more pressure on your existing
architecture than you want.

For example, is your architecture based on web services? If not, BPEL means
that you will need to express the endpoints that you are coordinating as
web services. This may mean you need to use adapters to the real sources.
This can be a lot of overhead. From the details you mention of your system
I think that maybe something like Apache Camel could help you out in a
lighterweight way : http://camel.apache.org

Antoine will no doubt answer the BPMN modeller related questions
you have :) One thing to point out is that STP doesn't provide a way to
turn BPMN to BPEL, so that will be an issue.

best regards
Oisin
Re: Is BPMN and BPEL the right choice for my problem? [message #621211 is a reply to message #375079] Tue, 09 June 2009 15:00 Go to previous message
David Wynter is currently offline David Wynter
Messages: 4578
Registered: July 2009
Senior Member
Hi,

For Antione, ETL is "Extract Transform Load" but this simplifies what we
do. Currently the FSM definition ( in XML ) just passes in the name of
the instance of the ETL task and ETL component then knows which one to use.

What you say about BPEL makes sense, and when I think about it
decorating the various components with the stuff needed to make them web
services adds complexity. It sounds like it might be easier to create my
own IM->FSM transformation and use the existing workflow engine I have.

So long as the annotation about which particular ETL task is to be run
in the workflow is passed through the BPMN->IM transformation then I can
take it from there.

Thx.

David

Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
Re: [stp-newsgroup] Re: Is BPMN and BPEL the right choice for my problem? [message #621212 is a reply to message #375080] Tue, 09 June 2009 15:05 Go to previous message
Antoine Toulmé is currently offline Antoine Toulmé
Messages: 31
Registered: July 2009
Member
--0015174c0fa85c3e14046bebb033
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Indeed. If you have questions when extending the modeler, please ask them on
the BPMN newsgroup, we'll help. We already provide a dozen of samples on how
to extend it, as well as our talk of EclipseCon 2008.
Thanks,

Antoine

On Tue, Jun 9, 2009 at 17:00, David Wynter <david@roamware.co.uk> wrote:

> Hi,
>
> For Antione, ETL is "Extract Transform Load" but this simplifies what we
> do. Currently the FSM definition ( in XML ) just passes in the name of the
> instance of the ETL task and ETL component then knows which one to use.
>
> What you say about BPEL makes sense, and when I think about it decorating
> the various components with the stuff needed to make them web services adds
> complexity. It sounds like it might be easier to create my own IM->FSM
> transformation and use the existing workflow engine I have.
>
> So long as the annotation about which particular ETL task is to be run in
> the workflow is passed through the BPMN->IM transformation then I can take
> it from there.
>
> Thx.
>
> David
>
>
> Oisin Hurley wrote:
>
>>
>> Hi David,
>>
>> David Wynter wrote:
>>
>>> I have a set of components that I use in workflows to basically do value
>>> added ETL work with reported outcomes etc. It currently has a Web interface
>>> and uses a FSM (blissed.codehaus.org) interleaved with the Quartz
>>> scheduler. All data about the ETL reports, scheduling etc is stored in the
>>> database, no files involved.
>>>
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a richer
>>> graphical representation of the workflow. The BPMN modeller and
>>> transformation to IM and thus to BPEL look like a good starting point. I
>>> would replace the Quartz and FSM workflow with a BPEL execution engine.
>>>
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
>> _______________________________________________
> stp-newsgroup mailing list
> stp-newsgroup@eclipse.org
> https://dev.eclipse.org/mailman/listinfo/stp-newsgroup
>

--0015174c0fa85c3e14046bebb033
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Indeed. If you have questions when extending the modeler, please ask them o=
n the BPMN newsgroup, we&#39;ll help. We already provide a dozen of samples=
on how to extend it, as well as our talk of EclipseCon 2008.<div><br></div=
>

<div>Thanks,</div><div><br></div><div>Antoine</div><div ><br><div class=3D"g=
mail_quote">On Tue, Jun 9, 2009 at 17:00, David Wynter <span dir=3D"ltr">&l=
t;<a href=3D"mailto:david@roamware.co.uk">david@roamware.co.uk</a>&gt;</spa=
n> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">Hi,<br>
<br>
For Antione, ETL is &quot;Extract Transform Load&quot; but this simplifies =
what we do. Currently the FSM definition ( in XML ) just passes in the name=
of the instance of the ETL task and ETL component then knows which one to =
use.<br>


<br>
What you say about BPEL makes sense, and when I think about it decorating t=
he various components with the stuff needed to make them web services adds =
complexity. It sounds like it might be easier to create my own IM-&gt;FSM t=
ransformation and use the existing workflow engine I have.<br>


<br>
So long as the annotation about which particular ETL task is to be run in t=
he workflow is passed through the BPMN-&gt;IM transformation then I can tak=
e it from there.<br>
<br>
Thx.<br><font color=3D"#888888">
<br>
David</font><div><div></div><div class=3D"h5"><br>
<br>
Oisin Hurley wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<br>
Hi David,<br>
<br>
David Wynter wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I have a set of components that I use in workflows to basically do value ad=
ded ETL work with reported outcomes etc. It currently has a Web interface a=
nd uses a FSM (<a href=3D"http://blissed.codehaus.org" target=3D"_blank">bl=
issed.codehaus.org</a>) interleaved with the Quartz scheduler. All data abo=
ut the ETL reports, scheduling etc is stored in the database, no files invo=
lved.<br>


</blockquote>
<br>
I&#39;m trying to picture this... you have a number of workflows, that are<=
br>
performing ETL tasks. The workflows are implemented using a FSM<br>
and a Quartz scheduler. There are reports generated as the result of<br>
a workflow happening. I&#39;m simplifying here a bit, I know, but I&#39;m<b=
r>
trying to cut it down to some core capabilities...<br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I want to now extend this to incorporate tasks for humans to handle through=
dedicated screens in amongst the workflow. So time for a richer graphical =
representation of the workflow. The BPMN modeller and transformation to IM =
and thus to BPEL look like a good starting point. I would replace the Quart=
z and FSM workflow with a BPEL execution engine.<br>


</blockquote>
<br>
The BPMN modeller would be a good choice for presenting your flows,<br>
I think. I&#39;m not so sure, however, about whether BPEL is really where y=
ou<br>
want to go. BPEL has a number of specific technology requirements (in<br>
the general case) that you might find puts more pressure on your existing<b=
r>
architecture than you want.<br>
<br>
For example, is your architecture based on web services? If not, BPEL means=
<br>
that you will need to express the endpoints that you are coordinating as<br=
>
web services. This may mean you need to use adapters to the real sources.<b=
r>
This can be a lot of overhead. From the details you mention of your system<=
br>
I think that maybe something like Apache Camel could help you out in a<br>
lighterweight way : =A0<a href=3D"http://camel.apache.org" target=3D"_blank=
">http://camel.apache.org</a><br>
<br>
Antoine will no doubt answer the BPMN modeller related questions<br>
you have :) =A0One thing to point out is that STP doesn&#39;t provide a way=
to<br>
turn BPMN to BPEL, so that will be an issue.<br>
<br>
best regards<br>
=A0Oisin<br>
<br>
</blockquote>
_______________________________________________<br>
stp-newsgroup mailing list<br>
<a href=3D"mailto:stp-newsgroup@eclipse.org" target=3D"_blank">stp-newsgrou=
p@eclipse.org</a><br>
<a href=3D"https://dev.eclipse.org/mailman/listinfo/stp-newsgroup" target=
=3D"_blank">https://dev.eclipse.org/mailman/listinfo/stp-newsgroup</a><br>
</div></div></blockquote></div><br></div>

--0015174c0fa85c3e14046bebb033--
Re: Is BPMN and BPEL the right choice for my problem? [message #621213 is a reply to message #375079] Tue, 09 June 2009 15:08 Go to previous message
David Wynter is currently offline David Wynter
Messages: 4578
Registered: July 2009
Senior Member
Hi again,

Camel looks like a good choice, except for two things.

1. Can the BPMN be transformed to suit two coordinated execution
environments, one for the ESB stuff in Camel and one for the human
interactions with screens that depend on certain outcomes from the ETL
tasks executing in the ESB?

2. Is there any transformation from the BPMN to one of Camels DSLs? (
Scala, Spring XML config or Fluent API) I could find nothing. I suppose
I could do a IM->Scala transformation...

thx.

David

Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
Re: Is BPMN and BPEL the right choice for my problem? [message #621214 is a reply to message #375077] Tue, 09 June 2009 18:58 Go to previous message
Marc Dutoo is currently offline Marc Dutoo
Messages: 71
Registered: July 2009
Member
Hi David

If you don't need web service integration nor composition (which seems
so, since you previously used a non distributed java FSM), rather than
using BPEL, I'd suggest to stay on the workflow side and take a look at
Scarbo ( http://scarbo.ow2.org ). It integrates the multipurpose (and
STP BPMN, IM compatible) Workflow Editor of the Eclipse Java Workflow
Tooling project ( http://eclipse.org/jwt ) to the XPDL-based Bonita 4
workflow engine and console ( http://bonita.ow2.org ).

I admit it is a shameless plug since I head JWT :) But I think it makes
sense, since it would provide full task extensibility (model and UI, see
what we've done on e.g. Groovy tasks), a true executable workflow format
(XPDL), workflow execution management and monitoring (in Bonita
console), still import / export with STP BPMN...

Regards,
Marc

David Wynter a écrit :
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
>
> 2. Can the annotation be transformed through to the BPEL?
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
>
> thx.
>
> David
Re: Is BPMN and BPEL the right choice for my problem? [message #621215 is a reply to message #375082] Tue, 09 June 2009 19:06 Go to previous message
Marc Dutoo is currently offline Marc Dutoo
Messages: 71
Registered: July 2009
Member
Hi again

(From my use of STP IM)

On this order of things (starting from BPMN and going in different,
complementary directions + code generation), have a look at what Spagic
( http://www.spagoworld.org/ecm/faces/public/guest/home/soluti ons/spagic
) does thanks to the STP BPMN editor, STP IM and some JET templating.
Either it'll give you ideas, or you'll want to write your own JET code
generation extension... Though I don't know exactly what it does for
human tasks, Andrea maybe ?

You could also use JWT as I wrote above and write a (simple or
composite) JWT transformation that exports parts of your model to Camel
DSLs, using the provided basis and samples (e.g. XSL transformation, ATL
model-to-model transformation)

Regards,
Marc

David Wynter a écrit :
> Hi again,
>
> Camel looks like a good choice, except for two things.
>
> 1. Can the BPMN be transformed to suit two coordinated execution
> environments, one for the ESB stuff in Camel and one for the human
> interactions with screens that depend on certain outcomes from the ETL
> tasks executing in the ESB?
>
> 2. Is there any transformation from the BPMN to one of Camels DSLs? (
> Scala, Spring XML config or Fluent API) I could find nothing. I suppose
> I could do a IM->Scala transformation...
>
> thx.
>
> David
>
> Oisin Hurley wrote:
>>
>> Hi David,
>>
>> David Wynter wrote:
>>> I have a set of components that I use in workflows to basically do
>>> value added ETL work with reported outcomes etc. It currently has a
>>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>>> is stored in the database, no files involved.
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a
>>> richer graphical representation of the workflow. The BPMN modeller
>>> and transformation to IM and thus to BPEL look like a good starting
>>> point. I would replace the Quartz and FSM workflow with a BPEL
>>> execution engine.
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your
>> system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
Re: Is BPMN and BPEL the right choice for my problem? [message #621216 is a reply to message #375077] Mon, 15 June 2009 07:37 Go to previous message
Andrea Zoppello is currently offline Andrea Zoppello
Messages: 48
Registered: July 2009
Member
Hi David,

If you are interested you could look at spagic platform here:

http://forge.ow2.org/projects/spagic

It's licensed under the LGPL paltform.

Actually we are using STP->IM stuff and then from the IM we are able to
provide the following two transformations

IM-> JBI Service Assemby for servicemix
IM-> BPEL ( In alpha stage )

Regards your questions:

1) In spagic we provide a JBI components that is able to integrate ETL
jobs designed with Talend, basically you must design your etl processes
with talend and then you could use the code generated by talend by our jbi
component

2) Regards workflow you could look at spagic workflow in spagic-2.5.x
where
workflow is simply another type of runtime you could use to annotate your
bpmn
editor..

Actually we are able to generate code for JBPM 3.2.x and in future we plan
to support JBPM 4.X.

We are working a lot on this and we are planning to integrate form support
for human based activities

for more information about this: Look at spagic-workflow documents.

3) Regarding BPEL things are a little more complex, this is because is
very difficult to produce a valid bpel file without haveing restrictions
on bpmn model ( this is basicallu because bpel is a structured language
instead bpmn is not ) and then bpel require partner links definitions and
so on, so the spagic approach is that the IM -> BPEL transformation
produce a bpel file that you must complete with bpel editor to complete.

For all the above you could have monitoring capabilities provided by
spagic console to keep your process monitored.

Andrea Zoppello

David Wynter wrote:

> Hi,

> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.

> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.

> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.

> Apologies if my language is not precisely aligned with all the STP
> buzzwords.

> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?

> 2. Can the annotation be transformed through to the BPEL?

> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?

> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)

> thx.

> David
Hi
Re: Is BPMN and BPEL the right choice for my problem? [message #621221 is a reply to message #375082] Thu, 18 June 2009 13:06 Go to previous message
Oisin Hurley is currently offline Oisin Hurley
Messages: 204
Registered: July 2009
Senior Member
Late getting back to this, my apologies.

David Wynter wrote:

> Camel looks like a good choice, except for two things.

> 1. Can the BPMN be transformed to suit two coordinated execution
> environments, one for the ESB stuff in Camel and one for the human
> interactions with screens that depend on certain outcomes from the ETL
> tasks executing in the ESB?

I think that you would be breaking new ground here in
bringing the two together.

> 2. Is there any transformation from the BPMN to one of Camels DSLs? (
> Scala, Spring XML config or Fluent API) I could find nothing. I suppose
> I could do a IM->Scala transformation...

Nothing like that exists at the moment, I'm afraid. I don't know
enough about BPMN to even give a reasonable guess as to how
they would work together.

--oh
Previous Topic:Traverse a BPMN diagram
Next Topic:SOPERA Validation: XSD Validation: cvc-elt.1: Cannot find the declaration of element 'wsp:Policy'.
Goto Forum:
  


Current Time: Wed Oct 22 01:46:17 GMT 2014

Powered by FUDForum. Page generated in 0.03496 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software