Home » Archived » Service Oriented Architecture Tools Platform (STP) » Is BPMN and BPEL the right choice for my problem?
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #375078 is a reply to message #375077] |
Tue, 09 June 2009 14:37   |
Eclipse User |
|
|
|
Originally posted by: atoulme.intalio.com
Hi David, see comments below:
David Wynter wrote:
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
Please clarify what you mean by ETL exactly, I'm not familiar with this
acronym.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
I suppose it would be correct to show your special tasks with an annotation.
>
> 2. Can the annotation be transformed through to the BPEL?
That would be a question for the STP-IM maintainers. My blind guess is
that you would need to hack something to achieve it.
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
I don't understand the question. Do you mean that you'd like the BPEL
engine to execute your component because the task would have an annotation ?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
You can do that in BPMN.
OK, it looks like you are trying to customize BPMN and BPEL to carry on
some ETL thingy. Neither BPMN nor BPEL are extensible when it comes to
executing, however you could probably represent an ETL task in BPEL and
model it as pattern in BPMN.
>
> thx.
>
> David
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #375079 is a reply to message #375077] |
Tue, 09 June 2009 14:47 |
 Oisin Hurley
Messages: 204
Registered: July 2009 |
Senior Member |
|
|
Hi David,
David Wynter wrote:
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
I'm trying to picture this... you have a number of workflows, that are
performing ETL tasks. The workflows are implemented using a FSM
and a Quartz scheduler. There are reports generated as the result of
a workflow happening. I'm simplifying here a bit, I know, but I'm
trying to cut it down to some core capabilities...
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
The BPMN modeller would be a good choice for presenting your flows,
I think. I'm not so sure, however, about whether BPEL is really where you
want to go. BPEL has a number of specific technology requirements (in
the general case) that you might find puts more pressure on your existing
architecture than you want.
For example, is your architecture based on web services? If not, BPEL means
that you will need to express the endpoints that you are coordinating as
web services. This may mean you need to use adapters to the real sources.
This can be a lot of overhead. From the details you mention of your system
I think that maybe something like Apache Camel could help you out in a
lighterweight way : http://camel.apache.org
Antoine will no doubt answer the BPMN modeller related questions
you have :) One thing to point out is that STP doesn't provide a way to
turn BPMN to BPEL, so that will be an issue.
best regards
Oisin
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #375080 is a reply to message #375079] |
Tue, 09 June 2009 15:00 |
David Wynter
Messages: 4624
Registered: July 2009 |
Senior Member |
|
|
Hi,
For Antione, ETL is "Extract Transform Load" but this simplifies what we
do. Currently the FSM definition ( in XML ) just passes in the name of
the instance of the ETL task and ETL component then knows which one to use.
What you say about BPEL makes sense, and when I think about it
decorating the various components with the stuff needed to make them web
services adds complexity. It sounds like it might be easier to create my
own IM->FSM transformation and use the existing workflow engine I have.
So long as the annotation about which particular ETL task is to be run
in the workflow is passed through the BPMN->IM transformation then I can
take it from there.
Thx.
David
Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
|
|
|
| Re: [stp-newsgroup] Re: Is BPMN and BPEL the right choice for my problem? [message #375081 is a reply to message #375080] |
Tue, 09 June 2009 15:05 |
Antoine Toulmé
Messages: 31
Registered: July 2009 |
Member |
|
|
--0015174c0fa85c3e14046bebb033
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Indeed. If you have questions when extending the modeler, please ask them on
the BPMN newsgroup, we'll help. We already provide a dozen of samples on how
to extend it, as well as our talk of EclipseCon 2008.
Thanks,
Antoine
On Tue, Jun 9, 2009 at 17:00, David Wynter <david@roamware.co.uk> wrote:
> Hi,
>
> For Antione, ETL is "Extract Transform Load" but this simplifies what we
> do. Currently the FSM definition ( in XML ) just passes in the name of the
> instance of the ETL task and ETL component then knows which one to use.
>
> What you say about BPEL makes sense, and when I think about it decorating
> the various components with the stuff needed to make them web services adds
> complexity. It sounds like it might be easier to create my own IM->FSM
> transformation and use the existing workflow engine I have.
>
> So long as the annotation about which particular ETL task is to be run in
> the workflow is passed through the BPMN->IM transformation then I can take
> it from there.
>
> Thx.
>
> David
>
>
> Oisin Hurley wrote:
>
>>
>> Hi David,
>>
>> David Wynter wrote:
>>
>>> I have a set of components that I use in workflows to basically do value
>>> added ETL work with reported outcomes etc. It currently has a Web interface
>>> and uses a FSM (blissed.codehaus.org) interleaved with the Quartz
>>> scheduler. All data about the ETL reports, scheduling etc is stored in the
>>> database, no files involved.
>>>
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a richer
>>> graphical representation of the workflow. The BPMN modeller and
>>> transformation to IM and thus to BPEL look like a good starting point. I
>>> would replace the Quartz and FSM workflow with a BPEL execution engine.
>>>
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
>> _______________________________________________
> stp-newsgroup mailing list
> stp-newsgroup@eclipse.org
> https://dev.eclipse.org/mailman/listinfo/stp-newsgroup
>
--0015174c0fa85c3e14046bebb033
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Indeed. If you have questions when extending the modeler, please ask them o=
n the BPMN newsgroup, we'll help. We already provide a dozen of samples=
on how to extend it, as well as our talk of EclipseCon 2008.<div><br></div=
>
<div>Thanks,</div><div><br></div><div>Antoine</div><div ><br><div class=3D"g=
mail_quote">On Tue, Jun 9, 2009 at 17:00, David Wynter <span dir=3D"ltr">&l=
t;<a href=3D"mailto:david@roamware.co.uk">david@roamware.co.uk</a>></spa=
n> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">Hi,<br>
<br>
For Antione, ETL is "Extract Transform Load" but this simplifies =
what we do. Currently the FSM definition ( in XML ) just passes in the name=
of the instance of the ETL task and ETL component then knows which one to =
use.<br>
<br>
What you say about BPEL makes sense, and when I think about it decorating t=
he various components with the stuff needed to make them web services adds =
complexity. It sounds like it might be easier to create my own IM->FSM t=
ransformation and use the existing workflow engine I have.<br>
<br>
So long as the annotation about which particular ETL task is to be run in t=
he workflow is passed through the BPMN->IM transformation then I can tak=
e it from there.<br>
<br>
Thx.<br><font color=3D"#888888">
<br>
David</font><div><div></div><div class=3D"h5"><br>
<br>
Oisin Hurley wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<br>
Hi David,<br>
<br>
David Wynter wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I have a set of components that I use in workflows to basically do value ad=
ded ETL work with reported outcomes etc. It currently has a Web interface a=
nd uses a FSM (<a href=3D"http://blissed.codehaus.org" target=3D"_blank">bl=
issed.codehaus.org</a>) interleaved with the Quartz scheduler. All data abo=
ut the ETL reports, scheduling etc is stored in the database, no files invo=
lved.<br>
</blockquote>
<br>
I'm trying to picture this... you have a number of workflows, that are<=
br>
performing ETL tasks. The workflows are implemented using a FSM<br>
and a Quartz scheduler. There are reports generated as the result of<br>
a workflow happening. I'm simplifying here a bit, I know, but I'm<b=
r>
trying to cut it down to some core capabilities...<br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I want to now extend this to incorporate tasks for humans to handle through=
dedicated screens in amongst the workflow. So time for a richer graphical =
representation of the workflow. The BPMN modeller and transformation to IM =
and thus to BPEL look like a good starting point. I would replace the Quart=
z and FSM workflow with a BPEL execution engine.<br>
</blockquote>
<br>
The BPMN modeller would be a good choice for presenting your flows,<br>
I think. I'm not so sure, however, about whether BPEL is really where y=
ou<br>
want to go. BPEL has a number of specific technology requirements (in<br>
the general case) that you might find puts more pressure on your existing<b=
r>
architecture than you want.<br>
<br>
For example, is your architecture based on web services? If not, BPEL means=
<br>
that you will need to express the endpoints that you are coordinating as<br=
>
web services. This may mean you need to use adapters to the real sources.<b=
r>
This can be a lot of overhead. From the details you mention of your system<=
br>
I think that maybe something like Apache Camel could help you out in a<br>
lighterweight way : =A0<a href=3D"http://camel.apache.org" target=3D"_blank=
">http://camel.apache.org</a><br>
<br>
Antoine will no doubt answer the BPMN modeller related questions<br>
you have :) =A0One thing to point out is that STP doesn't provide a way=
to<br>
turn BPMN to BPEL, so that will be an issue.<br>
<br>
best regards<br>
=A0Oisin<br>
<br>
</blockquote>
_______________________________________________<br>
stp-newsgroup mailing list<br>
<a href=3D"mailto:stp-newsgroup@eclipse.org" target=3D"_blank">stp-newsgrou=
p@eclipse.org</a><br>
<a href=3D"https://dev.eclipse.org/mailman/listinfo/stp-newsgroup" target=
=3D"_blank">https://dev.eclipse.org/mailman/listinfo/stp-newsgroup</a><br>
</div></div></blockquote></div><br></div>
--0015174c0fa85c3e14046bebb033--
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #375082 is a reply to message #375079] |
Tue, 09 June 2009 15:08 |
David Wynter
Messages: 4624
Registered: July 2009 |
Senior Member |
|
|
Hi again,
Camel looks like a good choice, except for two things.
1. Can the BPMN be transformed to suit two coordinated execution
environments, one for the ESB stuff in Camel and one for the human
interactions with screens that depend on certain outcomes from the ETL
tasks executing in the ESB?
2. Is there any transformation from the BPMN to one of Camels DSLs? (
Scala, Spring XML config or Fluent API) I could find nothing. I suppose
I could do a IM->Scala transformation...
thx.
David
Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #375083 is a reply to message #375077] |
Tue, 09 June 2009 18:58 |
Marc Dutoo
Messages: 71
Registered: July 2009 |
Member |
|
|
Hi David
If you don't need web service integration nor composition (which seems
so, since you previously used a non distributed java FSM), rather than
using BPEL, I'd suggest to stay on the workflow side and take a look at
Scarbo ( http://scarbo.ow2.org ). It integrates the multipurpose (and
STP BPMN, IM compatible) Workflow Editor of the Eclipse Java Workflow
Tooling project ( http://eclipse.org/jwt ) to the XPDL-based Bonita 4
workflow engine and console ( http://bonita.ow2.org ).
I admit it is a shameless plug since I head JWT :) But I think it makes
sense, since it would provide full task extensibility (model and UI, see
what we've done on e.g. Groovy tasks), a true executable workflow format
(XPDL), workflow execution management and monitoring (in Bonita
console), still import / export with STP BPMN...
Regards,
Marc
David Wynter a écrit :
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
>
> 2. Can the annotation be transformed through to the BPEL?
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
>
> thx.
>
> David
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #375084 is a reply to message #375082] |
Tue, 09 June 2009 19:06 |
Marc Dutoo
Messages: 71
Registered: July 2009 |
Member |
|
|
Hi again
(From my use of STP IM)
On this order of things (starting from BPMN and going in different,
complementary directions + code generation), have a look at what Spagic
( http://www.spagoworld.org/ecm/faces/public/guest/home/soluti ons/spagic
) does thanks to the STP BPMN editor, STP IM and some JET templating.
Either it'll give you ideas, or you'll want to write your own JET code
generation extension... Though I don't know exactly what it does for
human tasks, Andrea maybe ?
You could also use JWT as I wrote above and write a (simple or
composite) JWT transformation that exports parts of your model to Camel
DSLs, using the provided basis and samples (e.g. XSL transformation, ATL
model-to-model transformation)
Regards,
Marc
David Wynter a écrit :
> Hi again,
>
> Camel looks like a good choice, except for two things.
>
> 1. Can the BPMN be transformed to suit two coordinated execution
> environments, one for the ESB stuff in Camel and one for the human
> interactions with screens that depend on certain outcomes from the ETL
> tasks executing in the ESB?
>
> 2. Is there any transformation from the BPMN to one of Camels DSLs? (
> Scala, Spring XML config or Fluent API) I could find nothing. I suppose
> I could do a IM->Scala transformation...
>
> thx.
>
> David
>
> Oisin Hurley wrote:
>>
>> Hi David,
>>
>> David Wynter wrote:
>>> I have a set of components that I use in workflows to basically do
>>> value added ETL work with reported outcomes etc. It currently has a
>>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>>> is stored in the database, no files involved.
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a
>>> richer graphical representation of the workflow. The BPMN modeller
>>> and transformation to IM and thus to BPEL look like a good starting
>>> point. I would replace the Quartz and FSM workflow with a BPEL
>>> execution engine.
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your
>> system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
|
|
| | |
| Re: Is BPMN and BPEL the right choice for my problem? [message #621209 is a reply to message #375077] |
Tue, 09 June 2009 14:37 |
| Eclipse User |
|
|
|
Originally posted by: atoulme.intalio.com
Hi David, see comments below:
David Wynter wrote:
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
Please clarify what you mean by ETL exactly, I'm not familiar with this
acronym.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
I suppose it would be correct to show your special tasks with an annotation.
>
> 2. Can the annotation be transformed through to the BPEL?
That would be a question for the STP-IM maintainers. My blind guess is
that you would need to hack something to achieve it.
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
I don't understand the question. Do you mean that you'd like the BPEL
engine to execute your component because the task would have an annotation ?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
You can do that in BPMN.
OK, it looks like you are trying to customize BPMN and BPEL to carry on
some ETL thingy. Neither BPMN nor BPEL are extensible when it comes to
executing, however you could probably represent an ETL task in BPEL and
model it as pattern in BPMN.
>
> thx.
>
> David
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #621210 is a reply to message #375077] |
Tue, 09 June 2009 14:47 |
Oisin Hurley
Messages: 204
Registered: July 2009 |
Senior Member |
|
|
Hi David,
David Wynter wrote:
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
I'm trying to picture this... you have a number of workflows, that are
performing ETL tasks. The workflows are implemented using a FSM
and a Quartz scheduler. There are reports generated as the result of
a workflow happening. I'm simplifying here a bit, I know, but I'm
trying to cut it down to some core capabilities...
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
The BPMN modeller would be a good choice for presenting your flows,
I think. I'm not so sure, however, about whether BPEL is really where you
want to go. BPEL has a number of specific technology requirements (in
the general case) that you might find puts more pressure on your existing
architecture than you want.
For example, is your architecture based on web services? If not, BPEL means
that you will need to express the endpoints that you are coordinating as
web services. This may mean you need to use adapters to the real sources.
This can be a lot of overhead. From the details you mention of your system
I think that maybe something like Apache Camel could help you out in a
lighterweight way : http://camel.apache.org
Antoine will no doubt answer the BPMN modeller related questions
you have :) One thing to point out is that STP doesn't provide a way to
turn BPMN to BPEL, so that will be an issue.
best regards
Oisin
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #621211 is a reply to message #375079] |
Tue, 09 June 2009 15:00 |
David Wynter
Messages: 4624
Registered: July 2009 |
Senior Member |
|
|
Hi,
For Antione, ETL is "Extract Transform Load" but this simplifies what we
do. Currently the FSM definition ( in XML ) just passes in the name of
the instance of the ETL task and ETL component then knows which one to use.
What you say about BPEL makes sense, and when I think about it
decorating the various components with the stuff needed to make them web
services adds complexity. It sounds like it might be easier to create my
own IM->FSM transformation and use the existing workflow engine I have.
So long as the annotation about which particular ETL task is to be run
in the workflow is passed through the BPMN->IM transformation then I can
take it from there.
Thx.
David
Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
|
|
|
| Re: [stp-newsgroup] Re: Is BPMN and BPEL the right choice for my problem? [message #621212 is a reply to message #375080] |
Tue, 09 June 2009 15:05 |
Antoine Toulmé
Messages: 31
Registered: July 2009 |
Member |
|
|
--0015174c0fa85c3e14046bebb033
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Indeed. If you have questions when extending the modeler, please ask them on
the BPMN newsgroup, we'll help. We already provide a dozen of samples on how
to extend it, as well as our talk of EclipseCon 2008.
Thanks,
Antoine
On Tue, Jun 9, 2009 at 17:00, David Wynter <david@roamware.co.uk> wrote:
> Hi,
>
> For Antione, ETL is "Extract Transform Load" but this simplifies what we
> do. Currently the FSM definition ( in XML ) just passes in the name of the
> instance of the ETL task and ETL component then knows which one to use.
>
> What you say about BPEL makes sense, and when I think about it decorating
> the various components with the stuff needed to make them web services adds
> complexity. It sounds like it might be easier to create my own IM->FSM
> transformation and use the existing workflow engine I have.
>
> So long as the annotation about which particular ETL task is to be run in
> the workflow is passed through the BPMN->IM transformation then I can take
> it from there.
>
> Thx.
>
> David
>
>
> Oisin Hurley wrote:
>
>>
>> Hi David,
>>
>> David Wynter wrote:
>>
>>> I have a set of components that I use in workflows to basically do value
>>> added ETL work with reported outcomes etc. It currently has a Web interface
>>> and uses a FSM (blissed.codehaus.org) interleaved with the Quartz
>>> scheduler. All data about the ETL reports, scheduling etc is stored in the
>>> database, no files involved.
>>>
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a richer
>>> graphical representation of the workflow. The BPMN modeller and
>>> transformation to IM and thus to BPEL look like a good starting point. I
>>> would replace the Quartz and FSM workflow with a BPEL execution engine.
>>>
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
>> _______________________________________________
> stp-newsgroup mailing list
> stp-newsgroup@eclipse.org
> https://dev.eclipse.org/mailman/listinfo/stp-newsgroup
>
--0015174c0fa85c3e14046bebb033
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Indeed. If you have questions when extending the modeler, please ask them o=
n the BPMN newsgroup, we'll help. We already provide a dozen of samples=
on how to extend it, as well as our talk of EclipseCon 2008.<div><br></div=
>
<div>Thanks,</div><div><br></div><div>Antoine</div><div ><br><div class=3D"g=
mail_quote">On Tue, Jun 9, 2009 at 17:00, David Wynter <span dir=3D"ltr">&l=
t;<a href=3D"mailto:david@roamware.co.uk">david@roamware.co.uk</a>></spa=
n> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">Hi,<br>
<br>
For Antione, ETL is "Extract Transform Load" but this simplifies =
what we do. Currently the FSM definition ( in XML ) just passes in the name=
of the instance of the ETL task and ETL component then knows which one to =
use.<br>
<br>
What you say about BPEL makes sense, and when I think about it decorating t=
he various components with the stuff needed to make them web services adds =
complexity. It sounds like it might be easier to create my own IM->FSM t=
ransformation and use the existing workflow engine I have.<br>
<br>
So long as the annotation about which particular ETL task is to be run in t=
he workflow is passed through the BPMN->IM transformation then I can tak=
e it from there.<br>
<br>
Thx.<br><font color=3D"#888888">
<br>
David</font><div><div></div><div class=3D"h5"><br>
<br>
Oisin Hurley wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<br>
Hi David,<br>
<br>
David Wynter wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I have a set of components that I use in workflows to basically do value ad=
ded ETL work with reported outcomes etc. It currently has a Web interface a=
nd uses a FSM (<a href=3D"http://blissed.codehaus.org" target=3D"_blank">bl=
issed.codehaus.org</a>) interleaved with the Quartz scheduler. All data abo=
ut the ETL reports, scheduling etc is stored in the database, no files invo=
lved.<br>
</blockquote>
<br>
I'm trying to picture this... you have a number of workflows, that are<=
br>
performing ETL tasks. The workflows are implemented using a FSM<br>
and a Quartz scheduler. There are reports generated as the result of<br>
a workflow happening. I'm simplifying here a bit, I know, but I'm<b=
r>
trying to cut it down to some core capabilities...<br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I want to now extend this to incorporate tasks for humans to handle through=
dedicated screens in amongst the workflow. So time for a richer graphical =
representation of the workflow. The BPMN modeller and transformation to IM =
and thus to BPEL look like a good starting point. I would replace the Quart=
z and FSM workflow with a BPEL execution engine.<br>
</blockquote>
<br>
The BPMN modeller would be a good choice for presenting your flows,<br>
I think. I'm not so sure, however, about whether BPEL is really where y=
ou<br>
want to go. BPEL has a number of specific technology requirements (in<br>
the general case) that you might find puts more pressure on your existing<b=
r>
architecture than you want.<br>
<br>
For example, is your architecture based on web services? If not, BPEL means=
<br>
that you will need to express the endpoints that you are coordinating as<br=
>
web services. This may mean you need to use adapters to the real sources.<b=
r>
This can be a lot of overhead. From the details you mention of your system<=
br>
I think that maybe something like Apache Camel could help you out in a<br>
lighterweight way : =A0<a href=3D"http://camel.apache.org" target=3D"_blank=
">http://camel.apache.org</a><br>
<br>
Antoine will no doubt answer the BPMN modeller related questions<br>
you have :) =A0One thing to point out is that STP doesn't provide a way=
to<br>
turn BPMN to BPEL, so that will be an issue.<br>
<br>
best regards<br>
=A0Oisin<br>
<br>
</blockquote>
_______________________________________________<br>
stp-newsgroup mailing list<br>
<a href=3D"mailto:stp-newsgroup@eclipse.org" target=3D"_blank">stp-newsgrou=
p@eclipse.org</a><br>
<a href=3D"https://dev.eclipse.org/mailman/listinfo/stp-newsgroup" target=
=3D"_blank">https://dev.eclipse.org/mailman/listinfo/stp-newsgroup</a><br>
</div></div></blockquote></div><br></div>
--0015174c0fa85c3e14046bebb033--
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #621213 is a reply to message #375079] |
Tue, 09 June 2009 15:08 |
David Wynter
Messages: 4624
Registered: July 2009 |
Senior Member |
|
|
Hi again,
Camel looks like a good choice, except for two things.
1. Can the BPMN be transformed to suit two coordinated execution
environments, one for the ESB stuff in Camel and one for the human
interactions with screens that depend on certain outcomes from the ETL
tasks executing in the ESB?
2. Is there any transformation from the BPMN to one of Camels DSLs? (
Scala, Spring XML config or Fluent API) I could find nothing. I suppose
I could do a IM->Scala transformation...
thx.
David
Oisin Hurley wrote:
>
> Hi David,
>
> David Wynter wrote:
>> I have a set of components that I use in workflows to basically do
>> value added ETL work with reported outcomes etc. It currently has a
>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>> is stored in the database, no files involved.
>
> I'm trying to picture this... you have a number of workflows, that are
> performing ETL tasks. The workflows are implemented using a FSM
> and a Quartz scheduler. There are reports generated as the result of
> a workflow happening. I'm simplifying here a bit, I know, but I'm
> trying to cut it down to some core capabilities...
>
>> I want to now extend this to incorporate tasks for humans to handle
>> through dedicated screens in amongst the workflow. So time for a
>> richer graphical representation of the workflow. The BPMN modeller and
>> transformation to IM and thus to BPEL look like a good starting point.
>> I would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> The BPMN modeller would be a good choice for presenting your flows,
> I think. I'm not so sure, however, about whether BPEL is really where you
> want to go. BPEL has a number of specific technology requirements (in
> the general case) that you might find puts more pressure on your existing
> architecture than you want.
>
> For example, is your architecture based on web services? If not, BPEL means
> that you will need to express the endpoints that you are coordinating as
> web services. This may mean you need to use adapters to the real sources.
> This can be a lot of overhead. From the details you mention of your system
> I think that maybe something like Apache Camel could help you out in a
> lighterweight way : http://camel.apache.org
>
> Antoine will no doubt answer the BPMN modeller related questions
> you have :) One thing to point out is that STP doesn't provide a way to
> turn BPMN to BPEL, so that will be an issue.
>
> best regards
> Oisin
>
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #621214 is a reply to message #375077] |
Tue, 09 June 2009 18:58 |
Marc Dutoo
Messages: 71
Registered: July 2009 |
Member |
|
|
Hi David
If you don't need web service integration nor composition (which seems
so, since you previously used a non distributed java FSM), rather than
using BPEL, I'd suggest to stay on the workflow side and take a look at
Scarbo ( http://scarbo.ow2.org ). It integrates the multipurpose (and
STP BPMN, IM compatible) Workflow Editor of the Eclipse Java Workflow
Tooling project ( http://eclipse.org/jwt ) to the XPDL-based Bonita 4
workflow engine and console ( http://bonita.ow2.org ).
I admit it is a shameless plug since I head JWT :) But I think it makes
sense, since it would provide full task extensibility (model and UI, see
what we've done on e.g. Groovy tasks), a true executable workflow format
(XPDL), workflow execution management and monitoring (in Bonita
console), still import / export with STP BPMN...
Regards,
Marc
David Wynter a écrit :
> Hi,
>
> I have a set of components that I use in workflows to basically do value
> added ETL work with reported outcomes etc. It currently has a Web
> interface and uses a FSM (blissed.codehaus.org) interleaved with the
> Quartz scheduler. All data about the ETL reports, scheduling etc is
> stored in the database, no files involved.
>
> I want to now extend this to incorporate tasks for humans to handle
> through dedicated screens in amongst the workflow. So time for a richer
> graphical representation of the workflow. The BPMN modeller and
> transformation to IM and thus to BPEL look like a good starting point. I
> would replace the Quartz and FSM workflow with a BPEL execution engine.
>
> I have read a lot of the material covering STP over the last couple of
> days. But a few question have formed probably because I have not read
> everything yet.
>
> Apologies if my language is not precisely aligned with all the STP
> buzzwords.
>
> 1. Since I have many instances of an ETL involving a particular pair of
> input and output I need to have a clear display of which ETL we are
> dealing with. Is the annotation extension to the task icon the correct
> way of presenting these?
>
> 2. Can the annotation be transformed through to the BPEL?
>
> 3. Do the BPEL engines allow for execution of a component using the
> annotation to indicate to the component being run?
>
> 4. Since this is not a general business workflow is it acceptable to add
> specific task icons for the specialized tasks we carry out (e.g. an ETL
> task?)
>
> thx.
>
> David
|
|
|
| Re: Is BPMN and BPEL the right choice for my problem? [message #621215 is a reply to message #375082] |
Tue, 09 June 2009 19:06 |
Marc Dutoo
Messages: 71
Registered: July 2009 |
Member |
|
|
Hi again
(From my use of STP IM)
On this order of things (starting from BPMN and going in different,
complementary directions + code generation), have a look at what Spagic
( http://www.spagoworld.org/ecm/faces/public/guest/home/soluti ons/spagic
) does thanks to the STP BPMN editor, STP IM and some JET templating.
Either it'll give you ideas, or you'll want to write your own JET code
generation extension... Though I don't know exactly what it does for
human tasks, Andrea maybe ?
You could also use JWT as I wrote above and write a (simple or
composite) JWT transformation that exports parts of your model to Camel
DSLs, using the provided basis and samples (e.g. XSL transformation, ATL
model-to-model transformation)
Regards,
Marc
David Wynter a écrit :
> Hi again,
>
> Camel looks like a good choice, except for two things.
>
> 1. Can the BPMN be transformed to suit two coordinated execution
> environments, one for the ESB stuff in Camel and one for the human
> interactions with screens that depend on certain outcomes from the ETL
> tasks executing in the ESB?
>
> 2. Is there any transformation from the BPMN to one of Camels DSLs? (
> Scala, Spring XML config or Fluent API) I could find nothing. I suppose
> I could do a IM->Scala transformation...
>
> thx.
>
> David
>
> Oisin Hurley wrote:
>>
>> Hi David,
>>
>> David Wynter wrote:
>>> I have a set of components that I use in workflows to basically do
>>> value added ETL work with reported outcomes etc. It currently has a
>>> Web interface and uses a FSM (blissed.codehaus.org) interleaved with
>>> the Quartz scheduler. All data about the ETL reports, scheduling etc
>>> is stored in the database, no files involved.
>>
>> I'm trying to picture this... you have a number of workflows, that are
>> performing ETL tasks. The workflows are implemented using a FSM
>> and a Quartz scheduler. There are reports generated as the result of
>> a workflow happening. I'm simplifying here a bit, I know, but I'm
>> trying to cut it down to some core capabilities...
>>
>>> I want to now extend this to incorporate tasks for humans to handle
>>> through dedicated screens in amongst the workflow. So time for a
>>> richer graphical representation of the workflow. The BPMN modeller
>>> and transformation to IM and thus to BPEL look like a good starting
>>> point. I would replace the Quartz and FSM workflow with a BPEL
>>> execution engine.
>>
>> The BPMN modeller would be a good choice for presenting your flows,
>> I think. I'm not so sure, however, about whether BPEL is really where you
>> want to go. BPEL has a number of specific technology requirements (in
>> the general case) that you might find puts more pressure on your existing
>> architecture than you want.
>>
>> For example, is your architecture based on web services? If not, BPEL
>> means
>> that you will need to express the endpoints that you are coordinating as
>> web services. This may mean you need to use adapters to the real sources.
>> This can be a lot of overhead. From the details you mention of your
>> system
>> I think that maybe something like Apache Camel could help you out in a
>> lighterweight way : http://camel.apache.org
>>
>> Antoine will no doubt answer the BPMN modeller related questions
>> you have :) One thing to point out is that STP doesn't provide a way to
>> turn BPMN to BPEL, so that will be an issue.
>>
>> best regards
>> Oisin
>>
|
|
| | |
Goto Forum:
Current Time: Thu Apr 18 01:07:09 GMT 2024
Powered by FUDForum. Page generated in 0.03649 seconds |
] 
Search
Help
Register
Login
Home
