B2J Distribution Tutorial (Running your BPEL across multiple hosts)

Author: Antony Miguel, Last Updated 15th March 2006

Overview

This document explains how to use B2J to run BPEL processes distributed across multiple hosts.
This tutorial assumes that the reader has already read the B2J beginners tutorial to familiarise themselves with the basics of running a choreography. If you have not already read the beginners tutorial you can read it here.
B2J contains two reference implementations of the B2J framework. One is a simple BPEL engine capable of running a BPEL process within the JVM it is invoked from. The second is a more complicated distributed engine capable of running multithreaded BPEL processes (<flow> or <forEach parallel="yes"> activities) as a single process distributed across multiple physical hosts. The TPTP Choreography engine is capable of running as a single unit spread across multiple physical hosts.
Common reasons for distributing a BPEL process are:
- To run local resource intensive BPEL processes more efficiently by utilising resources across a group of hosts
- To simulate or invoke operations from different geographic locations without requiring a choreography of web services

Requirements

The user should have an installation of Eclipse and the B2J subproject plugins installed. See the B2J subproject main page for versions, downloads and installation information.
The user should have read the B2J Beginners Tutorial.

Distributing a BPEL process

To distribute a BPEL process, you must first set up a launch configuration as described in the beginners tutorial. Run the launch configuration with the default local engine to ensure everything is configured properly.

A good example process to run for this tutorial is the test_messaging sample BPEL process as it contains a significant degree of multithreading.

Once you have your launch configuration set up, click the Distribtion tab in the launch configuration dialog and select the Use distributed engine (one or more hosts) button.

From here you can use the buttons on the right hand side of the table to add and remove hosts from your distribution. Click the Add button once for each host you wish to run the engine on. Edit the hostnames in the first column of the table.

In our example, even though we only wish to run over two hosts, we have three entries in the table. This is because the first host in the table is the Coordinator host which hosts the part of the engine that coordinates the activities of the Worker hosts.

The Coordinator host can be on a separate host from the worker hosts or on the same host as one of the worker hosts.

In our example, we will use the same host as one of the worker hosts.

When we have specified all the hosts we wish to run the BPEL process over, we must run the B2J engine daemon on each of these hosts.

Note: If you plan to use the B2J engine in a distributed environment over a long period of time, you need only install and run any version of the B2J engine daemon once. The B2J engine will automatically update and choose the right version of the engine to run based on the version used in the client that executes the BPEL.

E.g. if you install and run B2J 1.0 on 10 physical hosts, then later you download the 1.2 plugin to your Eclipse workbench, you need not manually update all 10 physical hosts. The 1.2 plugin will automatically update any hosts used in the BPEL process execution.

To install the B2J engine daemon, copy or unzip the B2J plugin onto the target host and run either the b2j.bat file on Windows or the b2j.sh on Linux or other *nix variants.

Alternatively you can run the command:

java -cp b2j.jar org.eclipse.stp.b2j.core.jengine.internal.mainengine.SoapDaemon

From the B2J plugin directory.

Please Note: The choreography engine requires Java 1.4 or above to run. If you do not have Java installed you can download installation binaries for most operating systems from www.java.com.

The B2J engine daemon will try to listen on port 11000 for incoming SOAP/HTTP connections. Worker or coordinator hosts that the B2J engine daemon creates will scan ports above 11000 until they get to a free port (e.g. 11001, 11002...). If you have a firewall on your host you should configure it to allow incoming connections on port 11000+ (e.g. 11000 to 11100).

If you specify localhost as one of the hosts to run the engine on, the launch configuration will automatically run an engine daemon inside the Eclipse Workbench JVM in case you haven't started a separate engine.

When you have finished installing the B2J daemon on your hosts and you have specified the hosts in the Distribution tab Hosts table, click Run to run your launch configuration.

The initial (engine related, green and red) output from your launch configuration should look similar to:

Aborting a Run

If, for any reason you wish to abort the B2J process execution, you can open the Progress view and click the red terminate button pertaining to the B2J execution you wish to abort.

Note that the engine will not perform any cleanup during this abort - it will simply quit the processes hosting the engine (note that this does NOT include any engine daemons you may have running).

Headless Execution

If you run a distributed engine (even just on a single host) and you close the client to that engine (e.g. the Eclipse Workbench) the engine will automatically abort the run. If you want the engine to continue running after you have closed the workbench you can check the Leave engine running on client disconnect option in the Compilation tab of the launch configuration.

If you later wish to terminate one of these engines you can use the WTP web services explorer to invoke the engine daemon's SOAP/HTTP WSDL interface.

To see this, open a standard web browser and go to http://localhost:11000/ (substitute localhost with whatever host the engine daemon is running on). You should see something like:

Click on the Daemon Public WSDL link to see the WSDL interface to the engine daemon. You can use the address of this WSDL in the WTP web services explorer or with any generic web services UI to query the B2J engine daemon and terminate any engines running in it.

Network Layer Robustness

If you run a distributed engine over a non-LAN network link or over any poor network link there is a chance that the network link will break and cause the engine to abort the execution. If you want the engine to be robust against these network failures you can set the Reconnection and Reconnect Timeout options for each transport link in the B2J engine. These properties are in the B2J launch configuration under the Distribution tab.

For each host mentioned in the table, you can set individual properties for the link to that host from the coordinator host (or from the workbench, if you change settings for the coordinator host itself).

With transport link reconnection ON, if a network failure occurs, the B2J engine will pause any parts of the BPEL process which are dependant on that network link and try to re-establish the link, timing out after the period set under Reconnect Timeout.

Deployment and Load Balancing

Deployment and load balancing is performed within the BPEL program itself via the use of java bound engine web services. The example file test_deployment shows how parallel threads in a flow can be located either on the first host in the list of engine hosts, as an even spread across all hosts, or on a specific host identified by its index in the list of hosts. Child flows can then be used to create arbitrary combinations of these two deployment methods.

In the future it is expected that the deployment service will grow in functionality to specify a range of deployment strategies for any given flow or parallel activity.

The WSDL for the deployment service and a number of other services can be found in the plugin directory under the conf/Default/bpel directory.

As demonstrated in the test_deployment example, these WSDL files can be included into any BPEL process running in B2J via the namespace http://www.eclipse.org/stp/b2j/2006/02.

(E.g. <import ... location="http://www.eclipse.org/stp/b2j/2006/02/service_libraries/EngineDeploymentPort.wsdl" >).

Engine Deployment and Load Balancing BPEL

... <invoke partnerLink="engineDeploymentService" operation="setEngineDeploymentFirstHost" inputVariable="deployFirstHost" /> ... ... <invoke partnerLink="engineDeploymentService" operation="setEngineDeploymentEvenSpread" inputVariable="deployEvenSpread" /> ... ... <invoke partnerLink="engineDeploymentService" operation="setEngineDeploymentSpecificHost" inputVariable="deploySpecificHost" /> ...