Eclipse TPTP Version 4.2
File Transfer Service Design

 

 

Technical Owner:  Kevin O'Leary

Original contributers:  Hoang M. Nguyen, Guru Naragajan

 

Version 1.0

 

June 2006

 

 

Copyright © 2005, 2006 Intel Corporation.

Content provided under the terms and conditions of the Eclipse Public License Version 1.0.

Eclipse is a trademark of Eclipse Foundation, Inc.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Other company, product, and service names may be trademarks or service marks of others.

 

 

Revision History

Revision

Revision History

Date

0.1

Initial version

5/23/2005

1.0

Applied review comments to original and added a few notations with regard to the implementation as of v4.2.

6/29/2006

 

Table of Contents

Introduction.
    Scope
    Prerequisites
    Requirements
    Why do we need a new design of file transfer service?

File Transfer Service Design
    Design objectives
    Transfer Data format: DIME-compliant
    Use of DIME records on Data Channel

External Interface
    File Transfer APIs
        Basic File Transfer Operations
    Sample file transfer code
    File Transfer Agent  XML Command Protocol
        Commands

Figures

Figure 1.  High-level view of file transfer service.

Figure 2.  Multiple and configurable listening ports

Figure 3.  How the file transfer works

Figure 4.  Use of DIME in file transfer

Figure 5.  DIME records use on a data channel.

 

Tables

 Table 1.  Command Summary Table

Introduction

This document describes the design and interface of the file transfer service provided by the agent controller (AC) in TPTP 4.2

1.1 Scope

This document is intended for the following users:

1.2      Prerequisites

In order to have enough context and concepts of TPTP, readers should review the following documents and articles before examining this design:

1.3           Requirements

The requirements of the file transfer service are:

1.4           Why do we need a new design of file transfer service?

It is important to have this service redesigned to address the following constraints and limitations:

File Transfer Service Design

This section describes the internal components and processing flow of the transfer service.

2.1      Design objectives

The basic design objectives for the file transfer service are as follow:

Some of the important design points to keep in mind when using this document:

Figure 1 presents a high-level view of the file transfer servcie as implemented for the new technology Agent Controller in TPTP version 4.2 and later.

Figure 1.  High-level view of file transfer service

Note that the file transfer service is written and behaves just like any other agent. This means:

Consistency and uniformity of all services are key objectives. This in turn, helps reduce complexity in the implementation by leveraging the basic services provided in the base agent implementation. It avoids dealing with the details of the communication layer and allows developer focus to remain on implementing the service functionality itself.

Figure 2 shows how the Agent Controller allows transport layers to be configured to use multiple ports.

Figure 2.  Multiple and configurable listening ports

The new design of the AC enables the user to configure multiple pluggable transport layers such that each can listen to its own port number. As a result, the diagram highlights the ability for the file transfer service to be able to use a dedicated port number (like 10005 for compatibility with existing file transfer service) or make use of the same main port number as other agents (thus avoiding opening a new port number for file transfer service only).

In addition, the diagram shows that the agent implementation is shielded from the details of the transport mechanism itself, such as whether it has SSL encryption or not.

Figure 3 illustrates the flow for a basic file tranfer works.

Figure 3.  How the file transfer works

The flow of a basic transfer flow is as follows::

There are a couple of important points not illustrated in the diagram:

2.2           Transfer Data format: DIME-compliant

There are a couple of important quotes in the mentioned articles (see “Prerequisites” section for links) worth repeating to highlight why we choose to adopt the DIME format:

“The Direct Internet Message Encapsulation (DIME) specification addresses all these issues [data format and packaging] in a minimal and efficient manner.”

“While potentially less flexible than MIME, DIME prescribes a simpler message format, which in turn provides for faster and more efficient message processing.”

“When binding DIME messages to HTTP, the HTTP Content-type header field must specify "application/dime" instead of the usual "application/soap+xml" or "text/xml" defined respectively by the SOAP 1.2 and SOAP 1.1 protocols.

Why did we choose to adopt the DIME format?

Figure 4 shows the detail of a typical file transfer flow, illustrating how DIME is used in the system.

Figure 4.  Use of DIME in file transfer

The diagram shows a typical transfer flow and how DIME is used in the system:

2.3      Use of DIME records on Data Channel

Figure 5 shows how DIME is used to identify data chunks going across a single data channel.

Figure 5.  DIME records use on a data channel

Above is an example of how DIME is used to identify data chunks going across a single channel. This allows different data types, such as performance data collected by an agent monitoring an application and contents of a file, to be sent across the same channel in an interleaved fashion.  This also shows how different sets of same data type (i.e., multiple files) can be sent across the same channel simulatneously..

The DIME record contains sufficient information to allow the data to be properly grouped and correlated by the receiving component, client or agent, without knowing the form of the data itself.

External Interface

This section describes the external APIs that an application uses to access the file transfer service.

3.1       File Transfer APIs

3.1.1      Basic File Transfer Operations

The File Transfer agent provides the following operations.

The file transfer agent receives these requests on the command channel (any command responses are returned on the command channel) and subsequently returns the file on the data channel. The establishment of a data path  is handled by the client API. A data path will only be created if one does not currently exist. All file transfer functions check whether a data path has been created, and create one where one does not exit.

Multiple file transfer requests may be served by a single File Transfer agent.

The default File Transfer service agent can be replaced by a custom implementation and accessed just as any other agent.

Table 1 below summarizes the functions that can be called upon using a File Transfer agent object  The references to local and remote systems are relative to where the client (i.e. the calling application) is running.  

Table 1Command Summary Table

Command

Description

tptp_int32 getFile(char *destFile, char *srcFile)

Copy srcFile from the remote system to destFile on the local system.  See Note 1.

tptp_int32 putFile(char *srcFile, char *destFile)

Copy srcFile from the local system to destFile on the remote system.  See Note 1.

tptp_int32 deleteFile(char *File)

Delete File on the remote system. 
See Note 1.

Note 1:  The srcFile, destFile, and File names must be absolute pathnames that follow the
        operating system file naming conventions applying to that files's location.

 


 

3.2            Sample file transfer code

The following example for using the C++ File Transfer API is taken from the Sample code provided in the SDK package.

 

/**

 *********************************************************

 *                                                      

 * @brief                                               

 *    MyFileTransferClient

 * This is a sample client program that uses the Agent Controller

 * to get access to the FileTransfer agent.

 *

 * The FileTransferAgent is then sent 3 messages.

 *    getFile

 *    deleteFile

 *    putFile

 *

 * Usage : MyFileTransferClient                 <==defaults to localhost, port 10006

 *          : MyFileTransferClient <HostName> <Port>

 *          : For e.g. MyFileTransferClient

 *                            MyFileTransferClient "lab-system-001" 10006

 * Expected Output:

 *    Connected to Agent Controller message

 *    Running getFile test

 *    Get File X From File Y

 *

 *    Running deleteFile test

 *    Delete File X

 *

 *    Running putFile tests

 *    Put file X To File Y

 *    All finished

 *    Press any key to exit...

 *

 *  External Dependencies:

 *    1 file to test file transfers.

 *          The content of this file does not matter, it can be text or binary.

 *          This will be used to test the FileTransfer operations.

 *

 *          This Sample encodes the file path as followed.

 *          On Windows

 *                C:\big_simple.txt

 *          On Linux

 *                /tmp/big_simple.txt

 *   

 * Source Code Files:

 *    MyFileTransferClient.cpp

 *

 *

 *********************************************************/

int main(int argc, char* argv[])

{

      char* hostName = "localhost";

      int portNum = 10006;

      INode* TargetNode = 0;

      AgentController* ACProxy = 0;

     

      //Start of generic connect to AgentController code.

      if ((argc == 2) || (argc > 3))

      {

            cout<<"Usage: MyFileTransferClient or MyFileTransferClient <HostName> <Port> "<<endl;

            cout<<"Press any key to exit..."<<endl;

            getchar();

            return -1;

      }

 

      // Use the host name and port number if provided on the command line.

      if (argc == 3)

      {

            hostName = argv[1];

            portNum = atoi(argv[2]);

      }

     

 

      // Create a Node object for the target system of choice.

      TargetNode = NodeFactory::createNode(hostName);

      if (!TargetNode)

      {

            cout<<"ERROR: Failed to create the Node object."<<endl;

            cout<<"Press any key to exit..."<<endl;

            getchar();

            return -1;

      }

 

      // Establish a connection with an Agent Controller on the

      // target system.  The port number specified must match

      // that defined in the configuration file for the desired

      // Agent Controller.

      ACProxy = TargetNode->connect(portNum);

     

      if (!ACProxy)

      {

            cout<<"ERROR: Unable to connect to the Agent Controller running on "<<hostName<<" at port number "<<portNum<<endl;

            cout<<"Press any key to exit..."<<endl;

            getchar();

            return -1;

      }

     

      cout<<endl<<"Connected to the Agent Controller on \""<<hostName<<"\" at port number "<<portNum<<endl;

 

      //End of generic connect to AgentController code.

 

      //Start of FileTransferAgent specific code.

      // Create a client-side representative of a FileTransfer object from which

      // a MyFileTransferAgent is derived, specifying the name of the desired agent

      // as it has been installed in the agents directory.

      FileTransfer* MyFileTransferAgent

            = new FileTransfer("org.eclipse.tptp.FileTransferAgent");

 

     

      // Get a handle to the actual FileTransferAgent, storing it in the MyFileTransferAgent

      // object.  If the call is successful, the FileTransferAgent is running and

      // ready to respond to requests.

      // Note that the Agent Controller manages the life of an agent, so it will start

      // one running if needed.

      // We request controller access to the agent since we want to perform

      // actions like getFile and putFile. However, the FileTransferAgent is not checking

      // access levels on incoming requests at this time (TBD).

      if (-1 == ACProxy->getAgent(MyFileTransferAgent, TPTP_CONTROLLER_ACCESS))

      {

            cout<<"ERROR: Unable to get agent org.eclipse.tptp.FileTransferAgent"<<endl;             

            cout<<"Press any key to exit..."<<endl;

            getchar();

            return -1;

      }

 

      cout<<endl<<"The FileTransferAgent Agent ID: "<<MyFileTransferAgent->getAgentID()<<endl;

 

      //These next variables encode the file paths for the transfered files.

#ifdef _WIN32

      char *remoteFile = "C:\\big_simple.txt";          //This is the source file for getFile and putFile.

      char *localFile = "C:\\big_local.txt";            //This is the destination file for getFile and putFile.

                                            //Deleted by deleteFile operation.

#else

      char *remoteFile = "/tmp/big_simple.txt";         //This is the source file for getFile and putFile.

      char *localFile = "/tmp/big_local.txt";           //This is the destination file for getFile and putFile.

                                            //Deleted by deleteFile operation.

#endif

 

      cout <<"Running getFile test " << endl;

      cout <<"Get file " << localFile << " From File " << remoteFile << endl << endl;

 

      //Get the file specified in remoteFile to localFile.

      MyFileTransferAgent->getFile(localFile, remoteFile);

 

      cout<<"Running deleteFile test" <<endl;

      cout <<"Delete file " << localFile << endl << endl;

      MyFileTransferAgent->deleteFile(localFile);

 

      cout <<"Running putFile test " << endl;

      cout <<"Put file " << remoteFile << " To File " << localFile << endl;

 

      //Put the file specified in remoteFile to localFile.

      MyFileTransferAgent->putFile(remoteFile, localFile);

      // Wait for data to finish coming over channel.

      // Note: A protocol for recognizing end of data is TBD.

      Sleep(2000);

 

      // Release resources allocated by the Agent Controller

      MyFileTransferAgent->destroyDataConnection();

      MyFileTransferAgent->releaseAgent();

      ACProxy->disconnect();

     

      // Release client side resources

      delete MyFileTransferAgent;

      ACProxy->destroy();

      NodeFactory::deleteNode(TargetNode);

     

      cout<<"All finished"<<endl;

      cout<<"Press any key to exit..."<<endl;

      getchar();

      return 0;

}

3.3           File Transfer Agent  XML Command Protocol

This section describes the XML-formatted commands recognized by the File Transfer agent.  These are the client API uses these in its implement .

3.3.1      Commands

The following commands are supported through the File Transfer interface.    

Note that this is basically maintaining the same command format of the current protocol where there is no new information is required for the request.

Command:        

getFile / putFile / deleteFile

Parameters:

Length of file name

Int

The length of the fileName parameter

fileName

String

Name of file to be retrieved, uploaded or deleted, as located on the remote system.

Response:

No Data returned

Alternate Response (Error situation):

SECURE_CONNECTION_REQUIRED_EXCEPTION

FILE_NOT_FOUND_ERROR

IO_ERROR

Description:

In case of an error, one of the above error messages (Alternate Response) will be provided as a CBE formatted response.