Data Tools Platform (DTP) Project 1.0

SQL Development Tools Plan

Last Updated: 11/22/05



This document contains plans for the components of the SQL Development Tools for DTP 1.0. Currently, it only reflects plans through M4 (DTP 0.N).


The table below lists the deliverables planned for each milestone by component. Each component feature specifies a priority. Direct dependencies are also indicated either next to the component or feature. Click on the component to see a description of its scope and design goals.


Feature Priority:

(C ) Critical: feature is required in order to declare subcomponent completion for milestone

(H) High: very important feature, but could move out to next milestone if necessary

(M) Medium: good feature, but not necessary

(L) Low: nice to have feature, but could defer if necessary


Feature Dependencies: shown next to each feature in square brackets [ ]

[Conn] Connectivity

[DDM] Database Definition Model

[LPG] LALR Parser Generator (non-Eclipse component)

[SM] SQL Model

[SQM] SQL Query Model




MILESTONE 2: 12/22/05

Component

Iteration 1

11/21 – 12/02 (8)

Iteration 2

12/05 – 12/09 (5)

Iteration 3

12/12 – 12/15 (4)

Iteration 4

12/16 – 12/21 (6)

SQL Query Parser


[SQM, SM, LPG]



Initial code drop:

- Parses multiple statements (C)
- Table reference resolving (C)

- Optional quick parse mode (H)

- Configurable source format options: e.g. statement terminator, host variable prefix, parameter marker, delimited identifier quote, omit schema (H)

- Soft keyword support (“SELECT select FROM from;”) (H)

- Simple data type resolving (H)

- Test-driven development (junit testcases) (H)

Testing with Eclipse 3.2 M4

SQL Results View

Low Level Design
- Internal Review


Initial source code drop (for review only)
-Model and API code
-Core classes


Feature code drop
- Display execution status, summary and details, messages and result sets in different layout and modes ( C)


Low Level Design
- External Review


Feature code drop
- Save/export/print results in various file formats with user defined column delimiters and encoding (H)


Testing with Eclipse 3.2 M4

SQL/Routines Editor

Low Level Design
- Internal Review


API tutorial


Initial code drop:
-extension point
-SQL Editor Framework API


Feature code drop

Merge with RDB SQL Editor


Feature code drop

Initial SQL Editor Framework and Generic SQL Editor implementation build available for download


Low Level Design

- External Review


Testing with Eclipse 3.2 M4


Routines Debugger


Low Level Design
- Internal Review

Initial code drop:
-extension point
-Routine Debugger Framework API
- Debug Model classes

Low Level Design
-External Review


Testing with Eclipse 3.2 M4

SQL Execution Plan



Low Level Design
- Internal Review


SQL Query Builder

Note: Not targeted for delivery in DTP 1.0









MILESTONE 3: 01/20/06

Component

Iteration 1

01/03 – 01/06 (4)

Iteration 2

01/09 – 01/13 (5)

Iteration 3

01/16 – 01/19 (4)

SQL Query Parser


[SQM, SM, LPG]



Doc Drop

- Internal API Documentation (H)

SQL Results View

Feature code drop
- Execution history filters, e.g., to display execution history by connection profile, status, etc. (H)


Feature code drop
- Execution history sorting based on weight/execution frequency (H)

Feature code drop
- Re-execution of selected history script (H)

- Persistence of execution history (M)


SQL/Routines Editor

Feature code drop
- content assist (C ) [SM, Conn]
- syntax highlighting(C )
- code template (H)
- syntax validation (H)
- Outline view (H)
- statement execution ( C) [Conn, Results View]

Feature code drop
- Integration w/ Model Base and Connectivity:
- Life cycle management (C ) [SM, Conn]
- Routine launch configuration - run mode ( C) [SM, Conn, DDM]
- generic and extensible SQL editor preference pages (H)

Feature code drop
- Generic SQL editor implementation (Derby) ( C) [SM, Conn, DDM]
- Portability check (L)

Routines Debugger

Feature code drop
- Routine launch configuration - debug mode (C) [SM, Conn, DDM]
- Line breakpoint management (C ) [DDM, Conn]


Feature code drop
-
Integration w/ Model Base and Connectivity

Feature code drop
- Table view (M) [SM, Conn]

- External client's view (M) [DDM, Conn]

SQL Execution Plan

Initial build
- Tree-structure GUI execution plan (C)
- Generic and extensible Preference pages (C)

Feature code drop
- Integration w/ Model Base and Connectivity:
- Execution plan history (C)
- Save/load/print plans (C)

Feature code drop
- Import and export (M)

SQL Query Builder

Note: Not targeted for delivery in DTP 1.0

tbd

tbd

tbd





MILESTONE 4/DTP 0.N: 03/17/06

Component

RC1

01/23 – 02/24 (25)

RC2

02/27 – 03/03 (5)

RC3

03/06 – 03/10 (5)

0.N/M4

03/13 – 03/16 (4)

SQL Query Parser


[SQM, SM, LPG]

Plugin restructuring

- Separate out lexer


Testing with Eclipse 3.2 M5

Doc Drop

- Complete Internal API Documentation (H)



SQL Results View

Testing with Eclipse 3.2 M5




SQL/Routines Editor

Testing with Eclipse 3.2 M5




Routines Debugger

Testing with Eclipse 3.2 M5




SQL Execution Plan

Testing with Eclipse 3.2 M5




SQL Query Builder

Note: Not targeted for delivery in DTP 1.0

tbd

tbd

tbd

tbd


Components



Each component will have a focal point representative who will be responsible for driving the communications, design and development for that component. The focal point will interact with the other members of that team involved in the design and/or implementation and with other components as needed. Information coming from each component will be reviewed and shared with the overall project to ensure good communication, coherency and consistency.




SQL Query Parser




Scope:

The scope of the SQL Query Parser is syntactic and semantic validation of SQL statements. Initially, it will handle SQL Data Manipulation Language (DML) constructs: SELECT, INSERT, UPDATE and DELETE statements. Data Definition Language (DDL) constructs are outside the initial scope of the parser. The output of the parser is an instance of the SQL Query Model.


Design Goals:

- Based on LPG parser generator

- Quick parse feature without constructing model object to determine statement type

- Focus on test-driven development

- Target SQL Query Model directly as parse result

- Pluggable semantic action productions

- Generic, ISO SQL-based source generation, based on SQL-99/2003

- Architected support for multiple SQL dialects (inheritance)

- Use Eclipse extension-point mechanism

- Multiple parser plug-ins extend common base parser plug-in with separate LPG generated parser

- Reuse of grammar: overwrite, drop and add rules

- Along with the SQL Query Model, provide strong base for query tooling

- Enable “round-trip” editing for query builder tools

- Enable smart editing features for SQL text editors

- Provide rich information for query analysis

- Parser not dependent on a live database connection

- Two phase parsing:

- A purely syntactic parsing phase. Generate the Query model instance that represents the query. Completely independent of the database.

- A "semantic resolution" phase. Use information contained in an instance of the base SQL model (mainly table and column information) to associate the tables, columns, etc. contained in the query with the corresponding objects in a database. It doesn't matter to the parser how the catalog information in the base SQL model instance was populated (through a live database connection, cached catalog information, or in some other way).


Dependencies

LPG - used to generate SQL parser from SQL grammar; runtime jar also required

SQL Query Model - parser produces SQM objects, also used for semantic validation

SQL Model - used for semantic validation; create data type objects and hook into SQM objects


Consumers


SQL Results View


Scope


Design Goals


Dependencies

- Connection Profile: There is no strong dependency between connectivity layer and SQL Results View, other than the connection profile name, which is used for history filter and re-execution purposes only.



SQL/Routines Editor


Scope

The scope of the SQL Editor Framework is to:

- Provide the Data Development Perspective to support SQL application development lifecycle

- Help the user to develop SQL scripts by making most of eclipse IDE features and to allow vendor-specific features to be easily integrated in

- Allow the user to manipulate and run routine objects.


Design Goals

- Built using DTP infrastructure components: models--SM, DDM, connectivity layer.

- Provide generic SQL Editor support, but extendable for vendor-specific features, e.g., context menu/action, wizard invocation, etc.

- Reduce coupling between components, for example, avoid tightly coupled with SQL Results View and SQL Execution Plan View.

- Can be coupled with SQL Routine Debugger Framework and Debugger implementation.


Routines Debugger


Scope

- Allow the user to debug routine objects.

- The debug mechanism of each data server may vary, therefore the implementation is vendor specific and beyond the scope.


Design Goals

- Built using DTP infrastructure components: models--SM, DDM, connectivity layer, SQL Editor Framework, and SQL Editor implementation

- Flexible Debugger framework allows extendable vendor-specific implementations

- Reduce the coupling between SQL Editor framework and Routine Debugger framework'


Pending

- Exemplary debugger (M)

SQL Execution Plan


Scope

The scope of the Execution Plan View is to display SQL query execution plans. The execution plan retrieval mechanism is

outside the framework's scope and should be implemented by vendor-specific plug-ins.


Design Goals

- Flexible model to accommodate various execution plan formats

- Reusable UI components

- Reduce coupling between SQL Execution Plan View and SQL Editor framework and implementation.

SQL Query Builder


The SQL Query Builder will not be included in DTP 1.0. However, design and development work will go on in the 1.0 time frame, with an initial delivery targeted for the release after DTP 1.0. More details are forthcoming.