Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » BIRT » How about BIRT to provide "Multiple Flat Files Connection Manager"(Multiple Flat Files Connection Manager)
How about BIRT to provide "Multiple Flat Files Connection Manager" [message #717181] Fri, 19 August 2011 10:14 Go to next message
laijiazi  is currently offline laijiazi
Messages: 3
Registered: August 2011
Junior Member
Hi everyone,
I want to do like this:
1)Data Source = Flat file
2)The directory is
├─UserA
│ sales_20110801.csv
│ sales_20110802.csv
│ sales_20110803.csv
│ sales_20110804.csv
│ ...

└─UserB
sales_20110801.csv
sales_20110802.csv
sales_20110803.csv
sales_20110804.csv
...

3)I want to generate sumary reports of each user monthly, but i find that the dataset can only contain one csv file.

So, is it possible that one dataset contains multiple cvs files?

Note: No database available for me and all the file with the same format.

Thanks a lot.

[Updated on: Sat, 20 August 2011 09:09]

Report message to a moderator

Re: Is it possible that one dataset contains multiple cvs files [message #717186 is a reply to message #717181] Fri, 19 August 2011 10:32 Go to previous messageGo to next message
Jason Weathersby is currently offline Jason Weathersby
Messages: 9167
Registered: July 2009
Senior Member

Currently the flatfile datasource only supports one file. I think this
would be a good enhancement request. Can you log it? As a work around
you could look at just using a scripted datasource and iterate over all
the files and read them. Or you could use a scripted datasource to
concat all the files into one and then use a normal flat file data
source on the resultant. I have attached a files list scripted
datasource that should help you get started.

Jason

On 8/19/2011 10:14 AM, laijiazi wrote:
> Hi everyone,
> I want to do like this:
> 1)Data Source = Flat file
> 2)The directory is ├─UserA
> │ sales_20110801.csv
> │ sales_20110802.csv
> │ sales_20110803.csv
> │ sales_20110804.csv
> │ ...
> │
> └─UserB
> sales_20110801.csv
> sales_20110802.csv
> sales_20110803.csv
> sales_20110804.csv
> ...
>
> 3)I want to generate sumary reports of each user monthly, but i find
> that the dataset can only contain one csv file.
>
> So, is it possible that one dataset contains multiple cvs files?
>
> Note: No database available for me and all the file with the same format.
>
> Thanks a lot.
Re: Is it possible that one dataset contains multiple cvs files [message #717411 is a reply to message #717186] Sat, 20 August 2011 09:08 Go to previous messageGo to next message
laijiazi  is currently offline laijiazi
Messages: 3
Registered: August 2011
Junior Member
Thanks Jason.

I hope this will be provided in the future version of BIRT.
The details of my requirement is:
The data set of Flat File Data source should support to select a path, but not only one file.

It likes the "Multiple Flat Files Connection Manager" of MS SQL Server 2008

A Multiple Flat Files connection manager enables a package to access data in multiple flat files. For example, a Flat File source can use a Multiple Flat Files connection manager when the Data Flow task is inside a loop container, such as the For Loop container. On each loop of the container, the Flat File source loads data from the next file name that the Multiple Flat Files connection manager provides.

When you add a Multiple Flat Files connection manager to a package, SQL Server Integration Services creates a connection manager that will resolve to a Multiple Flat Files connection at run time, sets the properties on the Multiple Flat Files connection manager, and adds the Multiple Flat Files connection manager to the Connections collection of the package.

The ConnectionManagerType property of the connection manager is set to MULTIFLATFILE.
You can configure the Multiple Flat Files connection manager in the following ways:

Specify the files, locale, and code page to use. The locale is used to interpret locale-sensitive data such as dates, and the code page is used to convert string data to Unicode.

Specify the file format. You can use a delimited, fixed width, or ragged right format.

Specify a header row, data row, and column delimiters. Column delimiters can be set at the file level and overwritten at the column level.

Indicate whether the first row in the files contains column names.

Specify a text qualifier character. Each column can be configured to recognize a text qualifier.

Set properties such as the name, data type, and maximum width on individual columns.

When the Multiple Flat Files connection manager references multiple files, the paths of the files are separated by the pipe (|) character. The ConnectionString property of the connection manager has the following format:

<path>|<path>

You can also specify multiple files by using wildcard characters. For example, to reference all the text files on the C drive, the value of the ConnectionString property can be set to C:\*.txt.

If a Multiple Flat Files connection manager references multiple files, all the files must have the same format.

Re: Is it possible that one dataset contains multiple cvs files [message #717900 is a reply to message #717411] Mon, 22 August 2011 12:38 Go to previous message
Jason Weathersby is currently offline Jason Weathersby
Messages: 9167
Registered: July 2009
Senior Member

Can you log an enhancement request through bugzilla for this. This
looks very good.

Jason

On 8/20/2011 9:08 AM, laijiazi wrote:
> Thanks Jason.
>
> I hope this will be provided in the future version of BIRT. The details
> of my requirement is:
> The data set of Flat File Data source should support to select a path,
> but not only one file.
>
> It likes the "Multiple Flat Files Connection Manager" of MS SQL Server 2008
> A Multiple Flat Files connection manager enables a package to access
> data in multiple flat files. For example, a Flat File source can use a
> Multiple Flat Files connection manager when the Data Flow task is inside
> a loop container, such as the For Loop container. On each loop of the
> container, the Flat File source loads data from the next file name that
> the Multiple Flat Files connection manager provides.
>
> When you add a Multiple Flat Files connection manager to a package, SQL
> Server Integration Services creates a connection manager that will
> resolve to a Multiple Flat Files connection at run time, sets the
> properties on the Multiple Flat Files connection manager, and adds the
> Multiple Flat Files connection manager to the Connections collection of
> the package.
>
> The ConnectionManagerType property of the connection manager is set to
> MULTIFLATFILE.
> You can configure the Multiple Flat Files connection manager in the
> following ways:
>
> Specify the files, locale, and code page to use. The locale is used to
> interpret locale-sensitive data such as dates, and the code page is used
> to convert string data to Unicode.
>
> Specify the file format. You can use a delimited, fixed width, or ragged
> right format.
>
> Specify a header row, data row, and column delimiters. Column delimiters
> can be set at the file level and overwritten at the column level.
>
> Indicate whether the first row in the files contains column names.
>
> Specify a text qualifier character. Each column can be configured to
> recognize a text qualifier.
>
> Set properties such as the name, data type, and maximum width on
> individual columns.
>
> When the Multiple Flat Files connection manager references multiple
> files, the paths of the files are separated by the pipe (|) character.
> The ConnectionString property of the connection manager has the
> following format:
>
> <path>|<path>
>
> You can also specify multiple files by using wildcard characters. For
> example, to reference all the text files on the C drive, the value of
> the ConnectionString property can be set to C:\*.txt.
>
> If a Multiple Flat Files connection manager references multiple files,
> all the files must have the same format.
>
>
Previous Topic:Define printers for server side printing
Next Topic:(no subject)
Goto Forum:
  


Current Time: Wed Aug 27 05:07:09 EDT 2014

Powered by FUDForum. Page generated in 0.02346 seconds