Eclipse Community Forums: BIRT » How to best handle badly formatted data

Home » Archived » BIRT » How to best handle badly formatted data

How to best handle badly formatted data [message #182975]

Mon, 07 August 2006 06:10

Eclipse User

Originally posted by: exquisitus.transentia.com.au

I am curently handling CSV data using the flat file data source.

Turns out SOME rows in the data are 'bad' (ie don't match the
expected/required # fields).

I am open to suggestions as to how best to handle this situation in BIRT
2.1.

I anticipate developing a POJO that can reject bad lines (perhaps
providing a collection of 'BadLine' objects; at minimum a count of
rejected lines) while presenting a collection of 'GoodData' objects for
further processing.

Is this naive or reinventing the wheel?

Cheers,

Exquisitus

Re: How to best handle badly formatted data [message #183101 is a reply to message #182975]

Mon, 07 August 2006 12:13

Eclipse User

You may be able to use a filter. How do you know if a row is bad?

Jason

"exquisitus" <exquisitus@transentia.com.au> wrote in message
news:575750d809582c7e0379e1dfb0c27675$1@www.eclipse.org...
>I am curently handling CSV data using the flat file data source.
>
> Turns out SOME rows in the data are 'bad' (ie don't match the
> expected/required # fields).
>
> I am open to suggestions as to how best to handle this situation in BIRT
> 2.1.
> I anticipate developing a POJO that can reject bad lines (perhaps
> providing a collection of 'BadLine' objects; at minimum a count of
> rejected lines) while presenting a collection of 'GoodData' objects for
> further processing.
>
> Is this naive or reinventing the wheel?
>
> Cheers,
>
> Exquisitus
>

Re: How to best handle badly formatted data [message #183190 is a reply to message #182975]

Mon, 07 August 2006 14:59

Eclipse User

Originally posted by: ablancas_ at_pervasive_dot_com.x.y.zzz

You might want to do the data analysis ahead of running the report, perhaps staging the clean data to a second file for
input to BIRT (same for bad data or the results of the validation). Data analysis is probably to much of a burden for a
chart or report engine to handle.

"exquisitus" <exquisitus@transentia.com.au> wrote in message news:575750d809582c7e0379e1dfb0c27675$1@www.eclipse.org...
>I am curently handling CSV data using the flat file data source.
>
> Turns out SOME rows in the data are 'bad' (ie don't match the expected/required # fields).
>
> I am open to suggestions as to how best to handle this situation in BIRT 2.1.
> I anticipate developing a POJO that can reject bad lines (perhaps providing a collection of 'BadLine' objects; at
> minimum a count of rejected lines) while presenting a collection of 'GoodData' objects for further processing.
>
> Is this naive or reinventing the wheel?
>
> Cheers,
>
> Exquisitus
>

Previous Topic:	package selection
Next Topic:	Setting a timeout on a dataset query

Goto Forum:

-=] Back to Top [=-

Current Time: Mon Jun 09 08:52:44 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter