Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Test and Performance Tools Platform (TPTP) » Max Datapool records
Max Datapool records [message #42111] Wed, 23 November 2005 18:39 Go to next message
David Murley is currently offline David Murley
Messages: 3
Registered: July 2009
Junior Member
What is the maximum number of records that can be stored in a datapool? (my
largest test data set has approx 250,000 records)
Re: Max Datapool records [message #42593 is a reply to message #42111] Thu, 24 November 2005 15:45 Go to previous messageGo to next message
No real name is currently offline No real name
Messages: 404
Registered: July 2009
Senior Member
Hi David,

I'm not sure whether the datapools can scale that much but I would recommend
dividing the records between multiple datapools and multiple equivalent
classes. This will make the maintenance of your records a lot cheaper.
Aside that, the load time for a datapool will not be sluggish.
If you find that the datapools are not scaling to your standard, then feel
free to open features against the component.

Thanks,
Ali Mehregani

"David Murley" <david.murley@worldflow.net> wrote in message
news:dm2d38$gq7$1@news.eclipse.org...
> What is the maximum number of records that can be stored in a datapool?
(my
> largest test data set has approx 250,000 records)
>
>
Re: Max Datapool records [message #42743 is a reply to message #42111] Thu, 24 November 2005 17:46 Go to previous message
Paul Slauenwhite is currently offline Paul Slauenwhite
Messages: 970
Registered: July 2009
Senior Member
Hi David,
There are two problems with a single datapool containing 250,000
records:

1) Creating the CSV file using spreadsheet applications may be problematic
due to the upper bound for table rows. For example, Microsoft Excel limits
the number of rows to 65,536 and the number of columns to 256.

2) Although there is no theoretical upper bound to the number of records in
a CSV used to create a TPTP datapool (aside from memory and disk space
constraints), using the TPTP v4.1 driver the performance is not scalable.
For 65,536 records (one column), it took ~10 minutes for the import and
every save. I have opened defect
https://bugs.eclipse.org/bugs/show_bug.cgi?id=117932 to examine these
performance issues.

Paul
"David Murley" <david.murley@worldflow.net> wrote in message
news:dm2d38$gq7$1@news.eclipse.org...
> What is the maximum number of records that can be stored in a datapool?
(my
> largest test data set has approx 250,000 records)
>
>
Previous Topic:Can't see the Hyades URL test in the run configuration
Next Topic:Datapool - TPTP 4.0.1 missing jar files
Goto Forum:
  


Current Time: Mon Sep 15 09:44:31 GMT 2014

Powered by FUDForum. Page generated in 0.08568 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software