Home » Eclipse Projects » Standard Widget Toolkit (SWT) » Huge table, records, scrolling, etc.
Huge table, records, scrolling, etc. [message #462513] |
Thu, 13 October 2005 11:15 |
mr. burns Messages: 402 Registered: July 2009 |
Senior Member |
|
|
Hello,
I have a problem may other developers also have, but I did'nt find any
resonable solution yet.
My problem is I want to use SWT to show a table with many records to the
user.
Lets say I want to show 1.000.000 records, normally (on less recors) I can
load all records into table items and then I can show them.
But 1.000.000 records take a very long time to read from database, so there
must be another solution.
I read something about virtual tables and the propety SWT.SetData. But with
this I am not that familiar.
I suggest do do it like follows:
I only read as many records from database as fit on the screen, lets say
this are 25. Only if the cursor moves closer to the 25th record the next 25
records are preloaded and ready to show.
My 1st problem with this is, how can I make the scrollbar believe there are
1.000.000 records instead of 25? So the slider must be as small as 1.000.000
records instead of 25 records.
My 2nd problem is, which trigger / listener I should use to detect/trigger
the preloading of the next 25 records? I must trigger if the user uses the
keyboard to move down/up and/or if a user uses the scrollbar.
I believe there are standard solutions available, but sorry I could not
find.
Any help would be appreciated, thanks a lot!
|
|
|
Re: Huge table, records, scrolling, etc. [message #462524 is a reply to message #462513] |
Thu, 13 October 2005 13:16 |
Grant Gayed Messages: 2150 Registered: July 2009 |
Senior Member |
|
|
This is exactly how VIRTUAL works. For an example snippet see
http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.swt.sni ppets/src/org/eclipse/swt/snippets/Snippet144.java?rev=HEAD& amp;content-type=text/vnd.viewcvs-markup .
You just retrieve the records lazily when requested in the SetData callback.
Grant
"Mr. Burns" <Mr._Burns@web.de> wrote in message
news:dilfjv$lll$1@news.eclipse.org...
> Hello,
>
> I have a problem may other developers also have, but I did'nt find any
> resonable solution yet.
> My problem is I want to use SWT to show a table with many records to the
> user.
> Lets say I want to show 1.000.000 records, normally (on less recors) I can
> load all records into table items and then I can show them.
> But 1.000.000 records take a very long time to read from database, so
there
> must be another solution.
> I read something about virtual tables and the propety SWT.SetData. But
with
> this I am not that familiar.
> I suggest do do it like follows:
>
> I only read as many records from database as fit on the screen, lets say
> this are 25. Only if the cursor moves closer to the 25th record the next
25
> records are preloaded and ready to show.
> My 1st problem with this is, how can I make the scrollbar believe there
are
> 1.000.000 records instead of 25? So the slider must be as small as
1.000.000
> records instead of 25 records.
> My 2nd problem is, which trigger / listener I should use to detect/trigger
> the preloading of the next 25 records? I must trigger if the user uses the
> keyboard to move down/up and/or if a user uses the scrollbar.
> I believe there are standard solutions available, but sorry I could not
> find.
>
> Any help would be appreciated, thanks a lot!
>
>
|
|
| |
Re: Huge table, records, scrolling, etc. [message #462564 is a reply to message #462551] |
Thu, 13 October 2005 23:02 |
No real name Messages: 97 Registered: July 2009 |
Member |
|
|
To use VIRTUAL, you need to have the following two capabilities;
1. Should know the exact number of rows.
2. Able to identify records using index numbers, i.e., 0, 1, 2, ...
Of course, you also need records should not change sequence and
information during browsing, unless change using editors.
The rest is trivial.
Regards.
"Mr. Burns" <Mr._Burns@web.de> wrote in message
news:dilri5$6m6$1@news.eclipse.org...
> Thanks Grant (again),
>
> that was exactly what I needed. But what if I immediately want to go to
> the last record (and maybe back again to the first one) what is typical
> for database applications?
> Normally in any database application there are several buttons to move
> within the database records. Such as:
>
> 1 record forward
> 1 record back
> Jump to 1st record
> Jump to last record
>
> How can I achieve this using the VIRTUAL table?
>
> Thanks again for further information,
>
> best regards,
>
> Mr. Burns
>
>
|
|
|
Re: Huge table, records, scrolling, etc. [message #462592 is a reply to message #462564] |
Fri, 14 October 2005 10:00 |
mr. burns Messages: 402 Registered: July 2009 |
Senior Member |
|
|
Hello Joe,
thanks for reply, you say the rest is trivial, but if one is new to SWT its
not like database programming under C++ as I did do usually.
What you mean if you say: 'records should not change sequence and
information during browsing'?
And whaot you mean if you say 'unless change using editors'?
My main problem is what I should program within the SWT.SetData listener to
deal with the database?
To fill the table with the text "item " + index, as Grant show in the
snippet is trivial.
But to deal with database causes more problem. Should I place code there to
read only one record out of the database?
(like one record created by "Item " + Index)
But then if I scroll down, lets say for 25 records, the listener is executed
25 times reading only one record.
So the database receives 25 SQL Select ... statements.
Because of efficiency I would go a different way just sending only one
SELECT ... statment receiving 25 records within one resultset. This will
reduce database traffic. But of course than the code inside the listener is
more difficult, because it is executed once for every new record which is to
read, but only every 25th execution a new SELECT statement to the database
will be necessary.
Do you understand what I mean?
Thanks for any comments which way to go,
best regards,
Mr. Burns
|
|
|
Re: Huge table, records, scrolling, etc. [message #462593 is a reply to message #462592] |
Fri, 14 October 2005 12:07 |
Haris Peco Messages: 1072 Registered: July 2009 |
Senior Member |
|
|
i use JTable (JXTable from swingx) for this and Swing/AWT features
and implement list table model - list read data from database with
scrollable resultset
Mr. Burns wrote:
> Hello Joe,
>
> thanks for reply, you say the rest is trivial, but if one is new to SWT
> its not like database programming under C++ as I did do usually.
> What you mean if you say: 'records should not change sequence and
> information during browsing'?
> And whaot you mean if you say 'unless change using editors'?
>
> My main problem is what I should program within the SWT.SetData listener
> to deal with the database?
> To fill the table with the text "item " + index, as Grant show in the
> snippet is trivial.
> But to deal with database causes more problem. Should I place code there
> to read only one record out of the database?
> (like one record created by "Item " + Index)
> But then if I scroll down, lets say for 25 records, the listener is
> executed 25 times reading only one record.
> So the database receives 25 SQL Select ... statements.
> Because of efficiency I would go a different way just sending only one
> SELECT ... statment receiving 25 records within one resultset. This will
> reduce database traffic. But of course than the code inside the listener
> is more difficult, because it is executed once for every new record which
> is to read, but only every 25th execution a new SELECT statement to the
> database will be necessary.
>
> Do you understand what I mean?
>
> Thanks for any comments which way to go,
>
> best regards,
>
> Mr. Burns
|
|
| | |
Re: Huge table, records, scrolling, etc. [message #462601 is a reply to message #462596] |
Fri, 14 October 2005 15:17 |
Eclipse User |
|
|
|
Originally posted by: micasim.gmx.de
On Fri, 14 Oct 2005 15:28:13 +0200, Mr. Burns <Mr._Burns@web.de> wrote:
> Hello Haris,
>
> I want it to do using SWT instead of AWT/SWING!
> But thanks, anyway,
>
> Any other ideas?
>
>
Hello Mr. Burns,
I'm struggling with quite the same problem.
At the moment I see two solutions
First is to give KTable a chance. It's kind of a JTable port to SWT.
You might take a look at http://sourceforge.net/projects/ktable
The second is to put a org.eclipse.jface.viewers.TableViewer on top of the
SWT Table.
To actually work with a SWT.VIRTUAL table, you must connect a
LabelProvider and a ContentProvider to your TableViewer. While the
LabelProvider is easy (you stated you're a beginner, so don't hesitate to
ask about it, too); the ContentProvider is the interesting one when
content is stored in a database. Therefore you have to code a
ContentProvider that implements
org.eclipse.jface.viewers.LazyContentProvider. To do so you have to
implement the method
void updateElement(int index)
The API doc tells "Called when a previously-blank item becomes visible in
the TableViewer. If the content provider knows the element at this row, it
should respond by calling TableViewer#replace(Object, int)"
The second parameter is the index again, while the first one is the object
that the content provider retrieved from DB (please remind that the
presentation is done by the LabelProvider)
So when the user scrolls to the last record (e.g. using the scrollbar) you
have to return whatever you consider to be the last object fulfilling the
restrictions.
To batch the select statements as you're thinking about, your content
provider should cache the retrieved objects.
So when it receives a updateElement (X), it will first look in the cache
and if it isn't there,
load it together with N others, that are all reminded in the local cache.
You guess the idea.
There's one last pitfall: You must tell the table how much records there
are to come (count (*)) by calling TableViewer.setItemCount(int) before
displaying the first bunch.
Hope this helps,
Michael
|
|
|
Re: Huge table, records, scrolling, etc. [message #462651 is a reply to message #462596] |
Sun, 16 October 2005 22:58 |
No real name Messages: 97 Registered: July 2009 |
Member |
|
|
"Mr. Burns" <Mr._Burns@web.de> wrote in message
news:diobpf$7pi$1@news.eclipse.org...
> Hello Haris,
>
> I want it to do using SWT instead of AWT/SWING!
> But thanks, anyway,
It's the same. To make virtual table work,
(1) you need to know the exact number of rows,
(2) then be able to locate rows using index numbers.
Because of the first condition, even if you use Virtual tables,
it takes time DBMS to compute the numbers. Second,
you are using virtual table to handle large data. Unfortunately
if you use scrollable result tables, this will be huge burdon to
both client and dbms server. So unless you have well prepared
data for this purpose, performance will be always an issue.
For scrollable result tables, look at JDBC APIs. It has
nothing to do with AWT or SWT!
Regards.
|
|
| |
Re: Huge table, records, scrolling, etc. [message #462667 is a reply to message #462659] |
Mon, 17 October 2005 10:46 |
No real name Messages: 97 Registered: July 2009 |
Member |
|
|
"Mr. Burns" <Mr._Burns@web.de> wrote in message
news:divl3o$ijo$1@news.eclipse.org...
> Hello Michael,
>
> thanks for the detailed information.
> But is that really true? Seems for me (I come from C++) very complicated
> to properly display data using JAVA. ... What is the right way to store
> the cache table data? Should I use a vector, or should I prefer a 2nd
> (unvisible) Table and TableItems where I cache the data, and if needed in
> the visible table, copy from the invisible table to the visible one?
Note that it has nothing to do with C++ or D++ or even E++!
In ANY virtual table, system needs to know the exact number of rows!
Then system asks rows to show on the table window.
It asks using *index numbers* so your exit module should supply rows
of that index! If your database table has that index number or row
identifier
as primary key or as index, then you can easily fetch rows on demand.
Otherwise, whether you use virtual or not, if you have to cache all inside
your system, you may be better off with standard tables.
Regards.
|
|
| |
Re: Huge table, records, scrolling, etc. [message #462712 is a reply to message #462651] |
Mon, 17 October 2005 23:11 |
Haris Peco Messages: 1072 Registered: July 2009 |
Senior Member |
|
|
Joe Smith wrote:
>
> "Mr. Burns" <Mr._Burns@web.de> wrote in message
> news:diobpf$7pi$1@news.eclipse.org...
>> Hello Haris,
>>
>> I want it to do using SWT instead of AWT/SWING!
>> But thanks, anyway,
>
> It's the same. To make virtual table work,
> (1) you need to know the exact number of rows,
> (2) then be able to locate rows using index numbers.
> Because of the first condition, even if you use Virtual tables,
> it takes time DBMS to compute the numbers. Second,
> you are using virtual table to handle large data. Unfortunately
> if you use scrollable result tables, this will be huge burdon to
> both client and dbms server. So unless you have well prepared
> data for this purpose, performance will be always an issue.
>
> For scrollable result tables, look at JDBC APIs. It has
> nothing to do with AWT or SWT!
>
It is simple get number of row in scrollable result and it is quick
(for good databases) - go to last record and get row number
with hibernate I do next :
ScrollableResults scr = q.scroll();
scr.last();
size = scr.getRowNumber() + 1;
it is simple and with pure JDBC
If you have slow query, this will be slow too, but it is true for only one
(firts) row, too.
Any database load complete query in memory (haven't cursors) and then there
isn't solution - this is case with mysql, for example - postgresql, special
oracle, work good
Peco
|
|
| |
Re: Huge table, records, scrolling, etc. [message #462714 is a reply to message #462668] |
Mon, 17 October 2005 21:34 |
No real name Messages: 97 Registered: July 2009 |
Member |
|
|
"Mr. Burns" <Mr._Burns@web.de> wrote in message
news:dj00bi$2a8$1@news.eclipse.org...
> Thanks for reply,
>
> but this does not really answer my question, I wanted to know how I fetch
> now the records from DB and where to store.
> I want now each time the SWT.SetData trigger is executed, check if next 25
> records are to preload. If yes what is the best way to store/cache these
> records? Should I use a vector? Should I use a 2nd table where I cache the
> records in the TableIems and copy them into the virtual table?
How big are your tables?
If you have to cache all inside your program, it's better not to
use virtual table. In addition, can you access your tables
using indexes, say, 0, 1, 2, ..n? If you don't have keys,
then create temporary result tables inside dbms and
fetch partially. You may need to have some local cacheing
systems. Otherwise, for each row, there will be connection,
query and fetch statements executed resulting in slow speed.
So you need to develop some local caching capability
along with temporary tables.
>
> Thanks for further discussions!
>
>
|
|
| |
Re: Huge table, records, scrolling, etc. [message #462718 is a reply to message #462714] |
Tue, 18 October 2005 06:32 |
mr. burns Messages: 402 Registered: July 2009 |
Senior Member |
|
|
Hello Joe,
thanks for reply, I have tables that are in the beginning very small, but
then every day it will grow for arround 1000-2000 records.
You wrote: 'You may need to have some local cacheing'
Exactly that I wanted to know, I asked this in my prevous mail. HOW do I do
the caching, should I use a vector, should I use a 2nd table where I store
the records within the table items, and if needed I copy them to the virtual
table?
Thanks for further discussion.
|
|
| | |
Re: Huge table, records, scrolling, etc. [message #462725 is a reply to message #462718] |
Tue, 18 October 2005 10:13 |
No real name Messages: 97 Registered: July 2009 |
Member |
|
|
"Mr. Burns" <Mr._Burns@web.de> wrote in message
news:dj24t9$rso$1@news.eclipse.org...
> Hello Joe,
>
> thanks for reply, I have tables that are in the beginning very small, but
> then every day it will grow for arround 1000-2000 records.
> You wrote: 'You may need to have some local cacheing'
> Exactly that I wanted to know, I asked this in my prevous mail. HOW do I
> do the caching, should I use a vector, should I use a 2nd table where I
> store the records within the table items, and if needed I copy them to the
> virtual table?
Caching per se is quite straight forward. For example, you can cache
50 rows starting from, say, 200.
1. If a row requested is in the cache, then return from the cache.
2. If a row requested is before the starting row, then refill the cache
with 50 preceding rows ending with the row requested. and then
return the row from cache.
3. If a row requested is after the last row in the cache, then refill
the cache with the next 50 rows from the row requested.
But you should know the exact number of rows and able to retrive
rows using indices.
>
> Thanks for further discussion.
>
>
|
|
| |
Re: Huge table, records, scrolling, etc. [message #462745 is a reply to message #462659] |
Tue, 18 October 2005 18:32 |
Eclipse User |
|
|
|
Originally posted by: friederich.kupzog.de
Hi.
Mr. Burns wrote:
> I cannot catch the point what the advantage of KTable should be.
KTable was developed before the SWT Table got the VIRTUAL extension to
solve exactly this issue. Now that the SWT Table can also be virtual,
KTable's strength lies more in non-database-related features like fancy
selection modes, custom cell renderers, cell spanning etc.
I use KTable with MySQL (actually wrote it for that purpose). My
TableModel uses a scrollable ResultSet. My average customer has about
1000 records with up to ten (not very active) users connected to the
server and the performance is very good. There is a short delay when
fetching the first record but this seems to be nearly independent of the
number of records in the table.
If you are not sure whether you need caching or not, I would suggest a
simple implementation using SWT Table or KTable first and then, if it
fails to please, put more tuning effort into it later. I would say that
KTable gives you more flexibility for the Data-View Interface, but if
you started with SWT table, this might be the way for you to go.
Regards,
Friederich
--
Friederich Kupzog
Elektronik & Software
Neusser Str. 5-7
50670 Köln
Tel 0241 160696-1
Fax 0221 726670
www.kupzog.de/fkmk
|
|
| |
Goto Forum:
Current Time: Sat Sep 21 02:02:26 GMT 2024
Powered by FUDForum. Page generated in 0.06328 seconds
|