RE: [fltk.general] Has anyone used the table widget to display result sets from SQL? Any suggestions as to how about it? [General Use]

[MacArthur, Ian (Leonardo, UK)]

> I have been looking for a GUI tool to display data from a
> SQL source. Does anyone have any suggestions or direct me
> to examples of how FLTK can be used with SQL?

Have you looked in the fltk links pages?

http://www.fltk.org/links.php


There are a few things I remember seeing in the past as fltk add-ons to help with manipulating/displaying SQL material, but I can't say that I have ever used any of them.


I think SPTK is probably worth a look, anyway http://www.fltk.org/links.php?V118+Qsql and others, e.g.

Fl_SQLite http://www.fltk.org/links.php?V312+Qsql


LuaFLTK 1.0 (which also includes SQLite hooks) http://www.fltk.org/links.php?V390+Qsql


MySQLGUI http://www.fltk.org/links.php?V42+Qsql

Though whether any of those projects are actually still active I can not say.


Leonardo MW Ltd
Registered Office: Sigma House, Christopher Martin Road, Basildon, Essex SS14 3EL
A company registered in England & Wales. Company no. 02426132
********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

[Spacen]

Beware of large amounts of rows. FLTK keeps an array that remembers the size of each row, this uses a lot of memory. I once made a test version that did not have this problem by making the rows fixed height, otherwise you can just store when the hight differs from dfault. Also, I seem to remember another problem related to this, where the table code took ages to add up the size of each row. So there are a few size related limitations which may need a work around for a large amount of rows.

[Greg Ercolano]

On 09/28/16 06:49, Spacen wrote:
> Beware of large amounts of rows. FLTK keeps an array that remembers
> the size of each row, this uses a lot of memory.

Hmm, no.

There are only two arrays internal to Fl_Table, which are only integers:

_colwidths -- width of each column
_rowheights -- height of each row

This lets columns be different widths and rows be different heights.

So if you have 1000 rows and 1000 columns, that's 2000 integers.
The rest of the data in Fl_Table are small data; non-arrays.

There's also a separate class Fl_Table_Row, which derives from Fl_Table,
and only has extra a single array of chars, used to keep track of the
selection state of each row: _rowselect

So if you have 10,000 rows, that's an extra 10,000 chars in memory.
That's it really.. there's no large data in either of these widgets.

> I once made a test
> version that did not have this problem by making the rows fixed height,
> otherwise you can just store when the hight differs from dfault.

I don't think making the heights fixed should matter; there's
it still keeps track of them the same way, fixed or different.

I can only think you were using a really early version of
Fl_Table that needed some optimization.

It should be very snappy.

> Also, I seem to remember another problem related to this, where the
> table code took ages to add up the size of each row.

Shouldn't be.

Try the test/table program.

By default it creates a 500 column / 500 row table.

In the UI you can change the #rows and #cols.
Change them to 100,000 x 100,000 -- it should still be very snappy
when you scroll around.

And the memory footprint should be tiny.

I'm using 1.3 SVN current in my test.

Regarding table data, Fl_Table does not manage data at all,
it handles display only, so it doesn't have any memory implications
for data.

Data is managed by your derived class. In some applications one can
generate the data live on demand, and wouldn't have to keep data in memory at all.
(test/table is an example of this, since the cell's data is procedurally generated)

When you use Fl_Table, you override the function that draws each
cell, and it only draws cells visible on screen. So if you have a
100k x 100k table, but only 50 are visible on the screen, only those
50 cells will be requested to be drawn by your function.

Fl_Table tells you the row and column number,
and you just draw whatever data should be at that position.

[Spacen]

On Wednesday, 28 September 2016 17:29:38 UTC+1, Greg Ercolano wrote:

On 09/28/16 06:49, Spacen wrote:
> Beware of large amounts of rows. FLTK keeps an array that remembers
> the size of each row, this uses a lot of memory.

>        Hmm, no.

I am afraid so yes. I did try the test program with millions of rows. It is not usable. On the other hand it's not a usual usage case, but sometimes it's quite useful to do this so the user may scroll around a large set of data for purposes of orientation. 

[Edzard Egberts]

Spacen wrote:
>
>
> On Wednesday, 28 September 2016 17:29:38 UTC+1, Greg Ercolano wrote:
>
> On 09/28/16 06:49, Spacen wrote:

> > Beware of large amounts of rows. FLTK keeps an array that remembers
> > the size of each row, this uses a lot of memory.
>

> > Hmm, no.
>
> I am afraid so yes. I did try the test program with millions of rows. It
> is not usable. On the other hand it's not a usual usage case, but
> sometimes it's quite useful to do this so the user may scroll around a
> large set of data for purposes of orientation.

Change the design: The table should just cover the screen (not all the
data) and show the data section by scroll offset. This can be easy
realised with Fl_Table, because it doesn't hold data for itself, just
gives the frame (cells) for output of data.

[Greg Ercolano]

On 09/29/16 02:49, Spacen wrote:
>
>
> On Wednesday, 28 September 2016 17:29:38 UTC+1, Greg Ercolano wrote:
>
> On 09/28/16 06:49, Spacen wrote:
> > Beware of large amounts of rows. FLTK keeps an array that remembers
> > the size of each row, this uses a lot of memory.
>
> > Hmm, no.
>
> I am afraid so yes. I did try the test program with millions of rows.

> It is not usable. On the other hand it's not a usual usage case..

Yes, millions of rows is a lot.
But the test/table app runs fine even with 1M rows and 1M columns:

Rows: 1000000
Cols: 1000000

..scrolling is just as snappy as the defaults.

Scrolling does jump though; moving the vertical scrollbar one pixel
jumps 2000 entries at a time. I found I had to click on the scroll trough
to move a page at a time, and used the scrollbar buttons to browse
one cell at a time.

As for memory, using the above values and (top -a) to sort by memory use,
shows up pretty low compared to other processes on the machine:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
32405 erco 20 0 2714m 1.0g 49m S 6.3 27.7 544:55.98 firefox
4363 erco 20 0 724m 244m 21m S 0.0 6.5 92:21.52 thunderbird
7474 erco 20 0 985m 77m 19m S 1.0 2.1 52:00.47 plugin-containe
13127 root 20 0 52608 31m 1188 S 0.3 0.8 6:42.16 rushd
29629 root 20 0 185m 23m 4240 S 6.6 0.6 5452:54 X
29637 erco 20 0 557m 22m 5216 S 2.3 0.6 371:06.99 gnome-terminal
6094 erco 20 0 409m 19m 1824 S 0.0 0.5 0:34.07 eog
13363 erco 20 0 500m 15m 9232 S 1.0 0.4 1:49.35 mplayer
20156 root 20 0 56232 12m 3180 S 0.0 0.3 0:00.33 table <--
13558 root 20 0 236m 5768 2224 S 0.0 0.2 1:39.04 httpd
10919 erco 20 0 184m 4716 3548 S 0.0 0.1 0:00.60 notification-da
12057 root 20 0 68340 4492 2216 S 0.0 0.1 0:00.10 su
2624 erco 9 -11 408m 4336 3128 S 0.7 0.1 830:35.34 pulseaudio
^^^^

..seems reasonable to me.

But maybe there's something else going on in your app that
is causing the table to misbehave that the test program isn't doing.
If you figure it out, please make an STR. There's surely room for
optimizations.

But from what I can tell with the test program, just creating a table
with a million rows and a million cols and then scrolling all around it
seems perfectly reasonable.. not seeing any delays in the user's
scrolling ability.

> but sometimes it's quite useful to do this so the user may scroll
> around a large set of data for purposes of orientation.

If the integer arrays are a problem, I could turn the arrays off
when e.g. col_widths_all() is set. (fixed width for all cols).
Ditto for rows.

This way the arrays would only be created if e.g. col_widths(row,width)
is used to set individual widths.

Still, I wonder about your use case..

[Greg Ercolano]

On 09/29/16 09:41, Greg Ercolano wrote:
> On 09/29/16 02:49, Spacen wrote:
>> I am afraid so yes. I did try the test program with millions of rows.
>> It is not usable.
>

> Yes, millions of rows is a lot.
> But the test/table app runs fine even with 1M rows and 1M columns:
>
> Rows: 1000000
> Cols: 1000000
>
> ..scrolling is just as snappy as the defaults.

What version of FLTK you're using?

I'm using 1.3 svn current when I did my test, but I also tried with
the 1.3.3 release (Nov 2014) and got the same results.

Is your test on a release older than 1.3.3?