How to display very large tables

[luafox]

I have written a program which handles large data arrays inside lua tables. I would like to display them as a table in a spreadsheet like program. Of course the first idea is to use QTableView but as I understand it, you need to add the values of every cell into it. So I would need to copy all the data from the lua table into QTableView which would at least double the memory consumption. What I would like to have is a way to create an empty table with any number of columns and rows for which Qt requests only the amount of data currently shown on screen. Because the data is held in memory anyway there is no need to copy more of the data that what is currently shown on screen.

Does QSqlTableView help here? Could you use lua to catch SQL requests and redirect them to the data tables using metamethods? Or does QSqlTableView  also request the whole table at the beginning?

Can you suggest a clever way of doing this? Or do you think this is a question better ask on the Qt forums?

Thanks a lot!
Michael Gerbracht

[Michal Kottman]

> ... as I understand it, you need to add the values of every cell into it. So I would need to copy all the data from the lua table into QTableView which would at least double the memory consumption.

This is not entirely true. What you can do is create a Model (derived
from QAbstractTableModel [1]), which will access the elements of your
Lua table in a way QTableView can understand it.

What you need to do is to reimplement some methods dictated by the
documentation for QAbstractTableModel, i.e.:

* rowCount() and columnCount() to return the number of rows/columns, and
* data(index, role) to actually return the data at a given index

This will usually work like this:

local data = { ... some large 2d table }

local TableView = QTableView.new()
local DataModel = QAbstractTableModel.new(TableView)

function DataModel:rowCount() return 42 end
function DataModel:columnCound() return 3 end
local nothing = QVariant()
function DataModel:data(index, role)
if role == 0 then -- DisplayRole
local row = index:row()
local col = index:column()
return data[row+1][col+1] -- get the data from your Lua table
end
end

TableView:setModel(DataModel)
TableView:show()

This code is not tested (I do not have Lqt installed right now) but it
should work very similar to what I have written.


[1] http://doc.qt.nokia.com/4.7-snapshot/qabstracttablemodel.html

[luafox]

Thank you very much for your help and pointing me into the right direction, I will try to implement it soon. No problem if it might not be 100% correct - I hope this will be enough help for me to get started.

[luafox]

Some time ago you helped me with the QAbstractTableModel, I got it working in the meantime, but I now tried to open more than one table view at the same time and it crashed when you start "using" it. If you start it (code shown below), it shows three table views where you can select columns and rows as expected, but if you use the windows for a few seconds (or try to resize the windows) it crashes with the error "Abstract method rowCount not implemented! In =(tail call)" - sometimes it also misses another method like data. DO you have any idea where the problem could be? If you reduce the number of windows to 1 (for loop at the beginning) then everything seems to be fine.

require'qtcore'
require'qtgui'

app = QApplication(1 + select('#', ...), {arg[0], ...})

local TableView = {}

for n = 1,3 do

  local data = {}
  for i = 1,10 do
    data[i] = {}
    for j = 1,20 do
      data[i][j] = n
    end
  end

TableView[n] = QTableView.new()
local DataModel = QAbstractTableModel.new(TableView[n])
local parent = QModelIndex()

function DataModel:rowCount() return #data[1] end
function DataModel:columnCount() return #data end
function DataModel:parent() return parent end

local nothing = QVariant()
function DataModel:data(index, role)
    if role == 0 then -- DisplayRole
        local row = index:row()
        local col = index:column()

        return QVariant(data[col+1][row+1]) -- get the data from your Lua table
    else
      return nothing
    end
end

TableView[n]:setModel(DataModel)
TableView[n]:show()

end -- for n

app.exec()

thanks again,
Michael

[luafox]

On Saturday, August 4, 2012 4:38:18 PM UTC+2, Michal Kottman wrote:

function DataModel:data(index, role)
    if role == 0 then -- DisplayRole
        local row = index:row()
        local col = index:column()

        return QVariant(data[row+1][col+1)] -- get the data from your Lua table  

       elseif role == 8 then -- Background color
           local brush = QBrush(QColor(123,56,44,255))
           return QVariant(brush)

    end
end

I have got another question regarding the QTableView. I would like to color individual rows in a different background color. I found that role = 8 is used to set the background color. The color should be gives as QBrush and converted to QVariant. Unfortunately I get the following error message:

lua: QVariant::new(QBrush*): incorrect or extra arguments, expecting:  or QTime*, or QBitArray*, or QVariant*, or integer, or QDataStream*, or QRect*, or QStringList*, or QChar*, or QPointF*, or QLine*, or boolean, or QSize*, or QList<QVariant>*, or QDateTime*, or QUrl*, or QRegExp*, or QRectF*, or QPoint*, or number, or QLatin1String*, or QDate*, or QSizeF*, or QLocale*, or QLineF*, or QVariant.Type, or Qt.GlobalColor, or QString*, or string,.

This error suggest that QVariant can not handle QBrush, but [1] shows that QVariant should understand QBrush. So I wonder what I am missing here. Does anyone know and can help me? Or is there a more simple way to set the background color of rows (in the code above I left away the part that defines which rows to color, so I would expecte the code to set the background color of the whole table)?

Thank you very much!
Michael

[1] http://qt-project.org/doc/qt-4.8/qvariant.html#Type-enum

[Michal Kottman]

On 16 September 2012 12:04, luafox <in...@luafox.com> wrote:
> I have got another question regarding the QTableView. I would like to color
> individual rows in a different background color. I found that role = 8 is
> used to set the background color. The color should be gives as QBrush and
> converted to QVariant. Unfortunately I get the following error message:
>
> lua: QVariant::new(QBrush*): incorrect or extra arguments, expecting: or
> QTime*, or QBitArray*, or QVariant*, or integer, or QDataStream*, or QRect*,
> or QStringList*, or QChar*, or QPointF*, or QLine*, or boolean, or QSize*,
> or QList<QVariant>*, or QDateTime*, or QUrl*, or QRegExp*, or QRectF*, or
> QPoint*, or number, or QLatin1String*, or QDate*, or QSizeF*, or QLocale*,
> or QLineF*, or QVariant.Type, or Qt.GlobalColor, or QString*, or string,.
>
> This error suggest that QVariant can not handle QBrush, but [1] shows that
> QVariant should understand QBrush. So I wonder what I am missing here. Does
> anyone know and can help me? Or is there a more simple way to set the
> background color of rows (in the code above I left away the part that
> defines which rows to color, so I would expecte the code to set the
> background color of the whole table)?

The problem is that QVariant belongs to the QtCore module, but the
QBrush is from the QtGui module. You CAN use it in lqt, but it is not
as straight-forward. You should be able to use it like this:

local brush = QBrush 'red'
local brushVariant = QVariant()
brushVariant:setValue(brush)

ret = brushVariant:value()
print(ret.__type, ret:color():name()) -- QBrush* #ff0000

So essentially, create a variant, and use :setValue(v) to populate it.
Keep it somewhere and return it from data() when role ==
Qt.ItemDataRole.BackgroundRole then return this pre-populated variant
(to minimize memory allocations).

[luafox]

On Monday, September 17, 2012 12:56:11 PM UTC+2, Michal Kottman wrote:

On 16 September 2012 12:04, luafox <in...@luafox.com> wrote:
> I have got another question regarding the QTableView. I would like to color
> individual rows in a different background color. I found that role = 8 is
> used to set the background color. The color should be gives as QBrush and
> converted to QVariant. Unfortunately I get the following error message:
>

> lua: QVariant::new(QBrush*): incorrect or extra arguments [...]

The problem is that QVariant belongs to the QtCore module, but the
QBrush is from the QtGui module. You CAN use it in lqt, but it is not
as straight-forward. You should be able to use it like this:

    local brush = QBrush 'red'
    local brushVariant = QVariant()
    brushVariant:setValue(brush)

    ret = brushVariant:value()
    print(ret.__type, ret:color():name()) -- QBrush* #ff0000

So essentially, create a variant, and use :setValue(v) to populate it.
Keep it somewhere and return it from data() when role ==
Qt.ItemDataRole.BackgroundRole then return this pre-populated variant
(to minimize memory allocations).

Ok, thanks for the hint, I got it working now!
Michael

[luafox]

On Sunday, September 9, 2012 2:08:13 PM UTC+2, luafox wrote:

Some time ago you helped me with the QAbstractTableModel, I got it working in the meantime, but I now tried to open more than one table view at the same time and it crashed when you start "using" it. If you start it (code shown below), it shows three table views where you can select columns and rows as expected, but if you use the windows for a few seconds (or try to resize the windows) it crashes with the error "Abstract method rowCount not implemented! In =(tail call)" - sometimes it also misses another method like data. DO you have any idea where the problem could be? If you reduce the number of windows to 1 (for loop at the beginning) then everything seems to be fine.

Unfortunately the problem still exists and I did not find a solution, yet. I posted this also on the Qt forum but I do not fully understand the answer to my question and whether the suggested additional code is really necessary using lqt. Can anybody comment on the suggested solution? The thread can be found here:
http://qt-project.org/forums/viewthread/21514/

thank you very much!
Michael