Qt - loading resource from strings.

[Bacco]

Hi, Viktor

I have finally managed to load successfuly a binary compiled (
rcc.exe -binary ) Qt resource, that is embedded in a string in my
test.

Snippet of use:
re1 := QResource():new()
re1:registerResource_1(data,':/') /* data is embedded in the prg,
and not loaded */
lb1 := Qlabel():new()
lb1:setPixMap(QPixMap():new(':image1.png'));

I want to finish this, and share the code, however, to accomplish
this, I had to do the following things:

- I had uncommented these lines in QResource.qth
bool registerResource ( const uchar * rccData, const QString &
mapRoot = QString() )
bool unregisterResource ( const uchar * rccData, const QString &
mapRoot = QString() )

- Also had to add a little fix in the hbqtgen.prg, to fix uchar * ->
char conversion (that affects various Qt functions not in use now)
Original:
CASE aA[ PRT_CAST ] == 'uchar'
aA[ PRT_BODY ] := '( char ) hb_parni( ' + cHBIdx + ' )'
aA[ PRT_DOC ] := 'n' + cDocNM
New:
CASE aA[ PRT_CAST ] == 'uchar' .and. aA[ PRT_L_FAR ] .and.
!( aA[ PRT_L_CONST ] )
aA[ PRT_BODY ] := '( char * ) hb_parc( ' + cHBIdx + ' )'
aA[ PRT_DOC ] := 'c' + cDocNM

CASE aA[ PRT_CAST ] == 'uchar' .and. !( aA[ PRT_L_FAR ] )
.and. !( aA[ PRT_L_CONST ] )
aA[ PRT_BODY ] := '( char ) hb_parni( ' + cHBIdx + ' )'
aA[ PRT_DOC ] := 'n' + cDocNM

And here comes the problem: I also needed to add a cast into the
generated QResource.qth
from @287
hb_retl( ( p )->registerResource( hbqt_par_uchar( 2 ),
hbqt_par_QString( 3 ) ) );
to
hb_retl( ( p )->registerResource( (unsigned char *)
hbqt_par_uchar( 2 ), hbqt_par_QString( 3 ) ) );

And this into hbqt.h (only as an helper, as I dont know the right way
to deal with this)
#define hbqt_par_uchar( n ) ( hb_parcx( n ) )

So I need some help to deal correctly with uchar in the harbour api.
Of course I can simply add this cast in the generator, but I believe
that it's not the right way.
Please, give me the right directions, I believe that this will open
many possibilites not only in the resources usage, but in other Qt
parts. Also by fixing this I can send the finished embedded resource
sample and share the experience.

Thanks in advance
Bacco

[Bacco]

Hi, Pritpal

The last email was meant to be addressed to you also. I'll be glad if
you can help me, after all, It's directly related to your extensive
work. I just have the bad habit to send emails in a hurry.

Thanks.
Bacco

[Viktor Szakáts]

Hi Bacco,

All the changes you describe seem correct to me, assuming uchar
is a simple C 'unsigned char' value in QT. In the hbqt_par_uchar()
bit, it's enough to cast it.

Otherwise it looks good, no more roadblocks in the way. The
only missing bits are to create binary to .prg converter, plus
required glue logic in .prg code (which can f.e. be an
INIT PROC calling the QResource() bits), and to rework hbmk2
qt plugin to coordinate all these (in fact all of these can be,
and ideally should be included in the hbmk2 qt plugin script
itself).

Viktor

[Bacco]

Hi, Viktor

On Wed, Jun 30, 2010 at 10:16, Viktor Szakáts <harbo...@syenar.hu> wrote:
> Hi Bacco,
>
> All the changes you describe seem correct to me, assuming uchar
> is a simple C 'unsigned char' value in QT. In the hbqt_par_uchar()
> bit, it's enough to cast it.

So the cast logic should be inserted on the hbqtgen.prg, I assume.
I'll look into it.

> Otherwise it looks good, no more roadblocks in the way. The
> only missing bits are to create binary to .prg converter, plus
> required glue logic in .prg code (which can f.e. be an

The binary to prg that I used indeed is very simple, it just generate this:

qt_resource_data := hb_inline {
char qt_resource_data[] = {
0x89,0x50,0x4e,0x47,0x0d,0x0a,0x1a,0x0a,0x00,0x00,0x00,0x0d,0x49,0x48,0x44,0x52,
.... chunk of data ....
0x68,0x49,0x00,0x00,0x00,0x00,0x49,0x45,0x4e,0x44,0xae,0x42,0x60,0x82,
};
hb_retclen(qt_resource_data, sizeof ( qt_resource_data ) );
}

Of course I'll send a polished version.

> INIT PROC calling the QResource() bits), and to rework hbmk2
> qt plugin to coordinate all these (in fact all of these can be,
> and ideally should be included in the hbmk2 qt plugin script
> itself).

It's the logical next step. I don't have enough experience right now,
but the path is set.

Regards
Bacco

[Viktor Szakáts]

Hi,

>> Otherwise it looks good, no more roadblocks in the way. The
>> only missing bits are to create binary to .prg converter, plus
>> required glue logic in .prg code (which can f.e. be an
>
> The binary to prg that I used indeed is very simple, it just generate this:
>
> qt_resource_data := hb_inline {
> char qt_resource_data[] = {
> 0x89,0x50,0x4e,0x47,0x0d,0x0a,0x1a,0x0a,0x00,0x00,0x00,0x0d,0x49,0x48,0x44,0x52,
> .... chunk of data ....
> 0x68,0x49,0x00,0x00,0x00,0x00,0x49,0x45,0x4e,0x44,0xae,0x42,0x60,0x82,
> };
> hb_retclen(qt_resource_data, sizeof ( qt_resource_data ) );
> }
>
> Of course I'll send a polished version.

inline C is not allowed for Harbour SVN code, anyhow I'm
working on a .prg based solution to be included in the
plugin.

Viktor

[Viktor Szakáts]

Well, maybe a pure generated .c file will be better.
I've run into a Harbour compiler limitation:
Error E0030 Syntax error "memory exhausted at '+'"

That was with 427KB binary. Pbly there is a limit of
64K for strings in pcode.

So a combination of .c and .prg code could be a solution,
though I suspect there exist something more elegant.

F.e. in this case we might just not need the .prg init
code, but use Harbour .c level init functions. In this
case we'd almost completely replicate current solution,
except that there is no need anymore for QT .h headers,
which is a good thing.

Question: Would the registration functions work from
our own .c level init function?:

---
static void hbqt_res_Init( void * cargo )
{
HB_SYMBOL_UNUSED( cargo );

/* QResource init from here */
}

HB_CALL_ON_STARTUP_BEGIN( _hb_hbqt_init_ )
hb_vmAtInit( hbqt_res_Init, NULL );
HB_CALL_ON_STARTUP_END( _hb_hbqt_init_ )

#if defined( HB_PRAGMA_STARTUP )
#pragma startup _hb_hbqt_res_init_
#elif defined( HB_DATASEG_STARTUP )
#define HB_DATASEG_BODY HB_DATASEG_FUNC( _hb_hbqt_res_init_ )
#include "hbiniseg.h"
#endif
---

Viktor

[Bacco]

Hi, Viktor,

Sorry, I dont know enough of the compiling process now to give a
direct answer, but the resource loading is very straightforward, so I
believe that the simplest way of data block storage that gives a
pointer, the better. In the other hand, loading and unloading them in
the Qt context, should be user's task.

Pratical example: The user wants to compile 3 resource sets into the
final app, all 3 sets have same names, and only one will be in use at
a time (like to choose toolbar icon style: 'modern' or 'classic' or
'very fancy ones'). In this case, the user should deal with this as it
fits.

I believe that we should use some generic harbour data storage, the
only problem is to specify some meaningful name (maybe using the
filename as part of it?)

Alas, how can I add some raw binary data to harbour (not Qt specific)
and obtain a pointer to it? I really don't have this knowlege, any
pratical example will be welcome!

Regards
Bacco

[Viktor Szakáts]

Hi Bacco,

> Sorry, I dont know enough of the compiling process now to give a
> direct answer, but the resource loading is very straightforward, so I
> believe that the simplest way of data block storage that gives a
> pointer, the better. In the other hand, loading and unloading them in
> the Qt context, should be user's task.

> Pratical example: The user wants to compile 3 resource sets into the
> final app, all 3 sets have same names, and only one will be in use at
> a time (like to choose toolbar icon style: 'modern' or 'classic' or
> 'very fancy ones'). In this case, the user should deal with this as it
> fits.

Fair enough.

It's even better if such task can be left to users.
We can avoid the whole mess of generated init stuff.

> I believe that we should use some generic harbour data storage, the
> only problem is to specify some meaningful name (maybe using the
> filename as part of it?)

Yes, it can be created from the .qth filename with some
prefix. This method is used for compiled .ui resources, too.

Current prefix for .ui is simply "ui". Maybe something
like "__hbqt_ui" (and "__hbqt_res"), or at least "hbqtui_",
"hbqtres_" would be better.

I will make these modifications once we're there.

> Alas, how can I add some raw binary data to harbour (not Qt specific)
> and obtain a pointer to it? I really don't have this knowlege, any
> pratical example will be welcome!

Simplest is to put it in a .c source file.

I'm thinking of adding hb_retclen_const() API to make that
the most efficient.

Viktor

[Bacco]

Hi Viktor

Maybe something like this?

HB_FUNC ( __HBQT_QRC_(filename without extension) )
{
const char qt_resource_data[] = {
0x89,0x50,0x4e,0x47,0x0d,0x0a,0x1a,0x0a,0x00,0x00,0x00,0x0d,0x49,0x48,0x44,0x52,
..................
0x68,0x49,0x00,0x00,0x00,0x00,0x49,0x45,0x4e,0x44,0xae,0x42,0x60,0x82,
};
hb_retclen_const(qt_resource_data, sizeof ( qt_resource_data ) );
}

[Bacco]

Viktor,

I will focus on another subject now. Tonight or tomorrow by morning I
will download a fresh copy of the svn, revise/redo the changes I made
in HBQt / hbqtgen.prg and send them with the usage sample, so you can
revise and eventually commit.

Regards
Bacco

[Viktor Szakáts]

Okay, thank you. By then, I'll be done with discussed
modifications on my part.

Viktor

[Pritpal Bedi]

Hi

> I have finally managed to load successfuly  a binary compiled (
> rcc.exe -binary ) Qt resource, that is embedded in a string in my
> test.

This will really be a rewarding addition in Qt.

>    re1 := QResource():new()
>    re1:registerResource_1(data,':/')  /* data is embedded in the prg,
> and not loaded */
>    lb1 := Qlabel():new()
>    lb1:setPixMap(QPixMap():new(':image1.png'));

data => ?? - file name or function call ?

> - I had uncommented these lines in QResource.qth
>    bool registerResource ( const uchar * rccData, const QString &
> mapRoot = QString() )
>    bool unregisterResource ( const uchar * rccData, const QString &
> mapRoot = QString() )

Will not be an issue, it is simple but I could never
get this cast working especially as a return value of a function.

> - Also had to add a little fix in the hbqtgen.prg, to fix  uchar * ->
> char conversion (that affects various Qt functions not in use now)
> Original:
>             CASE aA[ PRT_CAST ] == 'uchar'
>                aA[ PRT_BODY ] := '( char ) hb_parni( ' + cHBIdx + ' )'
>                aA[ PRT_DOC  ] := 'n' + cDocNM
> New:
>             CASE aA[ PRT_CAST ] == 'uchar' .and. aA[ PRT_L_FAR ] .and.
> !( aA[ PRT_L_CONST ] )
>                aA[ PRT_BODY ] := '( char * ) hb_parc( ' + cHBIdx + ' )'
>                aA[ PRT_DOC  ] := 'c' + cDocNM
>
>             CASE aA[ PRT_CAST ] == 'uchar' .and. !( aA[ PRT_L_FAR ] )
> .and. !( aA[ PRT_L_CONST ] )
>                aA[ PRT_BODY ] := '( char ) hb_parni( ' + cHBIdx + ' )'
>                aA[ PRT_DOC  ] := 'n' + cDocNM

This seems OK if "uchar *" is not defined as an argument to a
function.
I will check if it can be managed like this.

NOTE: it may break a lot of wrappers.

> And here comes the problem: I also needed to add a cast into the
> generated QResource.qth

UPDATE: .qth is _NOT_a_GENERATED_ file. It is updated manually.
Probbaly you mean .cpp. The case below will automatically updated
if "generator" is modified to produce lines below.

>   from @287
>       hb_retl( ( p )->registerResource( hbqt_par_uchar( 2 ),
> hbqt_par_QString( 3 ) ) );
>   to
>       hb_retl( ( p )->registerResource( (unsigned char *)
> hbqt_par_uchar( 2 ), hbqt_par_QString( 3 ) ) );

As above.

> And this into hbqt.h (only as an helper, as I dont know the right way
> to deal with this)
>    #define hbqt_par_uchar( n )                         ( hb_parcx( n ) )

This is the right place to define.



Instead of baffling with "uchar" we can define a new function
in QResource.qth to honor any Harbour specific type like:

bool registerResource ( const uchar * rccData, const QString & mapRoot
= QString() )
bool unregisterResource ( const uchar * rccData, const QString &
mapRoot = QString() )

bool registerResource ( const hbuchar * rccData, const QString &
mapRoot = QString() )
bool unregisterResource ( const hbuchar * rccData, const QString &
mapRoot = QString() )

and then in the "generator"
CASE aA[ PRT_CAST ] == 'hbuchar'

aA[ PRT_BODY ] := '( char * ) hb_parc( ' + cHBIdx + ' )'
aA[ PRT_DOC ] := 'c' + cDocNM

It will be more convinient and also will not break anything.

Regards
Pritpal Bedi

[Pritpal Bedi]

Hi

I read the remaining posts later.
So no comment sfrom my side. Both Bacco and Viktor
are probably on the right path.

Regards
Pritpal Bedi

[Przemysław Czerpak]

On Wed, 30 Jun 2010, Viktor Szakáts wrote:

Hi,

> Well, maybe a pure generated .c file will be better.
> I've run into a Harbour compiler limitation:
> Error E0030 Syntax error "memory exhausted at '+'"
> That was with 427KB binary. Pbly there is a limit of
> 64K for strings in pcode.

In Harbour string size in the PCODE is limited to 16MB.
16MB is also the limit for maximum codeblock size. When
this limit is exceed "String too long" error message is
generated. Anyhow maximum jump size in current PCODE is
8MB so longer strings cannot be used inside conditional
expressions. In such case "Jump offset too long" error
is generated. These are also macro compiler limits and
they are checked during macro compilation. In Clipper
and xHarbour above limits are much smaller, i.e. maximum
string size in generated PCODE is 64KB and macrocompiler
can generate wrong if compiled PCODE is longer then 256
bytes in Clipper and 32KB in xHarbour.

Above E0030 error message is generated by bison grammar
parser and it's probably caused by some syntax error.
If you need more information then please send me the
compiled PRG code and I'll check it.

> So a combination of .c and .prg code could be a solution,
> though I suspect there exist something more elegant.
> F.e. in this case we might just not need the .prg init
> code, but use Harbour .c level init functions. In this
> case we'd almost completely replicate current solution,
> except that there is no need anymore for QT .h headers,
> which is a good thing.
> Question: Would the registration functions work from
> our own .c level init function?:
> ---
> static void hbqt_res_Init( void * cargo )
> {
> HB_SYMBOL_UNUSED( cargo );
>
> /* QResource init from here */
> }
>
> HB_CALL_ON_STARTUP_BEGIN( _hb_hbqt_init_ )
> hb_vmAtInit( hbqt_res_Init, NULL );
> HB_CALL_ON_STARTUP_END( _hb_hbqt_init_ )
>
> #if defined( HB_PRAGMA_STARTUP )
> #pragma startup _hb_hbqt_res_init_
> #elif defined( HB_DATASEG_STARTUP )
> #define HB_DATASEG_BODY HB_DATASEG_FUNC( _hb_hbqt_res_init_ )
> #include "hbiniseg.h"
> #endif
> ---

Yes they should.
HVM executed functions registered by hb_vmAtInit() just before
all CLIPINIT and other INIT PROCEDUREs.
See harbour/src/vm/hvm.c[1009]

best regards,
Przemek

[Viktor Szakáts]

Hi Przemek,

> In Harbour string size in the PCODE is limited to 16MB.
> 16MB is also the limit for maximum codeblock size. When
> this limit is exceed "String too long" error message is
> generated. Anyhow maximum jump size in current PCODE is
> 8MB so longer strings cannot be used inside conditional
> expressions. In such case "Jump offset too long" error
> is generated. These are also macro compiler limits and
> they are checked during macro compilation. In Clipper
> and xHarbour above limits are much smaller, i.e. maximum
> string size in generated PCODE is 64KB and macrocompiler
> can generate wrong if compiled PCODE is longer then 256
> bytes in Clipper and 32KB in xHarbour.

Nice to know, it was new information to me.

> Above E0030 error message is generated by bison grammar
> parser and it's probably caused by some syntax error.
> If you need more information then please send me the
> compiled PRG code and I'll check it.

Sample .prg uploaded here:
http://syenar.hu/harbour/download/memory_exhausted.zip

Viktor

[Bacco]

Hi, Pritpal

>>    re1 := QResource():new()
>>    re1:registerResource_1(data,':/')  /* data is embedded in the prg,
>> and not loaded */
>>    lb1 := Qlabel():new()
>>    lb1:setPixMap(QPixMap():new(':image1.png'));
>
> data => ??  - file name or function call ?
>

String with the rcc compiled resources.

>
>> - I had uncommented these lines in QResource.qth
>>    bool registerResource ( const uchar * rccData, const QString &
>> mapRoot = QString() )
>>    bool unregisterResource ( const uchar * rccData, const QString &
>> mapRoot = QString() )
>
> Will not be an issue, it is simple but I could never
> get this cast working especially as a return value of a function.

Now it works, lets see if we found side effects :)

>> - Also had to add a little fix in the hbqtgen.prg, to fix  uchar * ->
>> char conversion (that affects various Qt functions not in use now)
>> Original:
>>             CASE aA[ PRT_CAST ] == 'uchar'
>>                aA[ PRT_BODY ] := '( char ) hb_parni( ' + cHBIdx + ' )'
>>                aA[ PRT_DOC  ] := 'n' + cDocNM
>> New:
>>             CASE aA[ PRT_CAST ] == 'uchar' .and. aA[ PRT_L_FAR ] .and.
>> !( aA[ PRT_L_CONST ] )
>>                aA[ PRT_BODY ] := '( char * ) hb_parc( ' + cHBIdx + ' )'
>>                aA[ PRT_DOC  ] := 'c' + cDocNM
>>
>>             CASE aA[ PRT_CAST ] == 'uchar' .and. !( aA[ PRT_L_FAR ] )
>> .and. !( aA[ PRT_L_CONST ] )
>>                aA[ PRT_BODY ] := '( char ) hb_parni( ' + cHBIdx + ' )'
>>                aA[ PRT_DOC  ] := 'n' + cDocNM
>
> This seems OK if "uchar *" is not defined as an argument to a
> function.
> I will check if it can be managed like this.
>
> NOTE: it may break a lot of wrappers.

I have regenerated all files and did a quick file scan, at a first
glance all seem ok, and some other commented functions that I couldn't
enable before, are compiling now.

>> And here comes the problem: I also needed to add a cast into the
>> generated QResource.qth
>
> UPDATE:  .qth is _NOT_a_GENERATED_ file. It is updated manually.
> Probbaly you mean .cpp. The case below will automatically updated
> if "generator" is modified to produce lines below.

Sorry, read "generated FROM QResource.qth" instead. Just a typo.

>
>>   from @287
>>       hb_retl( ( p )->registerResource( hbqt_par_uchar( 2 ),
>> hbqt_par_QString( 3 ) ) );
>>   to
>>       hb_retl( ( p )->registerResource( (unsigned char *)
>> hbqt_par_uchar( 2 ), hbqt_par_QString( 3 ) ) );
>
> As above.
>
>> And this into hbqt.h (only as an helper, as I dont know the right way
>> to deal with this)
>>    #define hbqt_par_uchar( n )                         ( hb_parcx( n ) )
>
> This is the right place to define.

Ok, thanks!

>
> Instead of baffling with "uchar" we can define a new function
> in QResource.qth to honor any Harbour specific type like:
>
> bool registerResource ( const uchar * rccData, const QString & mapRoot
> = QString() )
> bool unregisterResource ( const uchar * rccData, const QString &
> mapRoot = QString() )
>
> bool registerResource ( const hbuchar * rccData, const QString &
> mapRoot = QString() )
> bool unregisterResource ( const hbuchar * rccData, const QString &
> mapRoot = QString() )
>
> and then in the "generator"
>    CASE aA[ PRT_CAST ] == 'hbuchar'
>        aA[ PRT_BODY ] := '( char * ) hb_parc( ' + cHBIdx + ' )'
>        aA[ PRT_DOC  ] := 'c' + cDocNM
>
> It will be more convinient and also will not break anything.
> Regards
> Pritpal Bedi

The uchar change affects positively other commented / unimplemented Qt
functions, I will look into your solution anyway, and It can be very
useful in many other places also. Thanks for the explanation, I am
sure your help will be of great value in my learning curve.

Regards
Bacco

[Przemysław Czerpak]

On Thu, 01 Jul 2010, Viktor Szakáts wrote:

Hi Viktor,

> > Above E0030 error message is generated by bison grammar
> > parser and it's probably caused by some syntax error.
> > If you need more information then please send me the
> > compiled PRG code and I'll check it.
> Sample .prg uploaded here:
> http://syenar.hu/harbour/download/memory_exhausted.zip

This example overflow bison stack.
The maximum size of the grammar stack used by bison is controled
by YYMAXDEPTH. By default it's 10000 and you code uses expression
which has over 50000 items.
If you modify it to use longer lines and smaller number of
items in expression then it can be compiled. You can also
change YYMAXDEPTH macro in harbour/src/compiler/harbour.yyc
We do not use execution stack area for bison grammar stack
so we can extend the maximum size and it should not be a
problem for current computers.
I'll commit such modification in a while.

best regards,
Przemek

[Viktor Szakáts]

Many thanks Przemek. I've seen the commit, perfect.

I'll tweak the code to use longer lines though to
lessen the strain on internals.

Viktor

[Bacco]

Hi Viktor

I am sending an unified diff with the resources / generator changes I
made so we can use embedded resources. What is the best way to send
this kind of changes?
I am redoing my resources demo .prg, and will send ASAP.

Bacco

[Viktor Szakáts]

Hi Bacco,

> I am sending an unified diff with the resources / generator changes I
> made so we can use embedded resources. What is the best way to send
> this kind of changes?

.diff file is perfect.

> I am redoing my resources demo .prg, and will send ASAP.

Ok. I've committed the integration changes, so now
the method is active.

Viktor

[Bacco]

Hi Viktor

Attached are some files intended to be extracted on
contrib/hbqt/tests, as resource demos:

resource/embedded.prg
Example of resource embedded inside the prg. Can be compiled
standalone, no file dependencies beside normal Qt ones.
I think it's good to allow users the possibility to embed the
resource directly in the .prg or .c file also, so I'll send my
embedder when I clean up the command line options. It's somewhat based
in my original test tool and your hbqt plugin also, more details
later.

resource/compiled.prg
Same example, intended to use your new #pragma (I need you to fix
the #pragma line, inside this file by the way)

resource/resource-text.xml
Original Resource file, meant to be compiled using rcc -binary
resource-text.xml -o resource-binary.qrc

resource/resource-binary.qrc
Precompiled as stated above, so one can test the .prg without
fiddling with rcc

resource/harbour-icon.png
resource/harbour-logo.png
Images referenced in resource-text.xml

Also, I didn't put the headers, because I don't know how the credits
should be handled in this case. Feel free to add the way you find
correct, If you believe this should be commited.

Regards,
Bacco