hbmk2, -inc and #include dependences in subfolder

[Sergy]

Hello friends

I was wondered few days ago, when did some tests:

This is the main module, test.prg:

#include "header.ch"

* ------- *

FUNC Main()

LOCAL cMode1,cMode2

 cMode1 := ShowMode1() // from this file

 cMode2 := ShowMode2() // from "library" file located in subfolder

 ? "cMode1 ==",cMode1

 ? "cMode2 ==",cMode2

 ?

 INKEY(0)

RETURN

* ------------ *

FUNC ShowMode1()

RETURN DEFINED_MODE

* --------------- *

This is a "library" subfolder\testlib.prg, which depends from header located in upper directory:

#include "d:\hb_test\header.ch" // direct link, the way: "..\header.ch" didn't work

* ------------ *

FUNC ShowMode2()

RETURN DEFINED_MODE

* --------------- *

This is the header.ch - common for both sources:

#define DEFINED_MODE "MODE ONE"

This is test.hbp file to make an executable:

-prgflag=/v

-inc

-workdir=OBJ

-run

-otest

test.prg

subfolder\testlib.prg

When I built this app at first time by command hbmk2 test.hbp, I saw the predictable result:

cMode1 == MODE ONE

cMode2 == MODE ONE

Then I modified the common header file to this value:

#define DEFINED_MODE "MODE TWO"

And made second build. I saw:

cMode1 == MODE TWO

cMode2 == MODE ONE

As I can understand - dependence wasn't checked: After *.ch file was changed - the "library" (in subfolder!) wasn't rebuilded.

Of course, I can:

1) remove -inc flag from *.hbp file...

or

2) delete all files from /OBJ...

or

3) move my "library" from subfolder to main app directory...

or

4) ... ?

But it's a simly stand-alone snippet and I faced with this issue in a big project.

What I did wrong ?

Thanks.

WBR, Sergy.

[Klas Engwall]

Hi Sergy,

> As I can understand - dependence wasn't checked: After *.ch file was
> changed - the "library" (in subfolder!) wasn't rebuilded.

If I understand your question correctly, you changed the header file but
not the prg file that uses it. And you expected the prg file to be
recompiled, but that did not happen. Right?

If you think about it for a moment, how would the compiler know which
prg files are affected by the change? Should it check every prg file to
see if there is a line saying #include "header.ch"? How many lines of
each prg file would it have to check? The entire file? That sounds more
like a complete rebuild of the project than an incremental build.

And which changed header files should trigger the check? The ones in the
current directory? The ones in your "library" subdirectory? The ones in
directories specified with -i? What about those specified with a full
path in certain prg files? It gets very complicated.

> Of course, I can:
>
> 1) remove -inc flag from *.hbp file...

Or, more easily, add -inc- on the commandline for the first compile
session after the change. It overrides the -inc in the hbp file.

> or
> 2) delete all files from /OBJ...

That is also a possibility, but -inc- is easier

> or
> 3) move my "library" from subfolder to main app directory...

That will not help much if you do not update library.prg after changing
the header file

> or
> 4) ... ?
>
> But it's a simly stand-alone snippet and I faced with this issue in a
> big project.
> What I did wrong ?

I think you were expecting too much from the compiler :-)

Regards,
Klas

[Sergy]

Hi Klas.

Yes, I updated only common header for two sources, and only one prg was recompiled.

If one file is includes the header and depends from it - why second not ?

If hbmk2 (not compiler!) can detect that one file depends from changed part of it (by #include) - why second not ?

Because second PRG stored in subfolder?

--

In common, I have two versions of my app - "debug" and "clean". And all source files have #include "common_header" - where I define which compile mode is. And I try to remove all files from /OBJ directory before make a "clean" version.

Few days ago my users said me about "never seen before" - some technical data on the screen. Small investigation show that some files (in subfolders) was not recompiled...

Then I made this atand-alone test. And now I try to understand - is it normal behavior or not.

Thank you.

WBR, Sergy.

вторник, 23 июня 2015 г., 14:38:55 UTC+3 пользователь Klas Engwall написал:

[Antonino Perricone]

Hello,

I tried your case, and I found a work-around:

In test.hbmk add:

-incpath=.

then in testlib.prg use:

#include "header.ch"

anyway, it is a strange bug...

Regards,

Antonino

[Antonino Perricone]

I just look the help of hbmk2, there is:

  -head=<m>           control source header parsing (in incremental build mode)

                      <m> can be: native (uses compiler to extract

                      dependencies), full (default, uses simple text parser on

                      the whole file), dep, off

maybe native or dep work better
Regards,

Antonino

[Sergy]

Hi, Antonio.

I tested suggested decision - it works fine, exactly I need.

Thank you.

WBR,

Sergy

вторник, 23 июня 2015 г., 16:37:36 UTC+3 пользователь Antonino Perricone написал:

[vszakats]

Fixed it here:

https://github.com/vszakats/harbour-core/commit/390562d7f16910c7fa8d5d5f590f19fc5ef26d40

Though as noted, referencing headers with absolute paths 

it not a recommended or good practice. Instead, header 

directories are best be added to the header search paths 

using `-i` option.

[Sergy]

Thank you, Viktor.

среда, 24 июня 2015 г., 13:37:01 UTC+3 пользователь vszakats написал:

[Antonino Perricone]

IMHO it is very strange that 

#include "..\header.ch"

does not work.

[vszakats]

It does work once you add the referring source file's 

directory to the list of header paths.

It's even better to add the directory of the header itself 

to the list of header paths, making even the relative 

paths unnecessary in the #include directive.

What is strange is that Harbour will allow to #include 

a header residing in the same subdirectory as the source 

itself, when being referred to _without_ paths, while Cl*pper 

will error. Cl*pper seems to be the consistent one here.

[Przemyslaw Czerpak]

On Wed, 24 Jun 2015, vszakats wrote:

Hi Viktor,

> It does work once you add the referring source file's
> directory to the list of header paths.
>
> It's even better to add the directory of the header itself
> to the list of header paths, making even the relative
> paths unnecessary in the #include directive.
>
> What is strange is that Harbour will allow to #include
> a header residing in the same subdirectory as the source
> itself, when being referred to _without_ paths, while Cl*pper
> will error. Cl*pper seems to be the consistent one here.

Can you create self contain example?

best regards,
Przemek

[vszakats]

Hi Przemek,

It's a half-false alarm. I did the mistake of using forward 

slashes in the Cl*pper command-line: `clipper sub/sub.prg`. 

Cl*pper did find the source itself (on Win98), but it failed 

to find the header in this case. When using `clipper sub\sub.prg` 

it found the source and the referenced header residing in 

the same directory.

This reduces this part to an OS platform difference, not an 

incompatibility per se.

But: I retried the relative path case, too and it turns out 

Cl*pper will find them correctly, while Harbour won't.

Unless I'm overlooking something obvious.

Test attached.

-Viktor

[Przemyslaw Czerpak]

Hi Viktor,

Fixed:
2015-06-24 20:32 UTC+0200 Przemyslaw Czerpak (druzus/at/poczta.onet.pl)

Now it should be fully Cl*pper compatible anyhow I do not think it's
good behavior. The highest precedence has current directory and this
may wrongly interact with nested projects.

best regards,
Przemek

> --
> --
> You received this message because you are subscribed to the Google
> Groups "Harbour Users" group.
> Unsubscribe: harbour-user...@googlegroups.com
> Web: http://groups.google.com/group/harbour-users
>
> ---
> You received this message because you are subscribed to the Google Groups "Harbour Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> Archive: /tmp/subhdr.ehWhaZ.zip
> Length Date Time Name
> --------- ---------- ----- ----
> 24 2015-06-24 18:07 cl.bat
> 24 2015-06-24 18:07 hb.bat
> 12 2015-06-24 18:05 par.ch
> 0 2015-06-24 17:55 sub/
> 12 2015-06-24 18:05 sub/sub.ch
> 54 2015-06-24 18:05 sub/sub.prg
> --------- -------
> 126 6 files

[vszakats]

Hello Przemek,

On Wednesday, June 24, 2015 at 8:36:56 PM UTC+2, druzus wrote:

Hi Viktor,

Fixed:
   2015-06-24 20:32 UTC+0200 Przemyslaw Czerpak (druzus/at/poczta.onet.pl)

Now it should be fully Cl*pper compatible anyhow I do not think it's
good behavior. The highest precedence has current directory and this
may wrongly interact with nested projects. 

I agree that mixing in the current directory is not good behavior.

But, couldn't see any evidence that Cl*pper uses it.

Such test was not included in the original example, but adding 

an `#include "par.ch"` line to `sub\sub.prg` _does_ fail in Cl*pper, 

while after your patch it succeeds in Harbour. It means that 

Cl*pper doesn't rely on current directory, and that it first checks 

the filename relative to the parent(s), next it looks it up in `-i` list.

I've updated the test to include multiple parent tests (Harbour 

is okay here), and a test for the current directory case, where 

Harbour now compiles, while Cl*pper rightfully fails.

Also included console outputs and Harbour/Cl*pper generated 

.ppo files.

-Viktor

[Przemyslaw Czerpak]

On Wed, 24 Jun 2015, vszakats wrote:

Hi Viktor,

> > Now it should be fully Cl*pper compatible anyhow I do not think it's
> > good behavior. The highest precedence has current directory and this
> > may wrongly interact with nested projects.
> I agree that mixing in the current directory is not good behavior.
> But, couldn't see any evidence that Cl*pper uses it.

I make more tests and now I think I found whole picture.
When I was working on PP code I made tests to find how
exactly #include works in Cl*pper and I fully replicated it.
It was correct implementation but only for example I tested:
file name without a path. Te recent example was different:
it's file name with relative path and in such case Cl*pper
works differently. In the way I've just committed.

> Such test was not included in the original example, but adding
> an `#include "par.ch"` line to `sub\sub.prg` _does_ fail in Cl*pper,
> while after your patch it succeeds in Harbour. It means that
> Cl*pper doesn't rely on current directory, and that it first checks
> the filename relative to the parent(s), next it looks it up in `-i` list.

Just simply change:
#include "par.ch"
to:
#include ".\par.ch"

and you will find that Cl*pper works like current Harbour
version: scan from current dir then the dir taken from first
compiled file. So to replicate Cl*pper behavior we have to
join current implementation with the previous one.

I'll make some tests yet with absolute paths (I want be very
surprised if Cl*pper use them like relative ones) and I'll
commit necessary modification.

best regards,
Przemek

[vszakats]

Hi Przemek,

Many thanks. On a next pass I'll need to revise the logic 

inside `hbmk2` to match that.

In the meantime I encourage everyone not to overuse paths 

in header names, for everyone's sanity and best interest :)

`-i` is good.

-Viktor