openssl 1.1.1 and harbour 3.2... a final word? (also for nightly)

[Francesco Perillo]

Now that 3.2 includes support for openssl 1.1.1 I'm trying to compile it...

... but with partial success....

I did some tests using Viktor build of openssl library... and with some changes to hbssl.ch/asking for static link I was able to link the test.prg app but it always needed the dll.

A friend was able to use another build, but only for a full static link, since it looks for ssley32...

Since I'm trying to update the nightly build it would be nice to find a way to add updated openssl support in a way that is shared and working for everybody.

So, please, if you can suggest the proper download for openssl libray and proper way to compile hbssl, tell me now.

Thank you

Francesco

[Klas Engwall]

Hi Francesco,

> Now that 3.2 includes support for openssl 1.1.1 I'm trying to compile it...
>
> ... but with partial success....
>

> I did some tests *using Viktor build of openssl library*... and with

> some changes to hbssl.ch/asking for static link
> I was able to link the test.prg app but it always needed the dll.
>
> A friend was able to use another build, but only for a full static link,
> since it looks for ssley32...

Can you provide a little more detail of what you did? Does "asking for
static link" mean setting the envvar HB_STATIC_OPENSSL=yes ?

What changes did you make in hbssl.ch?

Using the envvar above, libssl.a and libcrypto.a should be used by the
linker - and not libssleay32.a and liblibeay32.a

That is working for me (using Viktor's SSL build) ... except that I get
various error messages saying things like

l:/harbour/openssl111/libcrypto.a(bss_file.o):(.text+0x329): undefined
reference to `_imp____acrt_iob_func'

So the linker finds the correct lib(s) but something seems to be missing

When searching for the "_imp____acrt_iob_func" string in all files in
Viktor's SSL build I get four references in libcrypto.a but nothing
anywhere else. So far I have not been able to really figure out what to
conclude from that ... but Google finds a few discussions about
_imp____acrt_iob_func related to 64-bit MinGW (working) vs 32-bit MinGW
(not working), and to MSYS2, respectively, in other projects. A shot in
the dark, but is Viktor's SSL build compatible with generic MinGW 5.3.0?

Regards,
Klas

[Francesco Perillo]

Can you provide a little more detail of what you did? Does "asking for
static link" mean setting the envvar HB_STATIC_OPENSSL=yes ?

yes.

What changes did you make in hbssl.ch?

strictly speaking, changing references from ssleay32 and libeay32 to ssl and crypto also for dynamic

That is working for me (using Viktor's SSL build) ... except that I get
various error messages saying things like

l:/harbour/openssl111/libcrypto.a(bss_file.o):(.text+0x329): undefined
reference to `_imp____acrt_iob_func'

I tried with mingw 5.x and 7.2.

I'm in the process to revamp nigthly build with mingw from msys2. I found some problems (allegro binding fails, compilation stops) and hbssl seems to compile with old openssl but of course it requests .dll.

Then I found that msys2 has a openssl library inside mingw compiler, so tried to rebuild hbssl with HB_WITH_OPENSSL pointing to the include dir of \msys2\mingw32 and it compiled and run flawlessy.

Since my msys2/mingw setup was a mess (I installed the full toolchain, that included ADA, objective-c, fortran, python etc etc) I'm now cleaning up everything, trying to document it.

Unfortunately msys2 now includes openssl 1.0...m... and not 1.1... but it seems that it is possible to compile it quite simply from msys2/mingw...

[Diego Fazio]

I compiled the last Hb 3.2 with msys2(updated), openssl v1.1.1 without problems. 

Diego.

[Francesco Perillo]

Hi Diego, I know it is possible. What I'm looking for is a replicable list of instructions that I can put inside the mightly build script...

Did you use Viktor openssl build? How did you setup the build (HB:WITH_OPENSSL= ??)? Which files/dll are needed at runtime (and so must be copied into the installer)?

Francesco

--
--
You received this message because you are subscribed to the Google
Groups "Harbour Users" group.
Unsubscribe: harbour-user...@googlegroups.com
Web: http://groups.google.com/group/harbour-users

---
You received this message because you are subscribed to the Google Groups "Harbour Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to harbour-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Francesco Perillo]

msys2 includes openssl-1.1.1a-1 inside mingw32... (and I presume mingw64 also...)

It is a big step forward...

[Diego Fazio]

I used Viktor openssl build...

https://bintray.com/vszakats/generic/download_file?file_path=openssl-1.1.1-win32-mingw.tar.xz

put it in....C:\OpenSSL-Win32

set 

HB_WITH_OPENSSL=C:\OpenSSL-Win32\include

HB_STATIC_OPENSSL=yes

build HBSSL

then I compile the \contrib\hbssl\test\test.prg

"libcrypto-1_1.dll" and "libssl-1_1.dll" must be in the PATH 

and it works ok.

[Francesco Perillo]

Thank you Diego,

we got the same results... but I'd like to point you to these lines:

HB_STATIC_OPENSSL=yes

libcrypto-1_1.dll" and "libssl-1_1.dll" must be in the PATH

Is it normal that a static openssl needs 2 dll ?

Actually the set ...=yes just forces to link a different lib.... so I think that Viktor libs are dynamic only, we just fake the build system...

[Francesco Perillo]

Ok,

I was able to install msys2/mingw/mingw2-openssl-1.1.1a-1... (so that everything is installed from the same source (eg. pacman package))

Built with HB_STATIC_OPENSSL=yes

link is ok

at runtime it needs the dll....

In dynamic the link script looks for the .eia32.dll that probably don't exist anymore....

[Maurizio la Cecilia]

Hi Diego,

did you build 64bit flavour or 32bit.

Very strangely, only my build 32bit request the dlls, 64bit no.

Have you any idea about?

Best regards.

--

Maurizio

[Diego Fazio]

I only tried with the 32bit. Later will try with 64bit.

If you are testing with two dif machines, check if in the 64bit one you don't have the dlls in the PATH. 

Diego.

[Maurizio la Cecilia]

Hi Diego,

I'm building on the same machine, changing only the references to mingw compiler and openssl package (I'm using the Viktor distros for both the architectures).

I didn't set path to openssl folder at all.

I attached my log for bot 32 and 64 bits.

I'd set in both cases HB_STATIC_OPENSSL=yes

The result is:
. libraries and test.exe are built for both architectures

. the only unexpected message is, in both cases: hbmk2[hbssl]: Warning: No import library sources were found.

. test.exe (64bits) runs without errors

. test.exe (32bits) also runs, but it asks for dlls

I was expecting that, statically linking the libs, a path to the dlls shouldn't required.

Please, let me know if same behaviour occurs to you building the 64bits version of test.exe.

TIA

--

Maurizio

[Maurizio la Cecilia]

Please Diego, forgot my previous post.
The problem was due to the presence of only 64bit dlls in my system path, so these was loaded silently in 64 version of test.exe.

Obvioulsy, the 32 versione wasn't able to find the needed 32bit dlls...
Anyway, what's the mean of static linking if dlls are required?

Best regards.

--

Maurizio

Il 22/11/2018 16:07, Diego Fazio ha scritto:

[Francesco Perillo]

I found this thread: https://github.com/openssl/openssl/issues/7367

The last post reports:

If one has some experience with 1.0.2, then one has to recognize that default is changed in 1.1.x. In 1.0.2 static build was default and one had to pass 'shared' to build shared, in 1.1.x shared build is default and if one wants to build static library, one has to pass 'no-shared'. In other words pass 'no-shared' instead to Configure.

Probably we should compile openssl ourselves... or there si still something we don't get correctly...

[Diego]

that's what I meant with the PATH.

Diego

You received this message because you are subscribed to a topic in the Google Groups "Harbour Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/harbour-users/RJJ9g3raIl0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to harbour-user...@googlegroups.com.

[Reinaldo]

Hello Maurizio and Diego;

I keep coming to a road block trying to build harbour 3.2 hbssl with gcc 6.3.0.  Notice below I get a warning that no import libraries were found:

D:\harbour3.2-gcc6.3\contrib\hbssl>set HB_WITH_OPENSSL=d:\openssl-1.1.1-win32-mingw\include

D:\harbour3.2-gcc6.3\contrib\hbssl>set HB_STATIC_OPENSSL=yes

D:\harbour3.2-gcc6.3\contrib\hbssl>hbmk2 hbssl

hbmk2: Building sub-project (level 2): hbssls.hbp

hbmk2: Dependency 'openssl' found: d:\openssl-1.1.1-win32-mingw\include

hbmk2: Target up to date: libhbssls.a

hbmk2: Dependency 'openssl' found: d:\openssl-1.1.1-win32-mingw\include

hbmk2[hbssl]: Warning: No import library sources were found.

hbmk2: Target up to date: libhbssl.a

I hope that by "no import libraries" it is referring to  libcrypto-1_1.dll and libssl-1_1.dll  and not libssleay32.dll.   

I'm using Victor's openssl distribution and I must add that when built using gcc 7 or above, I don't get this problem.

What's wrong?  Why is it still looking for the wrong .dlls from previous OPENSLL versions?

Reinaldo.

[Maurizio la Cecilia]

Hi Reinaldo,

sorry for the late answer.

You're hoping right: the "no import libraries" is referred to the right version of openssl.

To reach the successful import  I did add the correct path into hbssl.hbp file.
I suspect that Przemislaw didn't build his test on Windows and the committed hbp file is misaligned with the new names of openssl dlls.

Anyway, despite the libraries are correctly imported and my test and production executables are working fine, also setting HB_STATIC_OPENSSL=yes the dlls are requested at runtime.
My knowledges stops on this point and I accepted to distribute the needed dlls  in my install.
Sorry.

Best regards.
--
Maurizio

[Pritpal Bedi]

Hi to all.

Harbour is linkable with static openssl libraries.

Download static libs from Viktors repository - https://bintray.com/vszakats/generic/openssl - and use these libs instead of any other.

Note that you need to build Harbour with -DHB_OPENSSL_STATIC against OpenSSL 1.1.1a. I did it with MinGW 8.1.0. Not sure if previous version will be helpful or not.

Hope it helps all.

Pritpal Bedi

a student of software analysis & concepts

 

[Maurizio la Cecilia]

Hi Pritpal,

please, can you control if you've the OpenSSL dlls in your system path, or are somehow reachable by your app?

I did the steps you suggested and:
a) the import libraries aren't created (but using msys2 distro they are)
b) the dlls are still required at runtime (as for msys2)

c) the linked executable isn't bigger than previous, so I suspect that anything changed...

To avoid any doubt about the need of dlls at runtime you could use this tool:
http://www.dependencywalker.com/

The output for my application shows the dependent libcrypto-1_1-X64.dll:

Thanks in advance for any further contribute you'll give about this problem.

Best regards.

--
Maurizio

[Pritpal Bedi]

Hi

Steps to test:

1. Copied excutable and needed .dlls ( not any openssl dlls )  into a separate folder

2. SET PATH=

3. Executed VouchDtCli.exe

Here is complete build environment for executable:

Build-env.bat

==========

VouchDtCli.hbp

============

-trace

-info

-inc

-gui

-mt

-gtwvg

-w0

-gc0

-m

-n

-a

-workdir=../../dev_objs/${hb_comp}/vouchDtCli

-o../../dev_projects/harbour/vouch/VouchDtCli

-L../../Image2PDF/V2-64

-L../../dev_libs/clean

-iInclude

-i../others/freewin/include

hbwin.hbc

hbzebra.hbc

hbssl.hbc

hbtip.hbc

hbmxml.hbc

hbct.hbc

-lgtwvg

-lgtgui

-lgtnet

-lV32LibL

-lFreeWin

-lImage2PDF.dll

-leztwain3

-lcrypto

-lssl

-lvouchqse

-lhbxpp

-lhbmemio

-lhbnf

source/clidt.prg

source/cli_prn.prg

source/wmiinfo.prg

source/activlbl.prg

source/b_client.prg

source/b_wvg.prg

source/b_ctools.prg

source/b_httpsrv.prg

source/hbi2pdf.c

source/hbeztwain.c

resources/vouch.res

-strip-

I believe that hbssl.hbc has to be fixed like - which I did at my local folder -

From

{!HB_DYNBIND_OPENSSL& (HB_STATIC_OPENSSL&!hbdyn)}libs=${_HB_DYNPREF}${hb_name}s${_HB_DYNSUFF}

To

{!HB_DYNBIND_OPENSSL& (HB_STATIC_OPENSSL&!hbdyn)}libs=${_HB_DYNPREF}${hb_name}${_HB_DYNSUFF}

Compiled Harbour with Mingw 8.1.0.

Copied libssl.a and libcrypto.a from Viktors repository into harbour/lib/win/mingw-582SH.

That's it.  

Just do not look at Qt specific environment variables. Qt 5.8 is not compilable statically with Mingw 8.1.0.

I will give it another try.

I had also struggled on OpenSSL issue then stumbbled upon Viktor's builds.

Hope it will help now.

[Maurizio la Cecilia]

Hi Pritpal and all,

after a lot of tries and some deeper study I too succeed to build statically OpenSSL.

Anyway I found some difference with your method:

1. The build succeed also using the MSYS2 distro, no need to use Viktor one (anyway, also with Viktor files the things are ok)

2. There's no need to patch hbssl.hbc. On the contrary, applying the change you suggested there's no way to distinguish between dynamic and static build.

3. There's no need to set -DHB_OPENSSL_STATIC, as this flag is set internally by Harbour when HB_STATIC_OPENSSL=yes. You can see the setting in hbssls.hbp which is called as subproject by hbssl.hbp in build process.

Thus, the steps I did are:

1. Leave the sources of Harbour untouched (but updated with my recent commits)

2. Copy from MSYS2 (or from the Viktor distro) the needed libs to the correspondent Harbour lib folder:

     libcrypto.a and libssl.a from <MSYSroot>/mingw32/lib to <HBroot>/lib/win/mingw for a 32bit build

     libcrypto.a and libssl.a from <MSYSroot>/mingw64/lib to <HBroot>/lib/win/mingw64 for a 64bit build

3. Build Harbour (or just hbssl contrib) with this setting:

     HB_STATIC_OPENSSL=yes

     HB_WITH_OPENSSL=<MSYSroot>\include   

I wasn't able to build statically so far because I was just omitting the step 2.

Please, give a try and report the result.

I hope the static OpenSSL build isn't more an headache...

Best regards.

--

Maurizio

[Pete]

Hi Maurizio,

On Friday, 18 January 2019 23:51:40 UTC, Maurizio la Cecilia wrote:

Please, give a try and report the result.

Seems that your build "formula" is working!
After following the steps you described, I was able to:
1) build static hbssl library
and
2) build and run the sample file: `\contrib\hbssl\tests\test.prg`

Since no extra dlls needed to run the executable, I suppose that
openssl is statically linked. Isn't it?

 

I hope the static OpenSSL build isn't more an headache...

Now, I wonder whether is also feasible, following some similar build process,
to link statically the curl library, (a long pending issue and headache cause too).

regards,
Pete

[Francesco Perillo]

Hi Pete and everybody.

The solution presented by Pritpal works but it is a workaround... he doesn't explain why it works, which behaviour corrects, etc.

Maurizio and I have been studying and testing a solution that will be valid for this and other libraries. We started trying to understand, from tests and docs, what really happens at link time, how hbmk2 and gcc/ld interact, and will be proposing a 4 lines patch to hbmk2 that will help solve the problem.

Later today we will present the proposal.

--

[Pete]

On Wednesday, 23 January 2019 14:43:42 UTC+2, fperillo wrote:

 

Later today we will present the proposal.

Ok, fine! we shall wait for it, then.
Thanks!

regards,
Pete
 

[Maurizio la Cecilia]

Hi Pete,

openssl is properly statically linked.

You can verify it, apart from the unneeded dlls, also confronting the size of the test executables between dynamically and statically linked version.

Going deeper, Francesco and me found a more interesting way to obtain the same result without copying the libraries to the Harbour lib folder.

Adding this simple row to hbssl.hbc:
{!HB_DYNBIND_OPENSSL& (HB_STATIC_OPENSSL&!hbdyn)&allmingw}ldflags=-static
does the job, as gcc gets the right libraries from openssl lib folder, discarding those based on dlls.
But that solution has some side effects as the -static is placed by hbmk2 before all the libs to be linked, making unsuccessful any attempt to mix dynamically and statically linked libs (a not so rare scenario).

Thus Francesco found a more robust solution, introducing a little patch to hbmk2, giving a general way to invoke the static libraries at link time.
Please, stay tuned, maybe also 'curl headache' will disappear...

Best regards.

--
Maurizio

[Anderson Cardoso Silva]

Where is "libcrypto-1_1.dll" and "libssl-1_1.dll"?

My app is asking for it and I can´t find it. 

Using hb32

[bernardm...@gmail.com]

Hello Anderson,

In my computer :

D:\hb32\comp\mingw\bin

Regards,

Bernard.