[Python-Dev] python3.10 compilation on OpenBSD: running into ssl issues

2,040 views
Skip to first unread message

Sandeep Gupta

unread,
Oct 5, 2021, 10:58:54 AM10/5/21
to pytho...@python.org
Trying to compile python3.10 on openbsd 7.0 on Pi4. It seems to run into several openssl issue.  I have installed openssl as I couldn't find libreSSL in the package manager. 

The configure seems to passthe ssl test. 

configure:17559: checking whether compiling and linking against OpenSSL works
Trying link with OPENSSL_LDFLAGS=; OPENSSL_LIBS= -lssl -lcrypto; OPENSSL_INCLUDES=
configure:17581: cc -pthread -o conftest      conftest.c  -lssl -lcrypto -lpthread  -lutil -lm >&5
configure:17581: $? = 0
configure:17583: result: yes


I get following errors: 

>> don't know why this happens
ldd: /usr/lib/libreadline.a: not an ELF executable

>>Errors when building _ssl module

cc -pthread -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/include -I. -I/usr/local/include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0 -c /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c -o build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.o
In file included from /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:390:
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl/debughelpers.c:180:5: error: implicit declaration of function 'SSL_CTX_set_keylog_callback' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    SSL_CTX_set_keylog_callback(self->ctx, NULL);
    ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:1842:29: error: implicit declaration of function 'SSL_get0_verified_chain' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    STACK_OF(X509) *chain = SSL_get0_verified_chain(self->ssl);
                            ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:1842:21: warning: incompatible integer to pointer conversion initializing 'struct stack_st_X509 *' with an expression of type 'int' [-Wint-conversion]
    STACK_OF(X509) *chain = SSL_get0_verified_chain(self->ssl);
                    ^       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2349:18: error: implicit declaration of function 'SSL_write_ex' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        retval = SSL_write_ex(self->ssl, b->buf, (size_t)b->len, &count);


/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2349:18: note: did you mean 'SSL_write'?
/usr/include/openssl/ssl.h:1415:6: note: 'SSL_write' declared here
int     SSL_write(SSL *ssl, const void *buf, int num);
        ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2501:18: error: implicit declaration of function 'SSL_read_ex' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        retval = SSL_read_ex(self->ssl, mem, (size_t)len, &count);
                 ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:2501:18: note: did you mean 'SSL_read'?
/usr/include/openssl/ssl.h:1413:6: note: 'SSL_read' declared here
int     SSL_read(SSL *ssl, void *buf, int num);
        ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:3542:30: error: implicit declaration of function 'SSL_CTX_get_num_tickets' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    return PyLong_FromSize_t(SSL_CTX_get_num_tickets(self->ctx));
                             ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:3560:9: error: implicit declaration of function 'SSL_CTX_set_num_tickets' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    if (SSL_CTX_set_num_tickets(self->ctx, num) != 1) {
        ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_ssl.c:3574:28: error: implicit declaration of function 'SSL_CTX_get_security_level' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    return PyLong_FromLong(SSL_CTX_get_security_level(self->ctx));
                           ^
cc -pthread -shared -fPIC build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_testcapimodule.o -L/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/lib -L/usr/local/lib -o build/lib.openbsd-7.0-arm64-3.10/_testcapi.cpython-310.so


> and errors building _hashlib module

cc -pthread -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/include -I. -I/usr/local/include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0 -c /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c -o build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.o
building '_sha256' extension
cc -pthread -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I./Include -I/home/kabira/DrivingRange/project_versa/Builds/Python-3.10.0/include -I. -I/usr/local/include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Include -I/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0 -c /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/sha256module.c -o build/temp.openbsd-7.0-arm64-3.10/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/sha256module.o -DPy_BUILD_CORE_MODULE
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:158:10: error: use of undeclared identifier 'NID_sha3_224'
    case NID_sha3_224:
         ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:161:10: error: use of undeclared identifier 'NID_sha3_256'
    case NID_sha3_256:
         ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:164:10: error: use of undeclared identifier 'NID_sha3_384'
    case NID_sha3_384:
         ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:167:10: error: use of undeclared identifier 'NID_sha3_512'
    case NID_sha3_512:
         ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:172:10: error: use of undeclared identifier 'NID_shake128'
    case NID_shake128:
         ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:175:10: error: use of undeclared identifier 'NID_shake256'
    case NID_shake256:
         ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:180:10: error: use of undeclared identifier 'NID_blake2s256'
    case NID_blake2s256:
         ^
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/_hashopenssl.c:183:10: error: use of undeclared identifier 'NID_blake2b512'
    case NID_blake2b512:


Any suggesstions?

Thanks
-S

Christian Heimes

unread,
Oct 5, 2021, 11:12:39 AM10/5/21
to pytho...@python.org
On 05/10/2021 16.40, Sandeep Gupta wrote:
> Trying to compile python3.10 on openbsd 7.0 on Pi4. It seems to run into
> several openssl issue.  I have installed openssl as I couldn't find
> libreSSL in the package manager.

Your installation is picking up header files from LibreSSL. Python 3.10
requires a fully OpenSSL 1.1.1 API compliant OpenSSL. LibreSSL is
missing a bunch of required features.

Please follow the instruction
https://docs.python.org/3.10/using/unix.html#custom-openssl to compile,
install, and use a custom installation of OpenSSL. I recommend that you
use the latest OpenSSL 1.1.1 version.

Regards,
Christian

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/BK22LJ5LICXUMIU6MYB746FSZ57KJX2T/
Code of Conduct: http://python.org/psf/codeofconduct/

Christian Heimes

unread,
Oct 6, 2021, 3:23:42 AM10/6/21
to Sandeep Gupta, pytho...@python.org
On 06/10/2021 09.06, Sandeep Gupta wrote:
> Tried with openssl. Some progress but no success.  The configure checks
> went through find.
> >configure:17536: checking for openssl/ssl.h in
> /home/kabira/DrivingRange//project_versa/Build
> >s/openssl-1.1.1l
> >configure:17543: result: yes
> >configure:17559: checking whether compiling and linking against
> OpenSSL works
> >Trying link with
> OPENSSL_LDFLAGS=-L/home/kabira/DrivingRange//project_versa/Builds/openssl-1.
> >1.1l/lib; OPENSSL_LIBS=-lssl -lcrypto;
> OPENSSL_INCLUDES=-I/home/kabira/DrivingRange//project_
> >versa/Builds/openssl-1.1.1l/include
>
> But for some reason module could not be imported. I could find any
> errors related to import.
> There were no compilation errors:
>
> >Following modules built successfully but were removed because they
> could not be imported:
> >_hashlib              _ssl                  readline

[...]

> *** WARNING: renaming "_ssl" since importing it failed: Cannot load
> specified object
> *** WARNING: renaming "_hashlib" since importing it failed: Cannot load
> specified object

OpenBSD uses clang C compiler. The new --with-openssl-rpath=auto option
was only tested with GCC. It turned out that Python's distutils package
didn't support rpath with clang. The bugfix
https://bugs.python.org/issue45371 will be available in upcoming release
Python 3.10.1.

In the mean time you either need to apply the patch from the issue
manually or figure out the right environment variables to add correct
rpath yourself.

I'm sorry for the inconvenience. We don't have any CI for OpenBSD.
Apparently this feature was never tested on OpenBSD during the release
candidate phase either.

Christian
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/RAKDWHTH7TMUYEWJJFNOPLKLBBDXZBED/

Sandeep Gupta

unread,
Oct 6, 2021, 3:31:56 AM10/6/21
to Christian Heimes, pytho...@python.org
Tried with openssl. Some progress but no success.  The configure checks went through find. 
>configure:17536: checking for openssl/ssl.h in /home/kabira/DrivingRange//project_versa/Build
>s/openssl-1.1.1l
>configure:17543: result: yes
>configure:17559: checking whether compiling and linking against OpenSSL works
>Trying link with OPENSSL_LDFLAGS=-L/home/kabira/DrivingRange//project_versa/Builds/openssl-1.
>1.1l/lib; OPENSSL_LIBS=-lssl -lcrypto; OPENSSL_INCLUDES=-I/home/kabira/DrivingRange//project_
>versa/Builds/openssl-1.1.1l/include

But for some reason module could not be imported. I could find any errors related to import. 
There were no compilation errors:

>Following modules built successfully but were removed because they could not be imported:
>_hashlib              _ssl                  readline


There area number of warnings regarding type conversions. Just attaching that for sanity check. 
This readline error is particulary wierd. ==> 
 ldd: /usr/lib/libreadline.a: not an ELF executable

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Python/pytime.c:160:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(time_t, intpart)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                `                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
Python/pytime.c:213:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
        if (!_Py_InIntegralTypeRange(time_t, intpart)) {
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
Python/pytime.c:398:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
3 warnings generated.
./Modules/_threadmodule.c:1644:26: warning: implicit conversion from '_PyTime_t' (aka 'long long') to 'double' changes value from 9223372036854775 to 9223372036854776 [-Wimplicit-const-int-float-conversion]
    double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6;
                         ^~~~~~~~~~~~~~~~~~~~~~~~~ ~
1 warning generated.
ldd: /usr/lib/libreadline.a: not an ELF executable
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/readline.c:1258:37: warning: assigning to 'char *' from 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
    completer_word_break_characters =
                                    ^
1 warning generated.
*** WARNING: renaming "readline" since importing it failed: dynamic module does not define module export function (PyInit_readline)
*** WARNING: renaming "_ssl" since importing it failed: Cannot load specified object
*** WARNING: renaming "_hashlib" since importing it failed: Cannot load specified object
renaming build/scripts-3.10/pydoc3 to build/scripts-3.10/pydoc3.10
renaming build/scripts-3.10/idle3 to build/scripts-3.10/idle3.10
renaming build/scripts-3.10/2to3 to build/scripts-3.10/2to3-3.10
*** Warning in /home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0: '$(LLVM_PROF_MERGER)' expands to '' while building build_all_merge_profile (Makefile:539)
Python/pytime.c:160:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(time_t, intpart)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
Python/pytime.c:213:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
        if (!_Py_InIntegralTypeRange(time_t, intpart)) {
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
Python/pytime.c:398:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
3 warnings generated.
./Modules/_threadmodule.c:1644:26: warning: implicit conversion from '_PyTime_t' (aka 'long long') to 'double' changes value from 9223372036854775 to 9223372036854776 [-Wimplicit-const-int-float-conversion]
    double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6;
                         ^~~~~~~~~~~~~~~~~~~~~~~~~ ~
1 warning generated.
ldd: /usr/lib/libreadline.a: not an ELF executable
/home/kabira/DrivingRange/project_versa/downloads/Python-3.10.0/Modules/readline.c:1258:37: warning: assigning to 'char *' from 'const char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
    completer_word_break_characters =
                                    ^
1 warning generated.
*** WARNING: renaming "readline" since importing it failed: dynamic module does not define module export function (PyInit_readline)
*** WARNING: renaming "_ssl" since importing it failed: Cannot load specified object
*** WARNING: renaming "_hashlib" since importing it failed: Cannot load specified object
renaming build/scripts-3.10/pydoc3 to build/scripts-3.10/pydoc3.10
renaming build/scripts-3.10/idle3 to build/scripts-3.10/idle3.10
renaming build/scripts-3.10/2to3 to build/scripts-3.10/2to3-3.10

Sandeep Gupta

unread,
Oct 6, 2021, 10:30:24 AM10/6/21
to Christian Heimes, pytho...@python.org
Thank you for super quick fix ). Works like charm. 

Thanks
S
Reply all
Reply to author
Forward
0 new messages