Custom Build Errors

[Mark Sigsbee]

I'm trying to build 3.6.0 with 3.2.1 for its approved FIPS library on Ubuntu 22.04. I know the default version of OpenSSL is 3.0 and basically leave it alone or the world will come to an end.

I'm building a newer version hoping to resolve issues that I'm getting with 3.0 and confirmed by Ubuntu Pro support.

I d/l'ed both into $ /PKI.
I followed the build instructions provided here:  openssl/README-FIPS.md at master · openssl/openssl

My check agrees with the check results in the readme.

When I execute:
$ sudo /PKI/openssl-3.6.0/apps/openssl list -providers -verbose

I get this:
/PKI$ sudo /PKI/openssl-3.6.0/apps/openssl list -providers -verbose
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.4.0' not found (required by /PKI/openssl-3.6.0/apps/openssl)
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.2.0' not found (required by /PKI/openssl-3.6.0/apps/openssl)
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.0.9' not found (required by /PKI/openssl-3.6.0/apps/openssl)
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by /PKI/openssl-3.6.0/apps/openssl)
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.5.0' not found (required by /PKI/openssl-3.6.0/apps/openssl)
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.6.0' not found (required by /PKI/openssl-3.6.0/apps/openssl)
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.4.0' not found (required by /PKI/openssl-3.6.0/apps/openssl)
/PKI/openssl-3.6.0/apps/openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.2.0' not found (required by /PKI/openssl-3.6.0/apps/openssl)

How do I correct this and get my desired result? 

[Neil Horman]

you don't have the library search path configured correctly on your platform.  Try running:

LD_LIBRARY_PATH=/PKI/openssl-3.6.0/lib PKI/openssl-3.6.0/apps/openssl list -providers -verbose

That will allow the application to run.  You may also need to set OPENSSL_CONF and OPENSSL_MODULES appropriately to find the needed configuration and provider DSO's

To avoid needing to set these values you need to configure openssl using the --prefix option to set the latter two variables automatically.  To avoid the need for LD_LIBRARY_PATH, you either need to configure your environment to add /PKI/openssl-3.6.0/lib[64] to the library search path globally, or install the library to a location already in the search path (though the latter isn't recommended if you are using a system with a default openssl already configured there).

Neil

--
You received this message because you are subscribed to the Google Groups "openssl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openssl-user...@openssl.org.
To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/CAPqvXLPc7dDaR3S05bmdSAaFJSZ0k5uSmCs0AY_juicFindorQ%40mail.gmail.com.

[Mark Sigsbee]

Thanks for the response.

/PKI/openssl-3.6.0/lib

Doesn't exist.

[Neil Horman]

did you run make install to put openssl-3.6 in that directory?  Or is that your build directory?  If its the latter then the libraries (libcrypto.so.3 and libssl.so.3) will be in the root of that subtree, and LD_LIBRARY_PATH should just be set to /PKI/openssl-3.6.0

Neil

[Mark Sigsbee]

Honestly, I've been following so may threads trying to resolve this, at this point I'm not sure what I did.  I do know that both of the lib*.so.3 files are in the root folder (openssl-3.6.0).

$ sudo echo $LD_LIBRARY_PATH

/PKI/openssl-3.6.0

I get the same error when re-executing the command.

[Neil Horman]

Mark-

     Based on what you've described, is it possible that, when you tried to make use of 3.1.2 provider with the 3.6 library that you, in addition to copying the 3.1.2 fips.so file from your 3.1.2 build, you also copied the 3.1.2 libcrypto.so.3 and libssl.so.3 files to the 3.6 directory?  That would also result in the errors that you are seeing.  Only the fips.so file should have been copied from the 3.1.2 build.

Neil

[Mark Sigsbee]

According to this:  openssl/README-FIPS.md at master · openssl/openssl

I executed this in the 3.6.0 directory. And I did it in the 3.6.0. directory based on the previous section:

$ cp ../openssl-3.1.2/providers/fips.so providers/.
$ cp ../openssl-3.1.2/providers/fipsmodule.cnf providers/.
// Note that for OpenSSL 3.1.2 that the `fipsmodule.cnf` file should not
// be copied across multiple machines if it contains an entry for
// `install-status`. (Otherwise the self tests would be skipped).

// Validate the output of the following to make sure we are using the
// OpenSSL 3.1.2 FIPS provider
$ ./util/wrap.pl -fips apps/openssl list -provider-path providers \
-provider fips -providers

// Now run the current tests using the OpenSSL 3.1.2 FIPS provider.
$ make tests

[Mark Sigsbee]

When I enter this: LD_LIBRARY_PATH=/PKI/openssl-3.6.0/lib PKI/openssl-3.6.0/apps/openssl list -providers -verbose

I get "command not found".

When I enter this: LD_LIBRARY_PATH=/PKI/openssl-3.6.0/lib /PKI/openssl-3.6.0/apps/openssl list -providers -verbose #added forward slash in front of second PKI

I get the crazy response that I shared above.

Thanks,

Mark

On Wed, Apr 15, 2026 at 9:31 AM Neil Horman <nho...@openssl.org> wrote:

[Neil Horman]

Adding the '/' makes sense, as I expect that PKI/openssl-3.6.0 isn't in your path

As for the linker errors, I'm fairly certain you've somehow mistakenly copied the libcrypto.so.3 library from your 3.1.2 build into /PKI/openssl-3.6.0/lib

If you run this command:

objdump -x /PKI/openssl-3.6.0/apps/openssl

you'll see output that includes this table:

Version References:
  required from libc.so.6:
    0x069691b3 0x00 16 GLIBC_2.33
    0x06969194 0x00 15 GLIBC_2.14
    0x069691b4 0x00 13 GLIBC_2.34
    0x069691b8 0x00 11 GLIBC_2.38
    0x0d696913 0x00 09 GLIBC_2.3
    0x09691a75 0x00 06 GLIBC_2.2.5
  required from libssl.so.3:
    0x06702f20 0x00 18 OPENSSL_3.4.0
    0x06702d20 0x00 05 OPENSSL_3.2.0
    0x06702b20 0x00 03 OPENSSL_3.0.0
  required from libcrypto.so.3:
    0x06702b29 0x00 17 OPENSSL_3.0.9
    0x06702b23 0x00 14 OPENSSL_3.0.3
    0x06702e20 0x00 12 OPENSSL_3.3.0
    0x06703020 0x00 10 OPENSSL_3.5.0
    0x06702120 0x00 08 OPENSSL_3.6.0
    0x06702f20 0x00 07 OPENSSL_3.4.0
    0x06702d20 0x00 04 OPENSSL_3.2.0
    0x06702b20 0x00 02 OPENSSL_3.0.0

That says the ELF file that is your openssl app requires that symbols from the OPENSSL_3.0.0 through OPENSSL_3.6.0 need to be resolved by this binary in libcrypto.so.3, as well as version 3.0.0, 3.2.0 and 3.4.0 from libssl.so.3.  This makes sense, because as we add api calls in minor versions the build system tags them with the version number in which they are added.

If you now run this command:

objdump -x /PKI/openssl-3.6.0/lib/libcrypto.so.3

What you _should_ see is a table that looks like this:

Version definitions:
1 0x01 0x06af4743 libcrypto.so.3
2 0x00 0x06702b20 OPENSSL_3.0.0
3 0x00 0x06702b23 OPENSSL_3.0.3
OPENSSL_3.0.0
4 0x00 0x06702b28 OPENSSL_3.0.8
OPENSSL_3.0.3
5 0x00 0x06702b29 OPENSSL_3.0.9
OPENSSL_3.0.8
6 0x00 0x06702c20 OPENSSL_3.1.0
OPENSSL_3.0.9
7 0x00 0x06702d20 OPENSSL_3.2.0
OPENSSL_3.1.0
8 0x00 0x06702e20 OPENSSL_3.3.0
OPENSSL_3.2.0
9 0x00 0x06702f20 OPENSSL_3.4.0
OPENSSL_3.3.0
10 0x00 0x06703020 OPENSSL_3.5.0
OPENSSL_3.4.0
11 0x00 0x06702120 OPENSSL_3.6.0
OPENSSL_3.5.0

Which define the versions that the openssl app needs from the library, and everything matches up.

What I expect you _will_ see however is a table that looks like this:

objdump  -x /PKI/openssl-3.6.0/lib/libcrypto.so.3

Version definitions:
1 0x01 0x06af4743 libcrypto.so.3
2 0x00 0x06702b20 OPENSSL_3.0.0

i.e. the library being pointed to by LD_LIBRARY_PATH doesn't contain the version definitions that the application requires.

I should be a bit more through here.  I asserted that somehow you inadvertently copied libcrypto.so.3 and libssl.so.3 from a non-matching build to /PKI/openssl-3.6.0/lib/ .  Its possible that didn't happen.  Instead it may also be possible that the needed libraries in /PKI/openssl-3.6.0/lib/ don't exist there at all.  If that were the case, the LD_LIBRARY_PATH lookup would fail and the runtime loader would fall back to the standard search path, loading /usr/lib64/libcrypto.so.3, which on ubuntu 22.04 would be a build from openssl-3.0.2, which would give you exactly the same error.

I would suggest the following:

1) confirm that /PKI/openssl-3.6.0/lib/libcrypto.so.3 and /PKI/openssl-3.6.0/lib/libssl.so.3 exist

2) Assuming that they do, run the above objdump commands to confirm the output I presented.

If the result of (1) is that those files don't exist in that location, the problem is that you're loading the system libcrypto/libssl and we need to adjust either your library locations or search path to find the right version that you want to use.

If the result of (1) is that those files do exist there in that location, then my initial theory is correct, and you've got an old version of libcrypto/libssl where it shouldn't be.

Regards

Neil

[Jan Just Keijser]

Hi Mark,

On 30/04/2026 17:10, Mark Sigsbee wrote:

Jan,

the screenshot is showing that you are NOT picking up your custom builds of libssl.so.3 or libcrypto.so.3.
As I suspect you have done a 'regular' build of the OpenSSL source code I recommend you try using
  LD_LIBRARY_PATH=/PKI/openssl-3.6.0

(no '/lib' at the end), so return with
  LD_LIBRARY_PATH=/PKI/openssl-3.6.0  /PKI/openssl-3.6.0/apps/openssl list -providers -verbose

This is all about the location of your newly built libssl.so.3 and libcrypto.so.3 - find where they are and you will find your solution.

HTH,

JJK

On Thu, Apr 30, 2026 at 10:05 AM Jan Just Keijser <jan.just...@gmail.com> wrote:

Hi Mark,

On 30/04/2026 15:10, Mark Sigsbee wrote:

When I enter this: LD_LIBRARY_PATH=/PKI/openssl-3.6.0/lib PKI/openssl-3.6.0/apps/openssl list -providers -verbose

I get "command not found".

When I enter this: LD_LIBRARY_PATH=/PKI/openssl-3.6.0/lib /PKI/openssl-3.6.0/apps/openssl list -providers -verbose #added forward slash in front of second PKI

I get the crazy response that I shared above.

please post the output of
  LD_LIBRARY_PATH=/PKI/openssl-3.6.0/lib ldd /PKI/openssl-3.6.0/apps/openssl


Also, keep in mind that combining LD_LIBRARY_PATH= and `sudo` in a single line is never going to work due to security reasons.

HTH,

JJK

To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/CAPqvXLPnz0tHo3Ob5bGmyWbxzubre%2BaJ7tW3AU2yXvTfB4fjcw%40mail.gmail.com.

[Mark Sigsbee]

Much better results: 

I also d/l'ed and compiled 3.2 for the FIPs cryptography only.  How do I make the environment understand that?

If I need to revert back to the "stock" out of the box configuration how do I reset (undo) what I did?

Thanks

[Mark Sigsbee]

Is this correct? This is after the success above. 

Mark

[Jan Just Keijser]

Hi Mark,

OpenSSL 3.0.2 is the 'stock' version of OpenSSL that comes with your Ubuntu 22 install

The output with LD_LIBRARY_PATH  set does seem to show better output - try running
   LD_LIBRARY_PATH=/PKI/openssl-3.6.0  /PKI/openssl-3.6.0/apps/openssl version

to get the version number of your custom build.

I would NOT recommend to replace the 'stock' openssl libraries with your custom build - it will most likely break your entire Ubuntu installation.
Also, any time Ubuntu publishes an update of their openssl package, an `apt update` or `apt upgrade` will overwrite your custom build.

HTH,

JJK

[Mark Sigsbee]

Here's what got me to this point:

1. Supporting a customer (T1) that has built a cloud solution for it's customers (T2). Like Azure in that T2 customers don't see each other.
2. T1 and T2 assets are required to have valid certs from a trusted CA.
3. Trusted CA is Microsoft AD CS.
4. AD CS is configured with NDES for remote automated certificate issuance.

5. T2 customers are predominantly Ubuntu, with a smattering of Rhel and Window mixed in.

6. Desire is to use Secure SCEP (sscep) to facilitate that.

Issue is Ubuntu Pro (FIPS enabled) stops with a segmentation fault error towards the end of the process.

I was trying to find an alternative OpenSSL/FIPs provider combination that could possibly resolve the segmentation fault.

I'm not confident that the baked in version of OpenSSL is playing nice with certificate issuance.

Mark

[Jan Just Keijser]

Hi Mark,

On 30/04/2026 18:36, Mark Sigsbee wrote:

Here's what got me to this point:

1. Supporting a customer (T1) that has built a cloud solution for it's customers (T2). Like Azure in that T2 customers don't see each other.
2. T1 and T2 assets are required to have valid certs from a trusted CA.
3. Trusted CA is Microsoft AD CS.
4. AD CS is configured with NDES for remote automated certificate issuance.

5. T2 customers are predominantly Ubuntu, with a smattering of Rhel and Window mixed in.

6. Desire is to use Secure SCEP (sscep) to facilitate that.

Issue is Ubuntu Pro (FIPS enabled) stops with a segmentation fault error towards the end of the process.

I was trying to find an alternative OpenSSL/FIPs provider combination that could possibly resolve the segmentation fault.

I'm not confident that the baked in version of OpenSSL is playing nice with certificate issuance.

segfaults + FIPS + certificates suggests to me that the certs being issued use a hashing algorithm (SHA1, MD5) that OpenSSL+FIPS does not accept.
To test this hypothesis, disable FIPS and see if it makes any difference; if it still crashes, then at least you know it is not the FIPS enablement

Also, what SSCEP client are you using? Have you tried https://github.com/certnanny/sscep  ?

I sincerely doubt that replacing the OS version of the OpenSSL libs is going to fix these issues - or it's not worth all the extra hassle you will introduce for yourself.

[Neil Horman]

On Fri, May 1, 2026 at 3:55 AM Jan Just Keijser <jan.just...@gmail.com> wrote:

Hi Mark,

On 30/04/2026 18:36, Mark Sigsbee wrote:

Here's what got me to this point:

1. Supporting a customer (T1) that has built a cloud solution for it's customers (T2). Like Azure in that T2 customers don't see each other.
2. T1 and T2 assets are required to have valid certs from a trusted CA.
3. Trusted CA is Microsoft AD CS.
4. AD CS is configured with NDES for remote automated certificate issuance.

5. T2 customers are predominantly Ubuntu, with a smattering of Rhel and Window mixed in.

6. Desire is to use Secure SCEP (sscep) to facilitate that.

Issue is Ubuntu Pro (FIPS enabled) stops with a segmentation fault error towards the end of the process.

I was trying to find an alternative OpenSSL/FIPs provider combination that could possibly resolve the segmentation fault.

I'm not confident that the baked in version of OpenSSL is playing nice with certificate issuance.

segfaults + FIPS + certificates suggests to me that the certs being issued use a hashing algorithm (SHA1, MD5) that OpenSSL+FIPS does not accept.
To test this hypothesis, disable FIPS and see if it makes any difference; if it still crashes, then at least you know it is not the FIPS enablement

Also, what SSCEP client are you using? Have you tried https://github.com/certnanny/sscep  ?

I sincerely doubt that replacing the OS version of the OpenSSL libs is going to fix these issues - or it's not worth all the extra hassle you will introduce for yourself.

HTH,

JJK

I agree, you're buying a lot more problems by replacing the system version of openssl.  If you are having problems with the system installed version of openssl, you likely want to contact canonical for support to fix the segfault (though they may tell you that the problem is in your application, likely due to invalid error checking after the cert is rejected).  If you can provide the segfault backtrace we may be able to confirm that.

Alternatively, if you're insistent on using a custom openssl build, I'd suggest that you containerize the applications and install your self-built openssl version to the container.

Neil

[Viktor Dukhovni]

On Wed, Apr 15, 2026 at 09:12:44AM -0400, Mark Sigsbee wrote:

> I'm trying to build 3.6.0 with 3.2.1 for its approved FIPS library on
> Ubuntu 22.04. I know the default version of OpenSSL is 3.0 and basically
> leave it alone or the world will come to an end.

I think you mean "3.1.2" which is the latest FIPS-validated release.

The attached build script will construct (from source) a custom 3.6
build installed /opt/openssl/3.6, and a 3.1.2 "fips" module installed
/opt/openssl/3.1, with the 3.6 runtime configured to load the "3.1"
module and use only FIPS-approved algorithms.

It is of course assumed that you have all the necessary build
dependencies (compiler, required Perl modules, ...) installed.

This does not by itself make your deployment be FIPS-conformant, you
still need to read and the OpenSSL FIPS security policy and follow all
relevant guidelines. My build is a demo, your deployment may need to
have appropriate adjustments

The source repo and build trees will be in the "openssl/" subdirectory
of whatever directory you're in when you execute the below "bash" shell
script.

Look carefully over the output to make sure all tests passed, the script
can be modified to abort on test failure, at your discretion. This is
not a "production" deployment script, it is a proof-of-concept demo.

--
Viktor. 🇺🇦 Слава Україні!

[Mark Sigsbee]

Really getting into the weeds here and I'm loving it!

Again, can't say enough how much I appreciate your input.

Jan's reply triggered me! LOL  I did a deep dive on Microsoft CA to make sure that it understood FIPS.  Turns out that my test lab was NOT FIPs compliant, but prod servers are. In my test lab,  I created a GPO just for FIPs and applied it domain wide.  Checked every server that is part of this process, 2 DC's, Issuing CA and my NDES server to confirm that they are now FIPS compliant:



Next I checked the crypto provider for each of the 3 cert templates that I can use. They are now configured to use the Key Storage Provider which is the recommended FIPS provider for Server 2012 and higher:

I reconfirmed that I was using the "out-of-box" (OOB) OpenSSL and crypto provider for Ubuntu Pro:

I'm tabling the customer OpenSSL build and crypto provider.

I re-gened the priv key:

I modified the .cnf file for sscep and attempted the process again. Same failure.

I manually copied the CSR to my CA and manually attempted to issue the cert. No problems. This rules out any FIPS issues, I think, between OpenSSL (Ubuntu Pro-OOB) and my CA. 

I think this clearly points to an issue with sscep.  And, yes, I am using the certnanny version of sscep.  I compiled it using the OOB version of OpenSSL. I made no changes to the Ubuntu Pro OS today while I was doing all of this testing. 

If you concur with my findings and troubleshooting, do you have any recommendations? Or do I need to rattle "CertNanny's" cage to see what they can do for me?

Thanks again!

Mark

[Mark Sigsbee]

Any thoughts on my previous post?

[Neil Horman]

To be clear, when you say the "same failure", are you referring to the segfault you mentioned earlier, or the symbol version mismatch?

If its the former, providing a backtrace might help narrow down the issue, but I would say at that point your most expedient path to resolving this would likely be to open an issue with canonical.  They carry their own patches for OpenSSL (IIRC to support their own FIPS module definition), and they would know best the ins and outs of their changes to help you resolve this.

neil

[Jan Just Keijser]

On 06/05/2026 14:39, Mark Sigsbee wrote:

Any thoughts on my previous post?

your question has changed from building your own version of OpenSSL (question more or less answered) to asking about debugging a third-part tool, sscep ... I'd say that is not entirely for this mailing list, but my recommendation is to build sscep against the 'stock' Ubuntu libssl with debug info, install the debug packages for libssl/openssl and then run the command inside gdb .  See where sscep crashes (use the 'where' command inside gdb) , then decide what to do next.

HTH,

JJK

[Mark Sigsbee]

Yup. Requirement creep! 

I'll end this thread with my thanks.

Mark