How to release ENGINE_load_private_key?

[Shudong Zhang]

Hi all,

    We use ENGINE_load_private_key to get key from PKCS#11 engine. And this API will increase the engine's struct_ref.

    But we could not find an proper API to decrease engine's struct_ref.

    Does anyone know how to release this correctly?

   Thansk for you in advance!

BRs,

Shudong

   

[Tomas Mraz]

When you release the EVP_PKEY object, the structural (and functional)
references to the ENGINE held by the EVP_PKEY will be decreased.

Tomas Mraz, OpenSSL

> --
> You received this message because you are subscribed to the Google
> Groups "openssl-users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to openssl-user...@openssl.org.
> To view this discussion visit
> https://groups.google.com/a/openssl.org/d/msgid/openssl-users/4d66794d.96a0.19305fc010a.Coremail.zsdclgc%40163.com
> .

--
Tomáš Mráz, OpenSSL

[Shudong Zhang]

Hi Tomas,

    Thansk for yur quick reply. The version that we use is 1.1.1.o.

    Unfortunately, our result indicate that EVP_PKEY_free can not decrese engine struct counter.

    Our code is as follow：

    ```

ENGINE *e = NULL;

    EVP_PKEY *pkey = NULL;

    X509_REQ *req = NULL;

    X509_NAME *name = NULL;

    BIO *bio = NULL;

    char *data = NULL;

    long len;

    std::string result;

    X509V3_CTX ctx;

    X509_EXTENSION *ext = NULL;

    STACK_OF(X509_EXTENSION) *exts = NULL;

    struct xp_engine_st *xet = NULL;

    /* init openssl engine */

    OpenSSL_add_all_algorithms();

    ERR_load_crypto_strings();

    ENGINE_load_dynamic();

    e = ENGINE_by_id("dynamic");

    if (e == NULL) {

        fprintf(stderr, "Could not find engine\n");

        return "";

    }

    if (!ENGINE_ctrl_cmd_string(e, "SO_PATH", PKCS11_LIB_PATH, 0)) {

        fprintf(stderr, "Set so path failed\n");

        return "";

    }

    if (!ENGINE_ctrl_cmd_string(e, "ID", PKCS11_ENGINE_ID, 0)) {

        fprintf(stderr, "Set engine ID failed\n");

        return "";

    }

    if (!ENGINE_ctrl_cmd_string(e, "LIST_ADD", "1", 0)) {

        fprintf(stderr, "LIST add failed\n");

        return "";

    }

    if (!ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0)) {

        fprintf(stderr, "Load engine failed\n");

        return "";

    }

    if (!ENGINE_ctrl_cmd_string(e, "MODULE_PATH", PKCS11_CA_PATH, 0)) {

        fprintf(stderr, "Set module path failed\n");

        return "";

    }

    if (!ENGINE_init(e)) {

        fprintf(stderr, "Engine init failed\n");

        return "";

    }

    xet = (struct xp_engine_st *)e;

    /* set default ec engine */

    ENGINE_set_default_EC(e);

    pkey = ENGINE_load_private_key(e, getPrivUri().c_str(), NULL, NULL);

    if (!pkey) {

        fprintf(stderr, "Error loading private key\n");

        goto end;

    }

    req = X509_REQ_new();

    if (!req) {

        fprintf(stderr, "Error creating X509_REQ object\n");

        goto end;

    }

    /* Set the subject of the CSR */

    name = X509_REQ_get_subject_name(req);

    X509_NAME_add_entry_by_txt(name, "C", MBSTRING_ASC,

                               (const unsigned char *)"CN", -1, -1, 0);

    X509_NAME_add_entry_by_txt(name, "ST", MBSTRING_ASC,

                               (const unsigned char *)"GD", -1, -1, 0);

    X509_NAME_add_entry_by_txt(name, "L", MBSTRING_ASC,

                               (const unsigned char *)"GZ", -1, -1, 0);

    X509_NAME_add_entry_by_txt(name, "O", MBSTRING_ASC,

                               (const unsigned char *)"XPENG PSO", -1, -1, 0);

    X509_NAME_add_entry_by_txt(name, "CN", MBSTRING_ASC,

                               (const unsigned char *)ecuid.c_str(), -1,

                               -1, 0);

    /* create extend attributes */

    X509V3_set_ctx_nodb(&ctx);

    X509V3_set_ctx(&ctx, NULL, NULL, req, NULL, 0);

    exts = sk_X509_EXTENSION_new_null();

    /* Add basic constraints extension */

    ext = X509V3_EXT_conf_nid(NULL, &ctx, NID_basic_constraints,

                              "critical,CA:FALSE");

    sk_X509_EXTENSION_push(exts, ext);

    /* Add key usage extension */

    ext = X509V3_EXT_conf_nid(NULL, &ctx, NID_key_usage,

                              "critical,digitalSignature,keyEncipherment");

    sk_X509_EXTENSION_push(exts, ext);

    /* Add extended key usage extension */

    ext = X509V3_EXT_conf_nid(NULL, &ctx, NID_ext_key_usage,

                              "clientAuth,emailProtection");

    sk_X509_EXTENSION_push(exts, ext);

    /* Add extensions to the certificate request */

    X509_REQ_add_extensions(req, exts);

    /* Set the public key for the CSR */

    if (!X509_REQ_set_pubkey(req, pkey)) {

        fprintf(stderr, "Error setting public key\n");

        goto end;

    }

    if (!X509_REQ_sign(req, pkey, EVP_sha256())) {

        fprintf(stderr, "Error signing CSR\n");

        goto end;

    }

    bio = BIO_new(BIO_s_mem());

    if (!bio) {

        fprintf(stderr, "Error BIO_new\n");

        goto end;

    }

    if (!i2d_X509_REQ_bio(bio, req)) {

        fprintf(stderr, "Error PEM_write_bio_X509_REQ\n");

        goto end;

    }

    len = BIO_get_mem_data(bio, &data);

    if (len <= 0) {

        fprintf(stderr, "Error BIO_get_mem_data\n");

        goto end;

    }

    result = toHexString((unsigned char *)data, len);

end:

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

    if (exts) {

        sk_X509_EXTENSION_pop_free(exts, X509_EXTENSION_free);

    }

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

    if (req) {

        X509_REQ_free(req);

    }

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

    if (pkey) {

        EVP_PKEY_free(pkey);

printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

    }

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

    if (bio) {

        BIO_free(bio);

    }

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

    if (e) {

        ENGINE_unregister_EC(e);

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

        ENGINE_remove(e);

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

        ENGINE_finish(e);

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

        ENGINE_free(e);

    printf("%d build csr %s %d\n", __LINE__, xet->name, xet->struct_ref);

        ENGINE_free(e);

        ENGINE_cleanup();

    }

    EVP_cleanup();

    ERR_free_strings();

    return result;

}

    Since engine is an oquare struce, so I copy an struct and rename to xp_engine_st to get struct counter.

    The log indicate that the counter not decrease.

    Is any error in my code?

BRs,

Shudong

.
>To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/fb620f6d03b9e75d593d6a9638b17decec072223.camel%40openssl.org.

[Tomas Mraz]

Are you sure the EVP_PKEY is really released, i.e. as it is also
refcounted, there is no other reference to it? If so, the engine struct
reference wouldn't be decreased only if the engine_finish function
returned failure. You'll have to use a debugger and build OpenSSL with
debug info to step through the EVP_PKEY_free() call to see what really
happens.

Regards,
Tomas Mraz, OpenSSL

On Thu, 2024-11-07 at 18:12 +0800, Shudong Zhang wrote:
>
>
>
> Hi Tomas,
>     Thansk for yur quick reply. The version that we use is 1.1.1.o.
>     Unfortunately, our result indicate that EVP_PKEY_free can not
> decrese engine struct counter.

--
Tomáš Mráz, OpenSSL

[Shudong Zhang]

Hi Tomas Mraz，

    Thank you so mush! I will debug it follow your guide!

BRs,

Shudong

> wrote:
>Are you sure the EVP_PKEY is really released, i.e. as it is also
>refcounted, there is no other reference to it? If so, the engine struct
>reference wouldn't be decreased only if the engine_finish function
>returned failure. You'll have to use a debugger and build OpenSSL with
>debug info to step through the EVP_PKEY_free() call to see what really
>happens.
>
>Regards,
>Tomas Mraz, OpenSSL
>
>
>On Thu, 2024-11-07 at 18:12 +0800, Shudong Zhang wrote:
>> 
>> 
>> 
>> Hi Tomas,
>>     Thansk for yur quick reply. The version that we use is 1.1.1.o.
>>     Unfortunately, our result indicate that EVP_PKEY_free can not
>> decrese engine struct counter.
>
>-- 
>Tomáš Mráz, OpenSSL
>
>-- 
>You received this message because you are subscribed to the Google Groups "openssl-users" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to openssl-user...@openssl.org.
>To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/d245d4c167097092496366399e026cdf1faa728d.camel%40openssl.org.

[Shudong Zhang]

Hi Tomas，

You are right! I add log to printf ref_counter.

And when I call EVP_PKEY_free, the ref_conunter is 1. So the function just return and not to release engine.

I will try to figure out where still have referance this pkey.

Thanks very much!

BRs,

Shudong

At 2024-11-07 18:54:38, "Tomas Mraz" <to...@openssl.org

> wrote:
>Are you sure the EVP_PKEY is really released, i.e. as it is also
>refcounted, there is no other reference to it? If so, the engine struct
>reference wouldn't be decreased only if the engine_finish function
>returned failure. You'll have to use a debugger and build OpenSSL with
>debug info to step through the EVP_PKEY_free() call to see what really
>happens.
>
>Regards,
>Tomas Mraz, OpenSSL
>
>
>On Thu, 2024-11-07 at 18:12 +0800, Shudong Zhang wrote:
>> 
>> 
>> 
>> Hi Tomas,
>>     Thansk for yur quick reply. The version that we use is 1.1.1.o.
>>     Unfortunately, our result indicate that EVP_PKEY_free can not
>> decrese engine struct counter.
>
>-- 
>Tomáš Mráz, OpenSSL
>
>-- 
>You received this message because you are subscribed to the Google Groups "openssl-users" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to openssl-user...@openssl.org.
>To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/d245d4c167097092496366399e026cdf1faa728d.camel%40openssl.org.

[Shudong Zhang]

Hi Tomas,

After add some log, I find code do some ops to add ref_counter.

```

ENGINE_load_private_key -----> add ref_counter to 2

if (!X509_REQ_set_pubkey(req, pkey)) ------> add ref_counter to 3

X509_REQ_free(req); ------> decrease ref_counter to 2

EVP_PKEY_free ------> decrease ref_counter to 2

```

    It looks like that I miss a free api to call. Do you konw which I miss?

BRs,

Shudong

To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/14653838.a869.19306aedc58.Coremail.zsdclgc%40163.com.

[Tomas Mraz]

That looks like a bug in the engine - the reference count in the
EVP_PKEY after the ENGINE_load_private_key() call should be just 1. As
the EVP_PKEY is created by the engine, there is not much OpenSSL can do
if the engine somehow keeps an additional reference to the key
somewhere.

Tomas Mraz, OpenSSL