Help with acme_certificate / letsencrypt and empty challenge_data

[Jason Wood]

I'm fairly new to this. I'm using ansible 2.9.6 on Ubuntu 20.04.

I followed the Digital Ocean tutorial to get started with letsencrypt/acme_certificate

The problem I run into is that if I run the play again, I get an error when I try to host the challenge data because challenge_data is an empty object.

I see in the official docs there is a "when" condition in the example:

when: sample_com_challenge is changed and 'sample.com' in sample_com_challenge['challenge_data']

...but I am unsure how to write this statement when the domain name comes from a variable. I tried both of the following:

when: acme_challenge_mydomain is changed and inventory_hostname in acme_challenge_mydomain['challenge_data']

and

when: acme_challenge_mydomain is changed and '{{ inventory_hostname }}' in acme_challenge_mydomain['challenge_data']

 

but in both cases the task executes even though I can see with a debug task that challenge_data is empty:

..."challenge_data": {},...

My second question is... I have remaining_days set to 91, and there are 89 days left, so shouldn't I have data in challenge_data? The docs for remaining_days states "If cert_days < remaining_days, then it will be renewed. If the certificate is not renewed, module return values will not include challenge_data."

[Felix Fontein]

Hi,

> I'm fairly new to this. I'm using ansible 2.9.6 on Ubuntu 20.04.

that version is really old and outdated. You should upgrade at least to
the latest 2.9.x release (2.9.27), or even to the latest Ansible
release (5.2.0).

This is the correct syntax (assuming your domain name is
inventory_hostname):

> when: acme_challenge_mydomain is changed and inventory_hostname in
> acme_challenge_mydomain['challenge_data']

Without knowing your playbook...

> but in both cases the task executes even though I can see with a
> debug task that challenge_data is empty:
> ..."challenge_data": {},...

...it's hard to say why this happens.

Same for this:

> My second question is... I have remaining_days set to 91, and there
> are 89 days left, so shouldn't I have data in challenge_data?

There should be. (You can also set `force: true` to force regeneration.)

Are you sure that you are really running the playbook that you
modified, and not something else?

Cheers,
Felix

[Jason Wood]

inventory_hostname is not the domain, it is a variable which contains the domain. That's the part where I'm not sure how to reference it.

I will try upgrading to see if that changes anything. Thanks!

[Parth Patel]

How inventory_hostname would contain domain ? 

Its just a special variable pre defined in ansible which contains name of the host for which task or play runs

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/d0750be4-1afe-4635-885e-8e377ff98d70n%40googlegroups.com.

[Stefan Hornburg (Racke)]

On 18/01/2022 18:52, Parth Patel wrote:
> How inventory_hostname would contain domain ?
>

If you put foo.example.com into the inventory, the inventory_hostname would include the domain example.com.

Regards
Racke

> Its just a special variable pre defined in ansible which contains name of the host for which task or play runs
>

> To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com <mailto:ansible-proje...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/d0750be4-1afe-4635-885e-8e377ff98d70n%40googlegroups.com <https://groups.google.com/d/msgid/ansible-project/d0750be4-1afe-4635-885e-8e377ff98d70n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google Groups "Ansible Project" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com <mailto:ansible-proje...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAEMGyL%2B2_ND0gOOHMv%3DyNi8LuXM4%2BU7mX0wRF3UFP5YzU43coA%40mail.gmail.com <https://groups.google.com/d/msgid/ansible-project/CAEMGyL%2B2_ND0gOOHMv%3DyNi8LuXM4%2BU7mX0wRF3UFP5YzU43coA%40mail.gmail.com?utm_medium=email&utm_source=footer>.


--
Ecommerce and Linux consulting + Perl and web application programming.
Debian and Sympa administration.

[Jason Wood]

I added these debugs:

- name: Debug when statement

debug:

var: acme_challenge_mydomain is changed and inventory_hostname in acme_challenge_mydomain['challenge_data']

- name: Debug the debug

debug:

var: acme_challenge_mydomain is changed and inventory_hostname in acme_challenge_mydomain['authorizations']

And these output false and true, as expected

But right after that, I have the exact same statement in when:

- name: "Implement http-01 challenge files"

copy:

content: "{{ acme_challenge_mydomain['challenge_data'][inventory_hostname]['http-01']['resource_value'] }}"

dest: "/opt/FileMaker/FileMaker Server/HTTPServer/htdocs/{{ acme_challenge_mydomain['challenge_data'][inventory_hostname]['http-01']['resource'] }}"

owner: root

group: root

mode: u=rw,g=r,o=r

when: acme_challenge_mydomain is changed and inventory_hostname in acme_challenge_mydomain['challenge_data']

...but this task is still returning this error:

fatal: [travelweek.ddb.space]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'travelweek.ddb.space'\n\nThe error appears to be in '/home/ubuntu/fms-ansible-master/letsencrypt-issue.yml': line 83, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n  - name: \"Implement http-01 challenge files\"\n    ^ here\n"}

Note: I have upgraded ansible and ansible --version now returns ansible [core 2.12.1]

> How inventory_hostname would contain domain ? 

> Its just a special variable pre defined in ansible which contains name of the host for which task or play runs

That is exactly what I want and it seems to be working as I expect. Notice the true result from my second debug task above. I also know through debug that inventory_hostname contains the domain I want the certificate issued for. I also debug the acme_challenge_mydomain object and I see the domain there in authorizations, which is why the statement returns true.

[Felix Fontein]

Hi,

> But right after that, I have the exact same statement in when:
>
> - name: "Implement http-01 challenge files"
> copy:
> content: "{{
> acme_challenge_mydomain['challenge_data'][inventory_hostname]['http-01']['resource_value']
> }}"
> dest: "/opt/FileMaker/FileMaker Server/HTTPServer/htdocs/{{
> acme_challenge_mydomain['challenge_data'][inventory_hostname]['http-01']['resource']
> }}"
> owner: root
> group: root
> mode: u=rw,g=r,o=r

> when: *acme_challenge_mydomain is changed and inventory_hostname in
> acme_challenge_mydomain['challenge_data']*

>
> ...but this task is still returning this error:

unfortunately in the text version of your email, all indentation is
gone. But in the HTML view online
(https://groups.google.com/g/ansible-project/c/cKmSS0VhyYo/m/oDCXtZFLAAAJ)
one can see that `when:` is indented at the wrong level. It is not an
option to the `copy` module, but belongs on the task level, i.e. the
same level as `copy:`.

If you remove two spaces before `when:` it should work.

Cheers,
Felix

[Jason Wood]

oh my! Thank you!!!

It appears that the first example in the docs has this error:

https://docs.ansible.com/ansible/latest/collections/community/crypto/acme_certificate_module.html#acme-certificate-module

So with that solved, I'm left with the problem where challenge_data is empty even if I specify remaining_days: 91 . I also tried adding force: yes or force: true

In the object created by acme_challenge task, I see that cert_days is 89

It works if I delete the /etc/letsencrypt directory, which includes the account key, certs, csr, etc (so basically we're starting over from scratch).

I only really care if it works when it gets to under 30 days. This is just for testing.

Here is my play from the top all the way down to the first run of acme_certificate

---
- hosts: fms
  become: true
  tasks:

  - name: "Create required directories in /etc/letsencrypt"
    file:
      path: "/etc/letsencrypt/{{ item }}"
      state: directory
      owner: root
      group: root
      mode: u=rwx,g=x,o=x
    with_items:
    - account
    - certs
    - csrs
    - keys

  - name: "Generate a Let's Encrypt account key"
    shell: "if [ ! -f {{ letsencrypt_account_key }} ]; then openssl genrsa 4096 | sudo tee {{ letsencrypt_account_key }}; fi"

  - name: "Generate Let's Encrypt private key"
    shell: "openssl genrsa 4096 | sudo tee /etc/letsencrypt/keys/{{ inventory_hostname }}.key"

  - name: "Generate Let's Encrypt CSR"
    shell: "openssl req -new -sha256 -key /etc/letsencrypt/keys/{{ inventory_hostname }}.key -subj \"/CN={{ inventory_hostname }}\" | sudo tee /etc/letsencrypt/csrs/{{ inventory_hostname }}.csr"
    args:
      executable: /bin/bash

  - name: "Begin Let's Encrypt challenges"
    acme_certificate:
      acme_directory: "{{ acme_directory }}"
      acme_version: "{{ acme_version }}"
      account_key_src: "{{ letsencrypt_account_key }}"
      account_email: "{{ acme_email }}"
      terms_agreed: 1
      challenge: "{{ acme_challenge_type }}"
      csr: "{{ letsencrypt_csrs_dir }}/{{ inventory_hostname }}.csr"
      dest: "{{ letsencrypt_certs_dir }}/{{ inventory_hostname }}.crt"
      fullchain_dest: "{{ letsencrypt_certs_dir }}/fullchain_{{ inventory_hostname }}.crt"
      remaining_days: 91
      force: yes
    register: acme_challenge_mydomain

[Felix Fontein]

Hi,

> It appears that the first example in the docs has this error:
> https://docs.ansible.com/ansible/latest/collections/community/crypto/acme_certificate_module.html#acme-certificate-module

oh, indeed! Thanks for spotting that! I've created a PR to fix it
(https://github.com/ansible-collections/community.crypto/pull/382).

> So with that solved, I'm left with the problem where challenge_data
> is empty even if I specify remaining_days: 91 . I also tried adding
> force: yes or force: true
>
> In the object created by acme_challenge task, I see that cert_days is
> 89
>
> It works if I delete the /etc/letsencrypt directory, which includes
> the account key, certs, csr, etc (so basically we're starting over
> from scratch).

That's not how it should be done :)

> I only really care if it works when it gets to under 30 days. This is
> just for testing.
>
> Here is my play from the top all the way down to the first run of
> acme_certificate
>
> ---
> - hosts: fms
> become: true
> tasks:
>
> - name: "Create required directories in /etc/letsencrypt"
> file:
> path: "/etc/letsencrypt/{{ item }}"
> state: directory
> owner: root
> group: root
> mode: u=rwx,g=x,o=x
> with_items:
> - account
> - certs
> - csrs
> - keys
>
> - name: "Generate a Let's Encrypt account key"
> shell: "if [ ! -f {{ letsencrypt_account_key }} ]; then openssl
> genrsa 4096 | sudo tee {{ letsencrypt_account_key }}; fi"

BTW, you can use `creates:` to avoid having to use the `if` construct
(https://docs.ansible.com/ansible/latest/collections/ansible/builtin/shell_module.html#parameter-creates).

> - name: "Generate Let's Encrypt private key"
> shell: "openssl genrsa 4096 | sudo tee /etc/letsencrypt/keys/{{
> inventory_hostname }}.key"
>
> - name: "Generate Let's Encrypt CSR"
> shell: "openssl req -new -sha256 -key /etc/letsencrypt/keys/{{
> inventory_hostname }}.key -subj \"/CN={{ inventory_hostname }}\" |
> sudo tee /etc/letsencrypt/csrs/{{ inventory_hostname }}.csr"
> args:
> executable: /bin/bash

Also you might be interested in using the openssl_privatekey module to
create the private keys, and the openssl_csr module to create the CSR.

> - name: "Begin Let's Encrypt challenges"
> acme_certificate:
> acme_directory: "{{ acme_directory }}"
> acme_version: "{{ acme_version }}"
> account_key_src: "{{ letsencrypt_account_key }}"
> account_email: "{{ acme_email }}"
> terms_agreed: 1
> challenge: "{{ acme_challenge_type }}"
> csr: "{{ letsencrypt_csrs_dir }}/{{ inventory_hostname }}.csr"
> dest: "{{ letsencrypt_certs_dir }}/{{ inventory_hostname }}.crt"
> fullchain_dest: "{{ letsencrypt_certs_dir }}/fullchain_{{ inventory_hostname }}.crt"
> remaining_days: 91
> force: yes
> register: acme_challenge_mydomain

This looks correct so far.

I guess afterwards you have the copy task, and then another
acme_certificate task. Which `when:` condition are you using for the
latter? I hope only `when: acme_challenge_mydomain is changed` and not
the same condition as for the copy task.

(The copy task is not always necessary - Let's Encrypt is caching valid
authorizations for some days -, but the other acme_certificate needs to
be run if you want a certificate.)

Cheers,
Felix

[Jason Wood]

Thanks - I'm understanding much better now.

The presence of challenge data is not necessarily required for certificate renewal, if previous challenge data is still valid, you can just skip the copy step and the second run will install a new certificate.

If no certificate is needed because remaining_days is lower than cert_days, the first run will return changed=false and the second run doesn't need to run.

I will also take your advice and implement creates: and the openssl_privatekey and openssl_csr modules!

Thanks for all your help!