Cannot use the pexpect module with Keybase pgp and multiple ':' characters

[Andrew Graham]

I'm wondering why the pexpect module is failing in this circumstance, perhaps my regex is not right, but the double ':' may be messing with things as well and I'm not sure how to approach handling it.

if I run-

keybase pgp encrypt -m 'secret' | keybase pgp decrypt

Then this prompt may occur, which I cant seem to catch.

Please enter your Keybase passphrase to unlock the secret key for:
PGP key andrew_graham <queglay@gmail.com> KJ2G34K4J3H1K234J

Reason: PGP Decryption:

here is an example of my attempts with expect:

- name: Case insensitive password string match

expect:

chdir: /vagrant

command: '/bin/bash -c "echo {{ secret }} | base64 --decode | keybase pgp decrypt"'

responses:

.*Please enter your Keybase passphrase to unlock the secret key for \[.*\]: "{{ keybase_secret }}"

.*Please enter your Keybase passphrase to unlock the secret key for.*: "{{ keybase_secret }}"

^Reason\[:.*\] PGP Decryption\[:.*\]: "{{ keybase_secret }}"

[Kai Stian Olstad]

On 28.08.2019 04:25, Andrew Graham wrote:
> I'm wondering why the pexpect module is failing in this
> circumstance, perhaps my regex is not right, but the double ':' may be
> messing with things as well and I'm not sure how to approach handling
> it.
>
>
> if I run-
>
>
> keybase pgp encrypt -m 'secret' | keybase pgp decrypt
>
>
> Then this prompt may occur, which I cant seem to catch.
>
>
> Please enter your Keybase passphrase to unlock the secret key for:

> PGP key andrew_graham <que...@gmail.com> KJ2G34K4J3H1K234J
>
> Reason: PGP Decryption:

I guess this output comes at once and the cursor is waiting after PGP
Decryption: for an input.

> here is an example of my attempts with expect:
>
>
> - name: Case insensitive password string match
> expect:
> chdir: /vagrant
> command: '/bin/bash -c "echo {{ secret }} | base64 --decode | keybase
> pgp
> decrypt"'
> responses:
> .*Please enter your Keybase passphrase to unlock the secret key for
> \[.*\]: "{{
> keybase_secret }}"
> .*Please enter your Keybase passphrase to unlock the secret key for.*:
> "{{
> keybase_secret }}"
> ^Reason\[:.*\] PGP Decryption\[:.*\]: "{{ keybase_secret }}"

You have a couple of problems.
First you are posting HTML so you formatting is screwed up.

.* at the start or at the end don't have any meaning since they will
always hit on nothing since pexpect in non-greedy.

Pexpect is just looking for a sequence of text that is matching, so you
only need to match uniquely as little as possible at the end of the
string.
So something like this should work

responses:
PGP Decryption: "{{ keybase_secret }}"

--
Kai Stian Olstad

[Andrew Graham]

I'll try this out Kai, thanks for the suggestion!

[Andrew Graham]

no luck with your suggestion @Kai - its still timing out here with the suggested changes.

currently the expect module isn't working here.  try to enter this in a shell to validate a password first.  "keybase pgp encrypt -m 'secret' | keybase pgp decrypt"
TASK [deadline-spot : Case insensitive password string match] ******************
Wednesday 28 August 2019  21:51:40 +1000 (0:00:00.042)       0:00:00.726 ******
fatal: [ansible_control]: FAILED! => {
    "changed": true,
    "cmd": "/bin/bash -c \"echo asdfljashflakewrva23lk432h42l3nr2tlkn32rt= | base64 --decode | keybase pgp decrypt\"",
    "delta": "0:00:35.439475",
    "end": "2019-08-28 21:52:16.698211",
    "rc": null,
    "start": "2019-08-28 21:51:41.258736"
}

STDOUT:

▶ INFO Starting keybase.service.

Please enter your Keybase passphrase to unlock the secret key for:
PGP key andre

w_graham <blahblah@gmail.com> seg4esehseh

Reason: PGP Decryption:


MSG:

command exceeded timeout

[Malcolm Hussain-Gambles]

I and a few others I know, have tried the pexpect module with similar results I ended up using the shell command unfortunately.

[Andrew Graham]

Would you be able to share an example of how you achieved it with a shell module?

[Kai Stian Olstad]

On 28.08.2019 13:54, Andrew Graham wrote:
> no luck with your suggestion @Kai - its still timing out here with the
> suggested changes.

There should not be a problem using expect on this, but to solve it it's
important to understand what is prompt and what is typed in by a human.

> currently the expect module isn't working here. try to enter this in a
> shell to validate a password first. "keybase pgp encrypt -m 'secret' |
> keybase pgp decrypt"
> TASK [deadline-spot : Case insensitive password string match]
> ******************
> Wednesday 28 August 2019 21:51:40 +1000 (0:00:00.042)
> 0:00:00.726
> ******
> fatal: [ansible_control]: FAILED! => {
> "changed": true,
> "cmd": "/bin/bash -c \"echo
> asdfljashflakewrva23lk432h42l3nr2tlkn32rt=
> | base64 --decode | keybase pgp decrypt\"",
> "delta": "0:00:35.439475",
> "end": "2019-08-28 21:52:16.698211",
> "rc": null,
> "start": "2019-08-28 21:51:41.258736"
> }
>
> STDOUT:
>
> ▶ INFO Starting keybase.service.
> Please enter your Keybase passphrase to unlock the secret key for:
> PGP key andre

> w_graham <blah...@gmail.com> seg4esehseh

>
> Reason: PGP Decryption:
>
>
> MSG:
>
> command exceeded timeout

Could you share a complete manual run including the command you run with
all it's output.
And if it's not clear what output from the command and what you are
typing inn please indicate what you are typing in.

And please refrain using HMTL since HTML formating screws up the format.

--
Kai Stian Olstad

[Andrew Graham]

An example of the chars used for the password entered (but not the actual pass obviously) is-

%9SuperSecretPassword

Here is the exact output below before password entry-

vagrant@openfirehawkserverdev:/vagrant$ keybase pgp encrypt -m 'secret' | keybase pgp decrypt

Please enter your Keybase passphrase to unlock the secret key for:

PGP key andrew_graham <blah...@gmail.com> KJ234K2J3RKJ3B2RKH

Reason: PGP

Decryption:

[Kai Stian Olstad]

On 30.08.2019 08:40, Andrew Graham wrote:
> An example of the chars used for the password entered (but not the
> actual
> pass obviously) is-
> %9SuperSecretPassword
>
> Here is the exact output below before password entry-
> vagrant@openfirehawkserverdev:/vagrant$ keybase pgp encrypt -m 'secret'
> |
> keybase pgp decrypt
> Please enter your Keybase passphrase to unlock the secret key for:
> PGP key andrew_graham <blah...@gmail.com> KJ234K2J3RKJ3B2RKH
>
> Reason: PGP
> Decryption:

Well you are still posting HTML and this output is different from your
first, so which is it.
In your first post you had this on one line

Reason: PGP Decryption:


And now it's on two lines like so

Reason: PGP
Decryption:


This is a huge different for expect and the reason you should always
post text only, HTML doesn't add any value, just confusion.


--
Kai Stian Olstad

[Andrew Graham]

Apologies.  I'm not clear on how to past a code block in here.  the shell output is being interpreted as html.  I'll just paste a screenshot for good measure!  ignore the leading \ from previous line.

[Kai Stian Olstad]

On 31.08.2019 13:56, Andrew Graham wrote:
> Apologies. I'm not clear on how to past a code block in here. the shell
> output is being interpreted as html.

If you are using Gmail web UI, you can send text only mail by pressing the three vertical dots and check "Plain text mode".

> I'll just paste a screenshot for good
> measure! ignore the leading \ from previous line.

A response like so should work.

responses:
Decryption: <password>

If not, it could be that is printing to stderr instead of stdout.
Another thin is you could turn up the verbose level by running

ANSIBLE_STDOUT_CALLBACK=debug ansible-playbook -vvv playbook.yml


--
Kai Stian Olstad

[Andrew Graham]

Apologies, I don't see the 3 dots you mention available to me in the google groups web ui. 

I pasted an image of the ansible error above which suggests that it is stdout output.  The suggested approach of just using this didn't work-

responses:

Decryption: "{{ keybase_secret }}"

[Zolvaring]

Looking at your posted output, I'm wondering if "Reason: PGP Decryption:" isn't the line you get after you enter a password? I noticed the extra line and nothing else on the cli to indicate a response. I don't fully understand the pexpect regex implementation here as it seems weird to me whenever I touch expect, but generally capturing as little as possible works. Instead of ".*Please enter your keybase passphrase to unlock the secret key for.*" I would try "key for" and if you need something more specific then work backwards from there.

There's the bizarre and unfortunate chance that it's also using the actual printed username as part of the prompt but I'm assuming that's not so, otherwise that'd be harder.

[Andrew Graham]

I have a workaround instead of ansible expect.  it looks like one of two issues below. I used autoexpect to help create a script instead of using ansible expect module.

two things of note was the white space required after the :

and that keybase has two possible outcomes.  if not initialized, the prompts are as we discussed.  if it has been initialized, then there will be eof, and the output will be printed.

These are the two scripts I used to do this considering each condition.

https://github.com/firehawkvfx/firehawk/blob/dev/scripts/pgp-expect.sh

https://github.com/firehawkvfx/firehawk/blob/dev/scripts/pgp-decrypt.sh

I would still love to figure out how to use ansible expect instead of these scripts to handle both conditions though, if it isn't capable of doing it then it might be raised as a bug.