Qrexec - Monero-Wallet-WS connection via Monerod-WS (Daemon)

[ogba...@gmail.com]

I have set two AppVMs and one TemplateVM for (split?) isolation of Monero. The Daemon has networking enabled and the GUI (monero-wallet-ws) does not. Qrexec seem to make this setup possible. I, however, am having trouble connecting to node.moneroworld.com:18089, let alone zdhkwneu7lfaum2p.onion:18099.

I am following the latest qubes-whonix monero guide from u/0xB44EFD8751077F97. https://github.com/0xB44EFD8751077F97/guides/blob/master/qubes-whonix-guides/monero/monero-wallet-daemon-isolation.md

Using command: 'sudo systemctl status ....' within monerod-ws, I can verify Daemon is running. It is set to start at boot.

tried sudo systemctl daemon-reloadtried sudo systemctl restart monerod-mainnet.service (in /rw/usrlocal/etc/qubes-rpc/monerod-mainnet.service:removed "fork" from Type, replaced with "simple"removed "PID" mentionsnow service file is identical to u/qubenix: https://github.com/monero-project/monero/issues/4468#issuecomment-429671939)

His tweaks seem to align with Monero's. https://github.com/monero-project/monero/blob/master/utils/systemd/monerod.service

Several different boards (whonix, tails, stackexchange, etc) provide instructions for full/local blockchain nodes only. As I gather I believe there may be some disargeement with IP binds, ports, and tor within my adjustments for remote.

In monero-wallet-ws; the /rw/config/rc.local contains: 'socat TCP-LISTEN:18081.fork.bind=127.0.0.1 EXEC:"qrexec-client-vm monerod-ws whonix.monerod-mainnet" '

In monerod-ws; the /rw/usrlocal/etc/qubes-rpc/whonix.monerod-mainnet contains: 'socat STDIO TCP:localhost:18081'

In monero-wallet-ws, both commands: 'torsocks monero-wallet-gui' and just 'monero-wallet-gui' run fine but the Network Status displayed remains "disconnected". The error "Wallet is not connected to daemon" is visble via GUI. Trying node.moneroworld.com and its port does nothing within the GUI except turn up an error in this VM's terminal.

"ERROR default src/wallet/api/utils.cpp:46 ... Host not found (non-authoritative), try again later"

tried --daemon-address, error "Unknown Option"tried --daemon-host, error "Unknown Option"

FYI -- the daemon, Monerod, is running but 'systemctl status' throws up blockchain errors. The command: 'sudo -u monerod monerod status' displays height numbers and percentage of "4.8% on mainnet." I am of the impression I shouldn't be downloading the blockchain or needing free diskspace for remote nodes. But the "uptime" agrees daemon is in fact running.

Does anyone know how to mend the connection between Monero's daemon VM and isolated wallet VM? Qubes 3.2.1 is what I have currently. Wifi is up.

P.S. If any logs are needed, I would love to know what should be omitted or obsfuscated. I am very recent to all this, qubes and linux in general.

P.P.S I want to post within this community before Monero's because I suspect qrexec and a misunderstanding on my part is at play. Please let me know if I am mistaken.

Thank you

[OGBaby]

On Monday, December 10, 2018 at 8:29:39 AM UTC-6, OGBaby wrote:
>
> Using command: 'sudo systemctl status ....' within monerod-ws, I can verify Daemon is running. It is set to start at boot.
>
> tried sudo systemctl daemon-reloadtried sudo systemctl restart monerod-mainnet.service (in /rw/usrlocal/etc/qubes-rpc/monerod-mainnet.service:removed "fork" from Type, replaced with "simple"removed "PID" mentionsnow service file is identical to u/qubenix: https://github.com/monero-project/monero/issues/4468#issuecomment-429671939)
>
>
>

Apologies. I'm seeing some formmating issues with this post and can't seem to edit.

Using command: 'sudo systemctl status ....' within monerod-ws, I can verify Daemon is running. It is set to start at boot.

- tried sudo systemctl daemon-reload

- tried sudo systemctl restart monerod-mainnet.service

(in /rw/usrlocal/etc/qubes-rpc/monerod-mainnet.service:

- removed "fork" from Type, replaced with "simple"

- removed "PID" mentions

- now service file is identical to u/qubenix: https://github.com/monero-project/monero/issues/4468#issuecomment-429671939)

This part should as so....

- In monero-wallet-ws; the /rw/config/rc.local contains: 'socat TCP-LISTEN:18081.fork.bind=127.0.0.1 EXEC:"qrexec-client-vm monerod-ws whonix.monerod-mainnet" '

- In monerod-ws; the /rw/usrlocal/etc/qubes-rpc/whonix.monerod-mainnet contains: 'socat STDIO TCP:localhost:18081'


And the last part...

- tried --daemon-address, error "Unknown Option"
- tried --daemon-host, error "Unknown Option"

[OGBaby]

Edit: Some missing info.

Monero v0.13.0.4
GUI v0.13.0.4

[qubenix]

ogba...@gmail.com:

> I have set two AppVMs and one TemplateVM for (split?) isolation of Monero. The Daemon has networking enabled and the GUI (monero-wallet-ws) does not. Qrexec seem to make this setup possible. I, however, am having trouble connecting to node.moneroworld.com:18089, let alone zdhkwneu7lfaum2p.onion:18099

I also use this setup without any problems. I did have to modify the
`systemd` unit after the recent Monero update[1].

Try to adjust your file on the template to match what I have in the
link, restart the daemon, and see if that helps your connectivity
issues. The major changes are adding `/usr/bin/torsocks` to the
`ExecStart` and changing the `Type` to `simple`.

As far as connecting to remote nodes, are you doing that in the daemon
or the wallet vm? The wallet vm has no networking and so will never be
able to make a connection to anything besides the daemon vm.

[1]:
https://github.com/monero-project/monero/issues/4468#issuecomment-429671939

--
qubenix

CODE PGP: FE7454228594B4DDD034CE73A95D4D197E922B20
EMAIL PGP: 96096E4CA0870F1C5BAF7DD909D159E1241F9C54
IRC OTR: DFD1DA35 D74E775B 3E3DADB1 226282EE FB711765

[OGBaby]

Greatly appreciate your help Qubenix.

I can confirm my systemd file (/lib/systemd/system/monerod-mainnet.service) matches yours now. I have removed the following to do so:

ConditonPathExists=/var/run/qubes-service/monerod-mainnet
After=qubes-sysinit.service

It is now identical to yours. Thank you.

Unsure if 'sudo systemctl restart monerod-mainnet.service' is sufficient so I've stopped the daemon and rebooted instead.

I did figure the Node settings wouldn't work via the GUI. I've tried '--daemon-address' as well as '--daemon-host' but it didn't work via monero-wallet-ws terminal. I've just attempted to pass these commands wihin the Daemon:

sudo -u monerod monerod --daemon-address node.moneroworld:18089

I'm sure that isn't a valid input for option. How would you connect to remote nodes within the Daemon vm?

I've only found the '--bootstrap-daemon-address' command listed here: https://monerodocs.org/interacting/monerod-reference/

[qubenix]

OGBaby:

> Greatly appreciate your help Qubenix.
>
> I can confirm my systemd file (/lib/systemd/system/monerod-mainnet.service) matches yours now. I have removed the following to do so:
>
> ConditonPathExists=/var/run/qubes-service/monerod-mainnet
> After=qubes-sysinit.service

That was not needed. Those will have no effect on your connectivity.
They just give you the ability to turn the service on/off from dom0 via
`qvm-service`. I don't remember why I omitted them from the Github post.

>
> It is now identical to yours. Thank you.
>
> Unsure if 'sudo systemctl restart monerod-mainnet.service' is sufficient so I've stopped the daemon and rebooted instead.
>
> I did figure the Node settings wouldn't work via the GUI. I've tried '--daemon-address' as well as '--daemon-host' but it didn't work via monero-wallet-ws terminal. I've just attempted to pass these commands wihin the Daemon:
>
> sudo -u monerod monerod --daemon-address node.moneroworld:18089
>
> I'm sure that isn't a valid input for option. How would you connect to remote nodes within the Daemon vm?
>
> I've only found the '--bootstrap-daemon-address' command listed here: https://monerodocs.org/interacting/monerod-reference/
>

The best thing for you to do is:

1. Start the daemon's AppVM fresh.
2. In a terminal on the daemon's vm do: `sudo tail -f
/home/monerod/.bitmonero/bitmonero.log`.
3. See what is happening in the log and report back. You can paste the
log to a site like paste.debian.net or similar if you have trouble
making sense of it.
a. **Only if the log is sitting completely dormant** do: `sudo
systemctl status monerod` and then: `sudo journalctl -xe`.
b. Report back the result of those two.

[OGBaby]

On Monday, December 10, 2018 at 5:16:01 PM UTC-6, qubenix wrote:
> The best thing for you to do is:
>
> 1. Start the daemon's AppVM fresh.
> 2. In a terminal on the daemon's vm do: `sudo tail -f
> /home/monerod/.bitmonero/bitmonero.log`.
> 3. See what is happening in the log and report back. You can paste the
> log to a site like paste.debian.net or similar if you have trouble
> making sense of it.
> a. **Only if the log is sitting completely dormant** do: `sudo
> systemctl status monerod` and then: `sudo journalctl -xe`.
> b. Report back the result of those two.
>

No problem.

The Monerod logs: paste.debian.net/hidden/5a996a7a

[Btw, I was interrupted during my session. I've marked #2 to mark the new logs upon restarting vm's fresh]

Also, could you clarify if this setup will leave the GUI as a view-only? Considering it's offline.

[OGBaby]

P.S. I see a lot of stacktrace, something I believe you've mentioned elsewhere in the past with torsocks, and plenty of blockchain errors.

Hopefully you can help me digest it all. Have to admit they all appear as walls of text at this stage. But I'm seeing what may be something to do with the initial blockchain sync?

[qubenix]

OGBaby:

From the logs: 'Free space is below 1 GB on /hom'.

Tells me your out of space on the daemon vm. Increase your private
storage size to at least 75G. You can do it from the qube manager gui,
or from a dom0 terminal with: `qvm-volume resize monerod-ws:private 75G`.

The monero gui will work 100% once your daemon is synchronized, from the
logs you are at 4% complete. You cannot use a remote node with it
because the wallet vm has no connection to the outside world, set it to
use a local node.

[OGBaby]

Thanks Qubenix,

Wouldn't a fully synced daemon mean the blockchain is downloaded onto disk? I was under the impression a benefit to remote node over local was to entirely avoid downloading the blockchain.

With the bootstrap-daemon address attempting to alleviate users from becoming discouraged or impatient during the long-wait by temporarily letting them use remote nodes while it syncs local. Am I misinterpretating Monero's remote node all together?

I can definitely increase the vm but there isn't 75G available. Please let me know your thoughts.

[qubenix]

OGBaby:

Using a remote node is a completely different scenario than what this
guide is doing. This guide is the safest way to use Monero that requires
no trust in third parties, using a remote node is near to the least
safest way and relies on third parties to give you information about the
state of the blockchain and thus your funds.

If you want to use a remote node I would suggest making a new vm, with
networking, and use that as your remote node wallet. You lose a ton of
security that way, as now your wallet is connected to the internet.

[OGBaby]

Wow, I had to reread a few materials first but It finally clicked. Security is the sole scope of this method. I had meshed a couple concepts together somewhere along the line. There's been a lot of learning (using linux/terminal included).

I've managed to successfully download the entire blockchain.

Just got the "You are now synchronized with the network. You may start monero-wallet-cli." message via status.

Monerod is running. The Height displays as "1727318/1727318 (100%) on mainnet, not mining"

The log: 'sudo tail -f /home/monerod/.bitmonero/bitmonero.log': "Synced 1727379/1727379 -- Synchronized Ok"

Only 1 error: Torsocks. Specifically, "connection refused to Tor Socks (in sock5_recv_connect_reply() at socks5.c.549)"

P.S. Must I wrap 'monero-wallet-gui' with "torsocks"? This is what I'm doing now.

Created the wallet and made time to write down all keys & seed. Connected to Local.

The GUI, I assume, will remain disconnected. You mentioned above that things should be handled within the Daemon vm, would you mind filling me in? (i.e confirming xmr recieved, sending xmr to address)

Or... does monero-wallet-cli commands take it from here? As you can my tell my fogginess is with the addition of the GUI.

I've also taken a look at './monerod --help'

The infomation online is a bit scattered however I completely understand if you'd rather point me some links or resource instead. Any more guidance is appreciated.

[qubenix]

OGBaby:

> Wow, I had to reread a few materials first but It finally clicked. Security is the sole scope of this method. I had meshed a couple concepts together somewhere along the line. There's been a lot of learning (using linux/terminal included).
>
> I've managed to successfully download the entire blockchain.
>
> Just got the "You are now synchronized with the network. You may start monero-wallet-cli." message via status.
>
> Monerod is running. The Height displays as "1727318/1727318 (100%) on mainnet, not mining"
>
> The log: 'sudo tail -f /home/monerod/.bitmonero/bitmonero.log': "Synced 1727379/1727379 -- Synchronized Ok"
>

Congrats! You're well on your way.

>
> Only 1 error: Torsocks. Specifically, "connection refused to Tor Socks (in sock5_recv_connect_reply() at socks5.c.549)"
>
>
> P.S. Must I wrap 'monero-wallet-gui' with "torsocks"? This is what I'm doing now.

No, only the daemon needs `torsocks`. The wallets (cli and gui) just
make a local connection to the daemon so no `torsocks` needed for them.

>
> Created the wallet and made time to write down all keys & seed. Connected to Local.
>
> The GUI, I assume, will remain disconnected. You mentioned above that things should be handled within the Daemon vm, would you mind filling me in? (i.e confirming xmr recieved, sending xmr to address)

I'm not sure what you mean by "disconnected", the gui will connect to
the daemon to retrieve information about the state of the blockchain to
derive your balance. However, it will not connect to the internet
directly to help protect against remote attackers.

I haven't used the gui in about a year, but if you look at left sidebar
there should be some indication that you are connected to the daemon and
it will also show your sync progress (the wallet also needs to sync your
balances, but it only takes a few minutes to an hour depending on hardware).

This might be helpful for you:
https://github.com/monero-ecosystem/monero-GUI-guide/blob/master/monero-GUI-guide.md.

>
> Or... does monero-wallet-cli commands take it from here? As you can my tell my fogginess is with the addition of the GUI.
>

You can use either the gui or the cli with the same wallet. It's really
just a matter of preference.

>
> I've also taken a look at './monerod --help>

That will give you the daemon's options. If you want the cli options you
need to run `monero-wallet-cli` from the wallet vm and the type `help`
at the prompt.

>> The infomation online is a bit scattered however I completely
understand if you'd rather point me some links or resource instead. Any
more guidance is appreciated.
>

Try here for further reading:
https://ww.getmonero.org/resources/user-guides/.

[OGBaby]

Great info!

I wasn't able to get 'torsocks ./monerod' running but found these options browsing:

1. 'DNS_PUBLIC=tcp torsocks monerod --p2p-bind-ip 127.0.0.1 --no-igd'

2. 'DNS_PUBLIC=tcp TORSOCKS_ALLOW_INBOUND=1 torsocks monerod --p2p-bind-ip 127.0.0.1 --no-igd'

I just wanted to run these by you or the qubes community to make sure I don't accidently mess with any default settings. (specifically the p2p option which we've removed from the systemd file).

'torsocks monerod' or 'torsocks ./monerod' and the ones I found above all attempt to resync the blockchain. Is this normal behavior?

[OGBaby]

By the way, yes the dicconected status is shown on the lower left via the GUI. Now I know when the torsocks connection is finally accepted, I will be able to see it change to connected and use the GUI normally.

[qubenix]

OGBaby:

> Great info!
>
> I wasn't able to get 'torsocks ./monerod' running but found these options browsing:
>

Why or where would you intend to run these? The monerod daemon should be
running in the background as soon as the vm is started.

You should never interact with the daemon like this. You should use
`systemctl` to start/stop the daemon. You can check if the daemon is
running by running `sudo systemctl status monerod` or using `tail` on
the debug.log (I gave you instructions in a previous mail on that).

>
> 1. 'DNS_PUBLIC=tcp torsocks monerod --p2p-bind-ip 127.0.0.1 --no-igd'
>
> 2. 'DNS_PUBLIC=tcp TORSOCKS_ALLOW_INBOUND=1 torsocks monerod --p2p-bind-ip 127.0.0.1 --no-igd'
>

You don't need these. They're for setups not as well designed as Whonix,
or if you're offering a remote node onion service.

> I just wanted to run these by you or the qubes community to make sure I don't accidently mess with any default settings. (specifically the p2p option which we've removed from the systemd file).>
> 'torsocks monerod' or 'torsocks ./monerod' and the ones I found above all attempt to resync the blockchain. Is this normal behavior?
>

As I said, don't run the daemon from the command line. `systemd` is
handling it already.

[OGBaby]

True, I figured as much. I'll just use the debug/start commands from now on. However the wallet should have a connection but there isn't one. Local node is "configured" but keep in mind the GUI's network status is "disconnected".

I'm not seeing what else could be wrong. Torsocks' General SOCKS server failure is the only error I have at the moment. When looking up the torsocks error (given in with status command), these and few other methods were thrown around. I will not manipulate the daemon any futher but I can confirm the systemd file looks good.

Everything else seem to be in place and running.

[OGBaby]

The last line of the bitmonero.log says: ERROR blockchain src/cryptonote_core/nlockchain.cpp:3728 Exception in cleanup_handle_incoming_blocks:Failed to commit a transaction to the db: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid.

[qubenix]

OGBaby:

> The last line of the bitmonero.log says: ERROR blockchain src/cryptonote_core/nlockchain.cpp:3728 Exception in cleanup_handle_incoming_blocks:Failed to commit a transaction to the db: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid.
>

That is a new issue or you are out of space again (I think I remember
seeing that in one of your original mails iirc).

Either way it's not specific to your setup or to Qubes. Best to check on
Monero channels (reddit, irc, etc).

[OGBaby]

Hey Qubenix, I have an update for you and anyone curious. The MDB_BAD_TXN error has been resolved. For anyone wondering this error most likely mean your blockchain was interrupted during sync and is now corrupt.

The solution advised by Monero channels is to delete the blockchain data ("lmdb") amd resync.

I was recommended to include the flag '--db-sync-mode=safe' to monerod upon startup to protect against unexpected interruptions or reboots during sync.

You should be able to locate the blockchain within /home/user/.bitmonero/

Keep in mind .bitmonero is a hidden directory.

After the resync everything regarding the daemon seem to be in place and running. I will repost the most recent logs and provide fresh ones as well.

I am being redirected back here since that particular monerod error appear to be resolved, there may be a minor issue with the separation setup of wallet and daemon VMs. The lower-left Network Status is 'disconnected' and of course "Wallet is not connected to daemon" is still visible above.

If there is anything specific I should verify and confirm, I would gladly take a look.

Thanks everyone

Monerod status (previous)
https://pastebin.com/v95gn1aB

Monerod Bitmonero.log (previous)
https://pastebin.com/3mzhuSHt

Monerod Status (fresh)
https://pastebin.com/XwAy91dh

Monerod Bitmonero.log (fresh)
https://pastebin.com/CHSHg9Zz

Hopefully it's just something I'm overlooking.

[qubenix]

OGBaby:

Your daemon issues seem solved, so that's a step in the right direction.
Can you come on irc to resolve this (either freenode or oftc)? I suspect
we're going to be doing a lot of back and forth to find out which step
in the guide wasn't completed correctly.

If you want to, you can go back through the guide first and pay special
attention that you have done the following correctly:

1. Set up qrexec policy (/etc/qubes-rpc/policy/whonix.monerod-mainnet)
in dom0. Guide section 1.4.
2. Set up qrexec file
(/rw/usrlocal/etc/qubes-rpc/whonix.monerod-mainnet) on the daemon vm.
Guide section 3.2.
3. Set up /rw/config/rc.local on wallet vm. Guide section 4.2.

If all of that is done correct, the gui is set to use a local node, and
you still have no connection then we will need to do a lot of back and
forth with running commands in vms and telling me the output.

[OGBaby]

Perfect.

Sound good to me. I'm going to quickly run through your checklist first to re-verify everything then I'll get on the IRC.

I've never been on either but I'll send a quick update here when I've got it going.

[OGBaby]

Created a Nick for OFTC. A few users appeared to be in the room. There was a message in #qubes similar to "Nothing to see here, go to freenode"

So I'm using the freenode irc.

I can confirm all the steps match the guide including your changes to the systemd file.