a few things about salt
277 views
Skip to first unread message
john.david.r.smith
Jan 16, 2017, 5:24:30 PM1/16/17
to qubes-users
hi.
i played around with salt and noticed a few things/ have some questions:
1)
even when some states fail for some vm, the cli tool displays ok. it
would be better, if it displayed error in case of an error (some errors
are displayed).
2)
what happens if i have not enough memory to start a management vm? is
there something to counter this problem? (e.g. detect this case, limit
the number of management vms started at the same time and retrying the
action)
3)
is there a plan to add some way of configuring dom0 (including service
permissions, default vm/template/etc, changing copy&paste shortcuts...)
4)
how much of the salt interface is subject to change?
5)
are there plans to add some functionality to the interface?
6)
currently i really don't like the way the configuration works.
i have a top file where i execute some states for dom0
these states create and configure my vms.
then in some top files i choose some vms and configure them again (but
this time it is some config i am doing in the domu).
so it kind of looks like this:
top.top
-----
base:
dom0:
- create-cfg-vm1
vm1:
- some-cfg-in-domu
----
now i have two layers of configuration (in top and sls).
for some config stuff i have to change a sls and for other i have to
change the top
is there a plan to change this?
e.g. some kind of virtual minions?
i would like to write something like this:
top.top
-----
base:
dom0:
- copy-sequence.Strg-Alt-Shift-C
vm1:
- create #this affects dom0
- color.red #this affects dom0
- netvm.sys-tor #this affects dom0
- mail #this affects domU
----
then i could see all my domU config in the top file.
i currently hacked something but this only works in a sls file and for
dom0 config (but has this style of syntax)
i am currently looking whether i can do the same in a top file (but i
doubt it, since there is no templating in top files)
concerning this i also have some other questions
how is the order of execution?
will dom0 always be executed before any domU is started?
when are the files for domU read?
after dom0 is configured? (then i could write state files during dom0
configuration)
(if you want i can upload my scripts when i am done)
7)
is there some documentation on the custom salt modules added by qubes?
(i may want to extend them)
8)
is there some way to execute some dom0 scripts after configuration of
domu? (e.g. trim-template)
9)
the fedora-24-min template can't really be configured with salt.
there is the package file missing.
after i installed the package i still got an error: "Target
'fedora-24-min' did not return any data, probably due to an error. exit
code 20"
any idea how i could fix this?
or where i could get more data. e.g. are there some salt logs in some vm
(e.g. in the domU)?
there probably are files in the management vm, but this vm gets deleted.
is there an option to stop the deletion of the management vm?
- john
i played around with salt and noticed a few things/ have some questions:
1)
even when some states fail for some vm, the cli tool displays ok. it
would be better, if it displayed error in case of an error (some errors
are displayed).
2)
what happens if i have not enough memory to start a management vm? is
there something to counter this problem? (e.g. detect this case, limit
the number of management vms started at the same time and retrying the
action)
3)
is there a plan to add some way of configuring dom0 (including service
permissions, default vm/template/etc, changing copy&paste shortcuts...)
4)
how much of the salt interface is subject to change?
5)
are there plans to add some functionality to the interface?
6)
currently i really don't like the way the configuration works.
i have a top file where i execute some states for dom0
these states create and configure my vms.
then in some top files i choose some vms and configure them again (but
this time it is some config i am doing in the domu).
so it kind of looks like this:
top.top
-----
base:
dom0:
- create-cfg-vm1
vm1:
- some-cfg-in-domu
----
now i have two layers of configuration (in top and sls).
for some config stuff i have to change a sls and for other i have to
change the top
is there a plan to change this?
e.g. some kind of virtual minions?
i would like to write something like this:
top.top
-----
base:
dom0:
- copy-sequence.Strg-Alt-Shift-C
vm1:
- create #this affects dom0
- color.red #this affects dom0
- netvm.sys-tor #this affects dom0
- mail #this affects domU
----
then i could see all my domU config in the top file.
i currently hacked something but this only works in a sls file and for
dom0 config (but has this style of syntax)
i am currently looking whether i can do the same in a top file (but i
doubt it, since there is no templating in top files)
concerning this i also have some other questions
how is the order of execution?
will dom0 always be executed before any domU is started?
when are the files for domU read?
after dom0 is configured? (then i could write state files during dom0
configuration)
(if you want i can upload my scripts when i am done)
7)
is there some documentation on the custom salt modules added by qubes?
(i may want to extend them)
8)
is there some way to execute some dom0 scripts after configuration of
domu? (e.g. trim-template)
9)
the fedora-24-min template can't really be configured with salt.
there is the package file missing.
after i installed the package i still got an error: "Target
'fedora-24-min' did not return any data, probably due to an error. exit
code 20"
any idea how i could fix this?
or where i could get more data. e.g. are there some salt logs in some vm
(e.g. in the domU)?
there probably are files in the management vm, but this vm gets deleted.
is there an option to stop the deletion of the management vm?
- john
Marek Marczykowski-Górecki
Jan 16, 2017, 10:29:30 PM1/16/17
to john.david.r.smith, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On Mon, Jan 16, 2017 at 11:24:13PM +0100, john.david.r.smith wrote:
> hi.
> i played around with salt and noticed a few things/ have some questions:
> 1)
> even when some states fail for some vm, the cli tool displays ok. it would
> be better, if it displayed error in case of an error (some errors are
> displayed).
Can you provide example error which wasn't detected? Regardless of the
result, output is logged to /var/log/qubes/mgmt-*.log in dom0.
> 2)
> what happens if i have not enough memory to start a management vm? is there
> something to counter this problem? (e.g. detect this case, limit the number
> of management vms started at the same time and retrying the action)
Currently number of concurrent instances is hardcoded to 4. We'll add an
option for that, but probably not soon. It's here:
https://github.com/QubesOS/qubes-mgmt-salt/blob/master/qubessalt/__init__.py#L191
This argument is not provided by /usr/bin/qubesctl, so default value (4)
is used. If you want, you can add it here:
https://github.com/QubesOS/qubes-mgmt-salt/blob/master/qubesctl#L79
In any case, you can retry, using --targets option and probably also
- --skip-dom0. So to retry only 'debian-8' configuration, execute:
sudo qubesctl --skip-dom0 --targets=debian-8 state.highstate
> 3)
> is there a plan to add some way of configuring dom0 (including service
> permissions, default vm/template/etc, changing copy&paste shortcuts...)
You can use any of standard salt module, there are plenty of them.
For example file.prepend (of file.managed for more strict cases) is
enough to configure /etc/qubes-rpc/policy/...
> 4)
> how much of the salt interface is subject to change?
Not much. While working on Qubes 4.0 we try to keep compatibility as
much as possible. But some minor changes probably will be unavoidable -
we'll list them in release notes.
> 5)
> are there plans to add some functionality to the interface?
Yes, "qvm" module will be extended for new features in Qubes 4.0. Is it
what you've asked about?
We were thinking about some own module for qrexec policy, but since
default 'file' module cover most of the cases, we've abandoned that
idea.
I think there is currently no sane way to setup global defaults (other
than cmd.run: qubes-prefs ...). So, we'll work on that too.
And the last sentence is exactly the reason why it's tricky to have it
in one place. Rendering sls files (may) require getting data (grains) from
target system and we don't want to parse that data from VM in dom0.
To limit attack surface. So, we can't render sls for VMs in dom0, we
need to decide what goes where at 'top' files level.
I think the only think that can be improved here, is some "automatic"
creation of VMs mentioned in top file - something like you've described
above. But it's tricky to do it, while keeping flexibility of salt...
Using valid salt syntax like yours, to achieve different effect looks
like asking for troubles. If going that way, IMO it would be better to
have something that isn't valid salt syntax here and have a pre-processor
script to create actual salt configuration.
> concerning this i also have some other questions
>
> how is the order of execution?
> will dom0 always be executed before any domU is started?
Yes. In particular you can create VMs using states for dom0, just to
have them configured a moment later using states for VM.
> when are the files for domU read?
> after dom0 is configured? (then i could write state files during dom0
> configuration)
Yes, those files are loaded just before configuring VM.
> (if you want i can upload my scripts when i am done)
> 7)
> is there some documentation on the custom salt modules added by qubes? (i
> may want to extend them)
Yes, see links at the end of https://www.qubes-os.org/doc/salt/
> 8)
> is there some way to execute some dom0 scripts after configuration of domu?
> (e.g. trim-template)
Currently no.
> 9)
> the fedora-24-min template can't really be configured with salt.
> there is the package file missing.
> after i installed the package i still got an error: "Target 'fedora-24-min'
> did not return any data, probably due to an error. exit code 20"
The important thing is what is your default template - it is used for
that intermediate VM from where target VMs are configured. Is it also
fedora-24-min?
salt-ssh requirements in the target VM are really minimal - I think any
shell + python should be enough. For me it works, but it's possible that
my minimal template is no longer such minimal...
Ok, tried on fresh minimal template and found the problem: sudo
So, packages needs to be installed:
- file
- sudo
And also cleanup ~/QubesIncoming/disp-mgmt-* in the template, because it
isn't removed after failed run...
> any idea how i could fix this?
> or where i could get more data. e.g. are there some salt logs in some vm
> (e.g. in the domU)?
/var/log/qubes/mgmt-*.log
That's the only thing that is preserved.
> there probably are files in the management vm, but this vm gets deleted.
> is there an option to stop the deletion of the management vm?
There is no option for that, but you can suspend qubesctl execution
(Ctrl-Z) to prevent that. You need to do that when you see that target
VM is being starting (at this moment dom0 have already send all required
data and all the execution is in management VM).
The above I've debugged exactly this way:
1. Ctrl-Z on qubesctl.
2. Open terminal in disp-mgmt-fedora-24-minimal.
3. Look at /etc/qubes-rpc/qubes.SaltLinuxVM - this is what is executed.
4. Get the last two lines and execute them, fix problems, repeat.
Those lines:
export PATH="/usr/lib/qubes-vm-connector/ssh-wrapper:$PATH"
salt-ssh "$target_vm" $salt_command
Of course adjust $target_vm (fedora-24-minimal) and $salt_command
(state.highstate).
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJYfY+RAAoJENuP0xzK19csEHcIAIQvqN9CldAxkY4OVUBBYB3b
C1tMuiYkGPEzyfcE+5J6QaNzGXOXf2i/lm+7HUKRnMN42bmYdLLVCFrHUcxn09lD
uDGid2kwOkhX17RhZf5FyMhhXETIutQGDw/gJK9bUl8k8gczYjY4ek1RGPSFnjYe
3T0BbUFKNeiJzD3i1hCNj5T2xOTpVcURWuIj4AuABl7QESg+OuEYJg2e4cJW+Ibl
8obVMLZcZ/rLT8IMzjHB1IOzuxI8UXiT3Qg4SLE8xmV+z8lqYI2wK3atCJfmWuYn
ZfgTuhxg9LPD/b8ILMmJQfykENefqBR549INpL8b1WdXqQZK7JME3Xa7veZkehQ=
=yUaE
-----END PGP SIGNATURE-----
Hash: SHA256
On Mon, Jan 16, 2017 at 11:24:13PM +0100, john.david.r.smith wrote:
> hi.
> i played around with salt and noticed a few things/ have some questions:
> 1)
> even when some states fail for some vm, the cli tool displays ok. it would
> be better, if it displayed error in case of an error (some errors are
> displayed).
result, output is logged to /var/log/qubes/mgmt-*.log in dom0.
> 2)
> what happens if i have not enough memory to start a management vm? is there
> something to counter this problem? (e.g. detect this case, limit the number
> of management vms started at the same time and retrying the action)
option for that, but probably not soon. It's here:
https://github.com/QubesOS/qubes-mgmt-salt/blob/master/qubessalt/__init__.py#L191
This argument is not provided by /usr/bin/qubesctl, so default value (4)
is used. If you want, you can add it here:
https://github.com/QubesOS/qubes-mgmt-salt/blob/master/qubesctl#L79
In any case, you can retry, using --targets option and probably also
- --skip-dom0. So to retry only 'debian-8' configuration, execute:
sudo qubesctl --skip-dom0 --targets=debian-8 state.highstate
> 3)
> is there a plan to add some way of configuring dom0 (including service
> permissions, default vm/template/etc, changing copy&paste shortcuts...)
For example file.prepend (of file.managed for more strict cases) is
enough to configure /etc/qubes-rpc/policy/...
> 4)
> how much of the salt interface is subject to change?
much as possible. But some minor changes probably will be unavoidable -
we'll list them in release notes.
> 5)
> are there plans to add some functionality to the interface?
what you've asked about?
We were thinking about some own module for qrexec policy, but since
default 'file' module cover most of the cases, we've abandoned that
idea.
I think there is currently no sane way to setup global defaults (other
than cmd.run: qubes-prefs ...). So, we'll work on that too.
in one place. Rendering sls files (may) require getting data (grains) from
target system and we don't want to parse that data from VM in dom0.
To limit attack surface. So, we can't render sls for VMs in dom0, we
need to decide what goes where at 'top' files level.
I think the only think that can be improved here, is some "automatic"
creation of VMs mentioned in top file - something like you've described
above. But it's tricky to do it, while keeping flexibility of salt...
Using valid salt syntax like yours, to achieve different effect looks
like asking for troubles. If going that way, IMO it would be better to
have something that isn't valid salt syntax here and have a pre-processor
script to create actual salt configuration.
> concerning this i also have some other questions
>
> how is the order of execution?
> will dom0 always be executed before any domU is started?
have them configured a moment later using states for VM.
> when are the files for domU read?
> after dom0 is configured? (then i could write state files during dom0
> configuration)
> (if you want i can upload my scripts when i am done)
> 7)
> is there some documentation on the custom salt modules added by qubes? (i
> may want to extend them)
> 8)
> is there some way to execute some dom0 scripts after configuration of domu?
> (e.g. trim-template)
> 9)
> the fedora-24-min template can't really be configured with salt.
> there is the package file missing.
> after i installed the package i still got an error: "Target 'fedora-24-min'
> did not return any data, probably due to an error. exit code 20"
that intermediate VM from where target VMs are configured. Is it also
fedora-24-min?
salt-ssh requirements in the target VM are really minimal - I think any
shell + python should be enough. For me it works, but it's possible that
my minimal template is no longer such minimal...
Ok, tried on fresh minimal template and found the problem: sudo
So, packages needs to be installed:
- file
- sudo
And also cleanup ~/QubesIncoming/disp-mgmt-* in the template, because it
isn't removed after failed run...
> any idea how i could fix this?
> or where i could get more data. e.g. are there some salt logs in some vm
> (e.g. in the domU)?
That's the only thing that is preserved.
> there probably are files in the management vm, but this vm gets deleted.
> is there an option to stop the deletion of the management vm?
(Ctrl-Z) to prevent that. You need to do that when you see that target
VM is being starting (at this moment dom0 have already send all required
data and all the execution is in management VM).
The above I've debugged exactly this way:
1. Ctrl-Z on qubesctl.
2. Open terminal in disp-mgmt-fedora-24-minimal.
3. Look at /etc/qubes-rpc/qubes.SaltLinuxVM - this is what is executed.
4. Get the last two lines and execute them, fix problems, repeat.
Those lines:
export PATH="/usr/lib/qubes-vm-connector/ssh-wrapper:$PATH"
salt-ssh "$target_vm" $salt_command
Of course adjust $target_vm (fedora-24-minimal) and $salt_command
(state.highstate).
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJYfY+RAAoJENuP0xzK19csEHcIAIQvqN9CldAxkY4OVUBBYB3b
C1tMuiYkGPEzyfcE+5J6QaNzGXOXf2i/lm+7HUKRnMN42bmYdLLVCFrHUcxn09lD
uDGid2kwOkhX17RhZf5FyMhhXETIutQGDw/gJK9bUl8k8gczYjY4ek1RGPSFnjYe
3T0BbUFKNeiJzD3i1hCNj5T2xOTpVcURWuIj4AuABl7QESg+OuEYJg2e4cJW+Ibl
8obVMLZcZ/rLT8IMzjHB1IOzuxI8UXiT3Qg4SLE8xmV+z8lqYI2wK3atCJfmWuYn
ZfgTuhxg9LPD/b8ILMmJQfykENefqBR549INpL8b1WdXqQZK7JME3Xa7veZkehQ=
=yUaE
-----END PGP SIGNATURE-----
john.david.r.smith
Jan 17, 2017, 10:40:48 AM1/17/17
to Marek Marczykowski-Górecki, qubes-users
>> 1)
>> even when some states fail for some vm, the cli tool displays ok. it would
>> be better, if it displayed error in case of an error (some errors are
>> displayed).
>
> Can you provide example error which wasn't detected? Regardless of the
> result, output is logged to /var/log/qubes/mgmt-*.log in dom0.
>
i somehow fail to reproduce the case. (i just noticed it when playing
>> even when some states fail for some vm, the cli tool displays ok. it would
>> be better, if it displayed error in case of an error (some errors are
>> displayed).
>
> Can you provide example error which wasn't detected? Regardless of the
> result, output is logged to /var/log/qubes/mgmt-*.log in dom0.
>
around with salt)
there were some states failed inside domu (i think some package
installation)
i will try to reproduce it later.
>> 5)
>> are there plans to add some functionality to the interface?
>
> Yes, "qvm" module will be extended for new features in Qubes 4.0. Is it
> what you've asked about?
> I think there is currently no sane way to setup global defaults (other
> than cmd.run: qubes-prefs ...). So, we'll work on that too.
i have a top file activating a dom0 sls
in this sls i do dom0 config, create vms and configure them (dom0 config
AND domU config).
all domU config is used to generate a generated.top file activating the
correct states for the correct minions.
then everything is in one file (not the top file, but this sls file has
the function of a top file)
the disadvantage would be that i always need to run dom0 to generate up
to date files for my minions. (but in my opinion the advantages beat the
disadvantages)
>> how is the order of execution?
>> will dom0 always be executed before any domU is started?
>
> Yes. In particular you can create VMs using states for dom0, just to
> have them configured a moment later using states for VM.
>
>> when are the files for domU read?
>> after dom0 is configured? (then i could write state files during dom0
>> configuration)
>
> Yes, those files are loaded just before configuring VM.
this (maybe because salt does things like this):
a) copy all files to some cache
b) run dom0 (using the files from the cache)
c) run domU (using the files from the cache)
in this case i would not be able to generate files in b to use in c
>> 8)
>> is there some way to execute some dom0 scripts after configuration of domu?
>> (e.g. trim-template)
>
> Currently no.
>> 9)
>> the fedora-24-min template can't really be configured with salt.
>> there is the package file missing.
>> after i installed the package i still got an error: "Target 'fedora-24-min'
>> did not return any data, probably due to an error. exit code 20"
>
> The important thing is what is your default template - it is used for
> that intermediate VM from where target VMs are configured. Is it also
> fedora-24-min?
> salt-ssh requirements in the target VM are really minimal - I think any
> shell + python should be enough. For me it works, but it's possible that
> my minimal template is no longer such minimal...
> Ok, tried on fresh minimal template and found the problem: sudo
> So, packages needs to be installed:
> - file
> - sudo
>
> And also cleanup ~/QubesIncoming/disp-mgmt-* in the template, because it
> isn't removed after failed run...
>
>> any idea how i could fix this?
>> or where i could get more data. e.g. are there some salt logs in some vm
>> (e.g. in the domU)?
>
> /var/log/qubes/mgmt-*.log
> That's the only thing that is preserved.
>> there probably are files in the management vm, but this vm gets deleted.
>> is there an option to stop the deletion of the management vm?
>
> There is no option for that, but you can suspend qubesctl execution
> (Ctrl-Z) to prevent that. You need to do that when you see that target
> VM is being starting (at this moment dom0 have already send all required
> data and all the execution is in management VM).
>
> The above I've debugged exactly this way:
> 1. Ctrl-Z on qubesctl.
> 2. Open terminal in disp-mgmt-fedora-24-minimal.
> 3. Look at /etc/qubes-rpc/qubes.SaltLinuxVM - this is what is executed.
> 4. Get the last two lines and execute them, fix problems, repeat.
>
> Those lines:
>
> export PATH="/usr/lib/qubes-vm-connector/ssh-wrapper:$PATH"
> salt-ssh "$target_vm" $salt_command
>
> Of course adjust $target_vm (fedora-24-minimal) and $salt_command
> (state.highstate).
i installed file and sudo in fedora-24-minimal, but it did not work.
when debugging i got an error and have no real idea what is going on.
(tmp-base-f24 == fedora-24-minimal) :
# salt-ssh tmp-base-f24 state.highstate
[ERROR ] An un-handled exception from the multiprocessing process
'MultiprocessingProcess-1' was caught:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/utils/process.py", line
644, in _run
return self._original_run()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 418, in handle_routine
stdout, stderr, retcode = single.run()
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 834, in run
stdout, retcode = self.run_wfunc()
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 880, in run_wfunc
opts_pkg = pre_wrapper['test.opts_pkg']() # pylint: disable=E1102
File
"/usr/lib/python2.7/site-packages/salt/client/ssh/wrapper/__init__.py",
line 117, in caller
stdout, stderr, retcode = single.cmd_block()
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 1134, in cmd_block
log.error('ERROR: Failure deploying thin, retrying:
{0}\n{1}'.format(stdout, stderr), stderr, retcode)
File "/usr/lib64/python2.7/logging/__init__.py", line 1193, in error
self._log(ERROR, msg, args, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 310,
in _log
self, level, msg, args, exc_info=exc_info, extra=extra
File "/usr/lib64/python2.7/logging/__init__.py", line 1285, in _log
record = self.makeRecord(self.name, level, fn, lno, msg, args,
exc_info, func, extra)
File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 344,
in makeRecord
exc_info, func)
File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 191,
in __init__
self.colormsg = '%s%s%s' % (cmsg, self.getMessage(), reset)
File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Process MultiprocessingProcess-1:
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in
_bootstrap
self.run()
File "/usr/lib/python2.7/site-packages/salt/utils/process.py", line
644, in _run
return self._original_run()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 418, in handle_routine
stdout, stderr, retcode = single.run()
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 834, in run
stdout, retcode = self.run_wfunc()
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 880, in run_wfunc
opts_pkg = pre_wrapper['test.opts_pkg']() # pylint: disable=E1102
File
"/usr/lib/python2.7/site-packages/salt/client/ssh/wrapper/__init__.py",
line 117, in caller
stdout, stderr, retcode = single.cmd_block()
File "/usr/lib/python2.7/site-packages/salt/client/ssh/__init__.py",
line 1134, in cmd_block
log.error('ERROR: Failure deploying thin, retrying:
{0}\n{1}'.format(stdout, stderr), stderr, retcode)
File "/usr/lib64/python2.7/logging/__init__.py", line 1193, in error
self._log(ERROR, msg, args, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 310,
in _log
self, level, msg, args, exc_info=exc_info, extra=extra
File "/usr/lib64/python2.7/logging/__init__.py", line 1285, in _log
record = self.makeRecord(self.name, level, fn, lno, msg, args,
exc_info, func, extra)
File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 344,
in makeRecord
exc_info, func)
File "/usr/lib/python2.7/site-packages/salt/log/setup.py", line 191,
in __init__
self.colormsg = '%s%s%s' % (cmsg, self.getMessage(), reset)
File "/usr/lib64/python2.7/logging/__init__.py", line 329, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
[ERROR ] Target 'tmp-base-f24' did not return any data, probably due
to an error.
- john
Marek Marczykowski-Górecki
Jan 17, 2017, 10:53:00 AM1/17/17
to john.david.r.smith, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
sls files (like {{ grains['os'] }}). Otherwise salt will render that
using dom0 data, not domU data. Unless you use some escaping...
> > > how is the order of execution?
> > > will dom0 always be executed before any domU is started?
> >
> > Yes. In particular you can create VMs using states for dom0, just to
> > have them configured a moment later using states for VM.
> >
> > > when are the files for domU read?
> > > after dom0 is configured? (then i could write state files during dom0
> > > configuration)
> >
> > Yes, those files are loaded just before configuring VM.
>
> i noticed that, but it could have been possible you do something like this
> (maybe because salt does things like this):
> a) copy all files to some cache
> b) run dom0 (using the files from the cache)
> c) run domU (using the files from the cache)
>
> in this case i would not be able to generate files in b to use in c
> > > 8)
> > > is there some way to execute some dom0 scripts after configuration of domu?
> > > (e.g. trim-template)
> >
> > Currently no.
>
> do you plan to add something like this?
log.error('ERROR: Failure deploying thin, retrying:
And some more helpful message will be also in journalctl of target VM
(tmp-base-f24). This is where I've found missing file and sudo.
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Yl+XOV6/V3oBvWYNymOcvCY+gZqvMMJK9oOYIrmAN4/4NoZLcg+HFZqKi0XGq3ry
PQHRK4kBVUSFIMUm+5eWRi5I096ttDalIqeI3deIED+8fJ1Yh4Ud6SZkCVQxGp0l
xYlC4DuRgcTu860ENfrnn84ucAiCvqvovvHJ1AEvBeSaeSABS9gk1Dtb0Q4ZKqQp
W+x9J+NF04LY9w50XjLwg+lQZPuaWlxPKNt9TwI0JqSk0kO9HDY/PZaLoI4xd+Id
k5WLNRs361ZoRoB1Au4DLP444fy+jYa4pPZGUq6z7hQtOGBVU9bhEdI54FFbr9s=
=Nsu+
-----END PGP SIGNATURE-----
john.david.r.smith
Jan 21, 2017, 6:52:19 PM1/21/17
to Marek Marczykowski-Górecki, qubes-users
>>>> i am currently looking whether i can do the same in a top file (but i doubt
>>>> it, since there is no templating in top files)
>>>
>>> And the last sentence is exactly the reason why it's tricky to have it
>>> in one place.
well it seems we were totally wrong.
>>>> it, since there is no templating in top files)
>>>
>>> And the last sentence is exactly the reason why it's tricky to have it
>>> in one place.
you can put jinja code in your top file.
i already wrote some python module to be able to manage everything from my top file.
after i tested it some more, i will post it here.
(if i ever manage to fix all my salt issues i will probably extend the salt section in the documentation, put that module there and document everything i learned about salt and jinja templates.)
>>> 8)
>>>> is there some way to execute some dom0 scripts after configuration of domu?
>>>> (e.g. trim-template)
>>>
>>> Currently no.
>>
>> do you plan to add something like this?
>
> We don't have such plans, but will accept a patch for this ;)
>
via a salt statement in dom0?
if not it should be possible to do this (just run the current script via cmd.run).
if a state dispatches all minions we could use requires to order states after domu salt configuration.
> The actual error is in the middle of this stack trace:
>
> log.error('ERROR: Failure deploying thin, retrying:
>
> (there is unrelated salt bug in logging code here...)
>
> And some more helpful message will be also in journalctl of target VM
> (tmp-base-f24). This is where I've found missing file and sudo.
it seems i was missing tinyproxy as well.
now it still does not work...
the journalctl contains no really useful information (at least noting i can understand as something useful
the only thing looking like some kind of error was (the test template is a clone of minimal and is called a):
Jan 22 00:25:20 a qubes.VMShell-disp-mgmt-a[1192]: WARNING: Unable to locate current thin version: /var/tmp/.root_62a99a_salt/version.
Jan 22 00:25:22 a qubes.VMShell-disp-mgmt-a[1324]: WARNING: Unable to locate current thin version: /var/tmp/.root_62a99a_salt/version.
the folder '/var/tmp/.root_62a99a_salt' does exist, but it is empty
the journalctl log is attached. (maybe someone with more knowledge can make sense of it)
does the salt-ssh command run some script on the minion i can execute manually line for line so i can (maybe) find the source of the error? (i could try to manually execute all this python code, but this would be very cumbersome and hard to understand)
how much of this execution differs from the default salt? (if nothing really differs i will ask on the salt ml)
@Marek:
were you able to configure a fresh fedora-24-minimal template at the end of your debugging?
Marek Marczykowski-Górecki
Jan 21, 2017, 7:19:05 PM1/21/17
to john.david.r.smith, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
On Sun, Jan 22, 2017 at 12:52:08AM +0100, john.david.r.smith wrote:
> > > > > i am currently looking whether i can do the same in a top file (but i doubt
> > > > > it, since there is no templating in top files)
> > > >
> > > > And the last sentence is exactly the reason why it's tricky to have it
> > > > in one place.
>
> well it seems we were totally wrong.
> you can put jinja code in your top file.
Ah, I totally forgot that. In fact we use that, for generating top file
> > > > > i am currently looking whether i can do the same in a top file (but i doubt
> > > > > it, since there is no templating in top files)
> > > >
> > > > And the last sentence is exactly the reason why it's tricky to have it
> > > > in one place.
>
> well it seems we were totally wrong.
> you can put jinja code in your top file.
based on qubesctl top.enable/top.disable (which just create symlinks in
_tops directory).
> i already wrote some python module to be able to manage everything from my top file.
> after i tested it some more, i will post it here.
> (if i ever manage to fix all my salt issues i will probably extend the salt section in the documentation, put that module there and document everything i learned about salt and jinja templates.)
> > > > 8)
> > > > > is there some way to execute some dom0 scripts after configuration of domu?
> > > > > (e.g. trim-template)
> > > >
> > > > Currently no.
> > >
> > > do you plan to add something like this?
> >
> > We don't have such plans, but will accept a patch for this ;)
> >
>
> how are the minions run?
> via a salt statement in dom0?
> if not it should be possible to do this (just run the current script via cmd.run).
> if a state dispatches all minions we could use requires to order states after domu salt configuration.
(or actually - at most 4 of them simultaneously). Here:
https://github.com/QubesOS/qubes-mgmt-salt/blob/master/qubessalt/__init__.py
and here:
https://github.com/QubesOS/qubes-mgmt-salt/blob/master/qubesctl#L71-L81
> > The actual error is in the middle of this stack trace:
> >
> > log.error('ERROR: Failure deploying thin, retrying:
> >
> > (there is unrelated salt bug in logging code here...)
> >
> > And some more helpful message will be also in journalctl of target VM
> > (tmp-base-f24). This is where I've found missing file and sudo.
>
> ok, i tried around some more.
> it seems i was missing tinyproxy as well.
needed only in your netvm... Or are you saying that tinyproxy
was missing in your netvm?
> now it still does not work...
> the journalctl contains no really useful information (at least noting i can understand as something useful
>
> the only thing looking like some kind of error was (the test template is a clone of minimal and is called a):
>
> Jan 22 00:25:20 a qubes.VMShell-disp-mgmt-a[1192]: WARNING: Unable to locate current thin version: /var/tmp/.root_62a99a_salt/version.
> Jan 22 00:25:22 a qubes.VMShell-disp-mgmt-a[1324]: WARNING: Unable to locate current thin version: /var/tmp/.root_62a99a_salt/version.
> the folder '/var/tmp/.root_62a99a_salt' does exist, but it is empty
This suggests you have not:
> Jan 22 00:25:21 a qrexec-agent[465]: executed root:QUBESRPC
> qubes.Filecopy disp-mgmt-a pid 1254
(...)
> Jan 22 00:25:21 a qrexec-agent[465]: pid 1254 exited with 17
17 is EEXIST (File exists).
Looking at all the troubles this caused, we should fix it somehow -
either automatically remove before uploading the file (as in case of
failure, it isn't removed after that attempt), or upload a file with a
unique name. The later will be safer (do not remove any file without
explicit user consent).
> the journalctl log is attached. (maybe someone with more knowledge can make sense of it)
>
> does the salt-ssh command run some script on the minion i can execute manually line for line so i can (maybe) find the source of the error? (i could try to manually execute all this python code, but this would be very cumbersome and hard to understand)
bash script calling basically one line:
python -c 'eval(base64.b64decode(BASE64_ENCODED_SCRIPT))'
And the BASE64_ENCODED_SCRIPT is
/usr/lib/python2.7/site-packages/salt/client/ssh/ssh_py_shim.py, with
some options (including salt-call cmdline) injected - encoded with
base64.
> how much of this execution differs from the default salt? (if nothing really differs i will ask on the salt ml)
salt-call there. But in your case it fails before this happens.
> @Marek:
> were you able to configure a fresh fedora-24-minimal template at the end of your debugging?
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
rbMBfvsTLMwML268LqD/oQCm7MxjLzrS57OtLItjke2jbgdXsea3hr94Z1A325ZP
H+r9KQDT6EIA4Ur8eX2GaI2xmeBMP9CFo7rLRaROwMUZPMvrSiAD5WM1hwBrg0X1
qxBT5GioG0HvtLOLbBJT8TyGHNFJQtIj1/04tiYzPl6KBnJGsplqH6riup8MXsX7
eu4wfOTvKymmdmx0QkwlBB//rNBkMeSnRFrXCPDtNjLHUSIBTObTlS/GIUxtSCjU
J6mAoCeja8eK/7cpfydNM547qEhfcUR82vb61Zh4B3Nb2wjjdNLeXrMdjzkFLN8=
=9i7Y
-----END PGP SIGNATURE-----
john.david.r.smith
Jan 21, 2017, 7:42:58 PM1/21/17
to Marek Marczykowski-Górecki, qubes-users
>
> Did you really need tinyproxy in the target template? It should be
> needed only in your netvm... Or are you saying that tinyproxy
> was missing in your netvm?
it was red in my fed24 minimal journalctl.
> Did you really need tinyproxy in the target template? It should be
> needed only in your netvm... Or are you saying that tinyproxy
> was missing in your netvm?
after i installed it, it went away.... (currently most things i am doing are kind of blind since i never done any sysadmin stuff before qubes and don't really know what i am doing)
i tried it again without the package and it worked.
this time i only needed to install sudo. (file was not necessary)
>> now it still does not work...
>> the journalctl contains no really useful information (at least noting i can understand as something useful
>>
>> the only thing looking like some kind of error was (the test template is a clone of minimal and is called a):
>>
>> Jan 22 00:25:20 a qubes.VMShell-disp-mgmt-a[1192]: WARNING: Unable to locate current thin version: /var/tmp/.root_62a99a_salt/version.
>> Jan 22 00:25:22 a qubes.VMShell-disp-mgmt-a[1324]: WARNING: Unable to locate current thin version: /var/tmp/.root_62a99a_salt/version.
>
>> the folder '/var/tmp/.root_62a99a_salt' does exist, but it is empty
>
> Have you cleaned QubesIncoming directory after failed attempt?
>
> This suggests you have not:
>> Jan 22 00:25:21 a qrexec-agent[465]: executed root:QUBESRPC
>> qubes.Filecopy disp-mgmt-a pid 1254
> (...)
>> Jan 22 00:25:21 a qrexec-agent[465]: pid 1254 exited with 17
>
> 17 is EEXIST (File exists).
>
> Looking at all the troubles this caused, we should fix it somehow -
> either automatically remove before uploading the file (as in case of
> failure, it isn't removed after that attempt), or upload a file with a
> unique name. The later will be safer (do not remove any file without
> explicit user consent).
>
also it would be nice if the min template would contain the sudo package (or rather all packages required of a minion to be configured via salt), since other users will probably run in the same problems (at least one already has)
especially since salt is supposed to become the default config tool for vms. (at least this is my current goal)
thanks for all your help!
tomorrow i will try to get the last things running and after that i will start writing a guide.
Reply all
Reply to author
Forward
0 new messages