Should I expect to be able to install/use Zun Container Svc on OpenStack?

[Gautam Thaker]

Hi:

I have run the standard openstack profile on emulab and it has worked for me in the past. (Today, however, the profile instructions somehow were just 2 lines and contained no openstack related instructions but I will try it again in the morning.)

My question is this: should I be able to install and use Zun container services of openstack? I am looking at steps at:

https://docs.openstack.org/zun/latest/install/index.html

In short, rather than spinning up VMs I am hoping I can start containers using all the openstack infrastructure/GUI etc.

Any comments appreciated.

Gautam

[David M. Johnson]

On 10/26/23 21:42, Gautam Thaker wrote:
> Hi:
>
> I have run the standard openstack profile on emulab and it has worked
> for me in the past. (Today, however, the profile instructions somehow
> were just 2 lines and contained no openstack related instructions but I
> will try it again in the morning.)

If you see this happen again, please leave the experiment in place and
send us a link to the experiment status page -- thanks!

> My question is this: should I be able to install and use Zun container
> services of openstack? I am looking at steps at:
>
> https://docs.openstack.org/zun/latest/install/index.html
>
> In short, rather than spinning up VMs I am hoping I can start containers
> using all the openstack infrastructure/GUI etc.

No, the current profile does not install `zun` nor `magnum`, sorry.

> Gautam

David

[Gautam Thaker]

https://www.emulab.net/portal/status.php?uuid=b9a06259-74e8-11ee-9f39-e4434b2381fc

Profile instructions are basically for me  (just swapped the profile in.)

I will attempt to follow instructions from openstack and see if I can install Zun or magnum, in theory it should be possible, no?

Gautam

[Gautam Thaker]

Sorry, meant to type "profile instructions are  basically empty for my openstack experiment"

GHT

--
You received this message because you are subscribed to a topic in the Google Groups "emulab-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/emulab-users/zjFnC29QV2Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to emulab-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emulab-users/4e09a251-9635-4791-812a-e35cb6dacaf4n%40googlegroups.com.

[Gautam Thaker]

It appears the issue that was resulting in empty "profile instructions" has been fixed.  Posting in case of value to others.

Given that openstack profile is working again, if I swap in an experiment with this profile, would it even be remotely possible for me to install 'zun' openstack service+docker in that environment? At first glance, and with some playing around and reading the instructions, this looks very difficult, esp. to get right the networking so that containers  that may be created would have properly working networking... Comments welcome.

https://docs.openstack.org/zun/0.1.0/dev/manual-devstack.html

Gautam

[David M. Johnson]

On 11/3/23 07:02, Gautam Thaker wrote:
> It appears the issue that was resulting in empty "profile instructions"
> has been fixed.  Posting in case of value to others.
>
> Given that openstack profile is working again, if I swap in an
> experiment with this profile, would it even be remotely possible for me
> to install 'zun' openstack service+docker in that environment? At first
> glance, and with some playing around and reading the instructions, this
> looks very difficult, esp. to get right the networking so that
> containers  that may be created would have properly working
> networking... Comments welcome.

Hi Gautam. I think this is achievable. Note that when we built this
profile years ago, we opted to use Ubuntu cloud archive Openstack
packages
(https://docs.openstack.org/install-guide/environment-packages-ubuntu.html)
to provide the most rapid deploy-from-scratch on bare metal that we
could; other approaches were far too slow. Our profile therefore
follows the per-service Ubuntu installation instructions (e.g., for zun,
https://docs.openstack.org/zun/2023.2/install/, starting at the "Install
and configure controller node" step). The per-component blocks in the
profile's setup-controller.sh basically follow the Ubuntu variants of
these instructions. I have found this documentation to be excellent.

> Gautam

David

> On Friday, October 27, 2023 at 1:13:45 PM UTC-4 Gautam Thaker wrote:
>
> Sorry, meant to type "profile instructions are  basically empty for
> my openstack experiment"
>
> GHT
>
>
> On Fri, Oct 27, 2023 at 12:58 PM Gautam Thaker <ghth...@gmail.com>
> wrote:
>

> https://www.emulab.net/portal/status.php?uuid=b9a06259-74e8-11ee-9f39-e4434b2381fc <https://www.emulab.net/portal/status.php?uuid=b9a06259-74e8-11ee-9f39-e4434b2381fc>

> https://groups.google.com/d/topic/emulab-users/zjFnC29QV2Q/unsubscribe <https://groups.google.com/d/topic/emulab-users/zjFnC29QV2Q/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email
> to emulab-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/emulab-users/4e09a251-9635-4791-812a-e35cb6dacaf4n%40googlegroups.com <https://groups.google.com/d/msgid/emulab-users/4e09a251-9635-4791-812a-e35cb6dacaf4n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "emulab-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emulab-users...@googlegroups.com
> <mailto:emulab-users...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/emulab-users/f0d567c6-4fba-48f2-93bc-5f9ddf4588d7n%40googlegroups.com <https://groups.google.com/d/msgid/emulab-users/f0d567c6-4fba-48f2-93bc-5f9ddf4588d7n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Gautam Thaker]

Hi David:

So, I swapped in Emulab openstack profile  with two d710 node (1 ctl and 1 compute), and then started to follow https://docs.openstack.org/zun/2023.2/install instructions.

Everything is going smoothly till I got to this part:
==============================================
Edit the /etc/zun/zun.conf:

In the [DEFAULT] section, configure RabbitMQ message queue access:

[DEFAULT]
...
transport_url = rabbit://openstack:RABBIT_PASS@controller
Replace RABBIT_PASS with the password you chose for the openstack account in RabbitMQ.

### I CAN'T DETERMINE VALUE of "RABBIT_PASS" in these instructions. Any idea how one goes about finding/setting  this password?
### I tried this:

root@ctl> rabbitmqadmin list users
*** Could not connect: [Errno 111] Connection refused

### ALSO, what shoudl be the "management interface IP addr" for my experiment 

###  https://www.emulab.net/portal/status.php?uuid=170596e9-7d59-11ee-9f39-e4434b2381fc

In the [api] section, configure the IP address that Zun API server is going to listen:

[api]
...
host_ip = 10.0.0.11
port = 9517
Replace 10.0.0.11 with the management interface IP address of the controller node if different.
============================================================================

[Gautam Thaker]

OK, I found this in /etc/nova/nova.conf, I think I can just use this.

transport_url = rabbit://openstack:b98200558c3cXYZZ@ctl

GHT

[David M. Johnson]

On 11/7/23 08:36, Gautam Thaker wrote:
> OK, I found this in /etc/nova/nova.conf, I think I can just use this.
>
> transport_url = rabbit://openstack:b98200558c3cXYZZ@ctl

These random passwords are generated by the profile's
`setup-controller.sh` script (e.g.,
https://gitlab.flux.utah.edu/johnsond/openstack-build-ubuntu/-/blob/master/setup-controller.sh?ref_type=heads#L4783
for the `designate` service configuration). They are persisted in
`/root/setup/settings`.

If you look at `setup-controller.sh`, each service configuration
procedure does similar things: creates random password, database state,
service endpoints; then starts writing configuration into service conf
files, usually using the `crudini` tool to edit ini-file entries.

> GHT

David

> On Tuesday, November 7, 2023 at 9:11:28 AM UTC-5 Gautam Thaker wrote:
>
> Hi David:
>
> So, I swapped in Emulab openstack profile  with two d710 node (1 ctl
> and 1 compute), and then started to follow
> https://docs.openstack.org/zun/2023.2/install
> <https://docs.openstack.org/zun/2023.2/install> instructions.
>
> Everything is going smoothly till I got to this part:
> ==============================================
> Edit the /etc/zun/zun.conf:
>
> In the [DEFAULT] section, configure RabbitMQ message queue access:
>
> [DEFAULT]
> ...
> transport_url = rabbit://openstack:RABBIT_PASS@controller
> Replace RABBIT_PASS with the password you chose for the openstack
> account in RabbitMQ.
>
> ### I CAN'T DETERMINE VALUE of "RABBIT_PASS" in these instructions.
> Any idea how one goes about finding/setting  this password?
> ### I tried this:
>
> root@ctl> rabbitmqadmin list users
> *** Could not connect: [Errno 111] Connection refused
>
> ### ALSO, what shoudl be the "management interface IP addr" for my
> experiment
> ###

> https://www.emulab.net/portal/status.php?uuid=170596e9-7d59-11ee-9f39-e4434b2381fc <https://www.emulab.net/portal/status.php?uuid=170596e9-7d59-11ee-9f39-e4434b2381fc>

> (https://docs.openstack.org/install-guide/environment-packages-ubuntu.html <https://docs.openstack.org/install-guide/environment-packages-ubuntu.html>)

> to provide the most rapid deploy-from-scratch on bare metal that we
> could; other approaches were far too slow. Our profile therefore
> follows the per-service Ubuntu installation instructions (e.g.,
> for zun,
> https://docs.openstack.org/zun/2023.2/install/

> <https://docs.openstack.org/zun/2023.2/install/>, starting at

> the "Install
> and configure controller node" step). The per-component blocks
> in the
> profile's setup-controller.sh basically follow the Ubuntu
> variants of
> these instructions. I have found this documentation to be
> excellent.
>
> > Gautam
>
> David
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "emulab-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emulab-users...@googlegroups.com
> <mailto:emulab-users...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/emulab-users/5b7e1875-4e3b-4e3b-b370-86e4a41b4025n%40googlegroups.com <https://groups.google.com/d/msgid/emulab-users/5b7e1875-4e3b-4e3b-b370-86e4a41b4025n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Gautam Thaker]

David:

Thanks for these pointers. setup-controller.sh weighs in at some 5300+ lines, but I see there is that repeating pattern you state for each service. 

`$PSWDGEN`  is called 40 times in this script, lots of passwords, but luckily only a few that I believe are used/consumed by humans.

I did also notice towards end there is "# May be install Magnum" section. We seem to enter it, many MAGNUM_* entries are present in /root/setup/settings. Could it be that magnum is being installed? (I  know you had said zun and magnum are not installed, so curious.)  I don't need magnum, if I can make zun work and spin up a couple of containers, one at a time, I am happy. 

Gautam

[Gautam Thaker]

So, I started with a clean openstack profile install on d710s and logged in to the 'ctl' node. I then tried to faithfully complete the steps at:

https://docs.openstack.org/zun/2023.2/install/controller-install.html#install-and-configure-components

when I finally got to step "7. Populate Zun database" it gives me a lot of errors:

root@ctl> su -s /bin/sh -c "zun-db-manage upgrade" zun
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 436, in resolve
    return _proxy.query(name, rdtype, raise_on_no_answer=raises,
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 392, in query
    return end()
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 371, in end
    raise result[1]
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 352, in step
    a = fun(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1222, in query
    return self.resolve(qname, rdtype, rdclass, tcp, source,
  File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1173, in resolve
    (nameserver, port, tcp, backoff) = resolution.next_nameserver()
  File "/usr/lib/python3/dist-packages/dns/resolver.py", line 626, in next_nameserver
    raise NoNameservers(request=self.request, errors=self.errors)
dns.resolver.NoNameservers: All nameservers failed to answer the query controller. IN A: Server 127.0.0.53 UDP port 53 answered SERVFAIL

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 613, in connect
    sock = socket.create_connection(
  File "/usr/lib/python3/dist-packages/eventlet/green/socket.py", line 44, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 529, in getaddrinfo
    qname, addrs = _getaddrinfo_lookup(host, family, flags)
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 502, in _getaddrinfo_lookup
    raise err
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 491, in _getaddrinfo_lookup
    answer = resolve(host, qfamily, False, use_network=use_network)
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 446, in resolve
    raise EAI_NODATA_ERROR
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 491, in _getaddrinfo_lookup
    answer = resolve(host, qfamily, False, use_network=use_network)
  File "/usr/lib/python3/dist-packages/eventlet/support/greendns.py", line 446, in resolve
    raise EAI_NODATA_ERROR
socket.gaierror: [Errno -2] Name or service not known

If anyone has any comments I welcome them.

Gautam

[Gautam Thaker]

I should have said my experiment is at:  https://www.emulab.net/portal/status.php?uuid=170596e9-7d59-11ee-9f39-e4434b2381fc (In case needed.)

A related question is this:

Rather than start with an openstack profile 'ctl' node and going thru installing 'zun' services, is an alternate (even if (much) slower) method to start with an ubuntu 22.04 image on 'smallLAN' profile and then follow steps at:

https://docs.openstack.org/install-guide/environment-packages-ubuntu.html

to create an openstack environment from scratch? I realize this may be a foolish goal for someone lacking guru level skills in general and about emulab specifically.  And it may be of limited use with no ability to save the work in any image etc.

Comments?

Gautam

[David M. Johnson]

Sorry I didn't get to this thread sooner, but perhaps the zun config
file entries were wrong? In particular, /etc/zun/zun.conf, in the
`database` section, the `connection` key's value should use the `ctl`
short host name. That would be my first guess.

> Gautam

David

> On Tuesday, November 7, 2023 at 1:13:00 PM UTC-5 Gautam Thaker wrote:
>
> David:
>
> Thanks for these pointers. setup-controller.sh weighs in at some
> 5300+ lines, but I see there is that repeating pattern you state for
> each service.
>
> `$PSWDGEN`  is called 40 times in this script, lots of passwords,
> but luckily only a few that I believe are used/consumed by humans.
>
> I did also notice towards end there is "# May be install Magnum"
> section. We seem to enter it, many MAGNUM_* entries are present in
> /root/setup/settings. Could it be that magnum is being installed?
> (I  know you had said zun and magnum are not installed, so
> curious.)  I don't need magnum, if I can make zun work and spin up a
> couple of containers, one at a time, I am happy.
>
> Gautam
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "emulab-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emulab-users...@googlegroups.com
> <mailto:emulab-users...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/emulab-users/ce06edfc-c8f3-4728-82a0-f63f4dea36d1n%40googlegroups.com <https://groups.google.com/d/msgid/emulab-users/ce06edfc-c8f3-4728-82a0-f63f4dea36d1n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[David M. Johnson]

This would be quite daunting to do manually, and the Ubuntu package
install path is exactly the strategy the OpenStack profile takes.

> Gautam

David