Replace mysql with etcd in ironic-baremetal-operator

[Shrijeet Aherkar]

Hi,

We are planning to use etcd instead of mysql in ironic-baremetal-operator. Wanted to confirm here, if we can proceed with this or there are any known limitations for this.

As an initial exploration, we have added the etcd writein ironic-api and ironic-conductor pods where the mysql create and update operation are getting called. Added etcd read changes during the get and list operations.

Things we observed are, in case mysql there are many default columns are created with value NULL or {}, those are missing in etcd. There will be separate handling needed to handle this in db/api.py to create a object for etcd.

Please do update us.

Thank you,

Shrijeet

[Dmitry Tantsur]

Hi,

On Thu, Dec 16, 2021 at 5:02 AM Shrijeet Aherkar <shri...@platform9.com> wrote:

Hi,

We are planning to use etcd instead of mysql in ironic-baremetal-operator. Wanted to confirm here, if we can proceed with this or there are any known limitations for this.

This sounds really interesting, although I'm not quite sure what exactly you mean. There is no ironic-baremetal-operator, there are ironic and baremetal-operator.

Also note that I'm currently working on making mysql optional in favor of sqlite. It may or may not be what you are looking for, depending on what your goal is.

 

As an initial exploration, we have added the etcd writein ironic-api and ironic-conductor pods where the mysql create and update operation are getting called. Added etcd read changes during the get and list operations.

Okay, so you want to update Ironic to be able to support etcd as the database backend? That would be absolutely amazing, but you're up for a rough ride. Ironic relies not only on NULL, but also on transactions and foreign keys.

That being said, the abstraction layer in db/api.py may be flexible enough to allow that.

 

Things we observed are, in case mysql there are many default columns are created with value NULL or {}, those are missing in etcd. There will be separate handling needed to handle this in db/api.py to create a object for etcd.

Not sure about {}, JSON fields are stored as strings in Ironic.

 

Please do update us.

If you, indeed, plan on adding etcd support in Ironic, it would be great to engage with the Ironic community (some of us are here, but many are not). We're on #openstack-ironic on the OFTC IRC network. We use http://lists.openstack.org/pipermail/openstack-discuss/ (with "[ironic]" in subject) for discussions.

Dmitry

 

Thank you,

Shrijeet

--
You received this message because you are subscribed to the Google Groups "Metal3 Development List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metal3-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/metal3-dev/5e47c508-12d4-4f50-9cb4-b00085eb3c71n%40googlegroups.com.

--

Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

[kfox...@gmail.com]

It might be interesting to target Kubernetes CRD's directly rather then use etcd. It would become even simpler to manage since you wouldn't need a whole other etcd.

Just a thought.

Thanks,

Kevin

[Dmitry Tantsur]

On Thu, Dec 16, 2021 at 6:07 PM kfox...@gmail.com <kfox...@gmail.com> wrote:

It might be interesting to target Kubernetes CRD's directly rather then use etcd. It would become even simpler to manage since you wouldn't need a whole other etcd.

I would assume the proposal is about using the Kubernetes' etcd rather than installing a new one?

Targeting CRDs may be more beneficial for us, but is also a tougher sell upstream in Ironic since it has limited usability outside of Metal3 and since testing/development will be harder (etcd is used elsewhere in OpenStack already).

Dmitry

 

To view this discussion on the web visit https://groups.google.com/d/msgid/metal3-dev/e9d22e8a-4e72-4b9d-8d7f-fcbc34110253n%40googlegroups.com.

[Mr. Mykola Dzham]

On Thu, 16 Dec 2021 at 18:12, Dmitry Tantsur <dtan...@redhat.com> wrote:
>
>
>
> On Thu, Dec 16, 2021 at 6:07 PM kfox...@gmail.com <kfox...@gmail.com> wrote:
>>
>> It might be interesting to target Kubernetes CRD's directly rather then use etcd. It would become even simpler to manage since you wouldn't need a whole other etcd.
>
>
> I would assume the proposal is about using the Kubernetes' etcd rather than installing a new one?

It is strongly discouraged directly accessing the etcd cluster used by
kubernetes and using it for anything else.
For various reasons, including security.
https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#limiting-access-of-etcd-clusters

> To view this discussion on the web visit https://groups.google.com/d/msgid/metal3-dev/CACNgkFzfrYFVjiqtr3H_q-_oPXzgggmJJZASeYwMP4fqyKs%2B%2Bg%40mail.gmail.com.



--
Mr. Mykola Dzham
+41 79 571 28 64

[kfox...@gmail.com]

While getting it all upstreamed would be great, not sure that would happen either way? Guessing what could be upsreamed would be a plugable layer where sql is plugged in by default. but could be replaced by etcd or k8s or something else.

sql vs etcd is pretty different. so is sql vs kube crds. So either way, guessing the plugable layer would have to be pretty generic to function.

Yeah, it would be easier to share the etcd, but I think that configuration is not usually recommended.

Just thinking from a users perspective, having everything in CRD's would be the easiest to manage. No extra etcd to have to maintain, or security worries around sharing an etcd and getting certs and networking all sorted out so that could work, etc.

 

Thanks,

Kevin

[Dmitry Tantsur]

On Sun, Dec 19, 2021 at 9:29 AM Mr. Mykola Dzham <mykola...@gmail.com> wrote:

On Thu, 16 Dec 2021 at 18:12, Dmitry Tantsur <dtan...@redhat.com> wrote:
>
>
>
> On Thu, Dec 16, 2021 at 6:07 PM kfox...@gmail.com <kfox...@gmail.com> wrote:
>>
>> It might be interesting to target Kubernetes CRD's directly rather then use etcd. It would become even simpler to manage since you wouldn't need a whole other etcd.
>
>
> I would assume the proposal is about using the Kubernetes' etcd rather than installing a new one?

It is strongly discouraged directly accessing the etcd cluster used by
kubernetes and using it for anything else.
For various reasons, including security.
https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#limiting-access-of-etcd-clusters

Hmm, makes sense, I guess.

Supporting CRDs may be a lot of work. The Ironic's database is quite large (Ironic has more features than we use here), we'll need to figure out what to do with things that require transactions. The speed may be really poor since Ironic stores a lot of temporary state in the database.

I guess someone should try building a proof-of-concept.

Dmitry