AlmaLinux integration

[Nikita Ivanov]

Hi!
Hi! We've prepared a tool to convert AlmaLinux errata information into shared OSV format. Serving shared format here. Could you review the data files we initialized out repository with? We tried to comply with OSV schema as much as possible, but would be pleased to receive any feedback.

The PR reserving AlmaLinux ecosystem name and advisory ID is already reviewed and merged into main. Is anything else required from us to publish our database at osv.dev? How soon can we expect our database to be published at osv.dev?

[Andrew Pollock]

Hi Nikita,

Very exciting!

I'm in the process of adding AlmaLinux to our test instance of osv.dev to make sure everything imports successfully, and once that's been verified, we can add it to osv.dev. Expect this to all happen in the order of the next 1-2 weeks.

regards

Andrew

--
You received this message because you are subscribed to the Google Groups "osv-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osv-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/f1690c0e-8be5-410d-8e81-d0f1f899b749n%40googlegroups.com.

--

Andrew Pollock Security Engineer, Google Open Source Security Team | +61419788191 | apol...@google.com Google LLC

This email is confidential. If you are not the right addressee, please inform the sender and please erase this email including any attachments.

[Nikita Ivanov]

Hi Andrew.

The last time you've said that you expected everything to happen in 1-2 weeks, but I can't see Almalinux on osv.dev still. Is everything alright? Were the tests successful?

[Andrew Pollock]

Hi Nikita,

Apologies for things taking a little longer than previously advised. I'd made an error in the initial addition, so importing wasn't working correctly. I corrected that error, and need to circle back with my colleagues on next steps. We've had a few public holidays here that have slowed us down. Please give me another week.

regards

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/24e7c332-a736-4f76-9b33-5f202faea3c8n%40googlegroups.com.

[Andrew Pollock]

Hi again Nikita,

Skimming through the published advisories, it looks like the identifier is not unique between different releases, e.g: almalinux8 and almalinux9 both have ALBA-2021:4228

We'll be configuring each release (i.e. 8, 9)  of AlmaLinux as a discrete source, and the identifiers need to be unique.

Can you tell me more under which circumstances each of the three prefixes are used as well? To give you an idea of what I need to configure, see the SourceRepository class at https://github.com/google/osv.dev/blob/2d681bfd5ed5885f28ecaf2aa07a68003986424b/osv/models.py#L696

Modulo the duplicate identifier issue, we seem to have the ALSA- prefixed advisories for almalinux8 importing successfully into our staging database.

regards

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/24e7c332-a736-4f76-9b33-5f202faea3c8n%40googlegroups.com.

[Nikita Ivanov]

Hi.

Why would you have a problem when Debian in OSV have kinda the same attitude as well? For example, https://osv.dev/vulnerability/DSA-2203-1 refers to both Debian 8 and Debian 6.
It is not duplicated identifier, it is the way the things were supposed to be, I believe. It is the way our Errata was organized (like any other RHEL-like errata). Each identifier addresses certain vulnerability/bug/issue that was found and were supposed to be fixed with some sort of update. Then, different products (like Almalinux 8, Almalinux 9) release product-specific update for the issue addressed by identifier.

Speaking of prefixes use cases:
- ALSA - security fixes
- ALBA - bug fixes
- ALEA - enhancements (package updates, I believe)
I might be wrong about it, so let me ask my colleagues and I'll come back with some detailed reply later.

[Oliver Chang]

Hi Nikita, 

In such cases we require the different versions/releases to be specified in the same advisory/entry with a unique ID, which is how it's done in Debian as well. 

E.g. right now there's https://github.com/AlmaLinux/osv-database/blob/746f577bc92b2d88984a90e7057b5416b46f83ce/advisories/almalinux9/ALBA-2021%3A4228.json#L4 and https://github.com/AlmaLinux/osv-database/blob/746f577bc92b2d88984a90e7057b5416b46f83ce/advisories/almalinux8/ALBA-2021%3A4228.json#L4 both with the same ID ("ALBA-2021:4228") but different contents referring to AlmaLinux 8 and 9. 

This is because we need the advisory ID to be a unique key to access the advisory. Would it be possible to merge these two entries in the OSV generation such that for "ALBA-2021:4228", there is only one JSON file, with two "affected" array entries covering the two AlmaLinux releases? 

Best,

Oliver

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/a27d7c8e-9e74-4df0-b200-35ec18d4aa25n%40googlegroups.com.

[Nikita Ivanov]

Hi Oliver,

Well, it would be a tough job to do. Mostly because release to Almalinux 8 and 9 might be at the different time. It would require to dynamically merge and modify OSV advisories which would make our release automation a lot more complicated. Can we come up with some solution which would still preserve separate OSV advisories for Almalinux 8 and 9 and would allow to bypass id uniqueness restriction as well?

[Oliver Chang]

Very sorry for the troubles here. Unfortunately this is a pretty hard requirement for us and difficult to change. 

As a middle ground solution: how about an out-of-band post-processing step that takes the contents in the AlmaLinux/osv-database's main branch and generates a "consolidated-osv" branch which has the format we need? This can run at a different frequency from the Almalinux 8/9 release schedule. 

Thanks,

--

Oliver

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/136a6b9f-aca2-421d-8385-af11bc51a66an%40googlegroups.com.

[Nikita Ivanov]

Okay, got you. The idea might be plausible. I will think about possible solutions and come back later with response.

ср, 26 апр. 2023 г., 13:13 Oliver Chang <och...@google.com>:

[Nikita Ivanov]

Andrew, speaking of identifiers use cases. That what my colleagues told me:

- ALSA - security fixes. Almost all the time they concern the actual CVEs assigned by MITRE and NIST.
- ALBA - bug fixes. It can be either a security fix or just a fix of any incorrect behavior.
- ALEA - enhancement releases. In most cases, package update.

--

Best Regards,

Nikita Ivanov | C developer

[Nikita Ivanov]

Speaking of identifier duplicates, It seems I was wrong and it's not an expected behaviour. I can't assure you, but most probably it is a bug which will be fixed in the meanwhile, so there will be unique ids for alma8 and alma9. With this in mind, it seems that there will be no need to change the repository structure at all. As soon as we resolve this issue and remove duplicates from the osv-database repository, I will contact you and tell you that everything is ready. In a meanwhile, as I see that the duplicates are only present for ALBA and ALEA ids, so you may safely release that to osv.dev, if possible.

[Nikita Ivanov]

Hi,
Erroneous advisories have been remove from osv-database repository, as I expected. Thanks for your report on this issue. Repo is ready to be released.

[Andrew Pollock]

Hello,

We've got our staging instance importing this now, and I'll check it later in the week, and if everything appears in order, we'll add it to our production database importer next week.

regards

Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/33607314-a825-4270-a398-79970873eeben%40googlegroups.com.

[Andrew Pollock]

Hello,

We didn't see any barriers to moving forward with integrating AlmaLinux this week, so I'm very pleased to advise that AlmaLinux is now available in our production database: https://osv.dev/list?ecosystem=AlmaLinux

Thank you for this collaboration and contribution!

We will have an upcoming blog post about this at http://osv.dev/blog, if you'd like to review the PR prior to publication, please let me know your GitHub username.

regards

Andrew

[Nikita Ivanov]

Hi,

Thanks, it is great news! My GitHub user name is "Roo4L": https://github.com/Roo4L.

It was a pleasure to work with you. Wish you good luck,

Nikita

пт, 5 мая 2023 г., 10:56 Andrew Pollock <apol...@google.com>:

[Michael Kedar]

Hi,

We've recently updated the OSV Schema to refine and clarify the usage of the 'aliases' and 'related' fields.

Currently, AlmaLinux seems to be using the 'aliases' field to list the multiple distinct CVEs a patch addresses. In our revised schema definition, these should come under the 'related' field instead. Could you please update your OSV records to use the 'related' field for these instead?
If you have any questions, I'd be happy to discuss further.

Thanks,
Michael

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/CAAJ4Ao_fzANkzxcmP5Dr_3LB0xfYCeig7KBuh2i9fYgs76o%3DDg%40mail.gmail.com.

[Nikita Ivanov]

Hi Michael,

Thanks for keeping us posted. Seems like you've done a great job formulating what the word `alias` really means. Nevertheless, while `symmetric` and `transitive` qualities of aliases are quite strict and formal (which is good), the last statement in the alias description might not be so obvious to a common reader:

> Aliases should not be used in records that bundle many different vulnerabilities in one patch of a distribution of a package.

I suggest you provide an example which would clearly explain why it is bad to use aliases in records that bundle multiple vulnerabilities. Something like that:

> For example, mentioning multiple CVEs in `alias` field would mean that those CVEs are identical (due to symmetric and transitive qualities of alias) instead of stating that the release has fixed multiple vulnerabilities.

Speaking of switching to `related` field in our OSV entries, we'll schedule the task and let you know as soon as we complete the transition.

Best regards,
Nikita

[Michael Kedar]

Hi,

Thanks for the example suggestion. Just letting you know that I've made created a pull request to add clarification: https://github.com/ossf/osv-schema/pull/197

Thanks again,
Michael

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/4192ea0f-d268-47b1-90d8-1bd3af4c9022n%40googlegroups.com.

[Oliver Chang]

Hi Nikita,

Thanks again for the suggestions on improving the spec! Just wanted to check up on if you've had a chance to migrate the existing AlmaLinux records to use `related` instead of `aliases`? 

Thanks,

--

Oliver

To view this discussion on the web visit https://groups.google.com/d/msgid/osv-discuss/CAFbZv58w8Zcg-k0EvwXXn7JaO%3D084MnOcjs15TPwQt6xB%2Bia-g%40mail.gmail.com.

[Nikita Ivanov]

Hi!

I've introduced the above-mentioned changes to our OSV entries generation script, but for whatever reason buildsystem team is struggling to integrate the changes into our release process. I'll let you know when the changes are going to take effect. Thanks for your patience.

[Nikita Ivanov]

Hi. The changes have been released. Nevertheless, I do not see any mentions of CVEs mentioned in the "related" field on osv.dev website. Is it a bug or is it not supposed to be shown? For example, one can check https://osv.dev/vulnerability/ALSA-2023:5453. "related" field in the import source is not empty, but information from it is not displayed.

[Oliver Chang]

Thanks a ton Nikita! 

This has been correctly ingested into OSV.dev (see https://api.osv.dev/v1/vulns//ALSA-2023:5453 for the full JSON ingested). 

Our UI indeed has a bug with not rendering the `related` field. @Holly Gong would you mind tackling this as well as part of the aliases cron work?

--

Oliver

[Holly Gong]

Sure, I will implement this and update it here. ref: https://github.com/google/osv.dev/issues/1734

Best,

Holly

[Holly Gong]

`Related` field has been added to the web UI. Please check: https://osv.dev/vulnerability/ALSA-2023:5453 

Best,

Holly

[Nikita Ivanov]

Looks great, thank you!