gs://resf-osv-data not being updated?

[Oliver Chang]

Hi Mustafa,

Hope you are well. 

We recently noticed that gs://resf-osv-data seems to be missing quite a few advisories. In particular, there doesn't appear to be any RLSA-2024-* advisories available in the bucket. 

Would you be able to help us look into this? Has the export not been running successfully? 

Thanks! 

--

Oliver

[Mustafa Gezen]

Hi, looks like the automation was never pushed. Sorry about that, I'll get that fixed right away

[Oliver Chang]

Thank you so much for the blazing quick response and helping fix this Mustafa!  

--

Oliver

[Mustafa Gezen]

Thank you for letting me know! I've been shifting focus a bit so haven't paid a lot of attention to errata. I'll deploy a cron job for automated syncs until we deploy triggers on the server side for real-time updates to the bucket.

[Oliver Chang]

Hey Mustafa,

Thanks again for fixing this. I just wanted to check if the automated syncs has been deployed? 

I noticed some new entries on https://errata.rockylinux.org/ that aren't available in gs://resf-osv-data yet. 

Kind regards,

--

Oliver

[Mustafa Gezen]

Hi,

Sorry about that. I was supposed to get that over the finish line but it slipped.

I have now deployed a nightly job that syncs, and it's tested and working.

I'll keep an eye on this for a few days to ensure that everything works as expected.

Again, sorry!

Mustafa

[Oliver Chang]

Thanks a ton again Mustafa!

How frequent are we able to make this run? Or rather, what's the highest latency we can get in terms of data freshness from upstream? 

Some of our consumers care a lot about data freshness so wanted to make sure we have an answer :) 

Thanks,

--

Oliver

[Mustafa Gezen]

This should run at midnight every day (CET). So at most 23 hours of stale status for OSV.

Which actually sounds very dramatic. I can up the frequency to every 3 or 6 hours too if that is preferred.

Upstream -> downstream is usually right away as a build appears for an advisory we can match.

I haven't worked on the errata system in a while either but will take up some work around EOL of OVAL v2

if someone else doesn't pick it up.

For the CSAFv2 work we can integrate the OSV delivery into the system to make it "real time". We also

expect higher precision for security advisories with that move (better matching).

Let me know the preferred frequency for now and I'll set it to that.

Mustafa

[Oliver Chang]

This sounds great. 3 hours would be amazing in the meantime if you could make that change :) 

Thanks,

--

Oliver

[Mustafa Gezen]

I looked at the execution time, and we're actually in a good spot to reduce the trigger time to every 15 minutes.

So hopefully that is an acceptable interval regarding staleness!

Feel free to reach out any time :)

Mustafa

[Oliver Chang]

This is incredible. Thanks Mustafa! 

--

Oliver