REST API common cases

[Kurt Seifried]

I want to start work on a basic public-facing API (so read-only for now), so I'm thinking what are the common cases. I'm pretty sure I might have missed something or overthought it, so here goes:

1) request current data on GSD entry X (essentially what raw.globalsecuritydatabase.org does)

2) request current data on all new/updated GSD entries since time

2a) request current data on all new GSD entries since time

2b) request current data on all updated GSD entries since time

I'm also thinking the API should, by default, serve only data described in the schema, e.g. the. stable stuff, and ignore any experimental/nonstandard data, so long term this means the GSD namespace for example would return well-formed OSV data only by default unless you specify some flag like "show_experimental_data" and the cve.org namespace would only return the well-formed CVE JSON data. In other words, by default, the API would be very boring and stable, but with flags to enable all the crazy stuff if you want it. 

Obviously, longer-term we want to support updates/delta views and all that, but I feel like for now giving people the ability to easily poll an API and get all the new/updated data since the last time they grabbed it would be a huge first step.

Kurt Seifried (He/Him) Chief Blockchain Officer and Director of Special Projects Cloud Security Alliance e: ksei...@cloudsecurityalliance.org

[Josh Buker]

I'm having trouble remembering who it was I was chatting with about it at GSVS, but having a REST API to easily post new GSDs would obviously be valuable, especially for folks enriching data with automated tools.

e: kseifried@cloudsecurityalliance.org

[Kurt Seifried]

On Wed, Jun 29, 2022 at 3:39 PM Josh Buker <jbu...@cloudsecurityalliance.org> wrote:

I'm having trouble remembering who it was I was chatting with about it at GSVS, but having a REST API to easily post new GSDs would obviously be valuable, especially for folks enriching data with automated tools.

Yes, but how do we handle deduplication? Do we even try? No matter how much deduplication we do up front there will always be future duplicates (e.g. CVE assigns something days/weeks/months later), my thinking is we don't try to deduplicate on the API, but we do deduplicate on the backend (manually right now, hopefully with automation and fancy ML/AI in future) and keep moving forwards as is. 

Obviously, this also makes the API simple, it's basically the web requests form (which... should maybe use the API itself longer term to submit? I have no idea how the auth flow works then).

Stupid question but why don't we just write some docs or a simple wrapper for the existing form, it takes POST input:

<form action="/gsdform/formsubmit" name="requestGSD" method="post" class="input-append">

I suspect the biggest problem is making the auth bits work (so having a wrapper for say... python? javascript? would be huge here).

-Kurt

[Josh Buker]

Easy for the updating existing ID case anyway - for now simply overwrite their namespace with the new blob wholesale.

On Wednesday, June 29, 2022 at 3:24:38 PM UTC-7 Kurt Seifried wrote:

[Kurt Seifried]

I mean deduplication at the "another GSD for this issue already exists" not deduplication within the GSD itself.

Kurt Seifried (He/Him) Chief Blockchain Officer and Director of Special Projects Cloud Security Alliance

e: ksei...@cloudsecurityalliance.org

On Wed, Jun 29, 2022 at 5:01 PM Josh Buker <jbu...@cloudsecurityalliance.org> wrote:

Easy for the updating existing ID case anyway - for now simply overwrite their namespace with the new blob wholesale.

On Wednesday, June 29, 2022 at 3:24:38 PM UTC-7 Kurt Seifried wrote: