extending Grafeas

[Gert Jan van Halem]

Hi all,

Can someone point me to some information on exteding Grafeas? We want to create our own metadata and provider. And I am looking for some documentation on how to start on that

Thanks!
Gert Jan

[Aysylu Greenberg]

Hi Gert,

Grafes eng lead here. For the new metadata kinds, please file a feature request via GH issues. Happy to discuss your use case there and please share more information about the provider you'd like to add!

Cheers,

Aysylu

[Gert Jan van Halem]

Hi Aysylu,

Thanks for your reply. We want to use in-toto with Grafeas. This has been build before but for an older version of Grafeas. And we wanted to bring it up to date. We were expecting that Grafeas would be pluggable. And by that, we understood that we could write our own extensions. But I understand that we have to fill out a feature request. That makes it less flexible for us. We will reconsider if this fits our purpose

thanks
Gert Jan

Op dinsdag 4 juni 2019 20:56:57 UTC+2 schreef Aysylu Greenberg:

[Aysylu Greenberg]

Hi Gert,

What metadata type were you using before, that it's no longer in Grafeas? I'd love to learn more about your use of in-tot- with Grafeas. And could you please clarify in what way you'd like Grafeas to be pluggable and extendable? If it's for new metadata kinds, it's certainly possible to add them.

Happy to continue the conversation to understand better your use case.

Cheers,

Aysylu

[Gert Jan van Halem]

Hi Aysylu,

in-toto enables us to describe our ci/cd pipelines upfront and verify if the described process was actually followed. That helps us big time in terms of audibility and SOx compliance. While still leaving room for teams to decide for themselves how to build and deploy their artifacts.

To do so, in-toto gathers metadata that we want to store in Grafeas

1. the layout. This is a signed json file with public keys of actors in the process, description of the steps in the process in terms of materials, products, commands and allowed actors. And information of how to do the verification. In the available demo this is stored in a Grafeas operation. But the operations seem to not exist anymore
2. We want to store the individual steps in the process as notes. And want to be able to add our own models to Grafeas. In this case a json with information on the step and the allowed actors
3. When running the pipeline, we want to create occurences based on these notes. So that we can see that a certain step in the process is based on the described step in the layout
4. In the end we want to combine these all to verify if all the steps were done as described and the end result is according to the expectation described in the layout

As said,  around 2 years ago a demo of this was created. https://github.com/in-toto/totoify-grafeas. We want to see if we can bring that to the current version of Grafeas. For us it is important that 

1. we can plug things like this in Grafeas without having to bother you with feature requests
2. we are not heavily depended on Grafeas version

Thanks
Gert Jan

Op dinsdag 4 juni 2019 21:58:56 UTC+2 schreef Aysylu Greenberg:

[Gert Jan van Halem]

Could you please tell us how we can add new metadata kinds?

Thanks!

Op dinsdag 4 juni 2019 21:58:56 UTC+2 schreef Aysylu Greenberg:

[Aysylu Greenberg]

Hi Gert,

Sorry for the delay in my response. Please see my responses below:

1. You're correct that operations are no longer part of the Grafeas API since v1beta1. In general, the layout doesn't seem like part of the Artifact Metadata API, as it represents the higher-level structure of the CI/CD pipelines, not the metadata about the artifacts generated by them.

2. What kinds of metadata in your process is not currently represented by Grafeas? You're welcome to file a FR as a GH issue for any that you'd like, to see if the community has a similar need.

3. Makes sense and it sounds like the right approach with Grafeas, to create occurrences based on the notes that represent metadata generated by the ci/cd pipelines.

4. This sounds like a separate process that'd query Grafeas for all metadata, and the verification will be done in that separate process.

5. One option is to create protos that represent your specific process, and use Grafeas protos inside them. If something is missing from Grafeas that the community can benefit from, we'd love to have your contribution!

6. We currently have no specific plans to retire v1beta1 API. We'll be releasing Grafeas server 0.1.0 soon, so you're welcome to use it or your own integration with the Grafeas API.

Happy to answer any other questions and hope this helps!

Cheers,

Aysylu