Content wrappers

[Charles Schmidt]

Hi all,

 

Per our conversation on June 10, I'm starting a thread to discuss specifying the set of Collector inputs. As a refresher:

- The group agreed that PCEs (such as the agents on endpoints) are unconstrained in the formats of the inputs they expect. The goal is to allow just about any collection mechanism to be integrated into the SCAP architecture regardless the input format/interface it uses, whether that be some standard or a proprietary design.

- This is possible because the Collector acts as a translator for PCEs, taking instructions sent from the Manager and turning them into instructions that the PCE understands. The same would be true with results - the Collector takes the PCE's results (in whatever format they might be) and turns them into results that meet Application requirements and can be stored in the Repository.

- By contrast, inputs to the Collector need to be standardized and limited. Every Collector needs to be able to ingest the same set of input formats in order to enable Collectors from different vendors to all work correctly in an implementation of the architecture. Since every Collector needs to be able to support all standardized input formats, it makes sense for this set of standardized input formats to be relatively limited.

- We identified two patterns for a Collector translating an input format into something that could be ingested by a PCE to which it is bound. In pattern 1, the Collector would parse its input and generate an equivalent set of instructions using the PCEs input syntax. In pattern 2, the input to the collector would consist of a wrapper that the Collector can understand enclosing instructions the PCE can understand. In this second scenario, "translation" of the input would simply consist of removing the wrapper and passing the enclosed instructions on to the PCE. The wrapper is necessary to ensure that the content has a unique identifier and that the content type is clearly specified in a way the Collector can understand. (Both are required by the SCAP architecture functions.) In this case, the wrapper serves as the "standard format" that is known by the Collector.

 

Based on this, my original proposal was that we use OVAL as a way to wrap content. Below is a more fleshed out version of this proposal (including some modifications based on feedback received).

 

In this idea, there are effectively three classes of OVAL: standard, custom, and wrapper. Collectors will be able to instantly distinguish between these classes by examining the names of OVAL tests and their associated namespaces.

 

Standard OVAL would be anything that used the official OVAL language and prescribed behaviors. While Collectors won't necessarily support every OVAL schema, it is expected that the core schemas will be supported. Collectors are expected to be able to parse supported OVAL schema fields. In particular, it is expected that Collectors will be able to evaluate State expressions, support importing of variable values into checks, and support exporting collected information into variables to be used elsewhere.

 

Custom OVAL represents checks that use the full OVAL format (States and Objects with sub-fields for individual parameters), but which are not part of the standardized OVAL language. They would be used for pre-standardized OVAL schemas as well as custom behaviors created by parties familiar with OVAL and seeking to follow the same paradigm. Custom OVAL would be recognized by the Test/State/Object names and namespaces. A Collector (or PCX) that supported a given Custom OVAL schema would be expected to understand how to parse and process instances of that schema. In particular, it is expected that Collectors will be able to evaluate State expressions, support importing of variable values into checks, and support exporting collected information into variables to be used elsewhere. In short, a Custom OVAL schema should be usable by a supporting Collector in all the ways a standardized OVAL schema is. (Of course, not all Collectors will support any given Custom OVAL schema.)

 

The third class of OVAL would be wrappers. A wrapper provides a way for black-box encapsulation of instructions. The only processing a Collector is expected to perform on a wrapper is: determine if/which PCE/PCX processes it, unwrapping the contents and passing those contents to a PCE, and wrapping PCE results into a wrapper Item as part of a result set. There is no expectation of being able to import or export variables or evaluate state involving wrapped content. A Collector MAY have additional abilities to interpret either the wrapped content or the wrapped results, but in general the purpose of a wrapper is to provide a vehicle for the passing of instructions to and results from a PCE without any need for the Collector to understand either. A wrapper would require the following:

 

- We create a new type of OVAL Test: the "wrapper_test". A wrapper_test would only have an Object, but no State.

- The Object referenced by a wrapper_test would have a type corresponding to the test type. E.g., it could be an "ansible_object", a "bash_object", a "charles_homebrew_check_system_object", or whatever.

- The body of the referenced Object would just be the relevant PCE instructions, encoded so as not to create XML compatibility issues. The Object would also have its id field to give this check its unique ID.

- As with other OVAL tests that are sent without State elements, the goal would be to gather information rather than make a state evaluation. In this case, the information gathered would be a string: the return value of the PCE check, encoded for transport within XML.

 

In all three cases, the standard OVAL identifiers attached to Tests and Objects (as well as enclosing Definitions) support the requirement for unique identifiers for each check. Likewise, the name and XML namespace given in Tests and Objects clearly indicate the type of check and allow Collectors to route and process the instructions correctly. This includes the ability for a Collector to quickly identify checks that none of its PCEs/PCXs support and a way to return a result that indicates that a check could not be run.  As a result, the proposal fits the needs of supporting custom extensions as well as a black-box wrapper for check content, all within the general framework of a checking language the architecture will support anyway.

Comments are welcome.

 

Thanks,

Charles

[Adam Montville]

Thank you for sending this out, Charles. 

I don’t like using OVAL as a wrapper based on OVALs current purpose - state assessment. We are talking about Collectors supporting a variety of state assessment languages, and this proposal would have OVAL, a state assessment language, serve as a wrapper for other state assessment languages. It’s cleaner and more intuitive if any state assessment language is carried in an envelope that is not particular to one of those languages. 

There may be some fabric transport assistance to be had here, as well. If the semantics of the interaction are inherent in the topic, then we could simply rely on that as the “envelope” in a sense. OCA, for example, has an Open DXL Ontology (forgive the misnomer) that defines a topic along with its expected payload. That payload could be the wrapper itself, into which OVAL, NETCONF, YANG, YAML, Ansible, or any other format could be placed, or we could do a per-topic convention and have a collection topic for each capability. This is probably where I need Bill, Danny, or David to chime in with implementation considerations. :-) 

Kind regards,

Adam

--
To unsubscribe from this group, send email to scap-dev-endpo...@list.nist.gov
Visit this group at https://list.nist.gov/scap-dev-endpoint
---
To unsubscribe from this group and stop receiving emails from it, send an email to scap-dev-endpo...@list.nist.gov.

[Charles Schmidt]

Hi Adam,

I hear your argument on being wary of turning "a checking language" into "a checking language and/or a vehicle for any other checking language". Something new that is a dedicated wrapper might make sense. 

Regarding your proposal, it would depend on the characteristics of a "topic", but I'm a bit skeptical that the message fabric can serve as the wrapper given the degree of flexibility we need. I would think it would require us to effectively have infinite topics. Then again, I will freely admit I'm still learning the paradigms of message fabric use.

Charles

[Adam Montville]

Charles,

Thanks for your response. I think sometimes we engineer types can think of extreme edge cases and that we tend to think in terms of absolutes. I don’t see the collection languages as an infinite set, but instead as a discrete set of a handful to start with the option to add more as they are invented.  Do you agree?

When I look at the list of “ontology” entries here, I see the opportunity to add things like “collect” or “monitor” under “Actions”. Then, we could define “collect/oval” or “collect/X” under “collect”. 

All that said, I’m going to need to lean on Bill and/or Danny and/or others who have worked with implementing OpenDXL (or even other topic-favored approaches - so the SACM hackathon work comes to mind) to help us understand where this thinking is accurate or not.

Kind regards,

Adam

[Charles Schmidt]

Hi Adam,

Yes - "infinite" was a bad choice of words. Bad call on my part.

I've still got some concerns about how this would work, but every time I try to write them up I realize I'm making assumptions about how the queues would be set up that I'm not sure would be right. I agree that exploring this idea is something that might best be done via implementation (or at least by someone familiar enough with how queues might be implemented for this that their opinions are not rank speculation, as mine would be). Definitely something to examine further.

Thanks,
Charles

On Thursday, June 18, 2020 at 8:14:36 AM UTC-5, Adam Montville wrote:

Charles,

Thanks for your response. I think sometimes we engineer types can think of extreme edge cases and that we tend to think in terms of absolutes. I don’t see the collection languages as an infinite set, but instead as a discrete set of a handful to start with the option to add more as they are invented.  Do you agree?

When I look at the list of “ontology” entries here, I see the opportunity to add things like “collect” or “monitor” under “Actions”. Then, we could define “collect/oval” or “collect/X” under “collect”. 

All that said, I’m going to need to lean on Bill and/or Danny and/or others who have worked with implementing OpenDXL (or even other topic-favored approaches - so the SACM hackathon work comes to mind) to help us understand where this thinking is accurate or not.

Kind regards,

Adam

On Jun 17, 2020, at 4:05 PM, Charles Schmidt <schmidt...@gmail.com> wrote:

Hi Adam,

I hear your argument on being wary of turning "a checking language" into "a checking language and/or a vehicle for any other checking language". Something new that is a dedicated wrapper might make sense. 

Regarding your proposal, it would depend on the characteristics of a "topic", but I'm a bit skeptical that the message fabric can serve as the wrapper given the degree of flexibility we need. I would think it would require us to effectively have infinite topics. Then again, I will freely admit I'm still learning the paradigms of message fabric use.

Charles

--

To unsubscribe from this group, send email to scap-dev...@list.nist.gov

Visit this group at https://list.nist.gov/scap-dev-endpoint
---

To unsubscribe from this group and stop receiving emails from it, send an email to scap-dev...@list.nist.gov.