Re: South Carolina Agency Accountability Reports

[Owen Ambur]

Gaya, no, I had not yet requested the latest versions of the spreadsheets because I was waiting to hear if you thought you could automate the process of converting them to StratML format.

However, in follow up to your message, I see that the updated reports are available at https://admin.sc.gov/budget/accountability and that the format has been somewhat improved.  See, for example, the report for the Department of Administration, which is responsible for providing guidance to the other agencies on preparation of the reports.  Obviously, however, there is still a lot of opportunity to make the data in the spreadsheets more usable and useful not only to citizens and taxpayers but also to legislators and even the agency administrators themselves.

Coincidentally, at a rally for U.S. Senator Tim Scott today, I had a chance to speak again not only with S.C. Representative Weston Newton, who chairs the Legislative Oversight Committee, but also our own S.C. Representative Jeff Bradley.  I reminded them of what I'm aiming to do and both remembered my previous conversations with them.  Jeff thanked me and Weston offered to help me again if we need assistance in getting copies of the updated spreadsheets but I doubt that will be necessary.  I will send a message to the Department of Administration and follow up with Weston if I don't hear back from them promptly.

I'm anxious to see what we can do with this data because we don't yet have many performance indicators available in StratML Part 2 format.  I'm copying Naval and Pradeep because after we have successfully developed and made a minimally viable StratML Part 1 query service available at https://aboutthem.info/ an obvious follow-on project will be to demonstrate how performance indicators available in StratML Part 2 format can be more usefully presented, including in graphical formats.  (With reference to performance "dashboards," see my September 12, 2017, article in Government Computer News entitled "A plethora of YADs: Let's hope they point to a more intelligent future.")

If and, hopefully, when we are able to do that, I'll make Senator Scott and well as U.S. Rep. Nancy Mace, with whom I've also spoken, aware of the opportunity to improve upon the Performance.gov site at the federal level.  I'll also remind officials of the Town of Hilton Head and Beaufort County of the opportunity to demonstrate leadership by example along these lines of pursuit at the local level.

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: gayaudeshani <gayaud...@gmail.com>
To: Owen Ambur <owen....@verizon.net>
Sent: Mon, Oct 3, 2022 1:06 am
Subject: Re: South Carolina Agency Accountability Reports

What is the update about this, did you check whether they have updated the format of the reports?

Regards

Gayanthika

On Mon, Aug 29, 2022 at 11:40 PM Owen Ambur <owen....@verizon.net> wrote:

Gaya, the plain XML renditions of the two files that I forwarded in .xslx format are at https://stratml.us/carmel/iso/part2/SCDE2018.xml & https://stratml.us/carmel/iso/part2/SCDNR2018wStyle.xml

The styled XML renditions are at the URLs listed in my message below.  Thanks to John Barker, they are also available in PDF at https://stratml.us/drybridge/index.htm#SouthCarolina or, more specifically, https://stratml.us/metaformixis/SCDE2018.pdf & https://stratml.us/metaformixis/SCDNR2018.pdf

Beauty is in the eye of the beholder but the intent is to make the information as readily discoverable, comprehensible, and useful as possible -- not merely in terms of presentation but also query services.

I agree that the conversion is not particularly complicated, just cumbersome to perform manually.  That's why I only converted a dozen rather than all of the reports listed on the SC Legislature's site.  As to be expected, however, there are also some data quality issues, e.g., text in numerical cells, etc.

BTW, I just noticed the last time that page was updated was December 3, 2018.  So if you think you can automate the process, before you put much effort into it, we should probably check to see if more recent reports are available in .xslx format.

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: gayaudeshani <gayaud...@gmail.com>
To: Owen Ambur <Owen....@verizon.net>
Sent: Mon, Aug 29, 2022 1:18 am
Subject: Re: South Carolina Agency Accountability Reports

I had a look into this, these files contain tables with data (not that complicated) so this information can be extracted into a CALS table, what is the expected output? Do you have a sample starXML for me to have a look at? may be a simple sample explaining these columns from the excel should go here in the output xml ....like that..

On Tue, Aug 23, 2022 at 8:46 AM Owen Ambur <Owen....@verizon.net> wrote:

Attached are a couple of the .xlsx files from a few years ago.  I have all of them in a .zip file, but if you think we can do something good with them, I'll request more current versions.

You can see what my StratML renditions look like at https://stratml.us/drybridge/index.htm#SouthCarolina or, more specifically,  https://stratml.us/carmel/iso/part2/SCDE2018wStyle.xml & https://stratml.us/carmel/iso/part2/SCDNR2018wStyle.xml

Owen

On 8/22/2022 1:29 PM, gaya udeshani wrote:

Could you please send me a sample xlsx file for me to have a look? Do they contain only tables?

Sent from my iPhone

On 22 Aug 2022, at 21:57, Owen Ambur <owen....@verizon.net> wrote:



Thanks for getting back to me, Gaya.

If NIST's catalog of security controls were actually machine-readable in the sense of having each discrete concept place in the appropriate columns and rows of the spreadsheet, I might be able to convert it to StratML format myself, by adding columns and inserting the StratML tags.  However, as per my exchange with Wendell Piez, that's not the case.  So converting the relatively unstructured contents of the "catalog" to actually machine-readable format is beyond my capability and I'll look forward to learning if it is anything you and your Typefi colleagues may wish to take on.

Of course, too, I'm still looking forward to hearing if you'd like to take a stab at converting the South Carolina agency accountability report data from .xslx to StratML format.

Owen

-----Original Message-----
From: gaya udeshani <gayaud...@gmail.com>
To: Owen Ambur <owen....@verizon.net>
Sent: Sun, Aug 21, 2022 3:10 am
Subject: Re: Standards Development & NIST's Strat Plan

Just to inform you I noticed your email, I’ll get back to you soon 

Sent from my iPhone

On 20 Aug 2022, at 21:42, Owen Ambur <owen....@verizon.net> wrote:



Thanks, Wendell, for your thoughtful response.

It's a bit more than I can productively comprehend right now.  However, I'm copying Pradeep and Megan in the event we might be able to make some sense of it in the context of the StratML query service that we aim to develop, leveraging the >5K plans already in the StratML collection.

Among them, a Google site-specific query reveals this reference to SP 800-53 in DOC's 2022 strategic plan.  

If you have any specific thoughts on how we might leverage OSCAL-facing work to make it less inscrutable and more useful in StratML format, please let me know.  

For example, while a catalog of security controls might be good, it is no substitute for performance plans and reports assigning stakeholder roles and documenting how well the controls are being applied.

BTW, since the catalog is available in .xlsx format, I'm copying Gayanthika Udeshani in the event that Typefi might be able to render it in StratML Part 2, Performance Plan/Report, format.

Owen

-----Original Message-----
From: Piez, Wendell A. (Fed) <wendel...@nist.gov>
To: Owen Ambur <Owen....@verizon.net>
Cc: Zubin Rustom Wadia <zu...@docugami.com>; Iorga, Michaela (Fed) <michael...@nist.gov>; Jeff Maynard <jmay...@turnkey.com.au>; Kurt Conrad <con...@sagebrushgroup.com>; Waltermire, David A. (Fed) <david.wa...@nist.gov>
Sent: Mon, Aug 15, 2022 8:55 am
Subject: RE: Standards Development & NIST's Strat Plan

Owen,

 

Forgive the latency I am now back from a week away. Maybe the robots were helpful.

 

From: Owen Ambur <Owen....@verizon.net>
Sent: Sunday, August 7, 2022 11:23 PM
To: Piez, Wendell A. (Fed) <wendel...@nist.gov>
Cc: Zubin Rustom Wadia <zu...@docugami.com>; Iorga, Michaela (Fed) <michael...@nist.gov>; Jeff Maynard <jmay...@turnkey.com.au>; Kurt Conrad <con...@sagebrushgroup.com>
Subject: Re: Standards Development & NIST's Strat Plan

 

Wendell, I'm not sure I understand the point you are making about "all kinds of metrics" but the core elements of all performance reports are the same.  All kinds of metrics can be reported in StratML Part 2, Performance Plans/Report, format.  What differs is the subject matter -- missions, goals, objectives, stakeholders, and performance indicators -- not the structure and semantics of the reports themselves.

Goodness – I can see we are already failing to communicate. Our nouns do not refer to the same things. To address this, I would have to go back and recapitulate the thread, anchoring it in context – maybe falling into more holes along the way. I think I won't. To find out more about OSCAL and its intended uses, you can consult any of our materials or indeed attend a workshop or public meeting. I am sure you would be very stimulated. If not encouraged.

Government Computer News invited me to write an article on that topic in 2017, in follow-up to two previous articles entitled "The open, efficient, machine-readable government" and "Government performance data: Let's make it open, machine-readable and permanent."

Thanks for the links! I will pass these along.

The NICE plan is now available in StratML Part 2 format at https://stratml.us/drybridge/index.htm#NICE  (There is a bit of latency associated with the Edit link but it should open the plan in my XForm for StratML Part 2, Performance Plan/Report, format.)  Presumably, someone is keeping track of progress on performance indicators associated with the goals and objectives it contains, but as per the good practice specified in section 10 of GPRAMA, it would be good if they were being published in open, standard, machine-readable format.

Indeed it would.

Are OSCAL's performance reports available on the Web, i.e., performance indicators for the objectives outlined in the plan for OSCAL? If so, I may wish to render them in StratML Part 2 format.

Our web site is not hard to find: http://pages.nist.gov/OSCAL. Go from there. It includes copious hand-authored content as well as machine-generated content. But it's probably not what you're looking for.

At the same time, I don't think the OSCAL documentation is where you should start, or indeed the OSCAL project (which after all is only one of many projects within our division, within an OU within an agency within a department). OSCAL itself is likely to be inscrutable (if not altogether baffling) if you do not first take on this:

https://github.com/usnistgov/oscal-content/blob/main/nist.gov/SP800-53/rev5/xml/NIST_SP-800-53_rev5_catalog.xml

which is an electronic rendering of the document found here:

https://csrc.nist.gov/publications/detail/sp/800-53/rev-5/final

which (in turn) is among the most widely downloaded pieces of public property on the Internet. (You can look up the figures.)

Any/all of this would be worthwhile transformation capabilities to have (in/out of StratML, which also serves as a bridge to much else, one should hope). An XSLT that could render this catalog (as we call it) into StratML could be used not only on this document but on any OSCAL catalog derived from it. This in turn leverages one of OSCAL's core features, namely that it offers an easy way for producing such catalogs programmatically, with the help of tooling. You would get all that for free.

As for the wider topic, it is of course very very extensive, to say nothing of deep. I think it would be interesting to see an argument for StratML in terms of the security controls that are described in SP 800-53. For example there is obviously SC-12, "Information Management and Retention" – although an open format does not in itself solve this problem, it is a sine qua non for solving it, is it not? But also check out SI-10, "Information Input Validation". (How can you even begin to do that without StratML or OSCAL or something analogous?) Fascinating stuff.

In any case we would be thrilled if you were to produce any OSCAL-facing work, as we are even now entering a phase of (we think) rapid growth wherein there are many opportunities for applications such as yours – both to help us realize the kinds of integrations we need (SC-29: Heterogeneity) and potentially to have an impact in your space (giving you another valuable connection for your pathway or "information bus").

Regards, Wendell

 

On 8/5/2022 10:12 AM, Piez, Wendell A. (Fed) wrote:

Hi Owen,

 

Cc:ing my supervisor Michaela Iorga here. 😊

 

The short answer is that all kinds of metrics not only performance metrics but also any kind of scan or log format etc., all sit outside the scope of OSCAL – directly outside, one might say, since they also constitute a very important (though not the only very important) segment of the widely disparate kinds of documents that must interface directly with OSCAL documents and systems. That is, OSCAL will represent metadata regarding all this stuff, summarize it and link to it, but does not seek to "subsume" it by representing it directly. (Not that we're against that: it's just not a goal.)

 

Largely because of the disparity of these formats but also because I'm not an expert (there is a lot to know), I can't answer your question regarding comparisons of those formats to StratML. With regard to OSCAL itself, my guess is that many or most kinds of OSCAL will convert fairly cleanly into StratML, but not the reverse, since OSCAL isn't actually a documentary format either. (It is more like an object model, supported by tagging, that is capable of supporting certain kinds of documentary information.)

 

To the extent this is the case (and OSCAL is not so large, at least considered as a tag set), any document published in OSCAL could be rendered in StratML for availability in systems that read StratML. But not all documents represented in StratML would convert easily into some or any form of OSCAL: that would depend on the case.

 

Best regards, Wendell

 

From: Owen Ambur <Owen....@verizon.net>
Sent: Friday, August 5, 2022 9:48 AM
To: Piez, Wendell A. (Fed) <wendel...@nist.gov>
Cc: Zubin Rustom Wadia <zu...@docugami.com>
Subject: Re: Standards Development & NIST's Strat Plan

 

Wendell, I had updated my address book to make your work address primary but neglected to check that my E-mail client used it on my last reply.

OSCAL's about statement has been in the StratML collection since June 2020, at https://stratml.us/drybridge/index.htm#OSCAL

W/re OSCAL Goal 3: Continuity - Enable Continuous Assessment, I'm curious to know: a) how performance metrics are being reported, i.e., in what format, and b) how that format compares to StratML Part 2.

Soon the NICE plan will also be available in StratML format.

Owen

 

On 8/5/2022 8:23 AM, Wendell Piez wrote:

Hi again Owen and Zubin,

 

The job email contact point for me is wendel...@nist.gov.

 

Currently I am playing much more of a support than a leadership role with respect to XML strategy (and everything else!) in the federal government, but in that capacity I am doing the best I can to promote the good work. You can research our project at pages.nist.gov/OSCAL (and related repositories).

 

Regards, Wendell

 

On Wed, Aug 3, 2022 at 7:56 AM Wendell Piez <wap...@wendellpiez.com> wrote:

Hi Owen,

 

Thanks for this. I'm cc'ing my day job email since a good response is likely to require a little research.

 

Write me back there too, anytime --

 

Cheers, Wendell

 

 

On Tue, Aug 2, 2022 at 3:40 PM Owen Ambur <Owen....@verizon.net> wrote:

Hey, Wendell, in follow up to Balisage discussion this afternoon, I'd like to render NIST's current strategic plan in StratML format for inclusion in my collection at https://stratml.us/drybridge/index.htm#NIST

However, I can't seem to find it.  This update is the best reference I've been able to turn up.

This reference indicates lots of room for improvement.

The most current version of NIST's Programmatic Plans seems to be for FY 2019, which is already available in StratML format at https://stratml.us/carmel/iso/NIST2017wStyle.xml

Perhaps you may have seen in the Balisage chat my reference to this objective in the Department of Commerce's plan:

Objective 1.2.2: Standards Development - Strengthen U.S. participation in technical standards development

However, the KPI associated with that objective is pretty weak:

KPI 1.2.3: Staff Trained - Number of U.S. Government staff trained to effectively coordinate, participate, and influence technical standards development

A better, more outcome-oriented metric would address the number of standards and SDOs in which Federal agencies are actually engaged.

I see the NTTAA reports at https://www.nist.gov/standardsgov/nttaa-reports but they leave a lot to be desired in terms of usability and utility.  Ideally, those reports would be published in StratML Part 2 format and the data from them would semi-automatically roll up to NIST's & DOC's performance reports.

Any chance you might be able to point me to a more current plan for NIST?

BTW, Google's top hit on "NIST strategic plan" is the NATIONAL INITIATIVE FOR CYBERSECURITY EDUCATION (NICE) Strategic Plan (2021-2025).  I'll probably convert it to StratML format.

Owen

[Owen Ambur]

Gaya, the StratML query service I hope to host at https://aboutthem.info/ hasn't been developed yet nor have I come to firm agreement with Naval and/or Pradeep on the cost and features.  However, a basic requirement is that the system must be capable of ingesting valid StratML (XML) instance documents.  

So if you can transform the SC agency accountability reports into valid StratML, that's all that should be necessary to "integrate" them into the query service.

The SC agency accountability reports that I have manually converted to StratML Parts 1 & 2 format are available at https://stratml.us/drybridge/index.htm#SouthCarolina

As far as the Excel files are concerned, it is probably best to wait to get the updated files in the event any changes have been made in them since I converted some of them nearly four years ago.

Beyond that, of course, I'd also be very interested to learn about any other value you think you and your Typefi colleagues might be able to add to data published in StratML format, bearing in mind not only the vision -- a worldwide web of intentions, stakeholders, and results -- but also the more explicit use cases and tool/app/service requirements documented thus far. 

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: gayaudeshani <gayaud...@gmail.com>
To: Owen Ambur <owen....@verizon.net>
Sent: Mon, Oct 3, 2022 11:35 pm
Subject: Re: South Carolina Agency Accountability Reports

Hi Owen

1. If we implement this in XSLT, how can we integrate this into your system? Is it possible to explain how you want to integrate this solution? 

2. I can remember I asked this earlier as well, but couldn't find your reply - Can you provide me with a sample excel file and the output star xml file so I can create a prototype using them?

Regards

Gayanthika

[Owen Ambur]

Gaya, I sent a request to the SC Department of Administration for a .zip file with the Excel spreadsheets for all of the agencies and will forward it to you as soon as possible.

Since I was able to manually convert the data to StratML by inserting new columns in the spreadsheet and placing the tags in them, I was pretty sure you could do that automatically but I didn't know whether you'd be up for the challenge of doing that.  I very pleased that you are.

Owen Ambur

https://www.linkedin.com/in/owenambur/

-----Original Message-----
From: gayaudeshani <gayaud...@gmail.com>
To: Owen Ambur <owen....@verizon.net>
Sent: Tue, Oct 4, 2022 12:29 am
Subject: Re: South Carolina Agency Accountability Reports

This is possible, let me know the next steps.

[gayaudeshani]

Thanks.