[WG-IDAssurance] Consideration of 'comparable alternatives'
Richard G. WILSHER (@Zygma)
A Federal agency has raised a question regarding how Kantara might handle ‘comparative alternatives’ in a 63A assessment, with specific reference being given to SP 800-63 rev.3 §5.4. This is my response to that question and a proposal as to how this could be addressed by Kantara. I submit this as an item for the IAWG’s consideration at this week’s meeting.
In response to the question, the first observation is that Kantara has no explicit criteria addressing normative statements in the SP 800-63 rev3 base document, only from 800-63A/B/C (rev.3). Most of these criteria are not so much directed at the implementation of proofing (63A), authenticating/managing (63B) or federating (63C) criteria per se as guiding (albeit in normative terms) Federal agencies as to how to perform their risk assessment within the overall scope of their implementation of the NIST RMF and then how to select appropriate ALs. The application of such guidelines/requirements to Federal agencies is therefore beyond the scope of Kantara’s remit, so I’m not sure that Kantara would/could/should have anything to state formally on the matter in the specific context of agencies.
However, what if an agency sought to have such changes as are addressed in -63 rev.3 §5.4 para. 3. to be implemented by a CSP which was assessed against Kantara’s criteria in 63A_SAC/63B_SAC (and potentially, 63C_SAC). None of -63x refer to ‘compensating controls’ but all three have requirements that the CSP employs “appropriately-tailored security controls, to include control enhancements, from […] controls defined in SP 800-53 or equivalent federal (e.g., FedRAMP) or industry standards”. Conveniently 800-53 defines ’compensating controls’ as “controls employed in lieu of the controls in […] 800-53 that provide equivalent or comparable protection for a system or organization”.
My IS27001 experience would lead me to expect an organization to perform a risk assessment, to describe the thresholds for acceptable risk, and to provide a defined and repeatable approach to quantify the perceived risks and then decide on a course of mitigation, those mitigations then to be approved and accepted by the organization’s ‘top management’ (to use IS27001 parlance). I.e. demonstrate quantatively how the compensating control(s) is(are) equivalent or comparable. An auditor should not be deciding whether an organization’s risk assessment and management approach is right or wrong, only that it leads to reasonable judgements about risk which management knowingly accepts. ‘Reasonable’ I think is not so easily defined, but is fairly obvious when it doesn’t exist – my pathetic example is ‘risk management by dice’, which would clearly be a random process rather than a judgmental one which would have repeatable or meaningful outcomes – except an average value over time, for all risks!
My approach to assessing such an instance would be that, if a CSP had an approach which was intended to compensate for a specific requirement of 800-63 I would ask them to define the nature of the risk which they believed NIST’s editors were trying to mitigate through the provision of the specific requirement and then to provide a risk analysis of their alternative approach to demonstrate that the risk assessment of their solution yielded, at least, the same level of residual risk as the original required control(s).
Frankly, if the argument appears to be rational and management accept it, then there is little the assessor can do but say OK. Again, it is not the auditor’s job to argue whether a risk likelihood is 70% not 75%, but only when it is being gauged as 40% when the 75% is an obviously more realistic value which perhaps can be justifiably expected (e.g. through empirical evidence).
From Kantara’s view, it at present does not have much visibility of any such assessment which might be made, but in future the auditor should be obligated to report on the existence of a compensating control and how they assessed and accepted it (or not, if that be the case).
I suggest that the best way for Kantara to address this is to introduce a new criterion which states:
If a CSP implements alternative controls for any specific proofing requirements for the applicable xAL, the CSP SHALL ensure that:
a) a quantitative analysis of the risk(s) addressed by the NIST-specified control and the degree of residual risk its implementation would achieve is determined and documented;
b) a justification of how the compensating measures deployed achieve equivalent or comparable mitigation of the same risks is determined and documented;
c) a justification of how different risks that may be introduced through the use of the alternative controls have been taken into account such that the same, or lower, residual risk can be achieved is determined and documented;
d) that the responsible management are aware of and accepting of that analysis and the implementation of the selected controls;
e) that the selected counter-measures are deployed;
f) how to configure the service to provide a risk mitigation level commensurate with the consumer’s own risk mitigation needs is documented and provided to users who wish to rely upon the service at the claimed xAL.
It is my consideration that such a criterion would apply to both IAL2 and IAL3, for CSP and Fed Agencies, and possibly the final text should include a phrase such as ‘commensurate with the applicable xAL’, but that’s fine-tuning.
Your thoughts are invited.
Richard G. WILSHER
Founder & CEO, Zygma Inc.
Operating independently since 1993
M: +1 714 797 99 42
E: R...@Zygma.biz
W: www.Zygma.biz
Richard G. WILSHER (@Zygma)
Valued input Nathan, as ever. Completely in accord, and I for one am not blasé in my auditing, but my clients seem to have me back, which is encouraging.
As we develop the Kantara Certification Board I believe that such measures as you propose, i.e. the auditors themselves being held a little more accountable, will be meaningful.
Richard G. WILSHER
Founder & CEO, Zygma Inc.
Operating independently since 1993
M: +1 714 797 99 42
E: R...@Zygma.biz
W: www.Zygma.biz
From: Nathan Faut [mailto:n_faut...@yahoo.com]
Sent: Monday, June 28, 2021 00:12
To: IA WG; Richard G. WILSHER (@Zygma)
Subject: Re: [WG-IDAssurance] Consideration of 'comparable alternatives'
Richard and colleagues -
I want to add color to one comment from below:
Frankly, if the argument appears to be rational and management accept it, then there is little the assessor can do but say OK. Again, it is not the auditor’s job to argue whether a risk likelihood is 70% not 75%, but only when it is being gauged as 40% when the 75% is an obviously more realistic value which perhaps can be justifiably expected (e.g. through empirical evidence).
I have reviewed a few risk assessments in my time, and several lack sufficient rigor. Using Richard's brief example above, I would agree the auditor cannot argue .70 likelihood v. .75 likelihood (keeping likelihood metrics b/w 0 and 1, not %s) - but the auditor should consider the discussion prior to or surrounding the assessments of comparable alternatives; often assumptions are made but not documented, for example. Or assumptions are incomplete, not fully considered.
As Richard indicates, after a point - at the auditor's discretion - analysis and discussion exhaust themselves and the auditor will have to accept the comparable alternatives risk assessment. But I don't want the auditors to be as blase as to accept whatever document may pass as this assessment just because it is called an assessment, has some figures and discussion, and has management signature. In the past I have pushed back - hard - on the lack of content, rigor, or well-rounded, thorough analysis, e.g., missing certain risks, missing mitigating approach/es discussion, etc.
(I say this b/c I am aware of a recent discussion at an organization with which I have some familiarity wherein a risk assessment will be forwarded to an auditor but it will be called a business impact analysis [BIA]. Management reviewed this risk assessment, and accepted it, but will turn around and forward it to the auditor as the BIA. So, the document has content, has some form of analysis, but a knowledgeable auditor will and should know the difference between a risk assessment and a business impact analysis. My understanding of this situation is such that the auditor likely either will not know the difference or despite knowing the difference will accept the BIA as it is offered at face value without rigor and internal analysis and probing. So, I raise this concern as a legitimate Warning Sign about which the ARB will be careful.
(Auditors may want to include a comment in their documentation for this element, stating perhaps something like, "The auditor inspected the comparable alternatives risk assessment, noted that the analysis documented # alternatives and analysis tied the comparable alternatives' metrics to the discussion. We also noted that management had reviewed and approved the analysis. We determined that the analysis appeared reasonable, well-considered, and approved by management," or similar in content.)
I thank you all for your time and attention. I now return you back to your regularly scheduled email message/s.
-Nathan =-=-=-=-=-=-=-
_______________________________________________
WG-IDAssurance mailing list
WG-IDAs...@kantarainitiative.org
https://kantarainitiative.org/mailman/listinfo/wg-idassurance
Jimmy Jung
Speaking for un-blasé assessors; I would suggest that the ARB should be more than careful, they should be significantly involved.
We have criteria, and we are well positioned to say something does or doesn’t meet that criteria – but if an assessee says we don’t meet the criteria, but we have something just as good; them I’m going to want more than just one assessor giving the OK. If an assessee can convince an assessor that their compensating controls actually compensate, I would still achieve the “visibility” Richard is asking for by reporting on the existence of a compensating control all over the S3A and the assessor’s report. I’m not about to accept a compensating control without making it VERY clear to the ARB that one is in place. Perhaps you see a future where the auditor is obligated to report on the existence of a compensating control, but that is how I’m living it now.
Federal Agencies are used to compensating controls as part of the SP 800-37 /800-53 process. and when 63-3 says, "Agencies MAY determine alternatives to the NIST-recommended guidance, for the assessed, based on their mission, risk tolerance, existing business processes ... that is very much NIST 800-37 language. In these, a system owner or perhaps CIO accepts the risk. Here I believe the risk is to the Kantara brand. The value of the Kantara brand is saying that we know specific criteria have been met. In instances where that is not precisely the case I think we need a lot of transparency before Kantara accepts that risk.
I think Richards proposed new criterion is in tweaking range and agree with the idea that assessors are not going to deliver fine grained quantitative risk numbers; but my emphasis would be on the transparency and visibility aspects of compensating controls. I wonder if the CSP should also provide a justification of why the original criteria cannot be implemented. I’m sure the answer will most likely be, “because this is how we built it before we even thought of Kantara, and we don’t want to spend money;” but still, this is the criteria they signed up for and we need to strike a balance regarding what our brand means and when people can go astray.
My apologies if I seem to rant, spent the morning in a fun little “waiver/risk acceptance” debate. Also my apologies; as I am doubtful for tomorrow.
Jimmy
|
|
Jimmy Jung vaxed, relaxed and ready for snacks |
