Zero-fault computer software in Healthcare

[Jack Ring]

Members,

Sharing this with youall. if you respond pls limit distribution to ASTEWG so that we do not become a burden on the Healthcare WG site.

Begin forwarded message:

From: Jack Ring <jri...@gmail.com>

Subject: Re: Zero-fault computer software

Date: December 23, 2016 at 12:42:39 AM MST

To: "Unger, Chris (GE Healthcare)" <Christop...@med.ge.com>

Cc: "Robert J. Malins" <rjma...@eaglesummittech.com>, Thomas Tenorio <teno...@gmail.com>

Wow! Thanks for the quick and clear response.

I will take you up on the ‘consult’ even of limited because it is clear that our WG comes at SE from a different perspective so we may learn a lot from the Healthcare WG.

For example, I note that your “customer” is the FDA. We have a similar situation in the DoD domain wherein the customer is the PEO in he Acquisition Phase rather than the warfighter in theater. 

By industry I meant the those your WG seeks to serve. Not to get too focused on GE I’ll just note that a key tenet as we developed GE strategy planning in the 1970’s was that GE should acknowledge and reflect the values of those it sought to serve. I understand that your WG does not practice medicine but I am interested in whether it seeks to help its clients avoid interfering with those who do practice medicine. In my perspective software containing bugs can do exactly that.

Unless it will be too burdensome we will keep you posted on our WG ideas, particularly opportunities for serving the INCOSE members and for aspects of healthcare that involve autonomous systems (which are coming on rapidly).

Again, thanks for your valuable time.

Jack Ring

On Dec 22, 2016, at 7:51 PM, Unger, Chris (GE Healthcare) <Christop...@med.ge.com> wrote:

Jack: quick answers to your three questions.  Each could justify a whole book I response!

1. Should all software associated with any facet of healthcare be certified as Fault-free?

If this were “free”, certainly, but there is a cost/benefit tradeoff.  The FDA assumes you have done a safety analysis, and you pay special attention to the safety critical elements of the SW.  However, they don’t insist that even those elements of the SW be “fault-free”.  They have a standard of ALARP (as low as reasonably possible).

Just as the safest plane is one that never flies, the safest procedure is one you never do.  The FDA wants the devices to bring more benefit than harm.  You’ve watched, I’m sure, as they get hammered by drugs that get released too soon (Vioxx…) and by the complaint that drugs that can save lives are held up in 7 year clinical trials.

2. Are the industry’s software practices and achievements consistent with the medical profession’s ethics?

“The industry” is rather vague, but I’d have to say no.  I’m sure there are companies with higher standards than others.

But you have an assumption that device developers SHOULD have the same ethics as “the medical profession” (not just “the engineering profession).  GE has a statement that “we don’t practice medicine.”  That isn’t cynical…the FDA once asked us to limit dose because radiologists were abusing the system and overdosing patients. But, then we would limit the ability to image a patient who is very sick.  The doctor has to make the tradeoff…is this a patient is well, and undergoing screening (very low dose), a recovering patient who is being monitored (moderate dose, since they will have repeat scans), or an acute patient (where every second counts, and you want the best image to make the right diagnosis in seconds…higher dose).  

This is a very subtle and complicated area. 

3. Does the Healthcare WG have an adequate focus and degree of intent toward Fault-free software?

Simply, no.  We can’t do everything; there are an almost unlimited number of good topics we are not addressing, and this is one of them.  If you would like to work on it, we could ‘consult’, but even that would be limited.  We are stretched and burning out our volunteers even with our current priorities.

 

Chris c: 262-424-9348

 

-----Original Message-----
From: Jack Ring [mailto:jri...@gmail.com] 
Sent: Thursday, December 22, 2016 7:56 PM
To: Robert J. Malins <rjma...@eaglesummittech.com>; Unger, Chris (GE Healthcare) <Christop...@med.ge.com>
Cc: Thomas Tenorio <teno...@gmail.com>
Subject: EXT: Zero-fault computer software

 

Gentlemen,

 

Just now saw your draft slides for IW17. Very impressive amount of work.

The Autonomous Systems Test and Evaluation WG is not as mature.

 

For our progress would you kindly respond to these three questions.

1. Should all software associated with any facet of healthcare be certified as Fault-free?

2. Are the industry’s software practices and achievements consistent with the medical profession’s ethics?

3. Does the Healthcare WG have an adequate focus and degree of intent toward Fault-free software?

 

The principals in the INCOSE Autonomous Systems Test and Evaluation WG would appreciate a dialog on these questions and any that you have.

 

How and when would someone in your WG be available to proceed?

 

Sincerely,

Jack Ring

Co-lead, ASTEWG

Fellow, INCOSE

[Jack Ring]

More info regarding scope of INCOSE Healthcare WG.

Begin forwarded message:

From: Jack Ring <jri...@gmail.com>

Subject: Re: Zero-fault computer software

Date: December 23, 2016 at 12:30:25 PM MST

Chris,

I suspect that our 2016 findings will not be very interesting by 2018 but am willing to follow your lead on that. I am behind schedule in summarizing the findings to date, particularly as a system model. 

We did note that there are a lot more agents involved than just doctors but chose to make the licensed doctor the focal point and rely on them to marshal whatever other 'health assurance’ assets and services are appropriate. 

The notion of health assurance rather than healthcare became widely popular. For example, one doctor said that 80% of health problems are due to diet so health assurance should start there.

I would love to encourage your WG about the role of Quality in the systemics of healthcare but will control myself. 

Greatly appreciate your attention.

Enough.

Jack

On Dec 23, 2016, at 12:00 PM, Unger, Chris wrote:

Jack:

 

The series of sessions on "The Ideal Doctor-Patient Visit" is interesting…that might be a good topic for a panel or roundtable at IS2018.  At IS2016 we had a session on “Keeping the patient out of the hospital” (how to deliver healthcare remotely).  As disease gets more chronic and as digital takes off, it might be interesting to generalize your topic to “Mapping the Ideal Patient-Caregiver Interaction” (not all caregivers are doctors, and not all interactions are a visit).  Do you think that might be a good topic to submit in November?

 

As for your last three questions, the WG doesn’t really have a position on any of them.  As I wrote initially, we don’t cover everything in healthcare, so we’ve made priority calls on what we discuss and work on.

 

Chris

 

From: Jack Ring [mailto:jri...@gmail.com] 
Sent: Friday, December 23, 2016 12:01 PM
Subject: EXT: Re: Zero-fault computer software

 

Chris,

Thanks for the clarification. Yes, customers are the source of revenue first and source of rules or constraints second. 

 

I read, perhaps misread, that the FDA was the main determinant of whether your WG pursued fault-free software. But thanks for mentioning them because it clarifies to me that our WG should be addressing the FDA directly as well as the FAA and DoD.

 

I should have addressed the notion of ‘free’ because like "Quality is Free" so also is fault-free software free, particularly when the cost of cyberattacks from external sources is close to a trillion dollars per year and growing 20% per year. We are pretty sure that fault-free software is much less vulnerable to such attacks.

 

As a member of ThinkTankPhoenx, not an INCOSE ASTEWG group, I recently facilitated several sessions on "The Ideal Doctor-Patient Visit" which specifically excluded other influences such as regulators, insurers, etc. Participants had hard time not thinking about such influences but responded quite well to a ‘systems engineering way of seeing the situation’ and became clear that these ‘third parties’ were part of the problem of sustaining and improving citizen health, not part of the solution. 

 

So, now that you have 'hugged the tar baby’ please tell us why a) your WG holds that the patient is not really the customer, b) whether fault-free software is free, and c) whether the producer of software is their own customer.

 

Meanwhile, lets be clear that none of this is a critique of GE.

 

Gratefully,

Jack Ring

 

On Dec 23, 2016, at 5:28 AM, Unger, Chris  wrote:

 

Jack:

 

One minor clarification.  I don’t know who my customer is, but it certainly isn’t the FDA.  They are a regulator, and a “constraint”.  (I could wax philosophical about what little I know about the relative strengths of the FAA and FDA as regulators, but I haven’t worked in aviation, so I only know indirectly).

 

GEHC sometimes gets confused about whether the user/buyer is the customer or the patient.  I was in a meeting a few days ago where marketing tried to map out the value stream from technician to radiologist to department chair and referring physician.  It didn’t take all that long to list the names (but they kept trickling in), but agreeing on what was the real value the system delivered to whom was not at all clear.  And who makes the buying decisions is very unclear…it used to be the radiologist.  Now, it can be the radiologist (implicitly), or the administration, or sometimes central (if the hospital is part of a chain or group).

 

I write this since we sometimes talk about the patient being the customer.  They aren’t really, but it is a very useful story.  I know Medtronic holds “customer appreciation days” where people who have their implants come in and visit the production lines (with engineers also present).  That makes “quality” real to everyone.  I assume others in healthcare field blur the concept of “customer” and “user/patient”.

 

Chris 

 

From: Jack Ring [mailto:jri...@gmail.com] 
Sent: Friday, December 23, 2016 1:43 AM
Subject: EXT: Re: Zero-fault computer software

 

Wow! Thanks for the quick and clear response.

 

I will take you up on the ‘consult’ even of limited because it is clear that our WG comes at SE from a different perspective so we may learn a lot from the Healthcare WG.

 

For example, I note that your “customer” is the FDA. We have a similar situation in the DoD domain wherein the customer is the PEO in he Acquisition Phase rather than the warfighter in theater. 

 

By industry I meant the those your WG seeks to serve. Not to get too focused on GE I’ll just note that a key tenet as we developed GE strategy planning in the 1970’s was that GE should acknowledge and reflect the values of those it sought to serve. I understand that your WG does not practice medicine but I am interested in whether it seeks to help its clients avoid interfering with those who do practice medicine. In my perspective software containing bugs can do exactly that.

 

Unless it will be too burdensome we will keep you posted on our WG ideas, particularly opportunities for serving the INCOSE members and for aspects of healthcare that involve autonomous systems (which are coming on rapidly).

 

Again, thanks for your valuable time.

Jack Ring

On Dec 22, 2016, at 7:51 PM, Unger, Chris wrote:

 

Jack: quick answers to your three questions.  Each could justify a whole book I response!

1. Should all software associated with any facet of healthcare be certified as Fault-free?

If this were “free”, certainly, but there is a cost/benefit tradeoff.  The FDA assumes you have done a safety analysis, and you pay special attention to the safety critical elements of the SW.  However, they don’t insist that even those elements of the SW be “fault-free”.  They have a standard of ALARP (as low as reasonably possible).

Just as the safest plane is one that never flies, the safest procedure is one you never do.  The FDA wants the devices to bring more benefit than harm.  You’ve watched, I’m sure, as they get hammered by drugs that get released too soon (Vioxx…) and by the complaint that drugs that can save lives are held up in 7 year clinical trials.

2. Are the industry’s software practices and achievements consistent with the medical profession’s ethics?

“The industry” is rather vague, but I’d have to say no.  I’m sure there are companies with higher standards than others.

But you have an assumption that device developers SHOULD have the same ethics as “the medical profession” (not just “the engineering profession).  GE has a statement that “we don’t practice medicine.”  That isn’t cynical…the FDA once asked us to limit dose because radiologists were abusing the system and overdosing patients. But, then we would limit the ability to image a patient who is very sick.  The doctor has to make the tradeoff…is this a patient is well, and undergoing screening (very low dose), a recovering patient who is being monitored (moderate dose, since they will have repeat scans), or an acute patient (where every second counts, and you want the best image to make the right diagnosis in seconds…higher dose).  

This is a very subtle and complicated area. 

3. Does the Healthcare WG have an adequate focus and degree of intent toward Fault-free software?

Simply, no.  We can’t do everything; there are an almost unlimited number of good topics we are not addressing, and this is one of them.  If you would like to work on it, we could ‘consult’, but even that would be limited.  We are stretched and burning out our volunteers even with our current priorities.

 

Chris