The GRADE Ontology Working Group Meeting was cancelled to allow participation in the GRADE working group meeting. The Tuesday Ontology Management meeting will include a discussion of the Spanish language translation of the GRADE Ontology.
The Risk of Bias Terminology Working Group found that of the 3 terms open for vote, language bias and geography bias received enough votes to be added to the code system. Publication bias had one negative vote with a request to change the comment for application. This was changed based on the comment and the term was reopened for vote. The group also discussed changes to the MyBallot software that make it much easier for administrators to tally and keep track of the votes.
Term
|
Definition
|
Alternative Terms
(if any) |
Comment for application
(if any) |
language bias-Passed
|
A bias in search strategy or study eligibility criteria that results from restrictions regarding the language of the study report.
|
|
Limiting the study reports included in a systematic review by language may result in an incomplete view of the truly available evidence. |
geography bias-Passed
|
A bias in search strategy or study eligibility criteria that results from restrictions regarding the geographic origin of the research.
|
|
Limiting the study reports included in a systematic review by country of origin may result in an incomplete view of the truly available
evidence. |
re-opened for voting |
A study selection bias in which the publicly available studies are not representative of all conducted studies.
|
|
Publication bias arises from the failure to identify all studies that have been conducted, either published (i.e., publicly available)
or unpublished. The term 'studies' means evidence or research results in any form where such studies would meet the study eligibility criteria without consideration of criteria regarding the form of publication. The phrase 'publicly available studies' means
the studies are available to the broad academic community and the public through established distribution channels in any form, including forms with restricted access. Established distribution channels include peer-reviewed journals, books, conference proceedings,
dissertations, reports by governmental or research organizations, preprints, and study registries. |
The Project Management Working Group created the proposed agenda for next week:
Day/Time (Eastern) |
Working Group |
Agenda Items |
Monday 8-9 am |
Project Management |
FHIR changes and EBMonFHIR Implementation Guide issues |
Monday 9-10 am |
Setting the Scientific Record on FHIR |
SRDR+ to FEvIR Review |
Monday 10-11 am |
ClinicalResearchOnFHIR (a CDS EBMonFHIR sub-WG) |
Create Evidence Resource examples |
Monday 2-3 pm |
Statistic Terminology |
Review SEVCO terms (2 terms open for vote) |
Tuesday 9 am-10 am |
Measuring the Rate of Scientific Knowledge Transfer |
Review Progress on Pilot Project and Software updates |
Tuesday 2-3 pm |
StatisticsOnFHIR (a CDS EBMonFHIR sub-WG) |
Review Baseline Measure Report Authoring Tool progress |
Tuesday 3-4 pm |
Ontology Management |
Review Objectives and Progress |
Wednesday 8-9 am |
Funding the Ecosystem Infrastructure |
Review progress on product launch readiness checklist for making guidelines computable |
Wednesday 9-10 am |
Communications (Awareness, Scholarly Publications) |
Global Evidence Summit, Study Design Paper |
Thursday 8-9 am |
EBM Implementation Guide (a CDS EBMonFHIR sub-WG) |
Review EBMonFHIR IG change requests |
Thursday 9-10 am |
Computable EBM Tools Development |
Review progress with Converter tools |
Friday 9-10 am |
Risk of Bias Terminology |
Review SEVCO terms (1 term open for vote) |
Friday 10-11 am |
GRADE Ontology |
Review GRADE terms (2 terms open for vote) |
Friday 12-1 pm |
Project Management |
Create Agenda for the Upcoming Week |
Releases on the FEvIR Platform:
FEvIR®: CodeSystem Builder/Viewer version 0.42.0 (May 3, 2024) creates and displays code system terms (concepts) in a CodeSystem Resource.
Release 0.42.0 (May 3, 2024) for CodeSystem administrators the vote count updates when you switch terms and now the vote tally also includes a count for duplicate votes and a vote count from before the open for voting date. The vote summary and list of voters are now automatically generated.
The FEvIR®: RAte of Dissemination Assessment Research (RADAR) Tool Version 0.14.0 (May 3, 2024) facilitates research conduct and project management for an investigation to measure the rate of scientific knowledge transfer.
Release 0.14.0 (May 3, 2024) in the Rate Articles table each article now has a link out to a PubMed query. Also an author’s last name no longer sometimes have a comma before the year in the article title list.
FEvIR® API version 0.8.0 (May 3, 2024) supports GET and POST requests to retrieve Resources from the FEvIR Platform, retrieve concept JSON from CodeSystem Resources on the FEvIR Platform, or convert a ClinicalTrials.gov record to a FHIR Bundle Resource.
Release 0.8.0 (May 3, 2024) now allows fhirEntry to be a JSON object in addition to a string.
Quote for Thought: “As I learned from growing up, you don’t mess with your grandmother.” — Prince William
To get involved or stay informed: HEvKA Project Page on FEvIR Platform, HEvKA Project Page on HL7 Confluence, or join any of the groups that are now meeting in the following weekly schedule:
Day
|
Time (Eastern)
|
Team
|
Monday
|
8-9 am
|
Project Management
|
Monday
|
9-10 am
|
Setting the Scientific Record on FHIR WG
|
Monday
|
10-11 am
|
Clinical Research on FHIR WG (a CDS EBMonFHIR sub-WG)
|
Monday
|
2-3 pm
|
Statistic Terminology WG
|
Tuesday
|
9-10 am
|
Measuring the Rate of Scientific Knowledge Transfer WG
|
Tuesday
|
2-3 pm
|
StatisticsOnFHIR WG (a CDS EBMonFHIR sub-WG)
|
Tuesday
|
3-4 pm
|
Ontology Management WG
|
Wednesday
|
8-9 am
|
Funding the Ecosystem Infrastructure WG
|
Wednesday
|
9-10 am
|
Communications(Awareness, Scholarly Publications) WG
|
Thursday
|
8-9 am
|
EBM Implementation Guide WG (a CDS EBMonFHIR sub-WG)
|
Thursday
|
9-10 am
|
Computable EBM Tools Development WG
|
Friday
|
9-10 am
|
Risk of Bias Terminology WG
|
Friday
|
10-11 am
|
GRADE Ontology WG
|
Friday
|
12-1 pm
|
Project Management
|
Term
|
Definition
|
Alternative Terms
(if any) |
Comment for application
(if any) |
---|
A measure of heterogeneity, based on the chi-square statistic, for reporting an analytic finding regarding whether two or more multinomial distributions are equal, accounting for chance variability.
|
|
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies.
Chi square for homogeneity assesses whether observed differences in results are compatible with chance alone. A chi square for homogeneity is a hypothesis testing measure that is testing the hypothesis of heterogeneity. A chi square for homogeneity is distinct from a chi square for independence (also called Pearson's chi square). A chi square test for homogeneity is based on testing whether the distributions across two or more populati |
Term
|
Definition
|
Alternative Terms
(if any) |
Comment for application
(if any) |
---|
Newly open for voting
|
A study selection bias in which included studies are limited to those that support one perspective.
|
|
|
Newly open for voting
|
A bias in the conduct of an analysis combining two or more studies or datasets.
|
|
A synthesis bias results from methods used to select, manipulate or interpret data for evidence synthesis.
|
Newly open for voting
|
A synthesis bias in which eligible studies were not included in the evidence synthesis.
|
|
|
Passed
|
A study selection bias in which the publicly available studies are not representative of all conducted studies.
|
|
Publication bias arises from the failure to identify all studies that have been conducted, either published (i.e., publicly available) or unpublished. The term 'studies' means evidence or research results in any form where such studies would meet the study
eligibility criteria without consideration of criteria regarding the form of publication. The phrase 'publicly available studies' means the studies are available to the broad academic community and the public through established distribution channels in any
form, including forms with restricted access. Established distribution channels include peer-reviewed journals, books, conference proceedings, dissertations, reports by governmental or research organizations, preprints, and study registries.
Publication bias often leads to an overestimate in the effect in favor of the study hypothesis, because studies with statistically significant positive results are more likely to be publicly available. The terms reporting bias and selective reporting bias are used to describe biases in study reports, i.e., reporting bias. To avoid confusion between biases in study reports and biases in study selection, when either 'reporting bias' or 'selective reporting bias' are used as alternative for 'publication bias', the term is appended to 'study selection bias due to' as shown below: • study selection bias due to reporting bias • study selection bias due to selective reporting bias |
Term
|
Definition
|
Alternative Terms
(if any) |
Comment for application
(if any) |
Differences between the outcome-related characteristics of the evidence and the intended target application.
|
|
Indirectness in outcome means there are differences between the outcomes in the relevant research studies and the outcome under consideration in a question of interest. Differences in outcome-related characteristics (such as definition, method of measurement,
and timing) may result in differences in the estimates derived from the evidence and those that would be seen in the intended target application. Outcomes in the evidence may differ from those of primary interest. For instance, changes in an intermediate or
surrogate outcome may only partially reflect or predict the outcome of interest.
|
|
The state in which the range of probable values for the estimate of effect includes values on both sides of a threshold.
|
|
In the context of making a decision, the threshold is a threshold for which the decision would differ. Outside the context of making a decision, the threshold should be specified in the research question. The range of probable values is commonly expressed as
a confidence interval or credible interval. In some circumstances, imprecision may occur when the sample size is less than the optimal information size, or the total sample size is not large enough or the number of events is too small to provide a reliable
estimate of effect.
|
Day/Time (Eastern)
|
Working Group
|
Agenda Items
|
---|
Monday 8-9 am | Project Management |
FHIR changes and EBMonFHIR Implementation Guide issues |
Monday 9-10 am | Setting the Scientific Record on FHIR |
SRDR+ to FEvIR Review | ||
Monday 10-11 am | ClinicalResearchOnFHIR WG (a CDS EBMonFHIR sub-WG) | Create Resource Examples |
Monday 2-3 pm | Statistic Terminology |
Review SEVCO terms ( 1 term open for vote) |
Tuesday 9 am-10 am |
Measuring the Rate of Scientific Knowledge Transfer |
Review Progress on Pilot Project, and GES Poster |
Tuesday 2-3 pm | StatisticsOnFHIR (a CDS EBMonFHIR sub-WG) |
Guideline Authoring Tool progress |
Tuesday 3-4 pm | Ontology Management |
Spanish Translation and Review Objectives and Progress |
Wednesday 8-9 am | Funding the Ecosystem Infrastructure |
Review Progress on product readiness checklist for making guidelines computable |
Wednesday 9-10 am | Communications (Awareness, Scholarly Publications) |
Global Evidence Summit and Study design paper |
Thursday 8-9 am |
EBM Implementation Guide (a CDS EBMonFHIR sub-WG) | Review EBMonFHIR IG change requests |
Thursday 9-10 am | Computable EBM Tools Development |
Review progress with Converter tools |
Friday 9-10 am | Risk of Bias Terminology |
Review SEVCO terms (3 terms open for vote) |
Friday 10-11 am | GRADE Ontology |
Review GRADE terms (2 items open for vote) |
Friday 12-1 pm | Project Management |
Create Weekly agenda |
Term
|
Definition
|
Alternative Terms
(if any) |
Comment for application
(if any) |
---|
A measure of heterogeneity, based on the chi-square statistic, for reporting an analytic finding regarding whether two or more multinomial distributions are equal, accounting for chance variability.
|
|
|
|
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies. Chi square for homogeneity assesses whether observed differences in results are compatible with chance alone. |
A chi square for homogeneity is a hypothesis testing measure that is testing the hypothesis of heterogeneity. A chi square for homogeneity is distinct from a chi square for independence (also called Pearson's chi square). A chi square test for homogeneity is
based on testing whether the distributions across two or more populations are the same.
|
|||
A measure of heterogeneity that estimates the proportion of variability across studies that is in excess of the expected variability due to chance.
|
|
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies. 𝐼2I2 = 100%×(Q - df)/Q, where Q is Cochran's heterogeneity statistic and df the degrees of freedom 𝐼2I2 is strictly
non-negative and lies between 0 and 100%. (If calculation results in a negative value, Q is less than df, 𝐼2I2 is defined as zero.) There are competing methods of calculating the confidence interval such as Hedges and Piggott (2001) and Higgins and Thompson
(2002).
|
10 people (AK, BA, CE , HL, JD, KS, KW, MA, MH, RC) participated in up to 3 active working group meetings
The Measuring the Rate of Scientific Knowledge Transfer Working Group discussed steps to complete the pilot project before the group meets again. (There will be no meeting next week due to the HL7 Connectathon.) The group then turned to discuss the protocol for the final project which needs to be in final form before we can start creating a poster to be presented at the September 2024 Global Evidence Summit.
The StatisticsOnFHIR Working Group (a CDS EBMonFHIR sub-WG) and the Ontology Management Working Group continued to review progress on a ResearchStudy example https://fevir.net/resources/project/195718, https://fevir.net/resources/195732.
Quote for Thought: “When you know, to know (recognize) that you know; and when you do. not know, to know (recognize) that you do not know; that is knowledge.” ( Analects 2.17)1. 1. Confucius
6 people (BA, CE , JD, JO, KS, MA) participated today in up to 2 active working group meetings.
The Funding the Ecosystem Infrastructure Working Group discussed a potential collaboration to produce a Learning Health Systems track at the September 2024 HL7 Connectathon which would focus on making guidelines computable and keeping existing computable guidelines up to date when additional information becomes available. We specifically suggested a demonstration of making the recent KDIGO guideline computable. This would require the help of a few subject matter experts to enter the data. We would also like to develop a use case to demonstrate the exchange of this information.
The Communications Working Group discussed logistics for the 4 posters and one special session that we will bring to the Global Evidence Summit in September 2024 in Prague.
Releases on the FEvIR Platform:
The Fast Evidence Interoperability Resources (FEvIR) Platform is available for use now, but is “pre-release”. The current version is 0.222.0 (May 15, 2024). Viewing resources is open without login. Signing in is free and required to create content (which can then only be edited by the person who created the content).
Release 0.222.0 (May 15, 2024) improves efficiency of editing Composition sections and retains Narrative text.status value as ‘empty’ when the text.div content contains instructional text instead of actual content.
Computable Publishing®: M11 Report Authoring Tool version 0.9.0 (May 15, 2024) creates and displays a Composition Resource with an M11Report Profile
Release 0.9.0 (May 15, 2024) limits editing to section text and empty reasons across all sections (reducing data entry for author, ordered by, and entry content in most sections); reshapes the Trial Objectives and Estimands section to allow adding sections for each of Primary, Secondary, and Exploratory Objectives; reshapes the Statistics section to support data entry for multiple endpoints; enables creation of Group Resource (StudyEligibilityCriteria Profile) in the Trial Population section, Group Resource (ExposureDefinition Profile) and ActivityDefinition Resource entries in the Trial Intervention section, Evidence Resource (EndpointAnalysisPlan Profile) entries in the Statistics section, CodeSystem and ValueSet Resource entries in the Glossary section, and Citation Resource entries in the References section.
The FEvIR®: RAte of Dissemination Assessment Research (RADAR) Tool Version 0.14.1 (May 15, 2024) facilitates research conduct and project management for an investigation to measure the rate of scientific knowledge transfer.
Release 0.14.1 (May 15, 2024) changed the clickable area for “Open for anyone to join” (when making a new investigation) to prevent mis-clicks.
Quote for Thought: "To walk safely through the maze of human life, one needs the light of wisdom and the guidance of virtue." – Buddha
7 people (BA, CE , IK, JD, JT, KS, XZ) participated in up to 2 active working group meetings.
The EBM Implementation Guide WG (a CDS EBMonFHIR sub-WG) reviewed an example of a summary of findings https://fevir.net/resources/Composition/244857 and answered questions about the Computable Publishing®: Summary Of Findings Authoring Tool. We also discussed participant flow profile name changes.
The Computable EBM Tools Development Working Group reviewed the latest developments for the Computable Publishing®: M11 Report Authoring Tool and made suggestions for improvements.
Releases on the FEvIR Platform:
The Fast Evidence Interoperability Resources (FEvIR) Platform is available for use now, but is “pre-release”. The current version is 0.223.0 (May 16, 2024). Viewing resources is open without login. Signing in is free and required to create content (which can then only be edited by the person who created the content).
Release 0.223.0 (May 16, 2024) adjusts multiple Profiles to match revisions to the EBMonFHIR Implementation Guide, including adjusting the type values across all Profiles of Composition Resources, creating a ParticipantFlowReport Profile of Composition, and modifying ParticipantFlowEvidenceVariable, ParticipantFlowEvidence, and ComparativeParrticipantFlowEvidence Profiles.
Computable Publishing®: M11 Report Authoring Tool version 0.10.0 (May 16, 2024) creates and displays a Composition Resource with an M11Report Profile
Release 0.10.0 (May 16, 2024) applies multiple improvements and corrections from user feedback
Quote for thought: "We have two ears and one mouth so that we can listen twice as much as we speak."--Epictetus
11 people (BA, CA-D, HL, JD, JT, KS, KW, PW, SL, SM, TD) participated in up to 3 active working group meetings.
Announcement: Many of our HEvKA meetings (including terminology and ontology meetings ) will occur next week despite announcements to the contrary! Please join us for these meetings, see HEvKA meeting schedule/agenda below.
The Risk of Bias Terminology Working Group found that synthesis bias passed with a vote of 5-0. The group discussed one-sided reference bias and synthesis missing eligible studies which received negative votes and changed the comments for application for both terms and the definition of one-sided reference bias. Both of these terms have been reopened for voting. This group will meet next week.
Term |
Definition |
Alternative Terms |
Comment for application |
A study selection bias in which included studies are limited to those that represent one perspective. |
The term "perspective" covers a variety of contexts, such as a side of an argument or point of view. |
||
A synthesis bias in which eligible studies were not included in the evidence synthesis. |
A typical use of this bias is when studies were included in the overall review but were not included in a specific meta-analysis. |
The GRADE Ontology Working Group discussed a comment from the expert working group regarding the term, "Indirectness in outcome" which resulted in a change in the comment for application. The term has been re-opened for voting even though the term initially received all positive votes 14-0. The group then discussed comments received during voting for the term "Imprecision" which received 4 negative votes. We will continue to discuss this term next week, it has not been re-opened for voting. We decided at the very end of the meeting that we will be able meet next week (May 24,2024). The Meeting was recorded and can be viewed here.
There is one term open for voting, if you have already voted on this term, please vote again, as its content has changed.
Term |
Definition |
Alternative Terms |
Comment for application |
Differences between the outcome-related characteristics of the evidence and the intended target application. |
|
Indirectness in outcome means there are differences between the outcomes in the evidence under consideration and the outcomes specified in the question of interest ("target application"). Differences in outcome-related characteristics (such as definition, method of measurement, and timing) may decrease confidence that the estimates derived from the evidence would be seen in the intended target application. Sometimes the outcomes of primary interest in the target application may differ from those available in the evidence. For instance, the evidence may only provide information about intermediate or surrogate outcomes that only partially reflect or predict the outcome of interest. |
The Project Management Working Group created the proposed agenda for next week: (Please note that a significant number of meetings are canceled for next week to allow participation in the HL7 Connectathon in Dallas, Texas.)
Day/Time (Eastern) |
Working Group |
Agenda Items |
Monday 8-9 am |
Project Management |
(CANCELED May 20 for HL7 WGM) |
Monday 9-10 am |
Setting the Scientific Record on FHIR |
(CANCELED May 20 for HL7 WGM) |
||
Monday 10-11 am |
ClinicalResearchOnFHIR WG (a CDS EBMonFHIR sub-WG) |
(CANCELED May 20 for HL7 WGM) |
Monday 2-3 pm |
Statistic Terminology |
Review SEVCO terms (2 terms open for vote) |
Tuesday 9 am-10 am |
Measuring the Rate of Scientific Knowledge Transfer |
(CANCELED May 21 for HL7 WGM) |
Tuesday 2-3 pm |
StatisticsOnFHIR (a CDS EBMonFHIR sub-WG) |
Authoring Tool progress |
Tuesday 3-4 pm |
Ontology Management |
Review Objectives and Progress |
Wednesday 8-9 am |
Funding the Ecosystem Infrastructure |
(CANCELED May 22 for HL7 WGM) |
Wednesday 9-10 am |
Communications (Awareness, Scholarly Publications) |
(CANCELED May 22 for HL7 WGM) |
||
Thursday 8-9 am |
EBM Implementation Guide (a CDS EBMonFHIR sub-WG) |
Review EBMonFHIR IG QA and update examples |
Thursday 9-10 am |
Computable EBM Tools Development |
Review progress with Authoring tools |
Friday 9-10 am |
Risk of Bias Terminology |
Review SEVCO terms (2 terms open for vote) |
||
Friday 10-11 am |
GRADE Ontology |
Review GRADE terms (Indirectness of outcome, Imprecision) |
Friday 12-1 pm |
Project Management |
(CANCELED May 24 for HL7 WGM) |
Releases on the FEvIR Platform:
Releases for May 17, 2024 2024 (Highlight in green once ready, on live site and AppTools updated)
The Fast Evidence Interoperability Resources (FEvIR) Platform is available for use now, but is “pre-release”. The current version is 0.224.0 (May 17, 2024). Viewing resources is open without login. Signing in is free and required to create content (which can then only be edited by the person who created the content).
Release 0.224.0 (May 17, 2024) revises the data entry for Narrative datatype elements to avoid non-compliant empty values when the content is deleted.
Computable Publishing®: M11 Report Authoring Tool version 0.11.0 (May 17, 2024) creates and displays a Composition Resource with an M11Report Profile
Release 0.11.0 (May 17, 2024) has a Send via API button that will transmit the FHIR Bundle Resource for the document, initially limited to a send to a test server for the Vulcan UDP Track of the HL7 FHIR Connectathon.
FEvIR®: Bundle Builder/Viewer version 0.3.0 (May 17, 2024)provides a human-friendly summary of a Bundle Resource.
Release 0.3.0 May 17, 2024) corrects the handling of the identifier element to have only one instance for Bundle Resources (instead of an array used for other FHIR Resources).
Quote For Thought: "When you synthesize evidence, there are opportunities for mischief" – Harold Lehmann
2 people (BA, KW) participated in 1 active working group meeting.
Several HEvKA meetings were cancelled to allow participation in the HL7 Connectathon and Working Group Meeting in Dallas, Texas.
During the Connectathon:
The Evidence Based Medicine Track worked closely with the Vulcan UDP Track. The FEvIR Platform was used to transfer a Composition Resource with the M11Report Profile (sub-profile of EvidenceReport) to the UDP server. The M11Report and EvidenceReport Profiles are part of the Evidence-Based Medicine on FHIR IG. Other than the Vulcan UDP collaboration, the Evidence-Based Medicine Track discussed user interfaces on FEvIR.net and ideas on how to represent data in the Evidence, EvidenceVariable, and Group Resources, especially considering the Inclusion and Exclusion Criteria using Group.characteristic where it references another Group Resource.
Participants: Khalid Shahin, Brian Alper, Sophie Klopfenstein, Irina Angel
The Vulcan UDP Track was the first of a series of Connectathons. The first iteration took a document-centric approach with 3 objectives:
1. Transfer of a clinical trial protocol from sponsor to regulator according to the development being done by ICH M11/M2, CDISC, and TransCelerate.
2. Look at use of profiles of Composition Resource to enable transfer of protocol document sections
3. Build experience and test implementation guidance
Participants: Hugh Glover, Vulcan (Track Lead); Brian Alper, Computable Publishing LLC; Matt Elrod, ONC; Dave Iberson-Hurst, CDISC; Dan Newingham, ZS Associates; Jimita Parekh, ICH M11/M2; Khalid Shahin, Computable Publishing LLC; Rik Smithies, HL7 UK; Kimberly Tableman, Espero; Panagiotis Telonis, ICH M11/M2; Stacy Tegan, Vulcan, TransCelerate
Notable achievements include:
Discovered issues include:
During the Working Group Meeting:
Several sessions attended today with the Structured Documents Work Group focused on the Composition Resource which is the basis for many Profiles in the EBMonFHIR Implementation Guide.
The Statistic Terminology Working Group revised the Comment for application for 1 term (I-squared) and drafted 1 term (tau squared). There are currently three statistic terms open for vote.
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies. 𝐼2I2 = 100%×(Q - df)/Q, where Q is Cochran's heterogeneity statistic and df is the degrees of freedom. 𝐼2I2 is strictly non-negative and lies between 0 and 100%. (If Q is less than df and the calculation would result in a negative value, then 𝐼2I2 is defined as zero.) There are competing methods of calculating the confidence interval such as Hedges and Piggott (2001) and Higgins and Thompson (2002). |
|||
A measure of heterogeneity that estimates the variance of the distribution of true effect sizes. |
|
A measure of heterogeneity is defined as a statistic that represents the variation or spread among values in the set of estimates across studies. The tau squared estimates the between-study variance in a random-effects meta-analysis or hierarchical multilevel model meta-analysis. |
To participate you can join the
Scientific Evidence Code System (SEVCO) Expert Working Group at https://fevir.net/resources/Project/27845.
Releases on the FEvIR Platform:
The Fast Evidence Interoperability Resources (FEvIR) Platform is available for use now, but is “pre-release”. The current version is 0.225.0 (May 20, 2024). Viewing resources is open without login. Signing in is free and required to create content (which can then only be edited by the person who created the content).
Computable Publishing®: Guideline Authoring Tool creates and displays a Composition Resource with a Guideline Profile.
Computable Publishing®: M11 Report Authoring Tool creates and displays a Composition Resource with an M11Report Profile.
Computable Publishing®: Recommendation Authoring Tool creates and displays a Composition Resource with a Recommendation Profile.
Quote for thought: “Stop being afraid of what can go wrong and start being positive about what can go right.” ―Unknown
Brian S. Alper, MD, MSPH, FAAFP, FAMIA
CEO, Computable Publishing LLC
Lead, HEALTH EVIDENCE KNOWLEDGE ACCELERATOR
President, Scientific Knowledge Accelerator Foundation
Read about The Mission Change of a Lifetime
"It only takes a pebble to start an avalanche."
3 people (AK, BA, SM) participated in up to 2 active working group meetings.
One HEvKA meeting was cancelled to allow participation in the Working Group Meeting in Dallas, Texas.
During the Working Group Meeting:
Joanne Dehnbostel provided an EBMonFHIR project update to the Clinical Decision Support (CDS), Biomedical Research and Regulation (BRR), and Clinical Quality Information (CQI) Work Groups.
The StatisticsOnFHIR Working Group reviewed developments to the Computable Publishing®: Baseline Measure Report Authoring Tool including the ability to create Group Resources to define the groups being measured, to create EvidenceVariable Resources for each outcome measured, and to view the Evidence Resources to be summarized in a table.
The Ontology Management Working Group reviewed progress on French translations of GRADE Ontology terms and spent considerable time reviewing the nuanced differences in definitions in French and English for the words inconsistency, heterogeneity, and incoherence.
Releases on the FEvIR Platform:
The Fast Evidence Interoperability Resources (FEvIR) Platform is available for use now, but is “pre-release”. The current version is 0.226.0 (May 21, 2024). Viewing resources is open without login. Signing in is free and required to create content (which can then only be edited by the person who created the content).
Computable Publishing®: Baseline Measure Report Authoring Tool creates and displays a Composition Resource with a BaselineMeasureReport Profile.
Quote for thought: “The capacity to learn is a gift; the ability to learn is a skill; the willingness to learn is a choice.” ―Brian Herbert
5 people (BA, GL, IK, SL, XZ) participated in up to 2 active working group meetings.
The EBMOnFHIR Implementation Guide Working Group applied changes to the Implementation Guide, specifically modifying the EvidenceReport Profile to include a binding to an Evidence Report Contributor Role Value Set for the attester.mode element, and creating that ValueSet to include a code for “recorder” to distinguish the entity that provides data entry from the “author” of the intellectual content in the Resource. The group also discussed approaches to simplify the Recommendation and RecommendationJustification Profiles and proposed a FHIR Change Request to separate the concepts with the recommendation statement, population, action, opposite action, and discussion in the Recommendation Profile and all other justification concepts in the RecommendationJustification Profile.
The Computable EBM Tools Development Working Group identified a series of working and non-working examples used for the Computable Publishing®: MAGIC-to-FEvIR Converter and prompting further investigation to determine the root cause, and reviewed developments to the Fast Evidence Interoperability Resources (FEvIR) Platform and Computable Publishing®: Baseline Measure Report Authoring Tool including the ability to create the Evidence Resources from within a table organized for more efficient data entry.
Releases on the FEvIR Platform:
The Fast Evidence Interoperability Resources (FEvIR) Platform is available for use now, but is “pre-release”. The current version is 0.227.0 (May 23, 2024). Viewing resources is open without login. Signing in is free and required to create content (which can then only be edited by the person who created the content).
Computable Publishing®: Baseline Measure Report Authoring Tool creates and displays a Composition Resource with a BaselineMeasureReport Profile.
Quote for thought: “The difference between who you are and who you want to be is what you do.” ―Unknown
Term
|
Definition
|
Alternative Terms
(if any) |
Comment for application
(if any) |
one-sided reference bias
re-opened for voting |
A reporting bias in which included citations are limited to those that represent only some of the perspectives.
|
|
The term "perspective" covers a variety of contexts, such as a side of an argument or point of view.
|
synthesis missing eligible studies re-opened for voting
|
A synthesis bias in which eligible studies were not included in the evidence synthesis.
|
|
A typical use of this bias is when studies were included in the overall review but were not included in a specific meta-analysis.
|
Term
|
Definition
|
Alternative Terms
(if any) |
Comment for application
(if any) |
---|
Reopened for voting
|
Differences between the outcome-related characteristics of the evidence and the intended target application.
|
|
Indirectness in outcome means there are differences between the outcomes in the studies under consideration and the outcome specified in the question of interest ("target application"). Differences in outcome-related characteristics (such as definition, method of measurement, and timing) may result in differences in the estimates derived from the evidence and those that would be measured in the intended target application. These potential differences in the estimates lead to a decreased confidence in the estimates. |
Sometimes the outcomes of primary interest in the target application may differ from those available in the evidence. For instance, the evidence may only provide information about intermediate or surrogate outcomes that only partially reflect or predict the outcome of interest. |
Reopened for voting
|
The situation in which the range of probable values for the estimate of effect includes values on both sides of a threshold.
|
|
The range of probable values is commonly expressed as a confidence interval or credible interval. |
The threshold should be specified. In the context of making a decision, the threshold should be specified as the value for which the decision would differ. Outside the context of making a decision, the threshold should be specified in the research question.
In some circumstances, imprecision may occur when the sample size is not large enough (less than the optimal information size) or the number of events is too small to provide a reliable estimate of effect. |
|||
Newly open for voting
|
The situation in which findings of a review are distorted due to systematic differences between the studies included and the studies that are eligible for the review but not identified due to the studies remaining unpublished or obscurely published.
|
|
Studies may be obscurely published if they are published in journals with limited access due to financial, language, or indexing barriers, or are only available as abstracts, theses, conference proceedings, preprints, or other less accessible formats. Publication bias is a study selection bias in which the publicly available studies are not representative of all conducted studies. |
Publication bias arises from the failure to identify all studies that have been conducted, either published (i.e., publicly available) or unpublished. The term 'studies' means evidence or research results in any form where such studies would meet the study eligibility criteria without consideration of criteria regarding the form of publication. The phrase 'publicly available studies' means the studies are available to the broad academic community and the public through established distribution channels in any form, including forms with restricted access. Established distribution channels include peer-reviewed journals, books, conference proceedings, dissertations, reports by governmental or research organizations, preprints, and study registries. Publication bias often leads to an overestimate in the effect in favor of the study hypothesis, because studies with statistically significant positive results are more likely to be publicly available. |
Day/Time (Eastern)
|
Working Group
|
Agenda Items
|
Monday 8-9 am
|
Project Management
|
FHIR changes and EBMonFHIR Implementation Guide issues (CANCELED May 27 for Memorial Day)
|
Monday 9-10 am
|
Setting the Scientific Record on FHIR
|
SRDR+ to FEvIR Review (CANCELED May 27 for Memorial Day)
|
||
Monday 10-11 am
|
ClinicalResearchOnFHIR WG (a CDS EBMonFHIR sub-WG)
|
Create Resource Examples (CANCELED May 27 for Memorial Day)
|
Monday 2-3 pm
|
Statistic Terminology
|
Review SEVCO terms (CANCELED May 27 for Memorial Day)
|
Tuesday 9 am-10 am
|
Measuring the Rate of Scientific Knowledge Transfer
|
SKAF Board Meeting and Pilot Project Progress
|
Tuesday 2-3 pm
|
StatisticsOnFHIR (a CDS EBMonFHIR sub-WG)
|
Authoring Tool progress
|
Tuesday 3-4 pm
|
Ontology Management
|
Review Objectives and Progress
|
Wednesday 8-9 am
|
Funding the Ecosystem Infrastructure
|
Review Progress on product readiness checklist for making guidelines computable
|
Wednesday 9-10 am
|
Communications (Awareness, Scholarly Publications)
|
Global Evidence Summit Posters, Special Session and Study design paper
|
Thursday 8-9 am
|
EBM Implementation Guide (a CDS EBMonFHIR sub-WG)
|
Review EBMonFHIR IG QA and update examples
|
Thursday 9-10 am
|
Computable EBM Tools Development
|
Review progress with Authoring tools
|
Friday 9-10 am
|
Risk of Bias Terminology
|
Review SEVCO Risk of Bias terms
|
Friday 10-11 am
|
GRADE Ontology
|
Review GRADE terms
|
Friday 12-1 pm
|
Project Management
|
Create Weekly agenda
|
Joanne, I believe Albert did say that artificial intelligence was no match for human stupidity, which is a related but not identical concept. I believe it was a way he used to address results of automated calculations that were not making sense.
I’d be happy to learn what others think
Alfonso
Iorio, Alfonso
From:
Joanne Dehnbostel <jdehn...@computablepublishing.com>
Date: Thursday, May 30, 2024 at 7:09 PM
To: Health Evidence Knowledge Accelerator (HEvKA) <HE...@computablepublishing.com>
Subject: Health Evidence Knowledge Accelerator (HEvKA) Daily Progress May 29, 2024
Caution: External email. |
Quote for Thought: “I’m not afraid of artificial intelligence, I’m afraid of natural stupidity”—Albert Einstein
5 people (BA, CE, JD, KS, MA) participated in up to 2 active working group meetings.
The Funding the Ecosystem Infrastructure working group worked on developments to support the user interface in the BaselineMeasureReport for table entry and viewing.
The Communications Working Group answered editor questions pertaining to the EBMonFHIR journal article which has been accepted but not yet published and discussed opportunities for grant funding.
Releases on the FEvIR Platform:
Computable Publishing®: Baseline Measure Report Authoring Tool creates a Composition Resource with a BaselineMeasureReport Profile. The current version is 0.7.0 (May 29, 2024)
Release 0.7.0 (May 29, 2024) provides a tabular display for editing and viewing the Baseline Measures represented in the BaselineMeasureReport as related artifacts.
FEvIR®: Citation Builder/Viewer version 1.28.1 (May 29, 2024) creates and displays a Citation Resource.
Release 1.28.1 (May 29, 2024) fixes a bug and now contributors are properly saved to the Citation resource.
Quote for thought: "A little more persistence, a little more effort and what seemed hopeless failure may turn to glorious success." —Elbert Hubbard.