Jerome  Carter MD,  April 25 2016

I have just finished reviewing workflow-focused reports from NIST and AHRQ. Some things I love and some simply leave me scratching my head. First, here is the list of reports:



Workflow, as these reports attest, is a highly elastic concept in health care. There are no standard task/activity lists, process lists, or ways of modeling workflow that are widely accepted within health care. Naturally, this makes for a mess when trying to compare the methods and outcomes of one group of researchers to another. Sharing process maps and task definitions between healthcare entities is equally problematic. These reports provide an excellent analysis of where we are now and point out in detail the many workflow issues that arise in clinical care as a result of HIT.

The oldest report by Carayon and Karsh et al. is a tremendous resource for anyone wanting to jump into clinical workflow research. It contains an extensive literature review, environmental scan and assessment of the field. The environmental scan includes extensive citations of workflow activity in many organizations as well as a toolkit of resources for organizations that are grappling with workflow disruptions secondary to HIT implementations. The authors’ assessment of the state of the field still stands six years later:

Although our literature review unearthed a great deal of information on (1) the effects of health IT implementation on workflow and (2) the use of health IT to analyze workflow, the quality of the findings is weak for many reasons. Most of the articles we found were not focused directly on workflow, so the quality of evidence related to workflow change varied substantially. Workflow measures also include such a variety of topics that comparisons and generalizations are difficult to make. Even the definition of a specific type of health IT (such as electronic prescribing) varied across articles, making comparisons even more challenging.

The majority of studies described research completed in large clinics affiliated with academic medical centers, health maintenance organizations or national health systems outside the US. This greatly limits the generalizability of our findings for the small and medium-sized clinics that are the end users of the toolkit. Also, although a substantial minority of articles were randomized controlled trials (RCTs), most of the studies did not use a scientifically rigorous design, limiting inferences of causality. As we discuss in Chapter 4, however, many barriers make it difficult to conduct a RCT to study health IT. Finally, most of the literature did not include descriptions of the socio-technical context of health IT implementations and use, making it difficult to understand the role of potentially conflating or mediating factors such as training, technical support, and organizational culture. Thus, although our findings on workflow change and analysis are suggestive, intriguing, and sometimes consistent across many studies, more research is needed to draw firm conclusions about the relationship between health IT and workflow.

I would suggest that this is the best document for getting a feel for HIT-related workflow issues, even six years later.

The next AHRQ report, from July 2015, contains the most extensive and detailed workflow implementation study I have encountered. Zheng et al. used five different data collection methods: ethnographic observations, time-motion studies, computer log analysis, semi-structured interviews and focus groups. The resulting data set is impressive in its completeness. Particularly interesting were the types of workflow disruptions and workarounds uncovered. While nothing new, they did serve to reinforce findings from other research studies on the problems that arise during EHR implementation and as a result of EHR design choices.

The implications/recommendations of the authors are fairly standard—no novel insights are offered. With the exception of the suggestion that vendors employ minimum specifications, nothing if offered beyond what any experienced EHR consultant would advise.

The Importance of Staff Engagement
The self-organizing process involved in implementing health IT changes in practices, the desire of many staff members for greater engagement in health IT planning and implementation, as well as the unique nature of each practice, make the case for greater involvement of staff in planning changes (so they can be tailored to unique local circumstances and so staff can understand rationale for changes), making sense of early implementation efforts (because there will inevitably be unexpected developments), and developing thoughtful modifications to health IT features and ongoing implementation efforts.

Consideration of Clinic Differences in Implementation Plans
This research found study sites had very different work environments and that these differences mattered in health IT implementation. This suggests that health IT plans and features should take into consideration the work environment of each practice site and that implementation plans should be created that support an engaged local culture (see point above about staff engagement).

Expect the Unexpected in Health IT Implementation
While implementing health IT changes there will inevitably be unanticipated developments, some small, like a workaround for scheduling, and some large, like the implementation of the Core Team model at Organization West Primary Care 2 and the manual importation of data from the old to the new EHR at Organization East. Recognizing that unexpected developments almost certainly will arise during health IT implementation efforts, and recognizing them when they do arise is important.

Employ Minimum Specifications
Vendors and health IT implementers should consider use of minimum specifications concept in health IT feature design and allow variation and flexibility around these core specifications. Health care organization leaders may be able to influence vendors in this regard.

Workload Considerations During Health IT Implementation
Consider reducing staff workload during the health IT implementation period to provide time for staff to incorporate health IT changes into practice and for staff to help one another.

I am not sure what I was expecting in terms of recommendations, but given the amount of work involved, I certainly expected to learn something new.

The use of the WEMS model as a conceptual framework decreased the utility of this report for me. WEMS is non-standard and highly conceptual, neither of which helps when trying to understand the report in terms of potential software design guidance.

The NIST report, unlike those from AHRQ, focuses specifically on workflow as an EHR design issue from the perspectives of usability and user-centered design. The NIST report also offers a workflow framework, the Systems Engineering Initiative for Patient Safety (SEIPS) model.

Process models are created for five scenarios and all are related to return patient visits: before the patient visit (approximately one to three days ahead), during the patient visit, physician encounter, discharge, and documentation.

Below is the authors’ description of the methodology employed for the report.

Process mapping and goal-means decomposition were selected to demonstrate that the application of human factors workflow modeling tools can improve EHR workflow integration into the clinical workflow. Based on the insights generated during collegial discussions with physicians, Subject Matter Experts (SMEs), and three interdisciplinary team meetings with clinical and human factors experts, we created process map visualizations and a goal-means decomposition diagram.

This approach was purposefully selected in order to illustrate human factors approaches to identify issues and opportunities with workflow that could potentially be addressed by EHR developers independent of implementation decisions at the local level, or by ambulatory care clinics independent of the particular EHR which is implemented.

In order to apply and exemplify these techniques, the human factors experts held the discussions with several physicians with experience in ambulatory care settings. The SMEs were presented with a description of the topics for discussion; the description explained that the purpose of the discussion was to utilize their subject matter expertise in order to better understand the workflow for a typical return patient grouped by the periods, “before the visit,” “during the visit,” and “after the visit.”

SMEs then discussed with interactive guidance from the investigator, a verbal walkthrough of a typical return visit and were asked to reflect upon and highlight challenging areas with the workflow that related to interactions with their EHR.

These physician SMEs had experience with different EHRs, represented different areas of specialty and primary care, and had a diverse perspective on the ideal level of integration of EHRs into routine and exceptional workflows. They included both males and females and an age range from approximately 30 years to 50 years old. A series of three focused interdisciplinary team meetings were held with human factors, informatics, and physician experts to generate the workflow models and accompanying insights for improving workflow. Notes during the discussions were taken by the human factors experts, and were shared within 24 hours following the discussion with the SMEs who had the opportunity to correct and augment the clinical information. Minor corrections were provided following two of the discussions, such as correcting the spelling of the blood condition eosinophilia (originally typed in the notes as eocenophillia). The notes across the discussions were compiled around related events or topics. Emerging insights were discussed among the authors of this report during scheduled meetings and as email discussions. Insights were supported, confirmed, and in some cases reframed by published studies in the literature and by related public posts to establish converging evidence.

The process model and goal-means decomposition were iteratively generated and revised over a series of meetings. The representations were constructed with a commercial flowchart program. All of the SMEs were provided the opportunity to review and make corrections to the final draft of the document. The draft included all of the workflow diagrams constructed from their input and the interdisciplinary team meetings, which represents a high-level depiction of the primary steps in the actual workflow using an EHR for an uncomplicated return patient. Four of the most important steps from the perspective of a physician provider are depicted in additional detail: ordering labs, ordering images, ordering medications, and ordering consults.

While the NIST report contains interesting information, I did not find it to be very useful for understanding the workflows as the process maps with too high-level. Knowing exactly how a user interacts with a system is essential to ensuring an appropriate design. Actual EHR system interactions do not seem to be a part of the study methodology. Without such information, how is one to determine how an EHR best fits within a workflow? In addition, direct observation does not seem to have been done for any of the processes-- everything seems to have been based on discussions with clinicians. In my experience, merely asking clinicians about complex behaviors results in many lost details.

Unfortunately, the lack of detail is a significant shortcoming. Too many workflow models skimp on details. However, it is the details that provide the information needed to see where disruptions actually occur. The narrative of Section 3 of the report provides more detailed information, but then one is left to mentally map those details back onto high-level process maps.

I am not quite sure what this are supposed to demonstrate concerning human factors methods. The topics raised by clinicians are not new. No methodological insights are offered vis-à-vis interview techniques, SME selection, the process maps have little detail, etc.

The authors offer EHR developers targeted recommendations for product changes based on the contents of Section 3 of the report.

For EHR developers, we recommend the following to improve EHR-related workflow during the patient visit:

Increase efficiency for these tasks:

  • Reviewing results with the patient,
  • Drafting pre-populated orders to be formally executed later, and
  • Supporting drafting documentation with shorthand notations without a keyboard;

Design for empathetic body positioning and eye contact with the patient while personally interacting with the EHR and while sharing information on the EHR screen with the patient and family members;

Support dropping tasks and delaying completion of tasks to help with daily flow; and

Verification of alarms and alerts and data entry without “hard stops.”

These suggestions are problematic because they are not tied to a detailed workflow model. For example, the suggestion: Increase efficiency for reviewing results with patient would be much more helpful with a detailed workflow model that captured the context of the specific tasks involved in reviewing results with a patient. Absent task-level detail, this becomes a generic suggestion.

Having reread these reports a few times, I offer the following observations.

1. Each report focuses on the patient visit as the fundamental unit of clinical work. While it may be the basis for payment, much work happens outside the encounter and systems that improve clinical workflow must recognize this fact. Results management and care coordination take significant amounts of time, yet occur outside of direct encounters.

2. For some reason, all NIST and AHRQ reports completely ignore the last 20 years of workflow research from the IT and business communities. Zheng borrows the WEMS framework from Unertl, and both ignore existing formal workflow languages. As a result, instead of using BPMN (a standard) or YAWL, flowcharts are used to model processes.

Workflow patterns are a widely-used for expressing control and flow of tasks along with the interactions -- people, machines, and data -- required to perform those tasks. It would be great if all clinical workflow modelers learned workflow patterns and a workflow language. Flowcharts simply do not have sufficient expressive power.

3. The role that architecture plays in usability issues seems to be largely ignored. Workflows are processes. Supporting processes with technology requires more than data. Giving users data requires them to keep a model of the process in their heads or on pieces of paper—classic workaround behavior. Systems that support processes must know what users tasks are performing and what information those tasks require. EHR systems are data-centric. They do not have process maps, and no concept of processes to work with. Workflow technology is real--it would be nice to see that fact acknowledged in one of the reports.

Overall, I found these reports to be helpful in the sense that confirmed what we already know about HIT and clinical work: EHR systems are poor at task support and should be altered to improve clinician productivity and efficiency. I cannot help thinking that if the human factors and clinical informatics communities adopted a formal workflow language, knowledge transfer among all communities interested in HIT and clinical work would improve significantly.

I leave you with my favorite quote from Thomas Kuhn that addresses the state of optics before Newton provided a common framework that was used by all researchers. I think it applies very well to the current state of clinical workflow research.

Being able to take no common body of belief for granted, each writer on physical optics felt forced to build his field anew from its foundations. In doing so, his choice of supporting observation and experiment was relatively free, for there was no standard set of methods or of phenomena that every optical writer felt forced to employ and explain. Under these circumstances, the dialog of the resulting books was often directed as much to the members of other schools as it was to nature.