Professional Documents
Culture Documents
Enabling Health and Healthcare Through Ict
Enabling Health and Healthcare Through Ict
THROUGH ICT
Studies in Health Technology and
Informatics
This book series was started in 1990 to promote research conducted under the auspices of the EC
programmes’ Advanced Informatics in Medicine (AIM) and Biomedical and Health Research
(BHR) bioengineering branch. A driving aspect of international health informatics is that
telecommunication technology, rehabilitative technology, intelligent home technology and many
other components are moving together and form one integrated world of information and
communication media. The series has been accepted by MEDLINE/PubMed, SciVerse Scopus,
EMCare, Book Citation Index – Science and Thomson Reuters’ Conference Proceedings Citation
Index.
Series Editors:
Dr. O. Bodenreider, Dr. J.P. Christensen, Prof. G. de Moor, Prof. A. Famili, Dr. U. Fors,
Prof. A. Hasman, Prof. E.J.S. Hovenga, Prof. L. Hunter, Dr. I. Iakovidis, Dr. Z. Kolitsi,
Mr. O. Le Dour, Dr. A. Lymberis, Prof. J. Mantas, Prof. M.A. Musen, Prof. P.F. Niederer,
Prof. A. Pedotti, Prof. O. Rienhoff, Prof. F.H. Roger France, Dr. N. Rossing,
Prof. N. Saranummi, Dr. E.R. Siegel, Prof. T. Solomonides and Dr. P. Wilson
Volume 183
Recently published in this series
Vol. 182. A.C. Smith, N.R. Armfield and R.H. Eikelboom (Eds.), Global Telehealth 2012 –
Delivering Quality Healthcare Anywhere Through Telehealth – Selected Papers from
Global Telehealth 2012 (GT2012)
Vol. 181. B.K. Wiederhold and G. Riva (Eds.), Annual Review of Cybertherapy and
Telemedicine 2012 – Advanced Technologies in the Behavioral, Social and
Neurosciences
Vol. 180. J. Mantas, S.K. Andersen, M.C. Mazzoleni, B. Blobel, S. Quaglini and A. Moen
(Eds.), Quality of Life through Quality of Information – Proceedings of MIE2012
Vol. 179. M. García-Rojo, B. Blobel and A. Laurinavicius (Eds.), Perspectives on Digital
Pathology – Results of the COST Action IC0604 EURO-TELEPATH
Vol. 178. A.J. Maeder and F.J. Martin-Sanchez (Eds.), Health Informatics: Building a
Healthcare Future Through Trusted Information – Selected Papers from the 20th
Australian National Health Informatics Conference (HIC 2012)
Vol. 177. B. Blobel, P. Pharow and F. Sousa (Eds.), pHealth 2012 – Proceedings of the 9th
International Conference on Wearable Micro and Nano Technologies for Personalized
Health
Vol. 176. T. Kotwicki and T.B. Grivas (Eds.), Research into Spinal Deformities 8
Vol. 175. S. Gesing, T. Glatard, J. Krüger, S.D. Olabarriaga, T. Solomonides, J.C. Silverstein,
J. Montagnat, A. Gaignard and D. Krefting (Eds.), HealthGrid Applications and
Technologies Meet Science Gateways for Life Sciences
Edited by
y
Karen
n L. Cou
urtney
Omiid Shabeestari
and
A Kuo
Alex o
S
School of Heealth Informa ation Sciencee
Univversity of Vicctoria
Victoria, Brritish Columbia, Canada a
All rights reserved. No part of this book may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, without prior written permission from the publisher.
Publisher
IOS Press BV
Nieuwe Hemweg 6B
1013 BG Amsterdam
Netherlands
fax: +31 20 687 0019
e-mail: [email protected]
LEGAL NOTICE
The publisher is not responsible for the use which might be made of the following information.
Preface
Information and communication technologies (ICT) can enable the delivery of
healthcare in ways unimaginable in a paper-bound world. Internationally, the use of
ICT in healthcare is growing rapidly as the need to provide efficient and effective care
becomes more urgent to patients, providers and payors alike. Research in new models
of healthcare delivery is starting to examine how ICT can increase the effectiveness
and efficiency of healthcare delivery.
Universal health care is still not the norm for many people in developing and de-
veloped countries alike. Socio-economic status and geography have been substantial
barriers to accessible care. Leveraging the diversity and ubiquity of available ICT may
be one of the keys in enabling accessible healthcare across geographic and socio-
economic divides.
Improvement in the health of nations begins with the individual. Recent develop-
ments in genomics and mobile networked information technologies have re-generated
interest in individualizing health care to leverage current knowledge for the greatest
individual benefit. In the not-so-distant past, it seemed unachievable to collect, store,
analyze and share information and advice tailored to a specific individual. At best, in
the past, clinical decision support systems made recommendations to clinicians about
how to apply population health probabilities based on only a few individual character-
istics; usually age and gender. Now researchers are testing personal decision support
systems, which provide recommendations directly to the individual based on a myriad
of characteristics, symptoms, signs and personal preferences. The inclusion of personal
genomic information as part of this cluster of characteristics is in the near future.
Governments, health systems and other payors are beginning to recognize not only
the subjective value of individualized care provided at a location of the patient’s choice,
but also the potential economic benefits as well. Globally, evidence is being gathered
that supports health care delivery models that embrace co-management of health be-
haviors between health care providers and patients and encourage “localized” health
care delivery. In this context, local care might take place in a community clinic, in the
home or even in the pocket, as telehealth takes healthcare mobile. The term, “house
calls”, may once again become familiar in 21st century healthcare, albeit with a differ-
ent implementation.
We hope that the knowledge shared between ITCH 2013 participants will generate
further discussions and collaborations and lead to breakthroughs in delivering effective
and efficient healthcare worldwide.
Karen Courtney
School of Health Information Science
University of Victoria
Victoria, British Columbia
Canada
This page intentionally left blank
vii
Program Co-Chairs
Elizabeth Borycki, Andre Kushniruk, Abdul Roudsari and Scott Macdonald
Editorial Committee
Karen Courtney, Alex Kuo and Omid Shabestari
Workshop Coordinator
Omid Shabestari
James Anderson
Purdue University
West Lafayette, IN USA
Riccardo Bellazzi
University of Pavia
Pavia, Italy
Joseph Cafazzo
University Health Network
Toronto, ON Canada
Dominic Covvey
National Institutes of Health Informatics
Waterloo, ON Canada
Eric Eisenstein
Duke Clinical Research Institute
Durham, NC USA
John Fox
Oxford University
Oxford, UK
Marilynne Hebert
University of Calgary
Calgary, AB Canada
Don Juzwishin
Alberta Health Services
Edmonton, AB Canada
Jonathan Kay
Oxford John Radcliffe Hospital
Oxford, UK
Craig Kuziemsky
University of Ottawa
Ottawa, ON Canada
x
James McDaniel
University of Victoria
Victoria, BC Canada
Christian Nohr
Aalborg University
Aalborg, Denmark
Grace Paterson
Dalhousie University
Halifax, NS Canada
Stephen Rees
Aalborg University
Aalborg, Denmark
Paul Taylor
UCL Centre of Health Informatics
London, UK
Maggie Wang
University of Hong Kong
Hong Kong
Jens Weber
University of Victoria
Victoria, BC Canada
Jennifer Zelmer
Canada Health Infoway
Toronto, ON Canada
xi
Contents
Preface v
Karen Courtney
ITCH 2013 Steering Committee vii
ITCH 2013 Scientific Review Committee ix
Tailoring Decision Support to Suit User Needs: A Diagnostic Imaging Example 132
Janessa Griffith, Elizabeth M. Borycki and Andre W. Kushniruk
Supporting Cystic Fibrosis with ICT 137
Erin Roehrer, Elizabeth Cummings, Paul Turner, Jenny Hauser,
Helen Cameron-Tucker, Sean A. Beggs, Nicole A. Micallef,
Claire Wainwright, Joyce Cheney, Mel Jessup, Heather Saddington,
Leonie Ellis, Haydn Walters and David W. Reid
Health Modeling
Issues and Considerations for Healthcare Consumers Using Mobile Applications 227
Elizabeth Cummings, Elizabeth Marie Borycki and Erin Roehrer
The Introduction and Evaluation of Mobile Devices to Improve Access to
Patient Records: A Catalyst for Innovation and Collaboration at BCCA 232
Jonn Wu, John Waldron, Shaina Hood, Adam Kahnamelli, Mohamed Khan,
Jeff Barnett, John French, Stacey Slager, Shadi Melhem and Omid Shabestari
Moving Mobile: Using an Open-Sourced Framework to Enable a Web-Based
Health Application on Touch Devices 238
Joseph Lindsay, J. Allen McLean, Amrita Bains, Tom Ying and M.H. Kuo
The Use of Social Media in Healthcare: Organizational, Clinical, and Patient
Perspectives 244
Mowafa Househ
Evaluation
Consumer Informatics
Nursing Informatics
Abstract. The open source cityEHR system was evaluated as a toolkit for
clinician-led development of an Electronic Health Record for management of
patients in the Ponseti clinic of a major London hospital. As a toolkit, it was found
that the ontology-driven approach of cityEHR was too complex for clinicians to
use. The toolkit was refined to use more familiar spreadsheet s to represent the
ontology and was then used successfully to create an effective clinical system,
generated automatically from the information model.
Introduction
The cityEHR electronic health records system has been developed as an integration of
'best of breed' open source software components, configured to support open standards
for clinical records such as ISO-13606 [1] and HL7 CDA [2]. It provides clinical users
with a toolkit that enables them to develop their own Electronic Health Records (EHR)
systems and then to deploy them on an enterprise-scalable platform. The system was
originally conceived to address the fundamental issues that caused the failure of the
EHR systems implemented as part of the National Programme for IT in England from
2004 to 2010 [3].
The platform for the deployed EHR is shown in Figure 1. It combines major open
source software components that run on an Enterprise Java platform and that are freely
available on the web: Apache Tomcat (tomcat.apache.org), eXist (exist-db.org), Mirth
Connect (www.mirthcorp.com), Orbeon (www.orbeon.com) and FusionCharts
(www.fusioncharts.com). These components are integrated and configured using the
XML-based languages XForms, XSLT and XQuery; there is no compiled code in the
cityEHR. The EHR deployed on this platform is configured entirely from an
information model represented as an ontology, in the Web Ontology Language (OWL)
[4]. This ontology uses a core architectural model (represented in OWL/XML syntax as
classes, object properties and data properties) to supply the building blocks for the
information model that contains specific individuals and property assertions.
1
Corresponding Author
4 J. Chelsom et al. / Clinician-Led Development of Electronic Health Records Systems
1. Requirements
The requirements for the Ponseti system were gathered without reference to the
capabilities of the cityEHR system or its modeling toolkit. A clinical analyst worked
with clinical staff (an orthopaedic surgeon and specialist clinic nurses ) to gather
requirements using structured interviews and analysis of existing paper-based data
records . The products of this analysis were a set of Use Cases, a System Requirements
Specification, a Data Dictionary and a set of User Interface Wireframes. The base
functional requirements (Table 1) were relatively simple and in our experience are
typical of the requirements for systems used in out-patient clinics and research studies .
J. Chelsom et al. / Clinician-Led Development of Electronic Health Records Systems 5
Table 1. Main Requirement s for the Electronic Health Records System in a Ponseti Clinic
Clinical Data Collection Other Requirements
Edit patient data systematically and store in a Secure authentication method to verify users
database
Clean, functional, form based user interface Basic role-based access control
Pre-defined datasets in a drop-down menu
Forms clearly sectioned into given headings Complex queries on the patient database
Attach/upload multimedia t o t he patient record Output result ant queries in a report
Conditional inclusion of sect ions/entries on forms Graphical or/and tabular format for reports
Enforce mandatory fields for data entry Display summary view(s) for existing records
Hints - short help assistance with data entry Generate standard letters (e.g. in MS word)
Hyperlinks to external web resources Query to assemble cohort s for research studies
Calculations and scoring on assessment forms Automatically calculate standard data (e.g. patient
age)
The basic functional requirements and user interface wireframes were used as the
starting point for configuration of the cityEHR. The data model elicited from clinical
users was coded as an ontology by the clinical analyst, using the open source
Protégé tooling from Stanford University [6]. Figure 2 shows the user interface of
Protégé with the basic architecture of the cityEHR represented as classes and the
specific in formation model for the Ponseti clinic system represented as individuals,
object property assertions and data property assertions.
The resulting OWL model was then loaded to the city EHR system and generated a
working EHR prototype which was compared with the original wireframes and
functional specification. It was found that the user functionality and general layout of
data entry forms could be created to the general satisfaction of users , but that a
number of specific functional requirements were not met by the automatically
generated system, specifically:
• complex calculations and s coring on assessment forms
• calculation and representation of patient age in days , weeks , months or years
depending on the age of the patient
• attach/upload multimedia to the patient record
These features were then added to the cityEHR system in a two stage process:
firstly adding the semantics to the cityEHR architectural ontology so that the features
could be specified in the information model and secondly adding functional support for
the features when the EHR system was generated from the model. Having added these
features to cityEHR it was then possible to generate a clinical EHR system that was
acceptable for piloting in the Ponseti clinic, as shown in Figure 3.
Although it was possible for the clinical analyst to create an information model using
Protégé and use that model to generate a clinic system in the cityEHR that met the
functional requirements, there were some major issues with the usability of the toolkit.
6 J. Chelsom et al. / Clinician-Led Development of Electronic Health Records Systems
4. Conclusions
Our study in the Ponseti clinic proved that it is possible to generate a usable EHR
system directly from an information model specified by clinical users. However, it also
showed that the complexity of the ontology-based approach (both conceptually and in
the available open source tooling) made it impossible for clinicians to develop that
model themselves directly using the cityEHR tools. Once a simpler spreadsheet-based
interface was introduced to specify the information model then it d id become possible
for clinical users to create and modify their information models directly.
In subsequent studies we have found that a variety of tooling can be used to create
the models (not just spreadsheets). This more general approach is shown in Figure 5.
The openness of the OWL/XML language, the widespread support of XML across
modeling tools and the capabilities of XSLT for trans formation combine to provide a
flexible approach to information modeling that can be used by clinical users to
automatically generate functional EHR systems using open source platforms such as
cityEHR.
8 J. Chelsom et al. / Clinician-Led Development of Electronic Health Records Systems
References
[1] ISO 13606-1:2008 Health informatics - Electronic health record communication - Part 1: Reference
model. Available at:
http://www.iso.org/iso/iso_catalogue/catalogue_t c/cat alogue_det ail.ht m?csnumber=40784
[2] Dolin RH, Alschuler L, Boyer S et al. HL7 Clinical Document Architecture, Release 2. J Am Med
Inform Assoc. 2006;13(1):30-9.
[3] Chelsom, J. Open Health Informatics–A Fresh Approach for NHS IT. City University Challenge Paper
(2011). Available at: http://www.openhealthinformatics.org
[4] World Wide Web Consortium. OWL Web Ontology Language Overview. W3C Recommendation 10
February 2004. Available at: http://www.w3.org/T R/owl-features/
[5] J. J. Chelsom, I. Pande, R. Summers, I. Gaywood. Ontology-driven development of a clinical research
information system. 25th IEEE International Symposium on Computer-Based Medical Systems
(CBMS), pp. 1-4, 2011 24t h International Symposium on Computer-Based Medical Systems, 2011
[6] John H. Gennari , Mark A. Musen , Ray W. Fergerson , William E. Grosso , Monica Crubzy , Henrik
Eriksson , Natalya F. Noy , Samson W. Tu. The Evolution of Protégé: An Environment for Knowledge-
Based Systems Development (2002) International Journal of Human-Computer Studies.
Enabling Health and Healthcare through ICT 9
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-9
Introduction
The Personal Health Record (PHR) has become a point of focus for a number of the
trends characterizing medical informatics, health information technology and their
intersection with public (community) health in the early decades of the 21st Century.
“Meaningful use” of the EHR stipulations required by the American Recovery and
Reinvestment Act for reimbursement for providers under US Medicare and Medicaid
programs, patient-centered care, patient-provider partnerships, accountable care
organizations (ACOs) all can be seen as converging on the ultimate goals of the
empowered consumer of health care and preventive services. Parallel to these factors is
the priority given to translational research in medicine and public health to inform and
justify evidence-based clinical practice to insure patient safety and cost-effective
delivery of health care and community health interventions to eliminate health
disparities and sustain the public’s health and wellness at the local, regional and global
levels.
The purpose of this conceptual paper is to provide a future vision of a “mature”
PHR (referred to subsequently as the MPHR), identify the potential benefits of and
1
Corresponding Author: Dr. Wesley Rohrer, Associate Dean, King Saud Bin Abdul Aziz University for
Health Sciences, College of Public Health and Health Informatics, Riyadh, Kingdom of Saudi Arabia; Email:
[email protected]
10 W. Rohrer and M.S. Househ / Implications of the Mature PHR for the Empowered Consumer
challenges to MPHR implementation, and present an agenda for health policy research
relevant to MPHR development. By doing so, we wish to focus attention on the MPHR
as a useful focus for health system reform from the perspective of the empowered
consumer.
Although various but similar definitions of the Personal Health Record (PHR) have
been proposed, the following language is succinct and consistent with the HIPPA
regulatory logic. The US Office of Civil Rights defines the PHR as “an electronic
record of an individual’s health information by which the individual controls access to
the information and may have the ability to manage, track, and participate in his or her
own health care.”[1] The PHR as defined and discussed above is viewed not only as a
repository for Personal Health Information but more importantly as a channel and
mechanism for enhanced communication between the individual consumer of health
care and related services and the provider system.
To suggest the potential capacities and benefits of the “mature” PHR (MPHR), we
provide the following brief scenario of the empowered health consumer using her
MPHR in 2020:
After checking her Outlook calendar in preparation for the work week ahead,
Maria realizes that she has a potential conflict with a mandatory staff meeting on
Monday at 11:00 and a previously scheduled routine follow-up visit with her PCP
three months after knee surgery. She accesses her PHR though her Apple
MyPhone/MyLife Pad. She places an instant message to the office of her PCP,
requesting a cancellation and new appointment on one of the available dates and
times highlighted in her on-line calendar. Within 10 minutes the confirmation of
the new appointment the following week is received along with a copy of an
inquiry to her physical therapist whether the range of motion data for Maria’s
knee is current. If not, Maria is asked to schedule a PT appointment prior to the
PCP visit. Also she is reminded to update her fitness and dietary plan with the
current week’s entries and to complete the six-month personal wellness survey.
Once she does so she will receive a table of her responses during the past three
years of key variables, e.g., weight, BM, HCL levels, blood pressure, pulse/OX
reading, mood (depression) scale, as compared to statistics for the health plan of
which to which she belongs and national benchmarks in the aggregate and by age,
gender and Regional cohort. If she had been interested Maria might have accessed
her cumulative use and cost of services history again with bench make
comparisons. Having requested a PT appointment later in the week, the PT office
Maria provides an on-line Pain Chart for her to complete on the touch screen, 24
hours prior to the confirmed appointment.
After her meeting Maria finished her workout at her employer’s fitness club.
During lunch she enters her weight training and aerobic exercise data and her
calorie count for the lunch she is enjoying, intending to trade off excess calories
W. Rohrer and M.S. Househ / Implications of the Mature PHR for the Empowered Consumer 11
for the slice of fruited cheesecake with a light dinner of fish and rice. While
completing her records, she receives an alert from the County Health Department
about a food-borne illness associated with a salad bar at a chain of family
restaurants in the Region with a request that anyone who dined at any of the
locations identified within the past 48 hours should complete an on-line survey
attached and if experiencing any severe gastric symptoms, contact the PCP or
receive expedited treatment at the closest ER. Finally a broadcast message in her
Healthy Community folder alerted subscribers of a current need for her blood type
at the Blood Bank and that first time donors would receive bonus points toward a
discount of their health plan premium payment .Next Maria logged into the MPHR
of her aging mother who had recently been diagnosed with early onset Alzheimer’s.
Having obtained power of attorney for all her mother’ affairs, Maria had complete
access to her PHR and the tethered EHR. She duly noted the alert that her mother
had missed her medication last evening, an issue that Maria intended to address
with the home health aide directly. Realizing that the time was drawing near when
her mother would require institutional long term care, Maria used the messaging
utility linked to her MPHR to contact the local Area on Aging to access their
quality ratings and prices for long term care facilities with dementia care unit.
The scenario described presents a PHR fully embedded in the consumer’s preferred
communication channels and serving as the nexus for linking medical care, personal
health behaviors and public health services and information potentially relevant to the
consumer and family members. The supporting technology is already in place and
almost certainly will become more powerful and available at lower cost within this
decade. So the technological infrastructure for the applications identified is neither
remote nor radical. The challenges presented by this scenario from a public health
policy perspective are related to use, acceptance and satisfaction with the MPHR on the
consumer side and the will and capacity for data system integration, provider system
commitment and inter-institutional bridge building on the provider side.
A mature and fully functional PHR entails an integrated health care system in which
medical services, in-hospital, ambulatory, rehabilitation and home health services are
coordinated with and mutually reinforce the disease prevention, environmental
protection and health promotion population-based services associated with public
health practice. Such system integration is inherently complex even if the political will,
appropriate incentives and regulatory mechanisms are in place to support it. The
daunting and ongoing challenges faced in implementing the Patient Protection and
Affordable Care Act (2010) in the US, at best a partial health care system reform, are
instructive and not encouraging for comprehensive health system integration and
reform, at least for systems that are non-nationalized hybrids. The existence of multiple
payers and conflicting reimbursement logics, e.g., fee-for-service, bundled, capitation
premiums, pay for performance and other managed care incentive plans, makes total
integration even more complex. The alignment of medical care provider practices and
incentives with community and population-based program designs and intervention
strategies while not inherently incompatible present the challenge of coordinating and
accommodating two different organizational cultures and institutional frameworks.
Perhaps an even greater concern from the public health perspective is that the
accelerating proliferation of information available electronically provides cover for
intentional generation of misinformation whether motivated by narrowly political,
financial or doctrinal purposes. In the health arena such misinformation masking as
scientific evidence and/or supported by (for example) celebrity endorsements can lead
to adverse health consequences for the individual or population targeted. The false
attribution of a link between immunization and autism, even in spite of the repudiation
of any causal link by medical researchers, can result in gullible, uniformed parents
refusing vaccinations for their children resulting in increased risks to the community’s
health by reduction in herd immunity.
The sophisticated health consumer may be expected to weigh the evidence and
reject implausible and unfounded claims and advice from unreliable sources. However,
the less educated and more vulnerable consumer will likely be at greater risk of basing
health behavior on bad information and fraudulent claims. Consequently, the ocean of
health related information potentially available to the consumer can pose risks to the
integrity of the MPHR and/or the consumer’s effective use of it in sustaining her/his
health and well-being. It should be noted that this situation is complicated by the rapid
release and distribution of translational research which may offer contradictory or
ambiguous findings which can baffle even the more well-informed consumer.
W. Rohrer and M.S. Househ / Implications of the Mature PHR for the Empowered Consumer 13
3. Conclusion
In summary, the evolution of the PHR beyond its current state of adoption, use and
functionality into a more mature and widely used tool for the empowered health
consumer is an exciting prospect but one that presents an array of challenges, both
regulatory, managerial, socio-cultural and technological. We conclude that the
potential pay-offs for enhancing the individual’s health and well-being and protecting
and sustaining the public’s health overall is worth a considerable investment in policy
analysis, technological R&D, partnership building and the expenditure of political
capital by influential advocates and decision makers.
Acknowledgements
We would like to thank the King Abdullah Institute for Medical Research for their help
in editing this document.
References
[1] US Department of Health & Human Services, Office for Civil Rights. Personal health records and the
HIPAA privacy rule, p. 1, undated)
[2] Curran WJ, Steams B, Kaplan H. Privacy, confidentiality and other legal considerations in the
establishment of a centralized healthdata system. N Engl J Med. 1968;281:241-8.
[3] Thompson LA, Black E, Duff WP, Paradise Black N, Saliba H, Dawson K. Protected Health
Information on Social Networking Sites: Ethical and Legal Considerations. J Med Internet Res
2011;13(1):e8. URL: http://www.jmir.org/2011/1/e8/ doi: 10.2196/jmir.1590 PMID: 21247862
[4] NAHIT. Report to the office of the national coordinator for health information technology. Defining
key health information technology terms. April 08, 2008.
14 W. Rohrer and M.S. Househ / Implications of the Mature PHR for the Empowered Consumer
[5] Shachak, A. and Jadad, A.R. (2010). Electronic health records in the age of social networking and
global telecommunications. JAMA. 303(5): 452-453.
[6] Househ M. Sharing sensitive personal health information through Facebook: the unintended
consequences. Stud Health Technol Inform. 2011;169:616-20.
Enabling Health and Healthcare through ICT 15
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-15
Abstract. This paper describes the application of the Clinical Adoption (CA)
Framework to evaluate the impact of a recently deployed electronic medical record
(EMR) in a Canadian healthcare organization. The CA Framework dimensions
evaluated were EMR quality, use and net benefits at the micro level; and people,
organization and implementation at the meso level. The study involved clinical and
support staff from two ambulatory care clinics, and managers and technical staff
from the organization. A number of issues were identified at both levels of the CA
Framework that had affected EMR adoption in the two clinics. Some perceived
benefits in care coordination and efficiency were reported despite challenges that
arose from early deployment decisions. There were five lessons that could be
applied to other ambulatory care settings. The CA Framework has proved useful in
making sense of ways that EMR can add value to the organization.
Introduction
Despite increased adoption of the electronic medical record (EMR) in primary health
care reported in recent years, its impact on provider performance and patient outcomes
has been mixed [1,2]. For instance, a systematic review by Lau et al. [3] of 43 EMR
evaluation studies has shown only 51.2% had positive impact, while 30.2% had no
effect. Given the recent attention to transform primary health care in Canada and the
great expectations on EMR as the enabler [4], the lack of positive evidence to date is
cause for concern. That said there are successful primary health care practices that have
shown improved care thru innovative use of their EMR. Examples are the use of
individualized decision support and reminder tools by Holbrook et al [5] to improve
diabetes care in a Canadian community, and the EMR supported quality improvement
project by Nemeth et al [6] in 99 primary care practices in the United States. In this
paper we describe the formative evaluation of a recently deployed EMR in two
ambulatory clinics of a regional healthcare organization in Canada. We applied the
Clinical Adoption (CA) Framework [7] to make sense of the impact of EMR adoption
in both clinics and the lessons learned.
1
Corresponding Author: Francis Lau, School of Health Information Science, University of Victoria,
P.O. Box 3050 STN CSC, Victoria BC, Canada V8W3P5
16 F. Lau et al. / Applying the CA Framework to Evaluate the Impact of an Ambulatory EMR
1. Objective
This formative evaluation study [8] applied the Clinical Adoption Framework (CA) to
examine the impact of EMR adoption in two ambulatory care clinics in a regional
healthcare organization, and the lessons learned to guide future implementation effort.
2. Methods
The research team employed a set of rapid evaluation methods [9] that consisted of
EMR adoption survey [10], user assessment, usability/workflow analysis, document
review, data quality review, project risk assessment, and group practice reflection. The
data collection took place over four weeks including review of project documents and
EMR data which took place concurrently. The EMR support staff organized sessions,
assembled relevant documents, and extracted EMR data for the research team. During
the study, the research team worked closely with EMR support staff to verify the
information collected to ensure its accuracy. All notes taken during interview,
usability/workflow, and focus group sessions were transcribed and analyzed for
common themes. The evaluation report was finalized in the last two weeks.
3. Evaluation Findings
Forty-three participants were recruited in the study that included providers and support
staff from both clinics, EMR support staff, and managers involved in the project. Over
4 weeks the team visited the clinics 3 times and completed 13 adoption surveys, 14
usability/workflow sessions, 13 user assessment interviews, 11 project risk assessment
interviews, 3 focus group sessions, and reviewed 65 project documents and 3 months of
EMR data. The findings were summarized according to the micro and meso dimensions
of the CA Framework. Due to space limitation only selected findings are included.
F. Lau et al. / Applying the CA Framework to Evaluate the Impact of an Ambulatory EMR 17
charting. For EMR functions in limited use, revisions and testing were needed to ensure
they work as intended. These included care plans and chronic disease management
forms. The EMR functions not yet in use that should be considered were flowsheets,
database queries and clinical decision support for drug interactions and allergy checks.
For care coordination, clinic staff saw value in capturing encounter notes, medications,
lab work, referrals and billing electronically in the EMR. The technical interface
between the HIS and EMR had created a potential patient safety issue. The HIS was the
source of truth for patient demographics, scheduling and allergy information. Though
the EMR had the ability to schedule appointments and record allergies, clinic staff were
not able to do so. For allergies, providers had to relay the information to the clerks who
then had to enter it into the HIS. This information would then be downloaded to the
EMR later. Clinic appointments had to be booked through the HIS, which did not allow
detailed reason for visit to be recorded or timely updating of patients who showed up
late at the clinic. The billing interface for physician services and labs required legal
patient names which sometimes were different from what was in the EMR. As a result,
clerks had to change the names for billing then matched the lab results.
For efficiency, clinic staff appreciated the EMR allowed them to find the chart and
the information needed at the time of patient visit. Yet providers faced challenges
working with fragmented charts. Clinical documents such as diagnostic imaging,
discharge summary and consultation reports were not stored in the EMR as they
were out of scope for the initial implementation. This had impeded workflow and
incented staff to find workarounds. Some consultation reports were supposed to be
stored in the HIS but were entered in the EMR since that was where the providers
needed the information. If a provider needed an imaging report and it was not found in
the EMR, they had to login to the HIS to look for it then returned to the EMR. Often
they would add a free-text comment in the patient’s medical history to indicate an
imaging result was in the HIS.
For people issues, clinic staff described having the opportunity for initial training but
with little follow-up afterward. These included existing users who wished to improve
their proficiency with additional EMR functionality and training for new staff that
joined the clinic after the system went live. Clinic staff recognized the EMR could do
much more for them if they only had time and resources dedicated to training them and
helping them move to the next level. For instance, the EMR had the ability to generate
automatic reminders to recall patients for follow-up lab work and investigations.
However this function was unavailable early on thus staff created workaround routines
which then had to be undone when it became available. At a more fundamental level,
some clinic staff indicated a lack of basic information management knowledge such as
ICD-9 coding that could be of value for EMR users.
For organizational issues, the two clinics had unique challenges in that the EMR
was deployed at the same time when the ambulatory care centre was being opened. As
such, there were lack of clearly defined staff roles on such tasks as scanning
documents, no clear clinical workflows on handling of scheduling, referral and billing,
and no detailed planning of the business requirements of how an EMR supported clinic
F. Lau et al. / Applying the CA Framework to Evaluate the Impact of an Ambulatory EMR 19
would function. These had led to different expectations of what the EMR should do and
how the system should be configured. For instance, the EMR had care plans,
custom templates and chronic disease management flowsheets that were not exploited
due to lack of defined requirements on where/how to use them.
For implementation issues, managers and EMR support staff involved with the
initial implementation rated their experience at 2.4/5 on a Likert scale of 1 to 5 with 5
being strongly positive. Most felt that there were insufficient resources available
and the EMR was not aligned with the clinic workflows. Interviews with staff
involved with the initial EMR implementation revealed their frustration that it was
viewed by others in the organization as a simple project. There was also a perception
that the EMR was a shrink-wrap product ready for plug-and-play with little training
required once installed. This view was reinforced by the project approach used to
implement the EMR, where an aggressive timeline was established to deploy the EMR
and the measure of success was that it got implemented and went live at the clinics.
The review of project documents confirmed the interview findings where similar issues
were reported.
4. Lessons Learned
The EMR adoption level achieved at the two ambulatory care clinics was comparable
to similar clinics that we had studied in the past two years that were within 1-year after
their EMR was implemented. Our experience suggested EMR users often had variable
experiences with the system depending on their expectations and prior IT experiences.
We should point out this evaluation represented only a point in time shortly after the
EMR was implemented at the two clinics. Despite the challenges reported, we found
clinic staff was resilient believing that the EMR was better than what they had before,
and that the system would improve over time and add further value to their work.
The lessons from this study that could guide future EMR implementation efforts are
to: (1) recognize EMRs as sophisticated clinical applications that require resource,
effort and commitment over time to adopt successfully; (2) have clearly defined staff
roles, clinical workflows and business requirements for the clinics involved to
maximize EMR-practice fit; (3) pay early attention to EMR data quality to minimize
subsequent database cleanup and realize benefits sooner; (4) provide incremental user
training over time to maximize EMR value based on a comprehensive training strategy
with different modalities; and (5) Redefine successful EMR adoption as being able
to demonstrate tangible clinical value beyond system going live. We believe the CA
Framework provided a useful organizing scheme for us to make sense of the EMR
impact from the dimensions of EMR quality, use and benefits, and contextual factors
involved.
References
[1]. C. Schoen, R. Osborn, M.M. Doty, D. Squires, J. Peugh, S. Applebaum. A survey of primary care
physicians in eleven countries, 2009: perspectives on care, costs and experiences. Health Affairs 28
(2009), w1171-w1183.
[2]. CMA. National Physician Survey 2010. http://www.nationalphysiciansurvey.ca. March 25, 2012.
[3]. F. Lau, M. Price, J. Boyd, C. Partridge, H. Bell, R. Raworth. Impact of electronic medical record on
physician practice in office settings – systematic review. BMC Med Inform Decision Making 12
20 F. Lau et al. / Applying the CA Framework to Evaluate the Impact of an Ambulatory EMR
(2012):1-10.
[4]. B. Hutchison, J. Levesque, E. Strumpf , N. Coyle. Primary health care in Canada: Systems in
motion.Milbank Quarterly 89 (2011), 256-88.
[5]. A. Holbrook, L. Thabane, K. Keshavjee, L. Dolovich, B. Bernstein, D. Chan, S. Troyan, G. Foster,
H.Gerstein, for the COMPETE II Investigators. Individualized electronic decision support and
reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ 181
(2009), 37-44.
[6]. L.S. Nemeth, A.M. Wessell, R.G. Jenkins, P.J. Nietert, H.A. Liszka, S.M. Ornstein. Strategies to
accelerate translation of research into primary care within practices using electronic medical records.
Journal of Nurs Care Qual 22 (2008), 343-9.
[7]. F. Lau, M. Price, K. Keshavjee. From benefits evaluation to clinical adoption: making sense of health
information system success in Canada. Healthc Q 14 (2011), 39-45.
[8]. J.J. McGowan, C.M. Cusack, E.G. Poon. Formative evaluation: A critical component in EHR
implementation. JAMIA 15 (2008), 297-301.
[9]. UVic eHealth Obsevatory. EMR evaluation toolkit v3.0. University of Victoria.
[10]. M. Price, F. Lau, J. Lai. Measuring EMR adoption: A framework and case study.
ElectronicHealthcare 10 (2011), e25-e30.
Enabling Health and Healthcare through ICT 21
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-21
Introduction
In the inpatient setting, handoffs occur several times a day for each patient. Handoff
communication failures are a known and frequent source of errors.[1-5] These errors
occur in almost all medical settings, e.g., emergency departments, among units in a
hospital, between hospitals and nursing homes. Yet, the role of healthcare information
technology (HIT) in handoffs has received little attention, especially in light of HIT’s
growing role in all medical settings and its potential to enhance handoff safety. Equally
important, the potential usefulness of HIT in handoffs when combining paper and HIT
has received even less attention—even though several HIT programs print out paper
handoff forms from programs within the EHRs themselves.[4]
In our observations, EHRs and other digital media are inconsistently incorporated
in handoffs even when the hospital is fully wired. Moreover, clinicians’ use of HIT
during handoffs is frequently an afterthought or a supplement that can distract as well
as illuminate—a reality affecting both departing and incoming clinicians. Oral
communication and paper remain the media of choice, and even paper is not
systematically employed. Integration of paper and HIT is less common and is even less
well thought out when it does occur. Hospitals and other medical settings lack a
coherent strategy for handoff processes that use HIT and/or HIT with paper documents.
In this paper, we examine handoffs that often incorporate HIT (usually EHRs)–
or—more commonly, handoffs that fail to incorporate HIT or incorporate HIT in
haphazard fashion. Until recently, HIT/EHRs were seldom a part of the handoff process.
But with the growing availability of HITs, ignoring its information in the handoff is a
1
Corresponding Author: email: [email protected].
22 R. Koppel and J.L. Telles / Role of Healthcare Information Technology in Handoffs
potentially significant loss of patient safety and clinical efficacy. Equally disconcerting
is allowing the HIT to structure the handoff process haphazardly – perhaps missing
needed emphases while mechanically repeating obvious or insignificant information.
Most often, handoffs involve pieces of paper that are subsequently discarded. Even
though the EHR is presumably the repository of the official and most accurate
information, it is often ignored or relegated to a marginal role. This may be because of
its widespread use is comparatively recent; because it is not part of the routine training
process; because it is seldom designed for handoffs; or because the use of HIT has
heretofore required proximity to a computer screen—a situation that has been altered
with computer tablets.
To create a context for analyzing handoffs, we explore some of the parameters within
which they occur, e.g., specific locations; whether they occur with or without paper,
structured formats, access to computers, participation of other clinicians, presence of
senior clinicians, and so on. The table (Table 1) presents a typology of factors that
influence handoffs’ clarity and accuracy. We refer to clinicians leaving the patient, shift,
or hospital as “departing” clinicians and to clinicians accepting care of the patient,
starting the shift, or managing the floor as “incoming” clinicians.
While the table reflects months of observations and interviews over several years,
we offer no systematic analyses of the frequency of each category or of the correlations
among the categories. Some relationships are obvious: Patient involvement is more
likely if handoffs are in or near patients’ rooms; proximity to computers is needed for
their incorporation in handoffs. Other physicians are more likely to be nearby if the
handoff is conducted in a group setting or at physicians’ workstations. Moreover, the
parameters provide a basis for examining additional issues such as: errors if the
documented information is wrong or outdated; errors of memory; staff sanitizing
phrasing due to concern about a patient overhearing prognoses; time required to create
a structured report versus information loss or misunderstandings with unstructured or
poorly documented reports; corrections provided by third parties and by those from
other disciplines; inefficiencies of ‘translating’ information to another discipline’s argot.
In addition, other categories could be included, e.g., fragmentation of the
information by other activities, role of clinician experience (head nurse, interns, senior
physicians), and delays caused by efforts to collect needed additional information. This
typology is elaborated in the following discussion, in which we use paintings and
etchings as metaphors to illustrate vulnerabilities involving handoff communications.
R. Koppel and J.L. Telles / Role of Healthcare Information Technology in Handoffs 23
TABLE 1. Sixteen Parameters of Handoff Interactions: Roles of Media (HIT, Paper), Memory, Type & Use
of Structured Formats, Location, Patient Presence, Other Artifacts, Dyadic or Group Setting, Intra- or Inter-
disciplinary, Face-to-Face or Mediated, & Involvement of Other Clinicians
1. Media Generally includes Oral. If also other media, they could be: Paper only; Paper
combined with HIT; HIT only. Different media might be used by incoming and
departing staff, e.g., HIT information from departing recorded on paper by
incoming; paper information entered in HIT and also on paper, etc. There might
even be a note in lieu of face-to-face communications.
2. Documentation Oral handoff with no recording of information in any medium. Paper, and even
more, HIT provide documentation accessed by more people. Many handoffs were
observed without reference to paper or HIT documentation.
3 Reliance on Extent to which information is provided based on what the departing clinician
Memory knows or remembers rather than/in addition to what is documented.
4. Incoming clinician Recipient’s questions and recipient looking up information. Reflects incoming
seeks information clinician’s active involvement in researching information and asking questions.
5. Degree and Type of Categories by problem, medication, organ system or temporal proximity, etc –
Structure in adhered to by departing clinicians. Use of structured discussion (such as SBAR)
Handoffs. may be informally recorded.
6. Use of Available Structured handoff reflected in templates or forms: HIT, paper or a combination.
Structured Forms Templates/forms may be used, partially used or ignored.
7. Accessibility & For paper: Small print summary sheets with little room for additions and notes.
Readability For computer: large-enough screen, viewable/findable by relevant parties
8. Handoff Location Patient Room; Nurse Station; Physician work area; Hallway; Other
9. Proximity of Role of patient’s presence in the handoff process. Role of other artifacts (e.g.,
patient & artifacts medications, IV bags)
10. Dyadic or Group Two clinicians alone or as part of team, rounds or other group settings
11. Intra- v. inter- MD to MD; RN to RN versus interdisciplinary, e.g., MD to RN or multi-
disciplinary disciplinary rounds.
12. Hierarchy Peers (resident to resident); or different levels (nurse manager tostaff nurse)
13. Face-to-Face or Traditional face-to-face, or involving the use of tape or voice mail.
Mediated
14. Consultations Others contacted or not; participation by others by chance or actively sought.
15. Speed Time allotted or available for handoffs.
16. Fragmentation Degree to which staff are interrupted and face other distractions to the handoff.
Hierarchy and status are always elements of handoffs (Figure 1), and their roles in
patient safety are critical—whether between peers (e.g., nurse to nurse; resident to
resident) or between those of different positions (e.g., resident to attending).
Figure 1 Figure 2
24 R. Koppel and J.L. Telles / Role of Healthcare Information Technology in Handoffs
The Kandinsky painting (Figure 2) depicts the often amorphous structure and freeform
presentation in handoffs--allowing clinicians much latitude in evaluating essential vs.
less important information as long as they use the two forms well and the incoming
clinician sees which is which. Many of the handoffs we observe combine areas of clear
and understandable information with opaque, and sometimes information that is missed
entirely or assumed away. These handoffs are largely unstructured either by the
departing clinician or the incoming clinician. Notes are sporadic and not organized in
any clear way. For example, we often observe clinicians referring to areas of a page or
screen that encompass medications, diagnoses, and laboratory reports. Some of the
information is specific (e.g., laboratory values) but other parts are suggestions for new
observations or tests. Of course, assumptions about the other clinician’s focus or
understanding may not be shared.
As suggested by the next Kandinsky (Figure 3), even when there is clear structure
in the handoff process, clinicians may differ about what data are most essential and
what should be covered. That is, one or both of the clinicians has a physical or mental
check list or set of items that is a backdrop to the interaction. Only a few of the items,
however, are discussed in the handoffs and no structured handoff record is maintained.
The rest of the checklist is taken for granted as known. This is not necessarily
dysfunctional: in the hands of experienced clinicians both of whom know what items
are essential and what are sufficiently routine or well known, much information can be
skipped.[6,7] (Some information might be intentionally excluded [e.g., prior drug use]
to protect patients by only allowing sanitized medical records.) Moreover, there is an
infinite amount of information that can be conveyed about the patient; only certain
items are news. It is impossible, however, to be sure both sets of clinicians are fully
aware of all patient issues, and such assumptions are risky. Even attentive and
experienced people might miss or forget an essential item, or they might assume a level
of understanding and familiarity that the other clinician does not possess. Patient status
and lab results change, sometimes unexpectedly—and clinicians who have been
treating a patient for a week or more may be believe there is little new information in
the chart. In addition, because not all relevant information is always systematically
recorded, some information might not be available for later review. In turn, systematic
documentation, while it can be reviewed and interrogated, is subject to the same
selectivity and need for assumptions about each participant.
Figure 3 Figure 4
R. Koppel and J.L. Telles / Role of Healthcare Information Technology in Handoffs 25
The disaffected woman in the Modigliani (Figure 4) reminds us that clinicians, like all
humans, vary in their level of attention to and involvement in the handoff process. Is
the departing clinician/team exhausted or eager to leave? Is the incoming clinician/team
a passive recipient due to overconfidence, unrealistic beliefs about the patient’s
condition, disciplinary presumptiveness or risky eagerness to start care? On the other
hand, perhaps the incoming clinician already knows a lot about the patient, so only new
information is needed. Adding more information would be wasteful or at least one
presumes so. In these cases, the failure to incorporate structured handoff forms, EHRs,
and/or paper records could produce unwanted consequences. Relying on memory,
when the initial transfer of information was poorly attended and documented, seems
unwise.
The image of two angels with Christ (Figure 5) poses the dilemma when an
authority figure (e.g., attending physician) offers insights and great learning, but may
also inhibit free (and needed) communication. Some handoffs are mediated by a
respected authority, such as a senior physician or a nurse manager. The respected
authority may help clarify ambiguous information, demand additional information,
raise new issues to increase understanding and improve treatment plans. As noted
above, however, the authority figure may prevent questioning and additional inquiry.
We often observed younger clinicians in the presence of older and respected authorities
use available HIT as a neutral means to broaden the discussion by referring to patient
information, clinical guidelines, or even recent articles on patients’ conditions. Here the
authority’s power is moderated by the HIT in respectful but helpful ways. We also
repeatedly observed settings where senior clinicians, while acknowledging the HIT-
provided information, offered additional insights to impart lessons on the complexity of
care and the need to balance factors missing in the standard protocols.
Figure 5 Figure 6
Ellsworth Kelly’s painting (Figure 6) highlights the role of structure. Highly structured
handoff forms & processes increase the probability that all items will be included. But
critical issues and consequential nuance may be lost. Highly structured forms or
templates can simplify and organize handoff information—making information entry
and retrieval faster and more secure than other methods. The downside of such forms is
that they often encourage a disgorging of routine and redundant data, in loss of
26 R. Koppel and J.L. Telles / Role of Healthcare Information Technology in Handoffs
emphasis and critical nuance. These advantages and disadvantages hold true for both
HIT-based templates and paper forms. One possible advantage of paper is that the
affordances of the medium allows easier annotation and comment than does the digital
form. Also, independent of the medium, each of these forms and templates are met with
varying levels of compliance.
We insert a computer screen into Durer’s Adam and Eve (Figure 7) to ask whether HIT
helps to convey needed information or is a distraction. EHRs, lab reports, I-O charts,
progress notes (in paper or HIT or both) are used in some handoffs. How can these
media be integrated to help focus clinicians on essential elements and present their
missing critical information? What are the risks these media will disperse focus and
lead clinicians to incorporate irrelevant or obvious information? Do they generate more
“busy work” that detracts from the clinicians’ thought-flow, workflow and time for
direct care? Are they used consistently or erratically? Does their use vary by unit,
individual, or supervisor? Most important: how are the paper and HIT (if both)
integrated? If they are hybrid systems – which most appear to be – do they maximize
the advantages of each medium? Are they redundant, complementary, reinforcing, or
distracting? While these are not new questions for any handoff analysis, the framework
suggested in Table 1 may help structure questions and a research agenda for further and
systematic investigation.
The meaning of Rembrandt’s Abraham and Isaac (Figure 8) depends on knowing
the biblical story. In addition to conveying information (in whatever form), a successful
handoff must reflect not only the recipient’s level of understanding--both of medical
knowledge and the patient’s condition--but also accommodate the participants’
interactions and relationships. There is always a context, expertise of each participant,
and a history of previous interactions in addition to the information exchange.
Efficient and effective handoffs reflect both needed information and awareness of the
participants’ context, backgrounds, and expectations.
Figure 7 Figure 8
3. Summary
Handoffs are consequential for patient care, clinician learning, patient safety, clinical
efficiency and liability. Improving our understanding of handoffs--especially how to
integrate HIT into handoffs--can improve outcomes. We suspect the use of HIT in
R. Koppel and J.L. Telles / Role of Healthcare Information Technology in Handoffs 27
handoffs will improve care and efficiency. To date, HIT use in handoffs appears
inconsistent, is frequently supplemental, and may distract as well as illuminate. Paper
remains the medium of choice and integration of oral, paper and HIT is not well
thought out. To address this essential patient safety concern, we need a coherent
strategy for handoff processes that use of HIT and/or HIT with paper documents. We
hope this typology and the illustrated discussion spur further examination of HIT in
handoffs, leading ultimately to efficacious use of HIT and other media to improve the
quality and efficiency of patient care.
References
[1] J. Johnson and V. Arora, Improving clinical handovers: creating local solutions for a global problem
Qual Saf Health Care 18:4 (2009), 244-245.
[2] I. Philbert, Use of strategies from high-reliability organisations to the patient hand-off by resident
physicians: practical implications Qual Saf Health Care 18 (2009), 261-266 .
[3] V. Arora V, J. Johnson, D. Lovinger, H. Humphrey, D. Meltzer, Communication failures in patient
sign-out and suggestions for improvement: a critical incident analysis Qual Saf Health Care 14 (2005)
401-407.
[4] M. Flanagan, E. Patterson, R. Frankel, B. Doebbeling Evaluation of a Physician Informatics Tool to
Improve Patient Handoffs, JAMIA 16:4, (2009), 509–515.
[5] L. Riesenberg, J. Leisch, J. Cunningham, Nursing Handoffs: A Systematic Review of the Literature ,
AJN, 110:4 (2010 ), 24-34.
[6] H. Garfinkel, ’Good’ organizational reasons for ‘bad’ clinic records in H. Garfinkel, Studies in
Ethomethodology, Prentice-Hall, Inc., Englewood Cliffs, 1967, 186-207.
[7] C. Heath and P. Luff, Documents and Professional Practice: 'bad' organisational reasons for 'good'
clinical records in Proceedings of the Conference on Computer Supported Cooperative Work, ACM
Press, Boston, 1996, 354-363.
28 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-28
Abstract. In 2000 Shaw and Kidd published an article on the lessons that could be
learned from the UK in general practice computerization. Over a decade later
many of these lessons remain yet to be learned. Hence Shaw & Bainbridge felt that
it was time to revisit these issues and review progress made against each in both
the UK and Australia in an effort to help Canada learn the lessons as it follows
behind both countries. Nine lessons are identified, reviewed and discussed in the
hope that Canada will choose to take note and leapfrog these jurisdictions by
learning from history, rather than being doomed to repeat it.
Introduction
“Those that fail to learn from history, are doomed to repeat it.” ~ Winston Churchill
At the turn of the century Shaw (formerly Ellis) and Kidd published an article on what
lessons Australia could learn from the United Kingdom (UK) in the area of general
practice computersisation [1]. Just over a decade later and it seems that many of these
lessons remain outstanding. Further, that other developed countries with comparable
medical systems, such as Canada, seem to be ignoring what has been learned through,
recent global experience of success as well as failure. We therefore thought that it was
an appropriate time to revisit the original article and draw attention to the lessons that
need to be learned, and acted upon, if we don’t wish to simply repeat the failures that
have gone before us.
Given also that the Canadian government has committed to primary care reform
and to a primary care led health care system and that Canada Health Infoway has
finally been allocated funding to support the adoption and implementation of Electronic
Records (ERs) in general practice it seemed especially appropriate to re-target these
lessons.
1
Corresponding Author
N.T. Shaw and M. Bainbridge / Computerisation in General Practice 29
developing data extraction and aggregation programmes but one must question both the
clinical and research value of these data without standardisation of definitions, coding,
structures or semantics. The Personally Controlled Electronic Health Record [14]
(PCEHR) for Australia went live in July 2012. It is based on a core ‘document’ set
indexed by a Unique National Individual Health Identifier (IHI) (shared health
summary, event summary, discharge summary, Medicare data, & consumer entered
data with pathology, specialist letters, prescriptions and dispensing notices to follow).
At present, the conformance criteria for some documents are low and permit free text
and ‘pdf’ content. However, the system is built to allow structured and coded data [15]
and will, over time, catalyze change and be delivered through increasing the level of
conformance criteria.
Lesson for Canada -The implementation and use of the PCEHR is a program that
Canada should follow closely. Minimal initial change from both vendors and clinicians
is needed at first but this approach also sets a path towards strategic goals which is
flexible.
Read Codes for diagnosis and symptoms were accepted as the standard for English
general practice computerisation in 1988 [17]. Their initial use wasn’t standardised
with slightly different versions being by different vendors. The editorial process was
seen as unresponsive and some vendors also authored and released their own additional
proprietary codes. This practice has now died out as the twin forces of interoperability
and a quality-based GP contract have emerged. As electronic records expanded beyond
general practice, the need for a more comprehensive terminology was recognised. This
led eventually to the international development of the SNOMED Clinical Terms
(SNOMED-CT) and its current instantiation as a ‘not for profit’ collaborative
organization between 19 Countries [18]. It has become a ‘fundamental’ standard for
clinical coding in England [19] with other countries set to follow this lead. Since the
advent of the 2003 GP contract the use of the computer at the point of care is almost
ubiquitous. There is also much more delegation and sharing of information between
Health Care Providers (beyond the scope of this paper). Both Canada and Australia are
participating members of the International Health Terminology Standards Development
Organisation (IHTSDO) and have named SNOMED-CT as their ‘preferred’
nomenclature nationally. However, in both countries, the proliferation of other coding
schemes and the confusion between nomenclature and classification has been allowed
to continue. Classifications such as ICD-10 and ICPC2 remain in use at the point of
care rather than being derived from a nomenclature such as SNOMED-CT. Many
vendor systems still don’t encourage or support clinical coding and rely heavily on
financial coding undertaken for billing purposes to derive clinical utility, audit and
reporting data.
Lesson for Canada – It is vital to set a course toward a mandated SNOMED and
Canadian National drug extensions as soon as possible. This effort has demonstrably
moved England forward. Australia has been less demonstrative but has set the same
direction of travel through conformance with CDA architecture [20]. Consideration
must be given for effective Vendor and Clinical engagement as the usability of systems
incorporating SNOMED will determine acceptability.
N.T. Shaw and M. Bainbridge / Computerisation in General Practice 31
Despite almost complete general practice computerisation since the mid 1980’s it was
only in 2007 that records were able to be transferred between practices using different
UK vendor systems. Five years after initial success nearly all clinical systems have
been deployed containing (or about to contain) accredited software [21]. To date
2,137,301 (27th June 2012) patient records have been transferred. There has been great
investment of time, effort and money in establishing data transfer standards, building
and then deploying a central ‘spine’ through which data would be transferred.
Awareness and acceptance of the need for record transfer amongst the medical
professional community who traditionally felt a great deal of ownership towards “their”
records has also had to be achieved on top of safe and professionally acceptable and
semantically correct transfer standards.
Despite the domination of the market by one single vendor there is still a complete
lack of interoperability in Australian general practice although the intention is that the
PCEHR will catalyse resolution in this area. The PCEHR is a document based
implantation of Clinical Document Architecture [15,16-20] with an ultimate goal of
fully structured and SNOMED coded content. Increasing levels of compliance will be
driven through the Standards-Australia process. There is considerable overlap with this
section and section 1.1 the common data model. The two are synergistic and
inseparable.
Lesson for Canada – Interoperability is the best catalyst for change as it forces
the re-examination and re-configuration in multiple areas from definitions to
infrastructure. There is an opportunity to shape the hard-won successes in this area for
local use in Canada. This, however, will only be achieved through a coordinated
national approach.
Developing a business case for electronic records has become the subject of many case
studies and classroom debates. Whilst more readily accepted by larger groups of
practitioners and those concerned with population level health, as opposed to individual
general practitioners, a true business case for adoption and implementation of
Electronic Health Records (EHR) has not yet been mad [22].
Despite this there is an increasing consumer demand for service, excellence and
availability that electronic record supporters purport that such systems can support and
achieve if only they were implemented. At the same time consumers are increasingly
sharing their own personal information electronically, through social media. Other
service industries are increasingly reliant on online consumer participation for much of
their service provision.
Likewise, there is a growing emphasis on clinical safety as a driver for
computerization. Whilst it makes inherent sense that computerisation could improve
safety, by such measures as forcing compliance with standardization and ensuring
legibility, there is little, to no, evidence that electronic records do actually improve
clinical safety beyond the area of prescribing [23,24]. Some of the reasons behind this
lack of evidence lie in the proprietary nature of the systems and the continued
difficulties in deriving interoperable and truly comparable data at a scale to prove the
hypothesis.
32 N.T. Shaw and M. Bainbridge / Computerisation in General Practice
Lesson for Canada – Consumer demand for excellence and safety will rapidly
mount if the current situation continues – It is also important to keep in mind that
ubiquity of implementation of the current systems will at best mechanize current
practice and this is insufficient to meet upcoming demands.
The quality of data within an organization has been a recognized problem for decades
but clinics rarely run simple data quality checks of their own accord. In the UK the
government attempted to address this issue by establishing the PRIMIS programme
[25] to provide localized training and data analysis. One of the primary components of
this program is to help general practitioners ensure that they have good data quality.
Australia has recently published training materials for Clinicians to use in support of
the PCEHR[26] and has recently funded the Royal Australian College of GPs
(RACGP) to intervene on the broader e-health agenda [28] .There is, as yet, however,
no national ‘bottom up’ support for e-health and information proficiency although the
eCollaboratives [27] set up to innovate in Chronic disease care and self-management
may grow to fulfill this role.
Lesson for Canada - Installing the computer into the practice with appropriate
infrastructure is the easy bit. What is less easy and traditionally less well done is to help
the practices to become more able with electronic clinical data. All the benefits derive
from high quality data. To achieve this requires a complex and personalised mix of
usability, infrastructure and culture.
Traditionally, most primary care clinical systems, their hardware, software and a
maintenance contract are sourced from a single clinical system vendor. In all three
countries in the past two decades this has slowly changed so that buyers can now buy
these items independently. The cost, for this freedom, however, is that clinics
embracing technology have had to become incredibly technically literate or have had to
hire specific IT staff. For larger clinics this may be a reasonable administrative cost;
not so much for those that are smaller. Often the response to this issue is for
practitioners to work in groups. However, this discriminates against those practitioners
choosing to work in the more isolated, rural regions where having a practitioner is a
luxury and a group of them merely a pipe-dream.
In the UK, hosted systems are becoming the norm. Clinical System, data,
infrastructure and maintenance are managed centrally. To date, this has not been
possible to replicate in Australia due to the high capacity internet requirements that
result. However, the Australian National Broadband Network [29] which will be
installed over the next 2-3 years may make this a more feasible option reducing the cost
of ownership, mean time between failures (MTBF) and also removing the need for
specialized knowledge at the clinic. Since the implementation of the 2003 Contract UK
GPs do not pay for their clinical systems and choose them from a ‘guaranteed’ choice
of conformant vendors. The Primary care trust / commissioning group are centrally
funded to provide these systems. Australian General Practice has not moved towards a
contractual change and still works on a strict ‘fee for service’ model with the GP
N.T. Shaw and M. Bainbridge / Computerisation in General Practice 33
Practice responsible for their own IT provision. A national incentive programme ePIP
[30] will come into operation in January 2013 and focuses on laying sound foundations
for the uptake of patient identifiers as well as coding of diagnosis and electronic
prescribing.
Lesson for Canada – There is considerable maturity of thinking in this area and
care should be taken to learn from both the successes and errors in England and
Australia. The requirements are virtually identical and implementation need not be
difficult if a mature and informed national approach is taken.
1.9. Research
Enormous investments in EHRs have been made over the last forty years in the belief
that they will bring cash releasing financial savings to an over-burdened health care
34 N.T. Shaw and M. Bainbridge / Computerisation in General Practice
2. Conclusions
For over three decades now the potential benefits of computerization in health care
have been touted. Despite this and despite billions of dollars of public funding being
invested, the fruits of these systems have not yet been fully realized. We believe that
until the nine lessons above are truly learned that this will be the status quo for another
quarter century. This, for those of us that work in health informatics and truly believe
in its capabilities when used appropriately is intensely disheartening. We hope that by
laying out these issues and pointing users in the direction of progress in these areas,
where it has been made, that internationally we can pick up the pace and put the
building blocks in place that will mean that Canada will truly be providing the world
class health care that we know its health care workers are striving to provide.
References
[1] Ellis NT, Kidd MR. General practice computerisation: lessons from the United Kingdom. Med J Aust
2000; 172(1):22-24.
N.T. Shaw and M. Bainbridge / Computerisation in General Practice 35
[2] Kidd MR. The computerisation of Australian general practice 1998–2001 – what did we get for AU$15
000 000? Informatics in Primary Care 2002; 10:25-29.
[3] UKCHIP The UK Council for Health Informatics Professionals [ 2012 [cited 2012 Oct 31]: Available
from: URL:http://www.ukchip.org
[4] NHS. eICE (Embedding Informatics in Clinical Education [ 2012 [cited 2012 Oct 31]: Available
from: URL:http://learning.nhslocal.nhs.uk/courses/careers/health-informatics/eice-embedding-
informatics-clinical-education
[5] NHS. NHS Connecting for Health. NHS [ 2012 [cited 2012 Sept. 14]; Available from:
URL:http://www.connectingforhealth.nhs.uk/
[6] EN 13606. The EN13606 Association [ 2012 [cited 2012 Oct 31]: Available from:
URL:http://www.en13606.org/
[7] Open EHR GEHR- The good European Health Record[ 2012 [cited 2012 Oct 31]: Available from:
URL: http://www.openehr.org/new-ws/site/resources/gehr
[8] HL7 Health LEvel Seven International [ 2012 [cited 2012 Oct 31]: Available from:
URL:http://www.hl7.org/
[9] IHTSDO International Health Terminology Standards Development Organisation [ 2012 [cited 2012
Oct 31]: Available from: URL:http://www.ihtsdo.org/
[10] NHS. General Practice Extraction Service. NHS [ 2012 Available from:
URL:http://www.ic.nhs.uk/gpes
[11] NHS. GP Systems of Choice (GPSoC). NHS [ 2012 [cited 2012 Sept. 14]; Available from:
URL:http://www.connectingforhealth.nhs.uk/systemsandservices/gpsupport/gpsoc
[12] Healthbase Australia Personally Controlled Electronic Health Record [ 2012 [cited 2012 Oct 31]:
Available from: URL:http://www.healthbase.info/pcehr/page5/page5.html
[14] DOHA About the PCEHR System [ 2012 [cited 2012 Oct 31]: Available from:
URL:http://www.ehealthinfo.gov.au/personally-controlled-electronic-health-records/about-the-pcehr-
system
[15] Dolin R, et al The HL7 Clinical Document Architecture Am Med Inform Assoc. 2006 Jan-Feb; 13(1):
30–39. doi: 10.1197/jamia.M1888
[16] Beeler JW, HL7 version 3--an object-oriented methodology for collaborative standards development.
Int J Med Inform. 1998 Feb;48(1-3):151-61.
[17] Chisholm J. The Read clinical classification. BMJ 1990; 300(6732):1092.
[18] IHTSDO Members of IHTSDO [ 2012 [cited 2012 Oct 31]: Available from: URL:
http://www.ihtsdo.org/members/
[19] DH (England) Common clinical languague for NHS will help improve patient care and safety[ 2012
[cited 2012 Oct 31]: Available from: URL:http://mediacentre.dh.gov.uk/2011/08/16/common-clinical-
language-for-nhs/
[20] HL7. Clinical Document Architecture (CDA). HL7 [ 2012 [cited 2012 Sept. 14]; Available from:
URL:http://wiki.hl7.org/index.php?title=Product_CDA
[21] NHS GP2GP [ 2012 [cited 2012 Oct 31]: Available from: URL:
(http://www.connectingforhealth.nhs.uk/systemsandservices/gpsupport/gp2gp
[22] Song PH, McAlearney AS, Robbins J, McCullough JS. Exploring the Business Case for Ambulatory
Electronic Health Record System Adoption. Journal of Healthcare Management 2011; 56(3):169-180.
[23] Brown M, Shaw NT, Grimm NA, Muttitt S, Gebran J. Electronic Health Records and Patient Safety:
What Lessons Can Canada Learn from the Experience of Others? Healthcare Quarterly 2008;
11(1):112-119.
[24] Brown M, Grimm NA, Shaw NT. The relationship between Electronic Health Records and Patient
Safety: Future Directions for Canada. iCARE, Canada Health Infoway, Canadian Patient Safety
Institute, editors. 2007. Canada Health Infoway.
[25] NHS/ PRIMIS Background to PRIMIS [ 2012 [cited 2012 Oct 31]: Available from:
URL:http://www.primis.nottingham.ac.uk/index.php/about-us/background-information
[26] DOHA eHealth records learning centre [ 2012 [cited 2012 Oct 31]: Available from:
URL:http://publiclearning.ehealth.gov.au/
[27] Improvement foundation eCollaborative-21st Century Patient Care and Self Management [ 2012 [cited
2012 Oct 31]: Available from: URL: http://improve.org.au/our-work/e-collaborative/
[28] RACGP Media Release[ 2012 [cited 2012 Oct 31]: Available from:
URL:http://www.racgp.org.au/yourracgp/news/media-releases/health-minister-announces-$255-
million-funding-to-support-gps-engage-with-the-pcehr/
[29] NBN Case Study [ 2012 [cited 2012 Oct 31]: Available from: URL:http://www.nbn.gov.au/case-
study/dr-stephanie-trust-health/
[30] RACGP Practice Incentive Programme e-health incentive (ePIP) [ 2012 [cited 2012 Oct 31]:
Available from: URL:http://www.racgp.org.au/your-practice/e-health/epip/
36 N.T. Shaw and M. Bainbridge / Computerisation in General Practice
[31] Salfordsoftware. NHS Smart Card Authentication. Salfordsoftware [ 2012 [cited 2012 Sept. 14];
Available from: URL:http://www.salfordsoftware.co.uk/identity-management/identity-solutions-for-
the-nhs/nhs-smart-card-authentication
[32] UNSW Use of information and communication technologies to support effective work practice
innovation in the health sector: a multi-site study. [ 2012 [cited 2012 Oct 31]: Available from:
URL:https://research.unsw.edu.au/projects/use-information-and-communication-technologies-support-
effective-work-practice-innovation-h
[33] NeHTA National Authentication Service for Health [ 2012 [cited 2012 Oct 31]: Available from:
URL:http://www.nehta.gov.au/connecting-australia/nash
[34] Delimeter IBM's NASH deal gets terminated [ 2012 [cited 2012 Oct 31]: Available from:
URL:http://delimiter.com.au/2012/10/23/ibms-nash-deal-gets-terminated/
[35] PRIMIS+ Guidelines. 1-114. 2001. Nottingham, UK, PRIMIS+ (Primary Care Information Services).
[36] Weed LL, New connections between medical knowledge and patient care BMJ 1997;315:231
!"#
!
Z
\
<^`
*{
*
`
**
|
<
|
}
**
|
``
<
**}
|
**
|\
<
<#
>
`
`!
<
>
!
!
*
<#
*
|
<
| *
|>
>>
><#
>
\
|
>
*
*`
*`
<
|
`
<
*!
*`
}
<
**} |
*
!
`
< } |
|
*
*
38 O. Shabestari and A. Roudsari / Challenges in Data Quality Assurance for Electronic Health Records
`
**
> |\
<
}
\
***
<
}
`* *
` |\
} | |
*
<
**
!
|
*!
*`<
`|}
\*
Z
`
*
| | *
}
} }*
!
<
\
!
} |
*
<
\*
` **
*
}
}
}
*
}
}
!
*
}`
\
`|<
*
\
*
<
*`
| !
\ |
*
**!
<
!
`
\
|
!
**
< |
`
*
}
|
\!
< *}
*
!
|
*
<
| `
`
<
`!
\|}
*
|
<
` |
`
< #
`
|
*
!
!
<
<
!
* `
|
|
O. Shabestari and A. Roudsari / Challenges in Data Quality Assurance for Electronic Health Records 39
`!*
<!
**
*
*
<
*
!
} |
! `
<
*
*!
<
<
!
*
!
|
|
|
<
!
<
`
| **
`
{
<
*
!
<
*
\| |
!
*
*!
!
|
*
*
`
!
<
* |
|
`**
**
!
|
*`
<
<
*
**
< `
!
** |
!
*
<
*
|*}|
*
<
|* !
**
!
<
|
*
|
*`
`
<!`
!
|
!
*
<
|
!
**
|
*
!
<
^
|
!
|
*
<
}
| `
**
*
!
*
<
*
!
` | |
<^`
\
*
*
}
>
`|
|
!
|
<
40 O. Shabestari and A. Roudsari / Challenges in Data Quality Assurance for Electronic Health Records
}
`
!*
*
><
*
*
!
*
*
<
**
*
* !
|
|
\
*
<
}
`!\
* *
*`
\
*
*
!
*
!
|\<
>
*
!
*
*
<
|
!
!
!
*
|!
* `>
*`
!
`
< >
`!
*
<
|
*
*
`
`<
*
!
` | |
O. Shabestari and A. Roudsari / Challenges in Data Quality Assurance for Electronic Health Records 41
<^`
\
>
` |
|
!
|
<
}
`
! *
*
*
<
**
*
*
!
|
|
\
*
<
}
`!\
* *
*`
\
*
*
!
*
!
|\<
!
>
! !
!
*
` |
*`
<
`
}
`
*
| ! *
!\
*
<
**Z
\*
<
*
*`
*
<
!
`Z
<
"
<
!
!
"
}
<
¢< } £<< !
}
!
#$
}<
<
}^<#<
}<
}#
>
*
|\
¤\
<
%#}<
}'
()*
"+}
|}
}<
\* ¥¤} }
"}<
¥
} >
!
:<
?
}
"¦}§<
¤} ¨
} ¤!
} } ^` !
<:
%}<
} #}
*
<
:-%'(}<
¥ #}
¤!
} ¨} | #} #*
}
**
*
"¦
` *
*
` `|
<:*'}<
42 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
1
Bogdan Motoc, [email protected].
Enabling Health and Healthcare through ICT 43
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-43
Keywords. child, in-care, foster, looked after, out of home, health record, review
Introduction
Children-in-care are among the most vulnerable populations and at risk for inadequate
health care compared to other groups of children [1]. When a child is taken into care,
the province or state has a legal obligation to provide reasonable health and dental care
[1-5]. Record keeping is required to provide tailored care to the child-in-care [6], to
procure records for the judiciary system [5,7,8], and for research on interventions for
good health outcomes [9]. The challenge is that “child care [traditionally] lacks an
information-driven culture” [10]. Many health records have been developed for adult
care [11]. Further, intake screening tools [12,13] and systems for child protection
research [9] have been implemented. A health record for a child-in-care needs to
capture additional data to meet statutory care requirements [7,9] and these data are also
needed for current and future health and social care planning [6,7]. Further, in the
United States, maintaining up-to-date foster care health records is legislated [4,7]. To
determine the current state of knowledge on these health records, this review focuses
on published academic studies on paper-based and electronic health records designed
specifically for child-in-care.
1
Corresponding Author - Email: [email protected]
44 C. Thompson and F.Y. Lau / A Scoping Review on Health Records for Child-in-Care
1. Methods
A scoping review was conducted to explore evidence on the current state of the child-
in-care health record in academic journals. Findings would inform whether to proceed
to include grey literature for other systems or to an in-depth systematic review [14].
Two concepts were searched: “child-in-care” and “health record”. The records or
“systems” could be either paper-based or electronic or both. Databases searched were:
Social Services Abstracts, Social Work Abstracts, FAMILY (Informit), Academic
Abstract Complete, and Web of Science (ISI – Web of Knowledge) from 1980 to
August 2012; and Medline (EBSCO) from 2002 to August 2012. Detailed search
protocols are available upon request from the authors.
One reviewer screened the title and abstract of each study for inclusion. Studies
included must 1) detail an electronic or paper-based child-in-care health record used by
providers at point of care, 2) describe a specific information system or record, and 3) be
a primary study. Studies were excluded if they were 1) a commentary or discussion, 2)
a secondary analysis, or 3) a system used for adults. Where it was not clear whether to
include a study, the full article was retrieved and screened. The included studies were
then reviewed.
The studies were reviewed for data elements specific to child-in-care health
records and compared against the data recommended for child personal health record
(PHR) [15]. Child PHR recommendations were chosen as a comparison as PHRs are
longitudinal summaries of key events from all providers, distinct from electronic health
records, and are held by the individual [15]. As a child-in-care moves the record should
move with the child [5,7,19]. Further, close linkages are needed between research,
policy and practice for positive changes [5,16]. Research findings must be presented
within a “what works” framework for research to effectively inform practice and policy
for positive changes [17]. “What works” were defined by the studies and reported as
“results” or “findings” and were thematically aggregated [18].
2. Results
We started with 1264 citations and, after applying the inclusion/exclusion criteria, eight
(8) studies met the criteria and are detailed in Table 1. One study was published in a
medical journal [3], and the other seven studies were published in social work journals.
No studies were published in informatics journals. The studies report on five unique
child-in-care health records. Four records were paper-based; two were paper-based
moving to electronic; one was electronic and generated a paper report; and one was
electronic. Overall, the eight studies reported positive findings on the implementations
and advocated for further research and development. The following sections summarize
the findings in terms of definitions, study descriptions, data elements, and key findings
on what worked for the implementations.
The Child, Family and Community Service Act [2] defines a “child-in-care” as a “child
who is in the custody, care or guardianship of a director or a director of adoption”. In
this context, "child" means “a person under 19 years of age and includes a youth” and
"care” means the “physical care and control of the child”. Different terms were found
C. Thompson and F.Y. Lau / A Scoping Review on Health Records for Child-in-Care 45
Table 1 lists the eight studies by first author and publication date (pub date), study aim,
setting and location, participants and records, child-in-care health record (CIC System),
paper or electronic. Adoption levels ranged from not implemented [10] to poor usage
[3] to fully integrated into practice [5,8,19] to able to use record data to generate
reports [7,20,21]. Current levels of adoption are outside the scope of this paper.
Hunter To investigate whether a Child care NHS staff, BAAF Health Record
[19]; 2008 specialist nursing service units; children, nursing Booklets
could improve health care for three areas of services; (carer-held health
these children Scotland 162 children in records);
residential care Paper
records
Child-in-care records contain additional data elements beyond those in a child PHR.
Data elements recommended for a child PHR include: demographic and insurance
information; contact information for family members, other support providers and
health care providers; advanced directives; and clinical information such as: problem
lists, encounters, procedures, chronic conditions, medications, immunizations, and
allergies, vital signs of weight, health, length, body mass index, laboratory results,
family history, birth history, durable medical equipment and supplies [15]. The child-
in-care health record also needs to include: placement information (e.g. at a foster
home), dental, abuse, education and court dates, among other data elements. These data
allow looking at the overall wellbeing and development of the child [21]. For instance,
low education level and high placement changes could be seen as co-existing factor
[21], multiple placements and behaviour issues may be identified concurrently [5],
medical history is known when planning for child adoption [5]. Further, at an aggregate
level, the data can inform policy and practice. For example, a service need for a
therapeutic environment for children who have experienced trauma [21].
3. Discussion
The paucity of studies that met the inclusion criteria suggests that there has been little
development specifically on child-in-care health records; however, studies were found
on other child records: pediatric, school, and personal health records. These records
might contribute to the further development of the child-in-care health record. Of the
eight studies included, three provided few details on the record or implementation
C. Thompson and F.Y. Lau / A Scoping Review on Health Records for Child-in-Care 47
[3,10,19] and two studies did not report positively on the implementation [3,10]. Four
of the eight studies were on the Looking After Children records. All studies anticipated or
reported benefits from using the records. One study reported that using the health
records enabled moving from reactive to proactive care for the child [5]. None of the
studies indicated that the records would no longer be used, except in Smart et al where
the system was decommissioned and replaced by a state-wide system. As several
studies referenced child-in-care health records not found during this review, a fuller
review is needed.
A key concept in the studies was the joining of the social work case management
record and child health status record [7]. Range in integration reached from one study
with six source records [3] to two studies with single source records and ability for
aggregate reporting [7,21] to a real time record with aggregate reporting ability [20].
There is clear direction to link research to policy and practice for evidenced-based
policy and practice changes. For example, identifying factors associated with
developmental progress or deterioration while in care [21]. Interagency cooperation
was found to be integral to developing and effective using the record in all the studies.
Further, to support individual care and research, complete, accurate and consistent
records are needed. Success factors to better recordkeeping included support roles,
audits, education, and results reporting.
4. Study Limitations
Review limitations include a single reviewer did the search, analysis and synthesis so
there could be selection, synthesis and reporting bias. Only studies in English were
included. No formal quality assessment of the eight studies was done. For terms,
through searching, it was found that “child welfare” may reach beyond the connotation
of “a child-in-care” to the broader connotation of overall child health and wellness. In
addition, all the terms relating to “child-in-care” may not have been included in the
searches.
5. Conclusion
References
[1] Kools S, Kennedy C. Foster child health and development: implications for primary care. Pediatr Nurs.
2003; 29(1): 39-46.
[2] Child, Family and Community Service Act, RSBC 1996 c 46.
[3] Bundle A. Health of teenagers in residential care: comparison of data held by care staff with data in
community child health records. Arch Dis Child. 2001; 84(1): 10-14.
[4] Risley-Curtiss C, Kronenfeld JJ. Health care policies for children in out-of-home care. Child Welfare.
2001; 80(3): 325-350.
[5] Kufeldt K, McGilligan L, Klein R, Rideout S. The Looking After Children assessment process:
promoting resilient children and resilient workers. Families in Society. 2006; 87(4): 565-574.
[6] Kelly S. Obtaining children's health information. Adoption & Fostering. 2007; 31(3): 80-82.
[7] Lindsay S, Chadwick, D, Landsverk J, Pierce E. A computerized health and education passport for
children in out-of-home care: the San Diego model. Child Welfare.1993; 72(6): 581-594.
[8] Knowles R, Blackburn M, Zahir M, Russell M, Carrier A, Nevrkla E. The implementation of a new
parallel child health record. Child Care Health Dev. 1999; 25(4): 253-266.
[9] Sternberg KJ, Knutson JF, Lamb ME, Baradaran LP, Nolan CM, Flanzer S. The child maltreatment log:
a computer-based program for describing research samples. Child Maltreat. 2004; 9(1): 30-48.
[10] Kerslake A. Computerization of the Looking After Children records: issues of implementation.
Children & Society.1998; 12(3): 239-237.
[11] Kim GR. (2011). Snapshots of child health and information technology. In: C. Lehmann, GR Kim, &
KB Johnson (Eds.), Pediatric Informatics: Computer Applications in Child Health. (pp. 3-4). New York,
NY: Springer.
[12] Anderson RL, Lyons JS, Giles DM, Price JA, Estle G. Reliability of the child and adolescent needs and
strengths-mental health (CANS-MH) scale. Journal of Child and Family Studies. 2003; 12(3): 279-289.
[13] Jee SH, Halterman JS, Szilagyi M., Conn AM, Alpert-Gillis L, Szilagyi PG. Use of a brief standardized
screening instrument in a primary care setting to enhance detection of social-emotional problems
among youth in foster care. Acad Pediatr. 2011; 11(5): 409-413.
[14] Rumrill PD, Fitzgerald SM, Merchant WR. Using scoping literature reviews as a means of
understanding and interpreting existing literature. Work. 2010; 35(3): 399-404.
[15] Zuckerman AE, Kim GR. (2011). Personal Health Records. In: C. Lehmann, GR Kim, & KB Johnson
(Eds), Pediatric Informatics: Computer Applications in Child Health. (pp. 293-301). New York, NY:
Springer.
[16] Davies C. (2003). Policy development: making research count. In: K. Kufeldt & B. McKenzie (Eds.),
Child welfare: Connecting research, policy and practice (pp. 377-386). Waterloo, ON: Wilfrid Laurier
Press.
[17] Jones H. (2003). The relationship between research, policy, and practice in delivering an outcome-led
child welfare service. In: K. Kufeldt & B. McKenzie (Eds.), Child welfare: Connecting research,
policy and practice (pp. 367-376). Waterloo, ON: Wilfrid Laurier Press.
[18] Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews.
BMC Med Res Methodol. 2008; 8(1): 45-__.
[19] Hunter D, McCartney G, Fleming S, Guy F. Improving the health of looked after children in Scotland.
Adoption and Fostering. 2008; 32(4): 51-56.
[20] Smart J, Russell J, Custodio C. Developing a computerized health record in a protective services system.
Child Welfare. 1998; 77(3): 347-362.
[21] Champion R, Wise S. Developing outcomes-based data from Looking After Children case records
completed for children in care in Victoria, Australia. Vulnerable Children and Youth Studies. 2009;
4(2): 107-113.
Enabling Health and Healthcare through ICT 49
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-49
Introduction
1
Jesdeep Bassi, Email: [email protected]
50 J. Bassi et al. / Application of the TPCK Framework in Integrating an Educational EMR
1. What is TPCK?
Figure 1. Original TPCK (left) (Source: http://tpack.org/) and TPCK in context of research (boxes at right).
Each integration context is unique. In exploring EMR use for health informatics
education, the three knowledge bases required were defined as follows.
2.1. Technology
2.2. Pedagogy
2.3. Content
The content piece reflects the competencies or knowledge that students are expected to
possess. For health informatics, several groups have established sets of competencies
for the health informatics professional that outline both the knowledge and skills
graduating students should have. In Canada, COACH produced the Health Informatics
Professionalism (HIP™) Competency Framework [9]. In this framework, the core body
of knowledge for health informatics comes from three source practices: health sciences,
information sciences, and management sciences. Within these three areas are seven
subcategories: clinical and health sciences; Canadian health system; information
technology; information management; project management; organizational and
behavioral management; and analysis and evaluation. The specific competencies for a
health informatics professional are organized under each of these subcategories.
This research utilized TPCK, both in study design and the resulting output. Previous
authors have expressed that technology should not drive pedagogy [10,11] and in
specifically referring to TPCK, Harris and Hofer[12] express that TPCK shouldn’t be
“technocentric”, that is, focused on the technology. They have advocated for the
exploration of technology to best suit learning needs. While in this research the
technology was preselected, the goal was not to fit the health informatics curriculum
around the educational EMR, but rather to determine where it fits best into the
curriculum.
In exploring EMR integration into health informatics education, the TPCK
framework was used to frame the research questions as well as instrument design i.e.
interview and focus group questions. Given that each construct could be defined for the
health informatics context, the next step was to look at each of the pieces in relation to
each other to effectively uncover all the considerations (i.e. types of knowledge)
required for effective integration in this context (see boxes at right in Figure 1). For
example, what learning activities (pedagogy) could employ the educational EMR
(technology) to give the instructor TPK? Or what topics (content) covered in health
informatics courses are directly related to EMRs (technology) to give the instructor
TCK? Instructors and students were recruited to provide answers to these questions (in
interviews and focus groups) using TPCK as a guide.
The result of this research is a proposed framework for integration that builds on
and extends the original TPCK framework [5]. The items identified in the research
within each knowledge area (technology, pedagogy, content) are included with two
additional pieces. The first added piece is a continuum for when to integrate the
technology into the education. From the research, points of integration emerged which
ranged from a single activity to integration across multiple programs including health
informatics in a multidisciplinary approach. The second added piece is contextual
considerations that emerged which “open-up” the context circle surrounding TPCK.
These include course, student, instructor, technical, and system considerations as well
as overall learning pedagogy.
J. Bassi et al. / Application of the TPCK Framework in Integrating an Educational EMR 53
Conclusion
Acknowledgements
References
[1] COACH Canada’s Health Informatics Association (COACH). Health Informatics Professional Core
Competencies Version 1.0. [Internet]. 2007. [cited 2009 Mar 17]. Available from:
http://www.coachorg.com/career_development/professionalism/core_competencies.htm
[2] Nagle LM. Informatics: emerging concepts and issues. Nurs Leadersh. 2007; 20(1): 30-32.
[3] Physician Information Technology Office (PITO). About PITO. [Internet]. 2009. [cited 2012 Aug 25].
Available from http://www.pito.bc.ca/about-pito/
[4] Mishra P, Koehler, MJ. Technological pedagogical content knowledge: A framework for teacher
knowledge. Teacher’s College Record. 2006; 108(6): 1017-1054.
[5] Bassi J. Towards the development of a framework for integration of an electronic medical record into
an undergraduate health informatics curriculum. [M.Sc. Thesis]. University of Victoria. 2011.
[6] Carter JH (Ed.). Electronic Medical Records: A Guide for Physicians and Administrators. Philadelphia:
American College of Physicians-American Society of Internal Medicine, 2001.
[7] Joe RS, Kushniruk AW, Borycki EM, Armstrong B, Otto T, Ho K. Bringing electronic patient records
into health professional education: Software architecture and implementation. Stud Health Technol
Inform. 2004; 150: 888-892.
[8] Weston C, Cranton PA. Selecting instructional strategies. The Journal of Higher Education. 1986;
57(3): 259-288.
[9] COACH Canada’s Health Informatics Association (COACH). Health Informatics Professional Core
Competencies Version 2.0. [Internet]. 2009. [cited 2009 Jun 20]. Available from
http://www.coachorg.com/career_development/professionalism/core_competencies.htm
[10] Lai K. ICT Supporting the Learning Process: The Premise, Reality, and Promise, In J Voogt & G
Knezek (Eds.), International Handbook of Information Technology in Primary and Secondary
Education. New York: Springer Science + Business Media; 2008: 215-230.
[11] Jefferies P. ICT in supporting collaborative learning: Pedagogy and practice. Learning, Media and
Technology. 2003; 28(1): 35-48.
[12] Harris J, Hofer M. Instructional Planning Activity Types as Vehicles for Curriculum-based TPACK
Development, In CD Maddux (Ed.), Research Highlights in Technology and Teacher Education 2009.
Chesapeake: Society for Information Technology in Teacher Education (SITE); 2009: 99-108.
[13] Koehler MJ, Mishra P. Introducing TPCK. In AACTE Committee on Innovation and Technology (Ed.),
Handbook of Technological Pedagogical Content Knowledge (TPCK) for Educators. New York:
Routledge; 2008: 3-29.
This page intentionally left blank
Telecare, Telemedicine and Telelaboratory
This page intentionally left blank
Enabling Health and Healthcare through ICT 57
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-57
Abstract. This paper briefly describes the main characteristics of the TeMaD
system, developed for the Saudi National Guard Hospital in Riyadh. TeMaD
attempts to improve current healthcare services for diabetic patients, and assists
healthcare givers in disease management. It strengthens communication channels
between patients and their healthcare givers, possibly leading to better health.
Introduction
Diabetes is rapidly spreading in Saudi Arabia, affecting almost one third of the
population [1]. The healthcare sector is struggling to cope with the growing numbers of
diabetics and associated healthcare costs, nor is it prepared to deal with the expected
rise in chronic diseases due to an aging population [2]. Reasons fuelling this problem
include poor dietary habits, inactivity, an aging population and associated social factors.
The Telecare for Managing Diabetes (TeMaD) system has been developed to help
improve healthcare services offered at King Abdulaziz Medical City (KAMC) in the
capital Riyadh. It presents a telecare system that is tailored for patients in Saudi Arabia
in order to overcome specific obstacles sometimes linked to local culture and traditions
[3]. TeMaD is also a highly available system offering a number of practical
communication channels between the patients and their healthcare givers.
This paper reviews the main characteristics of TeMaD, and examines the impact it
has had on diabetic patients, during a three month trial period between 2009 and 2010.
1
Corresponding Author
58 K. Alkadi and A. Roudsari / TeMaD System: Telecare for Managing Diabetes in Saudi Arabia
number of DEs remains the same, leading to a strain on hospital resources. Furthermore,
blood glucose levels are recorded and monitored in an inefficient manner. Patients log
their readings in a small booklet, kept with the patient at all times. During the hospital
visit (may vary from once a month to once a week depending on the patient), the DE
reviews all recorded readings and discusses any anomalies with the patient. Based on
this information, the DE determines the next step. The DE has no access to these
readings between hospital visits, which may put high risk patients in danger, delaying
required medical intervention when necessary.
2.1. Transportation
Saudi Arabia lacks an adequate public transportation system. In addition, Saudi law
prohibits females from driving. This combination has resulted in ‘Transportation’ being
amongst the highest obstacles in receiving healthcare services as rated by KAMC
patients during a survey conducted in 2010, mostly affecting female patients.
Saudi has a unique culture which combines social, traditional, and religious factors. It
is common for patients to insist on receiving care from same-sex healthcare givers.
Society’s strong endorsement of gender-segregation may also contribute to the problem.
This may be difficult to satisfy, especially in rural centres with limited resources.
3. TeMaD Design
TeMaD targets diabetic patients at KAMC Clinic between 16-70 years. It allows
patients and healthcare givers at the clinic to communicate using 3 different modes:
Internet, SMS, and Dedicated Landline. These modes make TeMaD an accessible
solution, allowing patients from different age groups to select the most suitable one.
Let us review the TeMaD interfaces for both Patients (Users) and DEs (Administrators).
TeMaD offers patients three different modes of access. Each one targets a different age
group and offers different functionality, making TeMaD more accessible.
The DE uses the TeMaD Web Interface to manage patients’ diabetes. The interface is
user-friendly (see Figure 2), and enables DEs to perform various tasks, such as:
Review existing readings uploaded by patients via Web Interface and/or SMS
Log patient readings collected during a landline call
Examine patient performance levels over a period of time (daily, weekly,
monthly, 3-month)
Communicate with patient (SMS sent via TeMaD)
4. TeMaD Study
TeMaD was used to manage the diabetes of 79 participating patients during the period
December 2009 and June 2010. Random sampling is used, with ages between 16 to 70
years. Only 52 patients completed the trial period while 27 patients dropped out for
various reasons.
60 K. Alkadi and A. Roudsari / TeMaD System: Telecare for Managing Diabetes in Saudi Arabia
where: Reading 1: is the Hba1c level taken before starting TeMaD, and
Reading 2: is the Hba1c level taken after completing TeMaD, and
Hba1c change: is the difference between Reading 1 and Reading 2.
K. Alkadi and A. Roudsari / TeMaD System: Telecare for Managing Diabetes in Saudi Arabia 61
5. Results
The gender ratio of the total number of participating patients is 46% male and 54%
female, closely resembling the ratio of patients at the clinic between 2009 and 2010,
being 42% male and 58% female. This indicates that our sample size closely reflects
the clinic environment in gender ratio.
Statistics showed that 69% of TeMaD patients fall under the 16-29 age group,
suggesting that younger patients are more likely to use alternative methods, especially
those methods relying on technology.
In addition, only 12% of participating patients lived in rural areas. However, the
urban area of Riyadh spans across 2435km2, with KAMC located at the eastern
outskirts of the city. With almost 5 million inhabitants, peak hours can result in some
patients requiring more than two hours to commute back and forth to the hospital.
TeMaD better suited many patients saving them both time and money.
Most importantly, the primary goal of the study was achieved, with TeMaD
succeeding in reducing HbA1c levels by more than 5% for 83% of participating
patients, with a greater impact on females (see Figure 3). The average HbA1c level of
patients was reduced from 9.2% down to an average of 8.4%.
Furthermore, out of the total number of participating patients recording drops in
their HbA1c levels, approximately 37% were able to lower it by 10% or more,
exceeding our expectations. HbA1c levels recorded three months after patients resumed
using traditional healthcare services indicate that progress made earlier through TeMaD
was reversed with an increased in the HbA1c levels of 42%of revisited patients.
6. Discussion
TeMaD has improved HbA1c levels of 83% of participating patients reducing the
average from 9.2% to 8.4%. Three months post-trial, tested HbA1c levels showed a rise
in 42% of revisited patients, indicating that TeMaD intervention contributed to
improved patient health. TeMaD had a greater impact on female patients between 16-
29 years, improved monitoring techniques, and patient–caregiver communication
channels. TeMaD may not be suitable for older patients, due to technology resistance.
Also, TeMaD is not recommended for newly diagnosed patients, since it minimises
personal contact with DE, sometimes essential to assess patient psychological aspects.
TeMaD has been useful to DEs in management of patient disease, time, and resources.
7. Conclusion
TeMaD is a telecare solution designed to help meet the growing needs of the Saudi
healthcare system. Although it is developed specifically for diabetes, it can be used to
manage other chronic diseases. TeMaD is tailored to address factors of the local culture
that may affect how healthcare services are offered. More research is required to
examine how technology can assist in overcoming these factors while maintaining local
culture. TeMaD is a highly available system which improved communication between
patients and healthcare givers and led to higher compliance with the diabetic treatment
plan. TeMaD had a positive impact on diabetes management indicating the need to
incorporate such solutions into existing healthcare practices.
8. References
[1] Al-Watan Newspaper, 28% are Diabetic in Saudi, November 16, 2007, 2604: 5.
[2] Gulf Cooperation Council Health Care: Challenges and Opportunities, McKinsey analysis based on
Global Insight data, 2006.
[3] Alkadi, K., Roudsari, A., Applying a methodological approach for a telecare solution in Saudi Arabia,
Stud Health Technol Inform, 2011; 164:243-7.
Enabling Health and Healthcare through ICT 63
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-63
Abstract. In 2011, there were more than 97,000 people living with Heart Failure
in British Columbia (BC) with a total of 17,592 within VIHA. To increase patient
accessibility to specialist care, the Vancouver Island Health Authority (VIHA)
implemented a telecardiology program that utilizes digital stethoscopes, telehealth
technology and collaboration to deliver cardiac care remotely. The program has
successfully completed 20 consultations to date in 6 communities within the
VIHA. This article outlines processes and outcomes of enabling the existing
VIHA cardiology program with the use of telehealth technologies.
Introduction
1
Corresponding Author
64 T. Horwood et al. / Telecardiology on Vancouver Island: Imagination to Implementation
1. Background
The Heart Function Clinic (HFC) provides HF patients with medical and nursing
assessments, education and support, exercise guidance, diagnostic interpretation (i.e.
electrocardiograms and blood tests), medication reconciliation and adjustment.
Currently, there are over 700 VIHA patients registered in the HFC. Many Vancouver
Island residents live in communities that do not have a HFC, and are asked to travel to
other communities to receive care (Table 1) [3]. On average, 7 patients per week
travel to Victoria, Nanaimo or Campbell River to visit a HFC. These patients absorb
the time, expense, risk and physical burden of travelling to the nearest centre.
Table 1. Communities without Heart Function Clinics: populations and distances (round trip) to HFCs
Community Total Distance to Distance to Distance to Campbell
Population Victoria HFC Nanaimo HFC River HFC
(Kms) (Kms) (Kms)
Duncan 57,761 130 54 204
Parksville 46,375 300 36 119
Port Alberni 31,442* 390 81 148
Tofino 632 203 270
Courtenay 65,340 440 106 45
Gold River 2,356* 710 239 92
Port Hardy 12,239* 1004 384 237
Port McNeill 926 347 200
Port Alice 1024 396 249
*Population Combined
2. Program Development
Recognizing the need for increased accessibility of service to patients with complex
cardiac conditions, VIHA in conjunction with representatives from BC health
authorities developed a provincial committee to foresee the implementation of a
standard digital stethoscope that could be used throughout the province. Digital
stethoscopes provide “…reliable and valid screening for congenital heart disease” [4],
T. Horwood et al. / Telecardiology on Vancouver Island: Imagination to Implementation 65
and can be use safely over internet-based software [5]. The intent of the committee was
to ensure compatibility, reliability and knowledge transfer through lessons learned.
The electronic Littmann 3M 32000 stethoscope was selected as the provincial standard
for telehealth programs within BC. The electronic stethoscope connects to a computer
using Bluetooth, and links the provider stethoscope with that of the patient at the
remote site. Once connected, providers can utilize the electronic stethoscope to assess
the heart and lung sounds of remote patients.
2.3. Requirements
The telehealth team conducted a thorough analysis of the current processes at the HFC.
The following items were identified as essential when conducting a session: capability
to capture a patient’s vital signs, ability for specialist to hear patient’s heart beat, and
ability to see and hear the patient in real time. The following workflow processes were
also examined: patient inclusion criteria, scheduling, visit types, pre-visit patient
preparation, diagnostic testing, patient assessment workflow, report generation, and
documentation.
3. Program Development
Human resource engagement was an essential component for the success of the
implementation of telecardiology. Engagement can “…be a primary antecedent to
successfully implementing an organizational change initiative” [6]. All stakeholders,
including executive support, physicians, assistants, and persons facilitating telehealth at
66 T. Horwood et al. / Telecardiology on Vancouver Island: Imagination to Implementation
The telehealth team developed a thorough multi-tier support plan for the various stages
of the project. The plan allowed for stakeholders to communicate issues or questions
through phone or email. If the issue could not be resolved immediately by the receiver,
the issue would be classified and directed to someone with specialized knowledge in
that area.
4. Findings
5. Next Steps
The success of the pilot and initial sessions has prompted the telecardiology service to
continue and expand. Other Health Authorities are expected to use the knowledge
gained from this project to implement similar use of the digital stethoscope. At VIHA,
the telecardiology model will expand to the Atrial Fibrillation Clinic. The clinic serves
up to 2,200 BC patients annually. Since there are only a few electrophysiology
cardiologists in the province the service is expected to save thousands of kilometers of
patient and provider travel.
VIHA is exploring the use of the digital stethoscope as well as other digital
medical peripherals (i.e. dermatoscope, otoscope and spirometer) to facilitate telehealth
for other disciplines such as respirology, speech language pathology, and dermatology.
T. Horwood et al. / Telecardiology on Vancouver Island: Imagination to Implementation 67
References
[1] J. Picot. Sector Competitiveness Frameworks Series: Telehealth Industry Part 1 –Overview and
Prospects. Industry Canada: Industry Sector Health Industries, Canada Health Infoway definition, 2004.
[2] Medical Service Economic Analysis, Medical Services and Health Human Resources Division, Ministry
of Health, 2011
[3] Local Health Area Profiles. Vancouver Island Health Authority. [cited 2012], Available from:
http://www.viha.ca/mho/stats/lha_profiles.
[4] J. Belmont, L. Mattioli, R. Ardinger, A.C. Thomas, Evaluation of Remote Stethoscopy for Pediatric
Telecardiology. Telemedicine Journal 1(2) (1995), 133-49.
[5] G. Fragasso, M.D. Benedictis, A. Palloshi, M. Moltrasio, A. Cappelletti, M. Carlino, et al, Validation of
heart and lung teleauscultation on an Internet-based system. Am J Cardiol 92(9) (2003):1138-9.
[6] Dicke, Colin, Employee Engagement & Change Management, Cornell University, 2007.
68 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-68
\ }
}*
*
}**
*Z
*
|
`
< ¤!} `
<<
`
<
|
**
*
`
Z
< <}"
}
"
¦ }
}
`*
*`
*
`
!!
*
*!
< <
|
¥<
*
||\
< <
}
}*
*}
|
*
!
!
!
<*
|
|\*
|*`
`
*
< *}
*
#<
D. Tuden et al. / Developing a Strategy for Studying Critical Thinking in a Nurse Telehealth Setting 69
`
`
!<
\
*
*
<
*!
*
<
|
¯ ¥¯
°
\ ±°
±}`|
<}
\
<**
**
!
`*
*
**
|
*
\ }
`*
|
|
\
!
**
`
<
**
|*`
!\
|\<
`*
`
*
*
}
<
| *`
*
|
***
<
} *
*
*
*
\|
`
<
*
`
*}}
!`
<
`*
!
*`
***
!
<
\
*
***
*
<
*
*
}
}
}
\
<
*
}
Z
*
|
**
<
*
`
}
*
`
<
\
*
*
}
*`
**
\
<
*
*
**|
\
|
|
\ !
*
Z
*`
Z
!
¢\<
*
70 D. Tuden et al. / Developing a Strategy for Studying Critical Thinking in a Nurse Telehealth Setting
*
``
*
<
`
!
!
` *
*
\
\ *
\
*
<
*
|\ * |
*
`
}
`
"`
¨
< *
*
<
*
! !
< *} }
}
*
}
*`
< <
*
}*
}
^
!
|
<
**
\| !
¦
}
!
|\
\
<
}
}
*} `
*
\
<
!
*
`*
\
*
}|
!
**
`*
\
±
¥
^
\ #`*
D. Tuden et al. / Developing a Strategy for Studying Critical Thinking in a Nurse Telehealth Setting 71
Z
\
*
!
\ <
} |\ *
`*
\
*
*
<
|
| !
|*
!
!
**
\ <
]^(
<'+<_--
!-!!
``
!
\*
|\
*<
|
}
`| |
\
|
\< !
*
`|
\
*
*
| * |\ *
<
}|\ *!
\
!
*
**
\ <
|\
`
°
\ ±|
|\|
|
<
°{
±°
{
±|
! |\ * !
< \ * !
!
!
*
|\< |
}
!
*`*
|
\ }|
|
|
|
|
|
<
|*|\ *
}
|`|!
* §
| *
^| *
< *
`|
} \ |
|
`
\ <
`*
*
`
<
\ <
`|
} |\ *
}
|
} `|}
`|
§
|
`*<
**
} |
\
!
!
!
!
!
`!
+"
<
j
<
"
<
< <
+< !!!
+"
<
!
<!
!!!
{<
?
<< < {<
+<
< !
" "
!
<
<{<
<
*
|\ *
``
\
*
<
*{
*
\
!
\
}
`
`
72 D. Tuden et al. / Developing a Strategy for Studying Critical Thinking in a Nurse Telehealth Setting
`
`!
¢\<
`
||\
!**
<
*
!
°
± | *`
<
`
}!
**
}!
*
| *`
*
**`
<
|\|
°
±}
*
*
} |
*
}
***
|``
!
| `
}
}
}
<
\
!
*
<*}
*
<
**
\
\
*
!
*
|
¥
*
*
°*
**
¦| ±<
`
|
!
|\ *
*
\
< <
`
*`
**
}
*
}
`
} !\
`
|
<
\
* !
{!
< *
}
|
*
`
\ }
**
\
\
<
¤
\
¤
\ \
|
*
<
*
±<
\
\
!
`
< |`}
°
\
²
!
\ |
* !
\| ±<
|\ *
`*
*
*
`*
\
\
}
!
*`
!! !
*
`
<
!
*|
*
<
} |
*
|\
\
*
***
`
<< *
< !
| *`
\ !
|\
*
*<
}
* *
`
|\ !\
³
|
D. Tuden et al. / Developing a Strategy for Studying Critical Thinking in a Nurse Telehealth Setting 73
*
`
`
*
{<
|\
*
*
<< !
}
`|}
!
`
`*
\ }
|
*
! `
|\
|
!{
`
<
|
\|
\ |
<
\ ]<
`
| ! ! **
\ {
<
`
|
|
* **
*
< *
*
|!
|
!
< |\ *
|
!
`* !
*
\|
|
*` !
<
| !
`| !
!
!
!
<\ |
*
|*
!
**
\ <
*
|\|*
\ ³ *}
} !
*
} !
*
|
|
`
<
*
|\
!
}|
*`
<
!` ¤
!
`
*
|
|**
\ <
**
`*
\ }
*
| !
°*
±
*
*
|
`
\| *
<
"
*
**
¦|
} <
* < `!
*
|||<!<
¥|^
^
¦| !
<
*
!<
\ |
*
Introduction
As we adopt e-health technologies into our health and health care environments, we
need a flexible, robust understanding of what works and where. Without this, the many
studies and evaluations of e-health technologies, in a variety of contexts, leads to
challenges in accumulating evidence for clear-decision-making. When a new health
challenge emerges, or e-health solutions become available, it may not be evident how
previous research applies, suggesting more research is needed.
There is no simple answer as e-health technologies are complex interventions
acting on complex social systems and therefore “are not 'magic bullets' that will always
hit their target” [1]. Rather their success is dependent on context and implementation.
This paper explores a “Realist Review” approach proposed by Pawson, et al. [2] to
understand the mechanisms for why e-health technologies work and in what contexts,
in order to better adapt future technologies in response to emerging health challenges.
1. Background
1
Corresponding Author: Marilynne A. Hebert, Department of Community Health Sciences, University
of Calgary, 3D41 - 3rd Floor TRW, 3280 University Drive NW, Calgary, Alberta T2N 4Z6
M.A. Hebert / A “Realist Review” Approach to e-Health: The Case of Type 2 Diabetes in Youth 75
adolescents were living with diabetes and over 400 new cases were identified in the
under-20-year-old population [3].
As well, obesity rates tripled among 12-17 year olds between 1979 and 2004 [4-5].
Compared to children of normal weight, overweight and obese children experience
higher blood pressure, more abnormalities in serum lipid levels and higher levels of
insulin resistance. These conditions are all hallmarks of early metabolic disease,
diabetes and susceptibility to cardiovascular diseases [5].
Coaching calls have been used primarily with families and children with Type 1
diabetes where adherence issues were a problem. Type 2 diabetes has been called a
“life style” disease and coaching calls have also been effective in helping seniors stay
active as well as regular telephone calls to assist adults with self-management and
education.
A variety of interventions have been used to automatically collect and send information
to people with diabetes via Short Message Service (or SMS), telephone and websites.
Daily text messages have been used to help youth with Type 1 diabetes to enhance self-
efficacy, stabilize insulin injections and improve overall self-management. Messages
have also been customized using patient biochemical profiles and clinical status on file.
Patients have frequently used phone calls and video-conferencing to connect with
diabetes educators. Another version of this interaction has been to customize education
as a result of submitting personal data. Examples include patients calling an automated
system weekly to receive tailored feedback and education on self care as well as
uploading step count data from pedometers to receive feedback and motivational
messages.
76 M.A. Hebert / A “Realist Review” Approach to e-Health: The Case of Type 2 Diabetes in Youth
There is a growing interest in technologies that may work particularly well with a
younger population. For example, using a Smart phone as both a glucose monitor and
insulin pump [6], cell phone voice and text message interventions [7], or a Personal
Health Application may address many of the factors in optimal Type 2 diabetes
management [8]. A promising mobile phone-based system includes an integrated suite
of applications including blood glucose meter, a tailor-made step counter, software for
recording food habits and providing feedback on how users performed in relation to
their own personal goals [9].
Traditional systematic review methods determine how many times an intervention has
produced the expected change. They typically do not provide information on why the
intervention worked or did not work when it was applied in different contexts,
deployed by different stakeholders, or used for different purposes. In contrast, Realist
Reviews ask how interventions bring about their effects and the limits of when and
where they work [1-2].
A Realist Review focuses on explaining the relationship between the mechanism
(M) by which an intervention works, the context (C) in which the intervention is
applied, and the outcomes (O) which are produced (Figure 1). A review cycle has four
stages [2, p.84-85]:
a. develop a theory around what would explain regularities in implementing the
intervention (including M,C,O);
b. hypothesize what might work for who and in what circumstances;
c. observe what works and for who using multi-methods to collect data and
analyses using M,C,O, which allows patterns to emerge;
d. specify program generalizations that identify what works for whom and in
what circumstances.
The goal is to look for evidence of “empirical uniformities,” i.e. regularities that occur
in the field, and in doing so establish limits to the intervention as well as inform theory.
2.2. Realist Reviews Applied to e-Health Used for Type 2 Diabetes in Youth
3. Conclusions
References
[1] Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review
designed for complex policy interventions. J.Health Serv.Res.Policy 2005 Jul;10(Suppl 1):21-34
[2] Pawson R & Tilley N. Realistic Evaluation, 1997; Sage.
[3] Alberta Diabetes Atlas 2011. Alberta Diabetes Surveillance System accessed July 5, 2012 at
http://www.albertadiabetes.ca/
[4] Shaw J. Epidemiology of childhood type 2 diabetes and obesity. Pediatric Diabetes, 2007: 8; 7.
[5] Alberta Health Report. Accessed July 5, 2012 at: http://www.health.alberta.ca/health-info/diabetes.html
[6] Brookes L. Why can’t a diabetes device be as cool as an iPod? Posted 04/07/2010. Accessed on-line
July 16, 2012 at: http://www.diabetesweb.org.uk/2010_Ipod%20designs%20for%20DM.pdf
[7] Krishna S, Austin Boren S, and Balas EA. Healthcare via cell phones: A systematic review.
Telemedicine and e-Health, April 2009: 15(3); 231-240.
[8] Fonda SJ, Kedziora RJ, Vigersky RA and Bursell SE Combining iGoogle and personal health records to
create a prototype personal health application for diabetes self-management. Telemedicine and e-Health,
2010: 16(4); 1-10.
[9] Årsand E, Tatara N, Østengen G, and Hartvigsen G. Mobile phone-based self-management tools for
Type 2 diabetes: The Few Touch Application. J Diabetes Sci Technol 2010;4(2):328-336
[10] Huus K and Enskär K. Adolescents' experience of living with diabetes. Paediatric Nursing, 2007;
19(3): 29-31.
[11] Arora S, Peters AL, Agy C, and Menchine M. A mobile health intervention for inner city patients with
poorly controlled diabetes: Proof-of-concept of the TExT-MED Program. Diabetes Technology &
Therapeutics, 2012; 14(6): 492-496.
[12] Norman, C. eHealth literacy 2.0: Problems and opportunities with an evolving concept. J Med Internet
Research. 2011 Oct-Dec; 13(4): e125. Published online 2011 December 23. doi: 10.2196/jmir.2035.
Accessed on-line July 16, 2012 at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3278111/
[13] Jefferson V, Jaser SS, Lindemann E, Galasso P, Beale A, Holl MG, and Grey M. Coping skills training
in a telephone health coaching program for youth at risk for Type 2 diabetes. Journal of Pediatric
Health Care, 2011; 25(3): 153-161.
[14] Davis RM, Hitch AD, Salaam MM, Herman WH, Zimmer-Galler IE and Mayer-Davis EJ. TeleHealth
improves diabetes self-management in an underserved community. Diabetes Care, 2010; 33(8); 1712–
1717.
[15] Katz R, Mesfin T and Barr K. Lessons from a community-based mHealth diabetes self-management
program: “It's not just about the cell phone.” Journal of Health Communication: International
Perspectives, 2012; 17(sup1): 67-72.
Enabling Health and Healthcare through ICT 79
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-79
Abstract. This paper discusses the design and development of a multi-user health
kiosk intended for independent use by underserved populations. The modular
integration of physiological sensors and psycho-social assessments provides an
extensible, customizable platform for research. We present the development of the
kiosk’s feature set and user interaction mechanisms through iterative user testing,
in addition to some technical challenges and solutions resulting from our design
choices.
Introduction
The Community Multi-user Health Kiosk (Figure 1) was designed to give people who
are currently underserved by the healthcare system a way to monitor their health and
communicate with their healthcare provider. The Quality of Life Technology
Engineering Research Center focuses on the elderly, those with disabilities, and people
of lower socio-economic status and supports the design and development of the health
kiosk.
Rather than a fixed function device, the kiosk is a platform that can support a
variety of psycho-social assessments and peripheral monitoring tools. It consists of a
mobile desk that has a computer running the Windows-based kiosk software program, a
printer, a movable touchscreen, and a drawer that contains any peripheral medical
devices needed for the kiosk's assessments. In the first use case, the kiosk was intended
for the common areas of senior high rise buildings. Potential sites have expanded to
include primary care offices, pharmacies, public libraries, and other community
gathering places.
In contrast to many commercially available telehealth monitors, the kiosk is
designed to be multi-user, individualized, customizable, and extensible. Each user's
self-monitoring health information is conveyed to their healthcare provider via multiple
possible mechanisms. Both the assessments and the results display can be customized
by clinicians to meet the specific needs of their patients, and an internal messaging
feature offers secure communication between the two. In addition, the modular design
makes it easy to switch hardware manufacturers/models and add or remove
1
Corresponding Author. Julie McMillan, email: [email protected]
80 J.M. McMillan et al. / Designing the Community Multi-User Health Kiosk
1. Features
"Assessment" is a term used broadly for any value measurable with the kiosk and can
include both physical as well as psycho-social measurements. In its current baseline
configuration, the kiosk has peripheral device-based, self-report and manual entry
assessments. The peripheral off-the-shelf devices include a pulse oximeter, blood
pressure monitor, seated scale, and hand dynamometer. The devices communicate with
the computer via USB or Bluetooth.
Self-report assessments are ones where a user answers questions. An example use
case would be a clinician scheduled annual depression screening. The default surveys
on the kiosk are the Geriatric Depression Scale [1] the Epworth Sleepiness Scale [2],
the Pittsburgh Sleep Quality Index [3], the Medical Outcomes Study 36 Item Short
Form Survey [4], the Morisky Medication-Taking Adherence Scale [5], and the Cost-
Related Nonadherence to Medications Survey [6]. Our community-based usability
testing has validated the reliability in translating these instruments to the kiosk.
To accommodate these various response formats, the kiosk supports a number of
response options: multiple choice, sliding scales, date and/or time entry, and free text
responses. The instrument engine also supports branches in the survey - that is, making
questions dependent on previous answers. The kiosk can automatically score
instruments with standard scoring metrics like summing. Automatic scoring of other
instruments with non-generic scoring algorithms is possible with minor software
modifications. Adding a new self-report assessment consists of formatting the content
according to a straight-forward XML specification. No programming knowledge is
necessary. In all cases, every user that sees, changes, or deletes an assessment result is
logged by an audit trail to ensure accountability and security.
J.M. McMillan et al. / Designing the Community Multi-User Health Kiosk 81
1.2. Customization
To increase user engagement, customization has been an ongoing theme in the kiosk
development. In addition to minor personalization features, such as welcoming each
user by name, clinicians can select assessments for individual patients and set patient-
level guideline parameters for assessment results. For example, each assessment can be
made available or unavailable globally (all kiosks), at selected kiosks, or for selected
individual users. The kiosk prompts the user to complete the assessment at the
clinician's specified interval. The kiosk can notify health care providers for alert values
via multiple contact strategies including text message, email, or fax.
A user's view of their own results is one of the clearest examples of software
tailored to the patient. Results are visible to the user in three ways –the current
assessment results, a summary of all a session's results, or as a historical view
encompassing the last 3, 6, 9, or 12 months. Both numerical and categorical results are
color-coded for easy comprehension; desirable values are green and less desirable are
yellow, orange, or red. In the default settings, the color-coding stems from population
guidelines, but a clinician can customize this to emphasize a "personal best" for a
patient (Figures 2 and 3).
Figure 2. Historical view of categorical result. Figure 3. Immediate view of numerical result.
1.3. Extensibility
From the beginning of the project, we defined our user base as broadly as possible to
include older adults, clinicians, and health care providers. Our emphasis on user-
centered design governed a number of initial decisions about the structure and content
of the health kiosk. For instance, concerns about participant falls led to use of a seated
scale rather than a traditional step-up model. This choice prompted the design of a
unique push-through drawer mechanism so wheelchair users could access the kiosk
82 J.M. McMillan et al. / Designing the Community Multi-User Health Kiosk
devices without moving the seated scale. Clinician concerns about blood-borne
pathogens resulted in a design choice to have participants type in blood glucose values
rather than perform testing on a kiosk-based glucometer. The target population for the
kiosk also led to some user interaction challenges. Many of our older users were not
familiar with computers or touchscreens (Figure 4). In early prototypes that labeled
buttons with the verb "Press", we observed people pressing but not releasing. Most
buttons are now labeled "Tap." Many common software metaphors had to be
eliminated or made more explicit.
3. Technical Challenges
The benefits of using off-the-shelf devices include easy replacement of hardware; low
cost; and the potential usability of consumer-oriented products. The disadvantages are a
lack of standardization and, potentially, poor reliability. To mitigate these issues, our
general strategy was intensive upfront development of a modular and fault-tolerant
architecture, component testing, and user-driven hardware and software modifications
to minimize error. Table 1 lists some of the complications we encountered in the
development process, their effects, and solutions.
For example, hardware defects were an unanticipated problem, and especially
difficult when the defect was subtle. Our first pulse oximeter occasionally displayed a
dramatically different result on the device screen than appeared on the kiosk, with
disastrous results for user confidence. After ruling out other potential sources of error,
J.M. McMillan et al. / Designing the Community Multi-User Health Kiosk 83
we concluded that the hardware was at fault and changed the brand of oximeter, which
was quick and painless due to the modular nature of the kiosk software.
4. Discussion
Lab and community-based user testing with over 40 individuals has led to a system that,
while continuing to evolve, is very usable for people in our target demographic. We
have also demonstrated that the initial investment in a modular, robust architecture
pays ample dividends when requirements change or the inevitable hardware problems
occur. Building upon this work, we are conducting pilot work for kiosk-based health
interventions and working with health organizations on kiosk data integration.
References
[1] Yesavage JA, Brink TL, Rose TL, Lum O, Huang V, Adey, M, Leirer VO. Development and validation
of a geriatric depression screening scale: A preliminary report. J Psychiatr Res. 1982; 17: 37-49.
[2] Johns MW. A new method for measuring daytime sleepiness: The Epworth sleepiness scale. Sleep.
1991; 14: 540-545.
[3] Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh Sleep Quality Index
(PSQI): A new instrument for psychiatric research and practice. Psychiatry Res. 1989; 28: 193-213.
[4] Hays RD. The Medical Outcomes Study (MOS) Measures of Patient Adherence. RAND Corporation.
1994.
[5] Morisky DE, Ang A, Krousel-Wood M, Ward HJ. Predictive validity of a medication adherence
measure in an outpatient setting. J Clin Hypertens (Greenwich). 2008; 10: 348-354.
[6] Soumerai SB, Pierre-Jacques M, Zhang F, Ross-Degnan D, Adams AD, Gurwitz J, Adler G, Safran
DG. Cost-related medication nonadherence among elderly and disabled medicare beneficiaries: A
national survey 1 year before the medicare drug benefit. Arch Intern Med. 2006; 166: 1829-1835.
[7] John, B., Prevas, K., Salvucci, D., & Koedinger, K. (2004) Predictive Human Performance Modeling
Made Easy. Proceedings of the SIGCHI conference on Human factors in computing systems (2004),
455-462.
This page intentionally left blank
Public Health Informatics
This page intentionally left blank
Enabling Health and Healthcare through ICT 87
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-87
Introduction
Health-care associated infections (HAIs) are a huge public health problem. Of roughly
39 million annual hospital admissions in the U.S., about 1.7 million (4.5%) result in
HAIs and 100,000 HAIs result in death [1]. The direct healthcare costs of HAIs are
estimated at $45 billion, and total annual costs likely exceed $100 billion [2, 3]. In
response, twenty-five states and the federal government currently require hospitals to
publicly report infection rates for at least some types of infections; other states have
1
Corresponding Author
88 A. Amini et al. / Public Reporting of Hospital Infection Rates
adopted laws that will require reporting in the near future; and private entities have
created their own public reports. The effectiveness of these reporting requirements
in informing consumer choice and reducing infections is unknown [4]. Moreover, and
the focus of this study, “we do not just want to know if public reporting works
(efficacy); we want to know who it works for and in what situations (effectiveness)”
[5].
HAI public reporting presents an ideal environment in which to examine the
effectiveness of public delivery of health care information. Each state maintains a
website; most also provide downloadable reports, but each uses its own unique
reporting format. There is wide variation in websites, and reports, across any metric
one might consider: how easy the websites and reports are to find, which infections
are reported; how infections or infection rates are measured; what information
about them is reported; time periods covered; level of technical difficulty,
readability levels, and so on.
We report here on work in progress in which we evaluate how effective each
state’s website and public reports are at communicating information. We focus on
website credibility, user-friendliness, and usefulness of information for three target
audiences with different needs and ability to understand technical information,
including typical healthcare consumers; physicians (and other more sophisticated
users); and infection control professionals.
1. Methodology
information
• Straightforward to find results for a particular hospital
• Website and report length and complexity is appropriate for consumer
audience
As with the Fogg guidelines, we developed a detailed description of the attributes
needed to receive each score. Each factor was scored on a 1-5 scale, for an overall 5~25
scale. We applied these factors separately to the website and to the formal reports on
state’s websites. In this paper, we focus on the formal reports.
Finally, for “usefulness,” we developed and then assessed a number of factors,
including:
• Was the state website easy to find?
• Within the website, was pertinent information easy to find?
• Are the explanations clear?
• Was it easy to find information on a particular hospital?
• Types of infections covered
• Type of rate reported (“raw” [crude], risk-adjusted, risk-stratified, or a
combination)
• Were number of infections reported?
• Is statistical significance reported?
• Is historical data available?
• If yes, is the format consistent over time?
• Can users compare hospitals online, how many, and using what selection
criteria?
• Are similar hospitals compared to each other?
• Overall, was the website understandable and useful?
We assess usefulness for three target audiences. One important audience is average
consumers. We assumed that these users have roughly a high school reading level and
limited understanding of statistics and medical terminology. A second audience is
physicians, who can counsel patients, make decisions on which hospital to admit their
patients to, and push hospitals for change. This audience is likely to have better than a
college graduate reading skills, moderate understanding of statistics and healthcare
terminology, but limited knowledge about particular HAIs. A third audience is
infection control professionals, who can use the reports to benchmark their hospital
against peers. These persons are likely to have a college graduate reading level; be
familiar with measurement and coding issues for HAIs, but have limited understanding
of statistical analyses. At many hospitals, infection control is assigned to registered
nurses. Only at major hospitals, often with academic affiliations, is the head of
infection control likely to be an epidemiologist with advanced statistical training.
Thus, we judged that a typical infection control professional would know more about
HAIs but less about statistics than a typical physician or other sophisticated user.
Manifestly, we must exercise judgment in deciding what criteria to measure, how to
measure them, and what target audiences to assume; we do so based on the expertise
developed in our previous research on these subjects. [10-11]
Thus, each state is scored on five metrics: website credibility, user friendliness of
website, user friendliness of public reports; usefulness of website for each target
audience, and usefulness of public reports for each target audience. Initial scoring was
by a law student at Northwestern. We are in the process of assessing inter-rater
90 A. Amini et al. / Public Reporting of Hospital Infection Rates
reliability. A spreadsheet including all criteria and results will be publicly posted on
Professor Black’s website at Northwestern University.
2. Results
We present here partial results from our analysis, focusing on Fogg website credibility
scores and user friendliness scores for the public reports from the perspective of an
ordinary consumer. Scores are based on the reports and websites as they appeared in
August, 2012. We also plan to assess changes in websites and reports over time. Higher
scores reflect better performance on the specified measures. Fogg scores have a
theoretical range from 5 to 40; in practice, they ranged from 24 (Vermont) to 39
(California). Public report usability scores have a theoretical range from 5 to 25; in
practice, they ranged from 9 (Rhode Island and Washington) to 25 (Massachusetts).
The close-to-maximum scores on each measure for some states reflect not perfection,
but our effort to define criteria that made a top score reasonably achievable.
Figure 1 indicates the scores on both measures for each state that currently has
public reporting of HAIs. Figure 1 also includes the Canadian province of Ontario. We
plan to expand our analysis to include the Hospital Compare website run by the
Department of Health and Human Services, and the websites operated by the Leapfrog
Group and Consumers Union.
It also provides a correlation line between the two measures. The Pearson
correlation coefficient is strongly positive at 0.58 (t = 3.46). There is thus a strong
tendency for states to do either well or poorly on both measures.
One might expect larger states, which usually have greater resources, to do better,
but this was not universally true. For example, New Hampshire did well on both
measures, while Florida did poorly on both measures.
Finally, rankings may change dramatically if states redesign their website. For
example, Washington’s current Fogg Score is 27 but an earlier version of Washington’s
website had a Fogg score of 36. The mean (median) Fogg score was 31.2 (30.5) for our
entire sample of 25 states and Ontario, so the redesign dropped Washington’s score
from substantially above-average to below-average.
3. Discussion
Public reporting of HAIs is widespread, but little attention has been paid to how to
design websites and public reports to reach the multiple target audiences. We report on
work in progress in which we systematically quantify the effectiveness of existing
websites and public reports, across a variety of general metrics (credibility, user
friendliness, and usefulness) for three different target audiences. For ordinary
consumers, we find substantial correlation between website credibility and user-
friendliness of each state’s public reports.
One issue that is missing from current discussions of credibility and user
friendliness, but substantially complicates any analysis of these issues, is the variety of
potential target audiences. HAI data can be quite technical, and assessing its value
requires an understanding of statistics and confidence intervals. To be readable and
understandable for a typical consumer, some website and reports have been deliberately
“dumbed down” to a level that is likely to prove inappropriate and frustrating for more
A. Amini et al. / Public Reporting of Hospital Infection Rates 91
sophisticated users.
Another issue that is missing from current discussions is “usefulness.” Assessing
the usefulness of HAI information requires in-depth knowledge of the goals of
reporting and the information being reported. For example, a time-series of data is
usually more reliable than a snapshot of a short time period. Reports that tell users
only whether a hospital is above, average, average, or below average (with 95% or
more graded as average) are easy to understand, but not very useful. Some states
provide more granular information, but most do not. A comparison of the CLABSI
(central line-associated bloodstream infection) rates at a small community hospital to
the rate at major tertiary hospitals may mislead, rather than inform. And so on.
Finally, many websites and reports seem to focus on a single target audience, even
though more tailored information would be useful. Serving multiple audiences through
a single website and report format is challenging. In the future, we will analyze the
extent to which states have recognized and addressed this problem.
Figure 1. Fogg Website Credibility Score vs. User- Friendliness Score for Public Reports
Fogg website credibility scores and HAI report user friendliness (for ordinary consumers) scores for 25 states
plus Ontario with public HAI reporting (as of August 2012). See text for scoring method.
References
[1] R.M. Klevens, J.R. Edwards, C.L. Richards, T.C. Horan, R.P. Gaynes, D.A. Pollock, and D.M. Cardo,
Estimating Health Care-Associated Infections and Deaths in U.S. Hospitals, 2002, Public Health
Reports 122 (2007),160-166
[2] R.D.II. Scott, The Direct Medical Costs of Healthcare-Associated Infections in U.S. Hospitals and the
Benefits of Prevention, [updated 2009], Available from:
www.cdc.gov/ncidod/dhqp/pdf/Scott_CostPaper.pdf.
[3] M.R. Eber, R. Laxminarayan, E.N. Perencevich, A. Malani, Clinical and Economic Outcomes
Attributable to Health Care-Associated Sepsis and Pneumonia, Archives of Internal Medicine 170
(2010), 347-353.
[4] B. Black, Public Reporting of HAI Rates: What We (Mostly Don’t) Know, Clinical Governance 17(2)
(2012), 124-133.
92 A. Amini et al. / Public Reporting of Hospital Infection Rates
[5] Agency for Healthcare Research and Quality (AHRQ), Public Reporting as a Quality
Improvement Strategy, Evidence Report/Technology Assessment No. 208. [updated
2012], Available from:
http://www.effectivehealthcare.ahrq.gov/ehc/products/343/1198/Evidencereport208_CQG-
PublicReporting_ExecutiveSummary_20120724.pdf.
[6] B.J. Fogg, C. Soohoo, D. Danielson, L. Marable, J. Stanford and E.R. Tauber, How do People Evaluate a
Website’s Credibility? Results from a Large Study [updated 2002], Available from:
http://www.consumerwebwatch.org/pdfs/stanfordPTL.pdf.
[7] K.M. Mazor, K.S. Dodd, A Qualitative Study of Consumers’ Views on Public Reporting of Health
Care- Associated Infections, American Journal of Medical Quality 24 (2009), 412-418.
[8] K.M. Mazor, K.S. Dodd, L. Kunches, Communicating Hospital Infection Data to the Public: A Study of
Consumer Responses and Preferences, American Journal of Medical Quality 24 (2009), 108-115.
[9] N. Bardach, J.H. Hibbard, R.A. Dudley, Users of Public Reports of Hospital Quality: Who, What, Why,
and How? AHRQ Publication No 12-0016-EF, [updated 2011], available from:
http://www.ahrq.gov/qual/value/pubreportusers/.
[10] D. Birnbaum, M.J. Cummings, K. Guyton, J. Schlotter, A. Kushniruk, Designing Public Web
Information Systems with Quality in Mind, Clinical Governance 15 (2010), 272-278.
[11] D. Birnbaum, R. Zarate, A. Marfin, SIR, You’ve Led Me Astray! Infection Control and Hospital
Epidemiology 32 (2011), 276-282.
Enabling Health and Healthcare through ICT 93
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-93
Introduction
1
Corresponding Author: Dr. med. Christian Juhra, MBA, University Hospital Münster, Department of
Trauma Surgery, Waldeyerstr. 1, D-48149 Münster, Germany; Email: [email protected]
94 C. Juhra et al. / Methodological Approaches to Comparing Information
British Columbia (along with the Yukon Territories) (22.1 per 100,000 population, age-
adjusted) [3]. A high rate of underreporting was revealed by a study in Münster,
Germany. The accident rates were up to 3 times higher than the officially reported rates
[4]. Head injuries were the main cause of hospitalization in Germany and Canada, and
one of the leading causes of death among cyclists. To get a better overview of the real
situation of bicycle accidents and related injury patterns, a prospective study funded by
the German Federal Highway Research Institute was initiated in May 2012. In this
paper, we will describe the methodology by which accident data from the two countries
with different healthcare systems and transport regulations can be collected.
Vancouver Island covers an area of 31,285 km2 (approximately 10% of the area of the
Federal Republic of Germany), and has a population of 748,937. Almost half of those
(367,572) reside in Victoria, the capital of British Columbia. Victoria has been named
repeatedly by the Canada’s Cycling Magazine “Pedal” as the “Bicycle Capital of
Canada” as a result of its bicycle-friendly infrastructure and the many biking (hiking)
trails available. Due to its geographical location, outdoor activities are very popular on
the island. This is also reflected by a high number of bicycle accidents.
In Germany, the accident data are collected within the Trauma Network North-
West (TNNW). The TNNW is a consortium of 37 trauma care hospitals in the regions
of North Rhine-Westphalia/Southern Lower Saxony. The TNNW represents an area of
approximately 14,000km2 and a population of about 4 million comprised from the
states of North Rhine-Westphalia, Lower Saxony, as well as part of the Netherlands,
and is one of the largest trauma networks in Germany. In contrast to the more urban-
centered trauma networks, such as that of Berlin, the TNNW also includes a large
number of rural regions.
Assuming that the area of the TNNW is representative of the Federal Republic of
Germany, approximately 3,000 bicycle accidents happen in this area per year. To
include the issue of under-reporting (i.e. approximately only 2/3 of actual cases are
reported), approximately 9,000 bicycle accidents involving personal injury occur in the
area of the TNNW per year. Based on the results of the Münster bicycle study, 11.1%
of all injured persons were hospitalized, which means that the expected number of
hospitalized patients was around 1,000.
The Münster bicycle accident study led by Münster University Hospital has shown that,
due to the high underreporting, the currently available statistics cannot provide reliable
information about the actual situation of bicycle accidents. However, the Münster
bicycle accident study was limited to the city of Münster, and is, therefore, not a
representative sample for Germany.
In order to allow for a detailed analysis of bicycle accidents, the data of all patients
requiring hospital medical care in the region of the TNNW and on Vancouver Island
C. Juhra et al. / Methodological Approaches to Comparing Information 95
due to bicycle accidents will be collected for a period of one year. The study aims to
provide answers to the following questions:
The study must thus combine data concerning medical issues and data concerning
issues related to the type and circumstances of the accident (Figure 1).
Currently, no standardized electronic data set exists for emergency departments. There
is a great variety ways in which physician and nurses document the reason for
admission, the health status and complaints of the patient, etc. in German hospitals.
This ranges from free text on paper to highly standardized electronic forms, which,
unfortunately, is often an exception to the rule. The police collect data on accidents and
hardly any detailed data on type of accident can be found in patient records. Due to
privacy and other legal issues, these databases cannot be linked. The nationwide
German Trauma Registry includes only the severely (life-threatening) injured patients
and does not contain detailed data on the type of the accident. Currently, no
standardized emergency dataset or electronic health record exists.
Thus data collection in Germany is heavily dependent on questionnaires.
Analogous to the established methodology of the Münster bicycle accident study,
accident data are collected by patient questionnaires and hospital chart reviews.
Questionnaires are distributed to the patients in the clinics of the TNNW. In addition,
the patient data can be collected on the spot, and can later be entered by a student
assistant (SA) into an anonymous database. 23 hospitals in all regions of the TNNW
have declared their commitment to take part in the study.
A FileMaker Advance Pro 12 ® database is used to collect the data. Each SA runs
a stand-alone application which can export the data to the main database. These stand-
alone applications are available in English and German.
The data will be entered into an anonymous database. Each record will be
identified by accident time, patient age, accident site, and hospital name. In cases
where patients will be relocated to another hospital within the TNNW, the records will
have to be updated accordingly. This method of collecting data in Germany has been
approved by the Ethics Committees of the University of Münster and Munich.
96 C. Juhra et al. / Methodological Approaches to Comparing Information
Due to the low density of population on Vancouver Island, injured cyclists can visit not
only hospitals, but also clinics throughout the island. As a result, using questionnaires
for data collection on Vancouver Island would not be feasible.
As compared to Germany, a standardized electronic ambulance record and dataset
exists in British Columbia, in which the circumstances of the accident (e.g. protective
gear, weather etc.) are also documented. In addition, every trauma patient admitted to a
hospital is entered into the BC Trauma Registry, which also includes data on accident
type and pre-clinical emergency care. In addition to this, the Vancouver Island Health
Authority (VIHA) has an Island wide hybrid, electronic patient record system with both
paper and electronic components. Patients who receive care following a bicycle
accident at VIHA have nearly all of their data residing in databases that are part of the
electronic patient record.
The School of Health Information Science of the University of Victoria will access
the data provided by the electronic data sources. The data will need to be accessed and
linked from database repositories located in different parts of the province. This
requires application to local ethics bodies and meeting legislative and privacy
requirements in the province of BC.
3. Discussion
In order to improve the safety of cycling and promote the use of bicycles, a deep and
thorough understanding of bicycle accidents is crucial. Unfortunately, most bicycle
accidents are not officially reported, so that using the official statistics does not show
the complete picture.
Despite the large number of accidents and fatalities, research on bicycle accidents
remains scarce. Most research focuses on severe or fatal accidents [5], while minor
injuries are often not investigated.
Different healthcare systems, transportation systems and regulations, and different
ways of documenting data are a challenge for collecting accident and medical data in
different countries. However, by identifying the needed variables and the possible
sources of information, ways can be found to collect the same data in different
countries.
An electronic health record can be a valuable source for health service research.
Using existing electronically available data avoids the need for paper-based
questionnaires with their drawbacks (single-source approach). eHealth can thus also be
useful to generate data for injury prevention.
The data collection of the described bicycle study will be finished at April 30 th,
2013. Preliminary results will be presented at the end of 2012. The final results will be
published in a BASt-report in German language and presented in Canada in the fall of
2013. We will be integrating our study of bicycle accidents in both countries with our
prior work in simulation modeling and other studies. [6].
References
1
Corresponding Author: Jonn Wu, BCCA, 600 West 10th Avenue, Vancouver, BC V5Z 4E6, Canada; E-
mail: [email protected].
J. Wu et al. / The Development of a Standardized Software Platform 99
Introduction
Although some BCCA Outcomes Units have been very successful, most use software
programs with limited functionality (ie. Microsoft Excel, Access). Frequently, data is
collected on paper or are extracted from the chart retrospectively, then manually
entered. This process is labor intensive, expensive and error-prone; ideally, data should
be entered once, and used multiple times. These programs often cannot support
multiple users across many sites, multiple modules/disciplines, customized role-based
security access, clinician/patient data entry, or direct data linkages.
Because data entry is a challenging and expensive barrier for any outcomes
project, there will be three processes to accommodate data entry:
This process will support the use of paper forms (ie. Staging and Follow-up).
Document distribution and data entry will be facilitated by the Data Quality & Registry
department. This is the traditional workflow used by most Outcomes Units currently,
and requires ongoing operational costs.
Clinicians (Oral Oncology, Pathology and Oncology Nutrition) and patients (side
effects) will be able to enter data directly into OaSIS. This will omit the use of paper
forms, thereby reducing costs and errors associated with distribution and entry. It is
well recognized that patient generated quality of life data is more accurate and less
prone to bias 3, 4.
The OaSIS system will support direct data linkages to existing data warehouses; this
provides the most efficient and accurate method for data collection. The system is
currently connected to the Cancer Agency Information System (CAIS, clinical
information), provincial cancer registry, and the provincial chemotherapy warehouse.
Additional linkages are in progress ie. direct access to dosimetric radiotherapy data.
In September 2008, the Lung and Head & Neck Tumour Groups invited
multidisciplinary leaders to develop their own modules ie. discipline-specific data
elements. Data dictionaries are mandatory for each module, and each data element is
limited to discrete variables. These multidisciplinary modules include:
• Patient Demographics (including Tumour Stage, Risk Factors)
• Follow-Up (Clinical Outcomes)
• Pathology (Histological Subtyping, Molecular Markers)
• Chemotherapy
• Radiotherapy (non-dosimetric datasets)
• Oral Oncology
• Oncology Nutrition
• Patient and Family Counselling (PSCAN Psychological Screening)
• Quality of Life Tools (Patient and Physician Generated Assessments)
Each module is a combination of imported data and additional data collected by
clinicians. The Patient Demographics module imports information from the registry (ie.
J. Wu et al. / The Development of a Standardized Software Platform 101
tumour site, histology) and CAIS (ie. BCCA ID). Additional factors are recorded on
paper (ie. tumour stage, smoking history, performance status) and entered later.
The Pathology and Nutrition modules primarily consist of clinician generated data.
Pathologists enter molecular markers, and histological sub-typing at the time of central
review, whereas nutritionists record weight, eating related side effects and feeding tube
complications in “real time” at the time of the patient encounter by the clinician.
The Chemotherapy module is unique since all of the data is imported from the
pharmacy system; this includes every chemotherapeutic drug that was ordered and
dispensed by the BCCA, including dose, and how, when and where it was
administered. Over 99% of all chemotherapeutic agents dispensed in BC is captured in
this system. Additional functionality includes filters (ie. dates, drug names, number of
cycles and days per cycle) used to automatically generate chemotherapy “courses”.
After an initial testing phase, prospective data entry was initiated in May, 2011. As
of October, 2012, OaSIS is supporting 12 unique research projects and includes:
• Patient Demographics: 7,374 records
• Follow-Up: 1,934 events (loco-regional and distant recurrence)
• Chemotherapy: 2,538 courses
• Oncology Nutrition: 189 assessments
• PSCAN Psychological Screening: 10,245 assessments
The Pathology and Oral Oncology Modules will launch in 2013.
The Radiotherapy (RT) Module introduced in Phase One extracts basic and generic RT
data from the Cancer Agency Information System (CAIS). These fields include RT
start and finish dates, total radiation dose, treatment technique, and treatment region.
Although these continue to serve as useful end points, they do not provide any spatial
information specific to a particular treatment course. To fully understand the effects of
radiotherapy dose on irradiated organs and target structures, investigators require
access to the full three-dimensional dose matrix ie. how much dose was delivered to a
particular part of a structure; additionally, the detailed coordinates of relevant structures
also need to be archived with their respective radiographic image sets (CT, MRI, and/or
PET/CT). For example, is radiation dose to a certain part of the tongue result in worse
dysgeusia than another part of the tongue? The second Phase will create a three-
dimensional dosimetric repository, or a “dose bank”, linked to clinical outcomes.
Although investigators have proposed the concept of a dosimetric library, they are
typically not linked to comprehensive population based clinical outcomes. Our ability
to correlate and analyze clinical endpoints, on a large scale, with three-dimensional
dosimetric data sets will provide investigators with unprecedented research potential.
This platform will allow investigators to answer important questions related to
radiation response and normal tissue durability. Initial work at the BCCA on a much
smaller scale (<100 patients) regarding salivary function and partial volume dosimetry
have already made a clinical impact5.
Software development was initiated in September, 2011, in conjunction with the
Faculty of Medical Physics, UBC. This included initial architectural design and review
of functional requirements ie. prior to any dosimetric analysis of a structure, the
module must be able to identify it correctly. Identification will be based on parsing the
name, and double checking its geographic location within the body. For example, if a
102 J. Wu et al. / The Development of a Standardized Software Platform
structure is labelled “L paro”, the module will correctly identify it as the “left parotid
gland” by parsing the name, and ensuring the structure exists in the correct geographic
location in the body. Automatic dose calculations within sub-segments of a structure is
now being developed. This work will result in improved understanding of the effect of
radiation dose to certain parts of normal structures, which will lead to fewer side effects
in irradiated patients.
Due to the uniquely disparate nature of the data elements in OaSIS, the Research
Module will include data mining tools using a combination of existing and custom
developed querying applications. This work will be done in conjunction with the
Faculty of Computer Science, UBC.
A multidisciplinary team will develop patient generated surveys which will capture
information during a patient’s cancer journey via a web-based Patient Portal. As cancer
therapies improve, quality of life becomes an increasingly significant endpoint.
The research integration phase will see the addition of other tumour groups who wish
to participate ie. the Testes Module will be added in early 2013. As well, linkages will
be created to other data repositories, including the BCCA Tumour Tissue Repository.
4. Summary
References
=? Köbel M, Kalloger SE, Huntsman DG, Santos JL, Swenerton KD, Seidman JD, Gilks CB; Cheryl Brown
Ovarian Cancer Outcomes Unit of the British Columbia Cancer Agency, Vancouver BC. Differences in
tumor type in low-stage versus high-stage ovarian carcinomas. Int J Gynecol Pathol. 2010
May;29(3):203-11.
[2] Panades M, Olivotto IA, Speers CH, Shenkier T, Olivotto TA, Weir L, Allan SJ, Truong PT. Evolving
treatment strategies for inflammatory breast cancer: a population-based survival analysis. J Clin Oncol.
2005 Mar 20;23(9):1941-50..
=? Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, Lenderking WR, Cella D, Basch E;
ISPOR ePRO Task Force. Recommendations on evidence needed to support measurement equivalence
between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good
Research Practices Task Force report. Value Health. 2009 Jun;12(4):419-29. Epub 2008 Nov 11.
J. Wu et al. / The Development of a Standardized Software Platform 103
[4] Hróbjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S.
Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both
blinded and non-blinded outcome assessors. BMJ. 2012 Feb 27;344:e1119.
[5] Moiseenko V, Wu J, Hovan A, Saleh Z, Apte A, Deasy JO, Harrow S, Rabuka C, Muggli A, Thompson
A. Treatment planning constraints to avoid xerostomia in head-and-neck radiotherapy: an independent
test of QUANTEC criteria using a prospectively collected dataset. Int J Radiat Oncol Biol Phys. 2012
Mar 1;82(3):1108-14. Epub 2011 Jun 2.
This page intentionally left blank
Clinical Decision Support Systems
This page intentionally left blank
Enabling Health and Healthcare through ICT 107
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-107
Introduction
1
Corresponding Author: Darren Hudson, email: [email protected].
108 D. Hudson / Neuroanatomical Basis for Recognition Primed Decision Making
The recognition primed decision making (RPDM) model was developed from cognitive
psychology in the mid-1980s by Klein et al. [2]. As a model, it has described decision-
making for time sensitive situations, employed by fire-fighters, tank commanders,
naval officers, and airline pilots [3]. However, there is a paucity of medical literature
using this model. The only published work in medicine involved an emergency
situation in an ophthalmology clinic [4] where the author described a successful
resuscitation based on early recognition and anticipation of patient deterioration.
RPDM was originally developed using fire command officers [5]. In post-incident
interviews, the researchers found that fire commanders followed a series of steps to
make decisions based both on available information and prior experience. The
investigators developed a model wherein the person presented with a situation made an
initial assessment and decided whether it fit a familiar pattern [5]. If the situation was
recognized, then the level of familiarity was graded by comparing it to previous cases
and cues present at that time. The individual then considered whether the intended
goal was feasible in the current circumstance, considered issues in the contemporary
case that needed to be accounted for, and finally considered possible actions.
If the current situation was not recognized, RPDM was unlikely to be effective,
further information was sought, and the situation reassessed in order to generate a
recognizable pattern [5]. Failing this, the decision-maker had the option to choose
other strategies to seek an answer. However, in the case where the situation was
recognized and a reasonable match found from past experience, the decision-maker
could mentally simulate the solution and try to anticipate problems that may occur. If
this simulation suggested a satisfactory outcome, it was enacted. If the simulation
suggested difficulties, then the situation required reassessment and plan modifications.
All these steps occurred automatically with little conscious effort. [5].
Several features make RPDM distinctive from other decision-making strategies
[5]. As a decision-maker develops more experience, only feasible options are
considered, and long lists of random options - many of which will not work - are
obviated. RPDM, then, renders faster, more efficient decisions. As well, decision-
making is not intended to find the best possible solution, but to find the first
satisfactory option for the current situation. Better options may present themselves
subsequently, but they do not delay action. This “satisficing” means that the goal in
the decision-making is to find the most reasonable, but not necessarily the best option,
to accomplish the goal. As opposed to other decision-making models, no formal
instruction in RPDM is required. The process is intuitive and built on experience.
Finally, the use of mental simulation also allows decision-makers to put the task into
context and allows for preliminary testing to ensure feasibility of the option.
A recent hypothesis has suggested that the main function of the cerebral cortex is to act
as a memory storage unit and recognition/prediction engine [6]. Consider the example
of a conversation in a noisy room. The only way to follow the conversation is to be
able to predict and fill in the unheard gaps. The memory of words and common
sentences with an ability to predict the next phrase makes this possible.
D. Hudson / Neuroanatomical Basis for Recognition Primed Decision Making 109
At only 4 mm thick, the cerebral cortex is a new evolutionary invention found only
in vertebrates. Mapping studies over the entire cortex have demonstrated different
functional regions throughout the brain. However, the microscopic structure of each
region is the same throughout the brain [7]. Thus, different regions are functionally
distinct because of their connections, not because of their structure.
The cortex can be divided into six layers - layer VI is the deepest, I is the most
superficial [6]. Inputs enter the cerebral cortex via VI and travel upwards for
processing. The different layers are structured such that cells in one layer will receive
multiple inputs from the cells below and send inputs to higher levels and laterally
through interneurons. Inputs arriving at the cortex will tend to stay within the same
region as they progress up the layers. Thus the cortex can be considered being divided
into columns. Sensory input sequences (vision, touch, sounds etc.) ascend up the
cortex through each cortical layer and interact with descending signals from higher
memory levels [6]. These higher memory level signals are representations of complex,
often multi-sensory sources used to predict incoming sensory input.
The cortex as a prediction engine has been modelled successfully by Rao and
Ballard [8] using an artificial retina attached to electronic simulation of a neuronal
column. In the model, a prediction of expected input based on previous input flows
downward from higher level neurons and interfaces with an input signal. The
difference between prediction and input signal generates an error signal which travels
back up the simulated cortical column through successive layers of interaction with the
prediction algorithm to modify the expected input. In this simple model, the simulator
successfully anticipated and predicted incoming inputs. Interlayer feedback interacted
with the incoming signal and was an efficient way to encode the visual image.
If the cortex can be considered a prediction/error recognition engine, then RPDM
has a neuroanatomical foundation. When faced with a situation, sensory stimulation
enters the cortex via level VI. These inputs ascend individual columns towards level I.
When the signal is recognized, associative interneurons connect to related columns.
These columns are now anticipating an incoming signal. When it arrives, the column
performs sequential error checking of the signal. If the incoming signal matches then it
is recognized and the process repeats with other associative interneurons. Consider the
example of hearing a series of odd numbers from one to nine. By the time the number
five is heard, the sequence is clear and interneurons have sent signals to the columns
involved in processing the numbers seven and nine. When seven is heard, there is a no
error signal in that column and it is considered to be recognized. If the next number is
ten, an error signal is generated which requires processing in higher cortical layers and
the sequence is now not recognized.
misuse of this decision-making model and associated potential cognitive errors as has
been pointed out by Kahneman [9].
RPDM also allows us to understand the phenomenon of the experienced nurse
calling for help before the crisis occurs. As a patient deteriorates, a series of stimuli are
generated. These inputs are processed in the lower cortical layers and recognised. The
associative interneurons then to connect to alarm centres that tell the nurse that there is
something wrong and to call for help. Often the nurse will be unable to explain the
reason for the recognition because it occurred in the lower layers and did not generate
error signals that required higher level processing. This re-enforces the importance of
experience and exposure in education. Reading a text about a patient deterioration will
not generate the same recognition pathways as experiencing it.
Conclusion
In this paper, recognition primed decision making as a model for medical decision
making has been described. Originally studied in non-medical situations, it represents
an important addition to understanding how experienced clinicians decide. Unlike other
decision making models, there appears to be a neuroanatomical mechanism for RPDM.
This is based on the hypothesis that the cerebral cortex functions as a memory storage
and recognition/anticipation engine. The implications for education and health
informatics are significant.
References
[1] Zakay D, Wooler S. Time pressure, training and decision effectiveness. Ergonomics.
1984;27(Compendex):273-84.
[2] Klein GA, Calderwood R, Clinton-Cirocco A. Rapid Decision Making on the Fire Ground. United
States: 1988.
[3] Schraagen JM. Naturalistic decision making and macrocognition. Aldershot, England ; Burlington, Vt.:
Ashgate; 2008.
[4] Bond S, Cooper S. Modelling emergency decisions: recognition-primed decision making. The literature
in relation to an ophthalmic critical incident. J Clin Nurs. 2006;15(8):1023-32. Epub 2006/08/02.
[5] Klein GA, Calderwood R. DECISION-MODELS - SOME LESSONS FROM THE FIELD. IEEE
TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS. 1991;21(5):1018-26.
[6] Hawkins J, Blakeslee S. On intelligence. 1st ed. New York: Times Books; 2004. 261 p. p.
[7] Creutzfeldt OD. Generality of the functional structure of the neocortex. Naturwissenschaften.
1977;64(10):507-17.
[8] Rao RPN, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-
classical receptive-field effects. Nature Neuroscience. 1999;2(1):79.
[9] Kahneman D. Thinking, fast and slow. New York: Farrar, Straus and Giroux; 2011..
Enabling Health and Healthcare through ICT 111
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-111
Introduction
Since 2005, when legislation enabling the nurse practitioner (NP) role was enacted in
British Columbia (BC), integrating the role into clinical practice settings has been a
priority. To date, $62.6 million has been invested to support and resource the
integration of NPs, however, no evaluation of their impact on the BC health system has
been conducted. At this time, it is imperative that we: a) begin to determine the changes
that occur for patients and the implications to the health care system when NPs become
part of the care process, b) assess the impact of adding an NP to the functioning of
collaborative health care teams, and c) continue to monitor the practice settings and
scope of practice of NPs working in BC. A mixed method, multi-year study is currently
being undertaken to evaluate the integration of NPs into the BC healthcare system, with
the intent to establish an evaluation framework. The overall study is designed to
address the following broad questions: (1) What are the practice settings and the scope
of practice of NPs working in BC? (2) What changes result for patients and, what are
the implications for the health care system when NPs become part of the care process?
(3) What is the impact of adding an NP on the functioning of collaborative health care
teams?
1
Corresponding Author
112 E. Sangster-Gormley et al. / Use of Knowledge Discovery Techniques
Although the NP role is new to BC, it has existed in other Canadian provinces and the
United States for more than 40 years. The role has been studied more than any other
role in health care [1]. The evidence accumulated over these years has consistently
demonstrated that NPs provide safe, effective care [2]. Researchers consistently find
that patients are satisfied with the care received from NPs, trust NPs [3], feel that NPs
take their problems seriously and discuss their concerns, have expert communication
skills and are approachable [4].
Sidani, et al. [5] found that, when other variables were controlled, patients cared for
by an NP in acute care were satisfied with their care, and also had higher physical and
social functioning than those cared for by other providers. Others compared NPs to
physicians and found that NPs had longer consultation times and patients received
more information and counseling [6]. In addition, NPs placed more emphasis on health
promotion and self care management of chronic and acute conditions than did their
physician colleagues [7]. Researchers have consistently found that NPs provide patient-
centred care, spend time with patients, and emphasize health promotion and chronic
disease self-care and, when compared to physicians, the physical outcomes of care are
similar [6, 8]. Nonetheless, there is a need to expand our evaluative efforts to
understand the value-added impact of NPs in the BC context. It is time to evaluate the
consequences or improvements in the overall health and wellbeing of patients that
might be attributed to such NP practice activities.
At the time the NP role was implemented in BC, the MoH developed encounter
codes to be used by NPs to track their activities. Encounter codes are unique numbers,
similar to ICD-10 codes, assigned to specific types of patient care services performed
by NPs. NPs are expected to submit an encounter code to the MoH at the end of each
patient encounter. Encounter code data are stored in Population Data BC databases.
Data mining is "an integral part of knowledge discovery in database (KDD), which is
the overall process of converting raw data into useful information" [9]. In other words,
it is the process of automatically discovering useful information in large data
repositories (e.g. data warehouses). The KDD process consists of four major
transformation phases. They include the: (1) problem definition phase, (2) data pre-
processing phase, (3) pattern recognition phase and (4) pattern validation phase (see
Figure 1) [10] .
In the problem definition phase we are attempting to answer the broad question:
What are the practice settings and scope of practice of NPs working in BC?
E. Sangster-Gormley et al. / Use of Knowledge Discovery Techniques 113
Support determines how often the data items in a rule are present together in a
transaction in a given data set and is simply the count of transactions that contain X and
Y. Support is used to determine if a rule is of interest since high support indicates that
the rule occurs often in the data. Confidence is used to determine the reliability of the
inference made by the rule and is an estimate of the conditional probability of Y given
X. It says “If X is present in a transaction, how likely is it that Y is also present”. The
generated rules suggest a strong co-occurrence relationship between the given data item
subsets. Finally, in the pattern validation phase we undertake the process of assessing
how well the mining models perform against real data. It is important that the mining
team validates the discovered patterns before deploying them into a production
environment. There are several approaches for validating the patterns [12]. For
example,
Use statistical validity to determine whether there are problems in the data or in
the model - A number of statistical methods may be used to evaluate the data
mining model quality or pattern accuracy, such as Cross Validation and Receiver
Operating Characteristic (ROC) curves.
Separate the data into training and testing sets to test the accuracy of patterns - It
is common for the data mining algorithms to find patterns in the training set which
are not present in the general data set. To deal with the so called "over-fitting"
issue, the validation uses a test data set which the data mining algorithm was not
114 E. Sangster-Gormley et al. / Use of Knowledge Discovery Techniques
trained on. The learnt patterns are applied to the data set and the resulting
output is compared to the desired output.
Ask domain experts to review the results of the data mining to determine
whether the discovered patterns have meaning in the targeted scenario – In this
study, the research team composed of domain experts in the fields of NP
education/research (ESG, RS) and nursing/health informatics (EB, MSK) will be
involved in the validation process to interpret the accuracy of the patterns.
We are planning to use data extracted from the MoH database from BC. In this paper,
since our human research ethics approval is pending, we use a test dataset with 25
records to test the feasibility of the proposed method. The data are encoded as numbers
for data analysis (see Figure 2). In the dataset, there are 4 health authorities (named
VIHA, VCH, FHA and NHA). Each health authority has 3 NPs whose age is between
20 and 59. The encounter types used by NPs are coded using unique number; for
example, 36360 is a prenatal visit, 36365 is the insertion of an intrauterine device,
36413 is repair of a minor laceration and 36255 is newborn care.
Applying the KDD techniques to this simple dataset has yielded some interesting
patterns/information as follows:
Pattern 1: Male NPs use more encounter types than female NPs (4-types/8-
encounters (50%) vs. 3-types/17- encounters (18%)).
Pattern 2: Older NPs (age group 4) use more encounter types than other age group
of NPs (age group 1~4: 1/2(50%), 4/7(57%), 3/12(25%) and 3/4(75%)).
Pattern 3: NPs in the Northern Health Authority (NHA) use more encounter types
than those of other health authorities (health authority 1~4: 3/7(43%), 3/9(33%),
3/5(60%) and 3/4(75%)).
Pattern 4: Female NPs insert more intrauterine devices (36365) than male NPs.
The confidence is 86%.
Pattern 4 suggests additional analysis is needed. This pattern shows that female
NP’s insert more intrauterine devices than male NPs. Further analysis will be
undertaken to determine if there are other activities that female NP’s undertake more
often.
4. Conclusion
Integrating the NP role into clinical practice settings is new in BC. An evaluation of its
impact on the BC health system is important. In this study we assessed the feasibility of
applying knowledge discovery techniques to analyze NP data and to generate some
interesting patterns/information. This work will lead to a better understanding of NP
practice patterns and will be used to answer the study questions. This analysis data
involved the use of test data (as we determined the feasibility of employing data mining
techniques). Future work will involve extracting data from BC MoH databases.
Patterns generated from our analyses in this study do not reveal the real nature of the
associations between NP demographics, geographic locations and encounter types.
Future work will use MOH NP data, and domain experts will be involved in the
validation process to interpret the mined patterns. This should increase the value of our
work in that we will be able to determine how generalizable the approach is.
References
[1] Bauer JC. Nurse practitioners as an underutilized resource for health reform: Evidence-based
demonstrations of cost-effectiveness. Journal of the American Academy of Nurse Practitioners,
2010;22(4):228-231.
[2] Horrocks S, Anderson E, Salisbury C. Systematic review of whether nurse practitioners working in
primary care can provide equivalent care to doctors. Brit Med J, 2002;324(6): 819 -823.
[3] Benkert R, Hollie B, Nordstrom C, Wickson B, Bins-Emerick L. Trust, mistrust, racial identity and
patient satisfaction in urban African American primary care patients of nurse practitioners. Journal of
Nursing Scholarship,2009; 41(2): 211-219.
[4 Thrasher C, Purc-Stephenson R. Patient satisfaction with nurse practitioner care in emergency
departments in Canada. Journal of the American Academy of Nurse Practitioners, 2008;10, 231-237.
[5] Sidani S, Doran D, Porter H, Lefort S, O’Brien-Pallas L, Zahn C., et al., Outcomes of nurse
practitioners in acute care: an exploration. Internet Journal of Advanced Nursing Practice, 2007;8(1), 3.
[6] Kinnersley P, Anderson E, Parry K, Clement J, Archard L, Turton P. et al. Randomizedcontrolled
trial of nurse practitioner versus general practitioner care for patients requesting “same
day”consultations in primary care. Brit Med J, 2000; 320, 1043-1048.
[7] Charlton C, Dearing K, Berry J, Johnson M. Nurse practitioners’ communication styles and their
impact on patient outcomes: An integrative literature review. Journal of the American Academy
of Nurse Practitioners,2008; 20, 382-388.
[8] Lenz E, Mundinger M, Kane R, Hopkins S, Lin S. Primary care outcome in patients treated by nurse
practitioners and physicians: Two-year follow-up. Medical Care Research and Review,2004;61(3), 332-
351.
[9] Tan PN, Steinbach M, Kumar V. Introduction to Data Mining. Addison Wesley;2005. ISBN0-321-
32136-7
[10] Kuo MH. Databases, Data Warehousing and Data mining for Healthcare" in Healthcare
Information Technology Essentials for the IT Professional, 2012, McGraw-Hill's. (in press)
[11] Kotsiantis S, Kanellopoulos D. Association Rules Mining: A Recent Overview, GESTS International
Transactions on Computer Science and Engineering. 2006;32 (1), 71-82.
[12] Richesson RL. Clinical Research Informatics. Springer;2012. ISBN: 1848824483
116 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-116
Introduction
The Institute of Medicine’s (IOM) Quality Chasm report highlighted 20 priority areas
in which improvements in care delivery could significantly impact health quality.[1]
Although evidence-based pharmacotherapies (EBP) are a principal component of
patient care in many of these areas (asthma, diabetes, depression, heart failure, COPD,
hypertension, ischemic heart disease and stroke), 30% to 50% of patients do not take
their medications as prescribed.[2]
Medication adherence is a result of two processes: (1) clinicians prescribing EBP
to their patients, and (2) patients taking their EBP medications as instructed by their
clinicians.[3] However, effective approaches for assuring that these processes are
completed have not been effectively integrated into routine clinical care. This lack of
1
Corresponding Author: Eric L. Eisenstein, DBA, Duke Clinical Research Institute, DUMC Box 3865,
Durham, NC 27710. Telephone: 919-668-8984; Fax: 919-668-7057; e-mail: [email protected]..
J.M. Willis et al. / Decision Support for EBP Detects Adherence Problems 117
effective integration has led to calls for new care models that address appropriate
medication use and make it a part of the patient care process. The present study sought
to use clinical decision support (CDS) to increase EBP adherence within a vulnerable
population and to assess the impact of this CDS technology on service use and medical
costs.
1. Methods
Subjects in the present study included NPCCN Medicaid beneficiaries with at least one
of six targeted IOM priority conditions who were continuously enrolled during the
118 J.M. Willis et al. / Decision Support for EBP Detects Adherence Problems
The two study interventions included CDS alerts delivered as reports to primary care
clinics (Figure 1) and email notices sent to network care managers (Figure 2). The first
intervention included reports with a summary list of filled prescription claims along
with a numeric calculation of medication adherence and a graphical depiction of “days
covered” over a 1-year period along with recommendations addressing possible
deficiencies in medication therapy prescriptions relative to EBP guidelines. These
medication summary reports were delivered to the point of care at the time of a
patient’s scheduled visit at their primary care clinic. The second intervention included
email notices to care managers for patients with 1) low medication adherence; 2) no
primary care clinic appointments in the previous 6 months; and 3) no appointments
scheduled for the future. Table 1 summarizes the pharmacotherapeutic rules used in this
study and their criteria for activation. Our methods for developing and testing these
interventions have been previously described.[7, 8]
NCPPN enrollees were randomly assigned using a 1:1:1 allocation by family unit using
a pseudo-random number generator. Subjects assigned to the first group received usual
care, those assigned to the second group received the first intervention (clinic reports)
and those assigned to the third group received both interventions (clinic reports and
care manager notices). In place of informed consent, potential study subjects were sent
a letter explaining the study and a response card to allow them to opt out of the study.
The Duke University Medical Center Institutional Review Board approved this study’s
protocol on December 18, 2007 (Registry Number: Pro00002524) and
ClinicalTrials.gov registration was obtained on August 27, 2009 (Identifier:
NCT00979225). The 12-month intervention period began on December 7, 2009, and
120 J.M. Willis et al. / Decision Support for EBP Detects Adherence Problems
ended on December 6, 2010. Study subject follow-up ended on August 30, 2011. The
primary study outcome measure was medication adherence across all drug classes and
conditions. Table 2 contains a complete list of study outcome measures. Estimates for
study outcomes and between-treatment group comparisons were generated using
generalized estimating equation models with a working correlation matrix to account
for clustering by family unit. Primary analyses were performed for the entire study
population, following the intention-to-treat principle. Secondary analyses were
performed for the subset of patients who generated an alert or medication summary
report to focus on patients who actually received an intervention (“touched” subjects).
2. Results
A total of 2910 subjects met this study’s inclusion criteria. Of these, 155 declined to
participate by returning their opt-out letter, 497 could not participate after their clinic
sites installed new electronic health record (EHR) systems and were unable to send
patient appointment scheduling information to the network HIE, and 39 transferred to
clinical sites not included in this study. A total of 2219 subjects were included, of
which 739 were assigned to usual care, 744 to have the clinic reports intervention and
736 to have both the clinic reports and care manager notices interventions (Figure 3).
Patients in the control and intervention groups were well matched with regard to
baseline characteristics. Overall, study subjects were predominantly female, African-
American and young. Most subjects had a history of persistent asthma, diabetes or
hypertension. Given this population’s age, it is not surprising that relatively few
subjects had a history of congestive heart failure (CHF), ischemic heart disease (IHD)
or stroke.
J.M. Willis et al. / Decision Support for EBP Detects Adherence Problems 121
Study interventions were generated for all study subjects meeting CDS criteria but were
only sent to those randomized to groups 2 or 3 (Table 3). Overall, the numbers of
medication management reports and care manager notices generated did not differ
significantly by treatment group; however, the volume of clinic reports was 5 times that
of care manager notices (5948 versus 1052) as anticipated because the criteria for
generating care manager notices were more stringent than for generating clinic reports.
122 J.M. Willis et al. / Decision Support for EBP Detects Adherence Problems
Note: * Clinic Report vs. Controls; ** Clinic Reports + Care Manager Notices vs. Controls
Medication Class % % P= % P=
All Classes 41.3 41.2 0.82 42.9 0.35
Statin All conditions (age >17) 54.0 61.0 0.12 56.3 0.63
ACEI (ARB) All conditions (age >17) 69.7 68.4 0.85 64.2 0.52
B-Blocker All conditions (age >17) 55.2 58.3 0.54 57.1 0.76
Warfarin All conditions (age >17) 48.3 56.1 0.54 37.8 0.26
Anti-Htn All conditions (age >17) 62.8 61.1 0.88 63.9 0.71
Anti-DM Diabetes (all ages) 54.3 53.8 0.90 57.1 0.49
Inhaled Steroid Persistent Asthma 36.9 36.4 0.73 41.8 0.23
+age>36 mo
B-Agonist Persistent Asthma + 25.4 23.9 0.85 24.7 0.84
age>36 mo
Six-month encounter rates per 100 subjects and medical costs per subject (total and by
type) were similar across treatment groups. As expected, the highest encounter rates
were for outpatient care and the greatest costs were for pharmaceuticals and outpatient
care.
J.M. Willis et al. / Decision Support for EBP Detects Adherence Problems 123
3. Discussion
This study sought to use CDS to increase EBP adherence within a vulnerable
population, and to assess the impact of this CDS technology on service use and medical
costs. We demonstrated significant technical advancement for CDS for promoting
medication adherence by intervening at both the level of the PCP clinic and the
population. Unfortunately, even though the interventions were successfully delivered,
neither the clinic-based intervention (point-of-care reports to clinicians), nor the
population-based intervention (notices to care managers) resulted in increased
medication adherence to EBP. From a positive perspective, we demonstrated that a
CDS intervention can successfully identify medication adherence issues relative to EBP
and mobilize the provider workforce. We detected that the notices sent to care
managers significantly increased the extent of contact that care managers had with
study subjects. One potential weakness of this population-level intervention was that
care manager contact was limited to encouraging patients to arrange follow-up with
their PCP clinic (where a medication management report would be available). Care
managers were permitted to inquire about possible reasons for medication
nonadherence, such as financial or transportation issues. However, they were not
permitted to give clinical advice because such an intervention exceeded their scope of
practice. Perhaps a more effectual population-level intervention would have been to
engage clinical pharmacists or advanced practice nurses to directly address
124 J.M. Willis et al. / Decision Support for EBP Detects Adherence Problems
Acknowledgements.
This study was funded by the Department of Health and Human Services, Agency for
Healthcare Research and Quality (R18-HS017072).
References
[1] I. O. M. Committee on Quality of Health Care in America, Crossing the Quality Chasm: A New Health
System for the 21st Century. Washington, D.C.: The National Academies Press, 2001.
[2] NEHI. (August 11, 2009). Thinking Outside the Pillbox: A System-wide Approach to Improving Patient
Medication Adherence for Chronic Disease. Available:
www.nehi.net/publications/44/thinking_outside_the_pillbox_a_systemwide_approach_to_improving_p
atient_medication_adherence_for_chronic_disease
[3 E. L. Eisenstein, et al., "Evaluating implementation fidelity in health information technology
interventions," AMIA Annu Symp Proceedings, pp. 211-215, 2007.
[4] J. Cook, et al., "Practice profile. Community collaboration to improve care and reduce health
disparities," Health Aff.(Millwood.), vol. 29, pp. 956-958, 2010.
[5] D. F. Lobach, et al., "Proactive population health management in the context of a regional health
information exchange using standards-based decision support," AMIA Annual Symposium Proceedings,
pp.473-477, 2007.
[6] K. Kawamoto and D. F. Lobach, "Design, implementation, use, and preliminary evaluation of
SEBASTIAN, a standards-based web service for clinical decision support," AMIA Symposium, pp. 380-
384, 2005.
[7] G. Del Fiol, et al., "Improving Medication Adherence in a Regional Healthcare Information Exchange
using a Scalable, Claims-Driven, and Service-Oriented Approach," AMIA Annu Symp Proc, pp. 137-41,
2010.
[8] K. Kawamoto, et al., "Development and evaluation of an improved methodology for assessing
adherence to evidence-based drug therapy guidelines using claims data," AMIA Annu Symp Proc, pp.
394-8, 2007.
[9] S.B. Calvert, et al., "Patient-focused intervention to improve long-term adherence to evidence- based
medications: a randomized trial," Am Heart J. 2012 Apr:163(4):657-65.e1.
[10] L.L. Wood, et al, "Utility of point-of-care medication management reports generated through a decision
support Web service," AMIA Annu Symp Proc, p.1312, 2010
126 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-126
Abstract. Maintaining blood oxygen saturation within the intended target range
for preterm infants receiving neonatal intensive care is challenging. Supplemental
oxygen is believed to lead to increased risk of retinopathy of prematurity and
hence managing the level of oxygen within this population is important within
their care. Current quality improvement activities use coarse hourly spot readings
to measure supplemental oxygen levels as associated with targeted ranges that vary
based on gestational age. In this research we use Artemis, a real-time online
healthcare analytics platform to ascertain if the collection of second by second data
provides a better representation of retinal exposure to oxygen than an infrequent,
intermittent spot reading. We show that Artemis is capable of producing more
accurate information from the higher frequency data, as it includes all the episodic
events in the activity of the hour, which provides a better understanding of oxygen
fluctuation ranges which affect the physiological status of the infant.
Introduction
Medical and technological advances in neonatal intensive care over the past 50 years
have increased survival rates for preterm infants whom are <37 weeks gestational age
(GA) and weigh <2500 grams. In 2009–2010, the Canadian Institute for Health
Information reported that one in 12 infants (8%) are born prematurely in Canadian
hospitals, and that more than 6% of those infants weigh <2500 grams [1].
Physiological immaturity is a precursor to many imminent complications involving the
lungs, brain, and eyes which result in the use of supplemental oxygen therapy to aid
with the pulmonary inadequacy [3]. In the preterm infant population, optimal oxygen
saturation ranges are still unknown and evidence based practices have set guidelines for
oxygen administration at 85-92% to maintain and reduce the risk of complications
while increasing the benefits of oxygen [8]. Supplemental oxygen exposure is an
established risk factor for Retinopathy of Prematurity (ROP) as the immature retina is
susceptible to injury [4, 5]. ROP is described as a vascular disorder of the retina, which
occurs after birth, among preterm infants and involves prolonged oxygen
1
Corresponding author
J. Cirelli et al. / Analysis of Continuous Oxygen Saturation Data for Accurate Representation 127
administration and fluctuation of oxygen, associated with hyperoxia and hypoxia [4].
At birth the immature retina is exposed to an excess of oxygen saturation which
decreases the sensitivity of cells due to the over exposure of oxygen therefore, reduces
the normal stimulus for development of the blood vessels and encourages abnormal
retinal vascular migration [5].
We use the Artemis platform, a real-time analytic system that captures and
analyses high fidelity physiological data streams from bedside medical devices in the
NICU. Its multidimensional approach uses time-series data and temporal data mining
to identify relationships between the data streams and specific clinical conditions that
affect the preterm neonate. The primary goal of this exploratory study was to determine
if the collection and analysis of high fidelity blood oxygen saturation levels (SpO2)
provides a better representation of the immature retina’s exposure to potentially
damaging oxygen levels than infrequent, intermittent spot recordings of SpO2.
1. Related Works
Recent research has shown that not only hyperoxia but also hypoxia and fluctuations in
retinal oxygen levels can contribute to the development of ROP [3,6,8]. A study by
Hagadorn et al., showed that SpO2 is in the intended target range for 50% of the time as
there is considerable fluctuation above and below the intended target range [8].
Hagadorn et al. have also shown that preterm infants less than 28 weeks gestational age
experience fluctuations in SpO2, but these episodic events are frequently transient
and/or have resolved by the time the nurse observes the baby, and therefore
opportunities for intervention are limited [8]. Current research is exploring how new
technologies can help to control oxygen delivery so to better understand fluctuations in
SpO2, and consequently, improve target range compliance [6, 8].
2. Methods
In this exploratory study, we used the Artemis platform to capture high fidelity data for
SpO2, to document retinal exposure to oxygen. We extracted SpO2 values both hourly
and at the frequency of 1Hz, for a preterm infant who was receiving supplemental
oxygen over a consecutive 4 day period. The representations of retinal exposure to
oxygen of these two approaches for capturing SpO2 were compared. The infant was
enrolled in a REB approved nosocomial infection research project being conducted in
the NICU at The Hospital for Sick Children, Toronto.
For this study, the Knowledge Extraction component of Artemis was used to extract
raw SpO2 values (collected by Masimo SET pulse oximeter) over days 27, 28, 29, and
30 of a study month, on a convenient sample data of a de-identified infant <32 weeks
GA [9]. The top of the hour spot readings were extracted as a subset of that dataset.
128 J. Cirelli et al. / Analysis of Continuous Oxygen Saturation Data for Accurate Representation
3600
3000
2400
readings
1800
1200
600
0
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
Hours of the day 16 17 18
19 20 21
22 23
3. Results
The characteristics of the continual collection of achieved SpO2 data using day 27 are
shown in Figure 1. The availability and capture of the infants’ machine acquired
bedside SpO2 transcutaneous oxygen saturation value achieved per hour was examined.
On day 27, ten of the 24 hourly periods showed oxygen saturation average values
which fluctuated more than 5% values from the top of the hour value, within this 24
hour period, one of those hours fluctuated 12%. Similar differences were noted on the
other three days within the study.
Achieved oxygen saturation readings between two shift windows, using the
Artemis second by second data readings (Figure 2) and the top of the hour spot
readings (Figure 3) were compared and examined for all days. Using the data on day 28
seen in figures 2 and 3, we observed that the appearances of the two histograms are
dissimilar. The percent values for each SpO2 range represent a different value in both
the day and the evening shift. Both values were represented by the frequency percent
which is derived from the number of times the achieved SpO2 value was captured in the
% range divided by the total data readings captured for the shift.
J. Cirelli et al. / Analysis of Continuous Oxygen Saturation Data for Accurate Representation 129
60% 53.31
55% 45.79
50%
45% 36.38 38.97
40%
35%
30%
25% 17.83
20%
15% 7.72
10%
5%
0%
7am - 7 pm Day Shift 7pm - 7 am Evening Shift
O2 saturation percent frequency using top of the hour spot readings, day 28
75% 66.67
70%
65%
60%
55%
50% 41.67
45%
40% 33.33
35% 25.00 25.00
30%
25%
20%
15% 8.33
10%
5%
0%
7am - 7pm Day Shift 7pm - 7 am Evening Shift
Figure 3. SpO2 percent frequency using top of the hour spot readings, day 28
4. Discussion
This exploratory study has shown potential that higher fidelity oxygen saturation data
collection provides greater detail of retinal exposure to oxygen than infrequent,
intermittent spot readings. The high frequency data collection provides a better
representation of all activity occurring in the full hour as episodic events are captured
and included to represent a clear picture of SpO2 occurring during each hour. As a
result this approach provides a greater ability to understand the retinal exposure to
oxygen values and percentages with more accuracy.
130 J. Cirelli et al. / Analysis of Continuous Oxygen Saturation Data for Accurate Representation
In addition to what has been reported in this paper, Artemis provides more in-
depth information than that of a single spot reading. It allows for detailed information
to be extracted from the raw data such as; ranges, times, frequency, hourly approaches
and outliers which can be used to help expose what is occurring at a particular time, for
any particular length of time. This aids in the tracking of hypoxia and hyperoxia and
provides a better understanding of the patient’s current level of oxygenation. Using this
detailed level of data can provide a more comprehensive data set inference from than
just a single value, in a single time frame which, would lack the connectivity and
relevance of the oxygen saturation which occurred during the hour. Analyzing rapid
and wide fluctuations in an hourly dataset can provide extensive information for the
many neonatologists, residents, fellows, nurses, and respiratory therapists to draw from.
This has great potential to improve adjustments in titrating the transcutaneous blood
oxygen saturation levels at the bedside. The ability to accurately reflect the activity of
the hour and convert copious amounts of data into visual charts and graphs, provides a
better understanding of the information which can be passed on to another health
professional at the end of the shift as a clinical and educational tool.
The Artemis platform has the capability to support clinical research to quantify
the relationship between retinal exposure to oxygen and ROP more precisely and
accurately. There is also potential for the discovery of other relevant causative factors.
5. Conclusion
This paper has presented an exploratory study to compare the target range analysis for
blood oxygen saturation levels SpO2 when data is only recorded as top of the hour
readings verses analysis of the readings each second. The analysis was performed using
data collected via the Artemis platform. High fidelity blood oxygen saturation data
provides a better representation of retinal exposure to oxygen than infrequent,
intermittent spot readings. In future research publications we will report on our
research to study the association of the SpO2 values recorded in Artemis and the
association with ROP risk scores.
References
[1] CIHI, Highlights of 2009-2010 selected indicators describing the birthing process in Canada. [cited May
12, 2011], Available from: www.cihi.ca.
[2] CIHI, Too early, too small: a profile of small babies across Canada.[ cited 2009]. Available from:
https://secure.cihi.ca/free_products/too_early_too_small_en.pdf.
[3] K. Johnson, S.D. Scott, K.D. Fraser, Oxygen use for preterm infants. Factors that may influence clinical
decisions surrounding oxygen titration. Advances in Neonatal Care 11, 1 (2011), 8-14.
[4] D.K. Wallace, K.A. Veness-Meehan, W.C. Miller, Incidence of severe retinopathy of prematurity before
and after a modest reduction in target oxygen saturation levels. . American Association for Pediatric
Ophthalmology and Strabismus 11 (2) (2007), 170-174.
[5] K.W. Wright, D. Sami, L. Thompson, R. Ramanathan, R. Joseph, S. Farzavandi, A physiologic reduced
oxygen protocol decreases the incidence of threshold retinopathy of prematurity. Trans Am Ophthalmol
Soc 104 (2006), 78-84.
[6] J.S. Greenspan, J.P. Goldsmith, Oxygen therapy in preterm infants: Hitting the target. Pediatrics 118, (4)
(2006) 1740-1741.
J. Cirelli et al. / Analysis of Continuous Oxygen Saturation Data for Accurate Representation 131
[7] S. Vanderveen, T. Mansfield, E. Eichenwald, Lower oxygen saturation alarm limits decrease the severity
of retinopathy of prematurity. American Association for Pediatric Ophthalmology and Strabismus 10
(2006), 445-448.
[8] J.I. Hagadorn, A.M. Furey, T. Nghiem, , C.H. Schmid, D.L. Phelps, D.M. Pillers, Achieved versus
intended pulse oximeter saturation in infants born less than 28 weeks’ gestation, Pediatrics 118(4)
(2006), 1574-82.
[9] M. Blount, M. Ebling, M. Eklund, A. James, C. McGregor, D. Sow, Real-time analysis for intensive
care, IEEE Engineering in Medicine and Biology Magazine, March/April (2010), 110-118.
132 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-132
Introduction
1
Corresponding Author: Janessa Griffith, School of Health Information Science; Email:
[email protected]
J. Griffith et al. / Tailoring Decision Support to Suit User Needs: A Diagnostic Imaging Example 133
1. The Prototype
In this paper we describe a DI DSS prototype. In the prototype, a tab on the horizontal
menu bar on the electronic health record supports physician navigation of the DI
portion of the system. By scrolling over this tab, physicians can access previous DI
examinations through the picture archiving and communications system, view current
guidelines, or order DI. When proceeding to order DI, physicians/nurse practitioners
are presented with a series of closed-ended questions in order to provide end-users with
the most relevant decision support. After answering these questions, the system
displays the recommendation. This feature is modeled after a DSS used at a hospital in
the US that produces a utility score to indicate the appropriateness of a DI examination
[7].
After a recommendation is produced in the DSS prototype, the physician or nurse
practitioner can accept or override the recommendation. If the recommendation is
rejected, the end-user must provide rationale in a free-text box. If the recommendation
is accepted, the physician may proceed to order the DI. Not only does the DSS suggest
the most suitable type of DI, it also recommends an urgency classification that is used
for booking the appointment. The order entry component of this DSS suggests multiple
dates and locations for the appointment. The patient will leave their consultation with a
confirmed DI examination appointment.
Based on findings from a previous study [see 4], and from a literature review, features
emerged that could be considered for inclusion in future DSS development. These
features included questioning the utility of the DI examination, increasing
communication between referring physicians/nurse practitioners and radiologists, and
134 J. Griffith et al. / Tailoring Decision Support to Suit User Needs: A Diagnostic Imaging Example
displaying previous DI examinations (or orders that are scheduled to take place) to
avoid duplicate orders. Other features include presenting a patient’s cumulative
radiation exposure, and having resources for information support to guide physicians
through challenging clinical scenarios. Overall, results from our previous work [see 4],
formed the basis for the features included in the DSS prototype described below.
In research conducted by the author [see 4], physician study participants suggested that
there is a need to increase opportunities for physician communication when completing
DI requisitions. Providing physicians with opportunities to communicate could help to
promote more appropriate DI ordering. More specifically, participants noted the
importance of providing sufficient information on the requisition; however, some
requisition formats (paper and electronic) restricted the amount of wording referring
physicians could enter. Thus, this prototype includes requisition order entry features
with a large free-text space for additional comment. Physicians can click a button if
they require more space than is provided (see Figure 1).
In previous research work [4], physician participants were asked how they determine if
a DI examination has already been performed or scheduled to take place. Interestingly,
67% expressed they would ask the patient directly [4]. The remaining participants
described using an electronic system to determine if the examination had been
performed or scheduled, but could only do this within their jurisdiction [4]. Thus, the
prototype includes a pop-up alert to warn physicians if a duplicate order has occurred.
This alert includes the date and type of DI examination. The alert does not cancel the
requisition being placed, as some duplicate orders may be necessary.
Certain DI examinations expose patients to radiation (for example, X-ray, CT, and
PET). For example, a CT examination of the abdomen or pelvis exposes a patient to
approximately 4.5 years worth of natural background radiation [3]. As noted previously,
J. Griffith et al. / Tailoring Decision Support to Suit User Needs: A Diagnostic Imaging Example 135
physicians may ask their patients directly to determine if previous imaging had been
performed. Having this information available could help physicians make more
informed decisions and promote patient safety.
Inappropriate DI ordering places stress on human, and financial resources and raises
patient safety concerns. Several Canadian DI DSS pilot studies have demonstrated
there is an interest in utilizing this technology to promote more appropriate ordering.
Results from previous research and a literature review formed the basis for the
development of this DSS prototype. The prototype included features such as having the
end-user identify the utility of the DI examination as well as increasing opportunities
for communication between referring professionals (physicians and nurse practitioners)
and radiologists. In order to prevent repeated DI ordering, this DSS prototype displays
136 J. Griffith et al. / Tailoring Decision Support to Suit User Needs: A Diagnostic Imaging Example
examinations that have already been performed or are scheduled to take place. The
prototype also displays the cumulative DI-related radiation exposure a patient has
received to help physicians make more informed decisions. As well, the top reported
methods of information support used by referring physicians (i.e. using UpToDate®,
Google®, specialty journals, and contacting other physicians) were incorporated into
the prototype to support decision making involving challenging cases. As inappropriate
DI ordering raises concerns for patient safety and places a strain on human and
financial resources, investigating ways to reduce such ordering has become critical. As
described in this paper, based on interviews from our previous study, and from a
literature search, several themes arose that influenced the introduction of specific
features into this DSS prototype. Future work will focus on the usability testing of the
prototype described in this paper.
References
[1] Canadian Association of Radiologists. Do you need that scan? [updated 2005], [Cited Aug 8, 2012],
Available from: http://car.ca/uploads/news%20publications/car_cat_scan_eng.pdf.
[2] Canadian Association of Radiologists. As many as 30% of diagnostic imaging procedures deemed
unnecessary: Canadian Association of Radiologists helps physicians choose best test first, [updated
2010], [Cited Nov 3, 2010], Available from:
http://www.car.ca/uploads/news%20publications/car_news_release_20100927.pdf .
[3] Canadian Association of Radiologists, Diagnostic imaging referral guidelines: A guide for physicians,
1st ed. Saint Laurent, Quebec, 2005.
[4] Griffith JK. Diagnostic imaging ordering practices by referring physicians: A qualitative approach
[thesis]. Victoria (BC); University of Victoria; 2012.
[5] Health Council of Canada. Decisions, decisions: Family doctors as gatekeepers to prescription drugs
and diagnostic imaging in Canada, [updated 2010], [Cited 2012 Sep 5]. Available from:
http://healthcouncilcanada.ca/docs/rpts/2010/HSU/Decisions.
[6] Atlantic Health Sciences Centre, Canadian Association of Radiologists, Medicalis. Demand-side
control of diagnostic imaging through electronic clinical decision supports: A pilot using
appropriateness guidelines. [updated 2005], [Cited Oct 6, 2010]. Available from:
http://www.car.ca/uploads/nb_approp_051005_finaldraft.pdf .
[7] D. Rosenthal, J. Weilburg, T. Schultz, J. Miller, V. Nixon, K. Dreyer, J. Thrall, Radiology order entry
with decision support: Initial clinical experience, JACR 3 (2006), 799-806.
A. Bautista, J. Amorosa, A. Burgos, B. Nickel, A. Tilara, J. Yoon. Do clinicians use the American
College of Radiology Appropriateness Criteria in the management of their patients? Am J Roentgenol
192 (2009), 1581-1585.
Enabling Health and Healthcare through ICT 137
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-137
Introduction
1
Corresponding Author: Erin Roehrer, School of Computing & Information Systems, University of
Tasmania, Private Bag 87, 7001, Hobart, Tasmania.E-mail: Erin,[email protected]
138 E. Roehrer et al. / Supporting Cystic Fibrosis with ICT
Tasmanian Community Fund Project (myCF pilot). Each project is discussed from a
health informatics perspective.
1. Pathways Home
Pathways Home for Respiratory Illness was a joint research project involving the
Schools of Medicine, Nursing and Midwifery, and Computing and Information
Systems at the University of Tasmania. The project was funded by the Commonwealth
Department of Health and Ageing, under the Australian Health Care Agreement.
A major aim of the Pathways Home project was to investigate interventions that
offered the potential to facilitate the development of self-management skills and self-
management self-efficacy in people with cystic fibrosis (CF). The project responded to
a range of challenges presented to people with CF in Tasmania including: the high birth
incidence of CF in Tasmania; a geographically dispersed CF population receiving care
through regional healthcare facilities rather than via the conventional centralised CF
Centre care model and the resultant impact on regular, physical clinic visits; and the
fact that people with CF are surviving well into adulthood.
The CF arm of the Pathways project, which also assessed ICT support for older
patients with chronic obstructive disease (COPD)[5], aimed to investigate whether the
introduction of an ICT tool in CF, specifically a mobile phone application, could be
used for daily symptom monitoring to encourage the development of self-management
self-efficacy. The study explored whether individuals with CF would use the tool and if
its use would make a difference to their self-management behaviours and quality of life.
A total of 19 participants across Tasmania participated in the investigation. These
participants were randomised into 3 groups;
Intervention 1: Given access to health-mentoring to foster self-efficacy for
self management (face-to-face and via telephone).
Intervention 2: Given access to health-mentoring to foster self-efficacy for
self management (face-to-face and via telephone) and also provision of an ICT
supported self-monitoring program (via mobile phone).
Control Group: Continued to receive normal level of care.
The project ran for 6 months with a 6-month washout. Both qualitative and
quantitative data was collected at commencement; 3 and 6 months; and 6 months post
completion. The initial evaluation of the results of Pathways Home indicated that there
was sufficient evidence to undertake further evaluation of these intervention types. The
project also demonstrated that mentoring did assist in developing self-efficacy for self-
management behaviours, but the pilot study was too small to make definitive
conclusions on the ICT arm, but the feasibility of this approach was confirmed [5].
The second project was funded by the Australian Cystic Fibrosis Foundation and
extended the Pathways Home project that had been conducted in adults into
adolescents and older children with CF. The project was conducted in Queensland
through collaboration with the Royal Children’s Hospital and their CF outreach service.
This project examined two self-management related strategies, task-specific self-
E. Roehrer et al. / Supporting Cystic Fibrosis with ICT 139
3. myCF
4. Discussion
The succession of three projects building on each projects’ findings enables the
research team to ensure the myCF project takes its place as a translational research
project in the model of CF care in Tasmania. The feedback from the series of projects
suggested that not only was online symptom monitoring feasible but that it is most
likely to be beneficial when combined with access to health-mentoring opportunities
and educational resources. The progression from single platform offerings to that of a
platform (myCF) with multiple functions was aided by knowledge from the previous
projects. The myCF project pilot additionally combined the web platform and ability to
be mobile in order to overcome some of the limitations highlighted by participants
from the sole use of the mobile phone or PC.
E. Roehrer et al. / Supporting Cystic Fibrosis with ICT 141
The myCF project pilot was trialled by representatives of the three generations;
older children, adolescents and adults. This application allowed for a holistic
representation of the CF community, removing the adult only or adolescent only focus
of the previous two projects. The recruitment strategy reflects the applied nature of
research into practice of the myCF project pilot.
The evolution of Pathways Home, CFA Project and myCF represents a natural
progression and the challenge now is to determine which components of the myCF
project pilot are of most benefit and why. By monitoring usage of myCF we will be
able to explore how patients and their families interact with particular components of
the ICT tool and gain better understanding of how myCF may be used to support
chronic disease self-management.
The size of the sample populations and at times difficulty with ensuring
sustainable use of the IT interventions were some of the limitations of these projects.
This paper does not explore the findings of each project in detail, rather presents an
overview of each project to highlight the method of building on the previous projects.
5. Conclusions
ICT use in chronic disease support has increased in use from a health consumer’s point
of view. This paper follows the journey of three projects involving clinicians and
eHealth researchers that explored health mentoring supported in some cases by ICT
with the aim of improving self-management and self-monitoring activities for those
with chronic disease, in particular, cystic fibrosis.
The potential for ICT use in self-monitoring, particularly using mobile devices, has
been highlighted by all three projects. The myCF pilot combines an on-line symptom
diary with educational material and peer support with the aim of improving care for
people with cystic fibrosis who live remote from specialist CF services such as the
patient population residing on the island state of Tasmania.
References
[1] K. Akesson., B. Saveman & G. Nilsson. Health care consumers’ experiences of information
communication technology – A summary of the literature, International Journal of Medical Informatics.
2007; 76: 633-645.
[2] D. Dorr., L.M. Bonner., A.N. Cohen., R.S. Shoai., R. Perrin., E. Chaney & A.S. Young. Informatics
Systems to Promote Improved Care for Chronic Illness: A Literature Review, Journal of the American
Medical Informatics Association. 2007; 14: 156-163.
[3] M.R. Solomon. Information Technology to Support Self-Management in Chronic Care, Disease
Management and Health Outcomes. 2008; 16: 391-401.
[4] B. Celler., N. Lovell & J. Basilakis. Using information technology to improve the management of
chronic disease, MJA. 2003; 179: 242-246.
[5] E.A. Cummings, J. Hauser, H.L. Cameron-Tucker, P. Fitzpatrick, M. Jessup, H. Walters, D.W. Reid & P.
Turner. Enhancing Self-Efficacy for Self-Management in People with Cystic Fibrosis. Studies in Health
Technology and Informatics. 2011; 169(1): 33-37.
This page intentionally left blank
Human Computer Interaction
This page intentionally left blank
Enabling Health and Healthcare through ICT 145
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-145
Introduction
The creation of health information silos, and the generation of thousands of Electronic
Healthcare Record (EHR), Electronic Medical Record (EMR) and Personal Health
Record (PHR) systems around the globe are seemingly unstoppable. Furthermore, the
explosion of information sharing using public social networking and related links is
accelerating in an ever-increasing rate. Recent statistics show that around 3.7 billion
people around the globe are using the Internet [1] (this is >50% of world population),
while 40 billion photos are deposited [2] in Facebook. An important aspect of capturing
the positive impact of such developments is to integrate those social activities with
health information exchange (HIE), a powerful tool [3] yet to be realised fully. Health
information sharing in a general context is not new, however, in the digital world, this
increasing social interaction demands further scrutiny for several reasons. Health
1
Corresponding Author: Dr. Tony Sahama, QUT. Email: [email protected]
146 T. Sahama et al. / Information Accountability and Usability: Are There Any Connections?
1. Information Accountability
2. Usability
Broadly speaking, usability can be defined as a measure of ease of use and usefulness
of an information system in terms of its: (1) effectiveness, (2) efficiency, (3)
enjoyability, (4) learnability, and (5) safety [8]. These attributes or dimensions of
usability are useful in focusing attention on key aspects of the use of systems in the
design and evaluation of health information systems. For example, to lead to uptake
and adoption by end users, health information systems need to both effective and
T. Sahama et al. / Information Accountability and Usability: Are There Any Connections? 147
In considering the broad definition of usability given above, it is argued in this paper
that adding information accountability to the list of attributes/dimensions of the concept
of “usability” could practically lead to improved systems and consequently more
effective user adoption of HIT. From a practical point of view, studies can be designed
to assess the following from the perspective of end users interacting with HIT: (1)
effectiveness, (2) efficiency, (3) enjoyability, (4) learnability, (5) safety, and (6)
information accountability (this non-functional requirement is a critical consideration
for HIT applications that adequately protect information privacy and security). This
will be essential as systems that have been deemed to have met requirements of the first
5 attributes above may still not be accepted by end users without explicit consideration
of making information accountable to all classes of end users. Nowhere is this more
important than in designing systems intended for end users who are patients or citizens,
as described in the next section.
148 T. Sahama et al. / Information Accountability and Usability: Are There Any Connections?
6. Discussion
References
[13] Liang, J., Iannella, R. and Sahama, T. (2011). Synchronised Integrated Online e-Health Profile. Studies
in Health Technology and Informatics, 168; 96—103.
[14] Leap, L.L. and Berwick, D.M. (2005). “Five Years After to Err is Human: What Have We Learned?”,
JAMA, 293(19):2384—90.
Enabling Health and Healthcare through ICT 151
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-151
1. Introduction
Information systems have been designed to increase the efficiency and safety of
healthcare processes. The predominant paper technology is inadequate for meeting the
needs of modern healthcare [1]. Mobile devices, such as smartphones and tablets, are a
pervasive technology. They can facilitate messaging, time management, education and
telephony. They also have the ability to easily retrieve and run health-related software
‘apps’ from online repositories. Consequently, mobile devices are now capable of
many assistive roles, particularly the ability to display and manipulate medical
information in clinical settings. In Canada, health information systems (HIS) adoption
rates lag behind those of other countries [2]. Considering the low HIS adoption rates
and the rapid mobile device adoption rates, it is not unreasonable to suggest that
adoption of these pervasive devices will outpace the adoption of HIS systems – if they
have not already done so. The issue for safety is the evidence that new technologies,
despite their clear advantages over paper technology, can be the root cause for new
1
The author may be contacted at [email protected]
152 S. Minshall / A Review of Healthcare Information System Usability & Safety
kinds of errors. Research in this area has indicated that the design of new technology,
in particular the user interface (UI), may be related to increased incidence of new types
of error [3], [4]. In this paper, the relationship between human factors and usability
will be considered in the context of designing safe and effective healthcare applications,
with a focus on computing devices.
2. Methods
Medline was searched for the specific terms listed below and restricted to the date
ranges 2006-01-01 through to 2011-03-03:
1. (error AND technology AND human factors)
2. (error AND (CPOE OR
(Computerized AND provider AND order AND entry)))
3. (Technology AND Induced AND Error)
3. Findings
Table 1. The Medline search resulted in 239 papers. After reviewing the title and abstract, 186
papers were rejected and 51 papers remained for review.
Inclusion Criteria Title Exclusion n Abstract Exclusion n
User Errors Not data-related 37 Wrong device 20
Data entry Other tools 27 Not medical app 17
Data access Tablet is medication 16 Not clinician 9
Computing Surgical devices 13 Review off topic 17
HCI Not medical 5 Inaccessible 4
User interfaces Education 3 Duplicates 4
Usability Workflow 3 Education 13
In English
Remaining 239 135 51
Table 2. The remaining papers were classified into the following sets. Countries abbreviated as per ISO
3166 standard
Paper Type n Research n Technology n Country n
Critical Reviews 1 Case Studies 4 CPOE 27 US 21
Lit. Reviews 10 Interventions 10 Devices 2 CA 10
Methods 10 Observations 3 HIS 18 NL 9
Research 30 Prospective 1 Paper 3 AU 4
Retrospective 4 Workflow 1 DK 2
Simulations 6 IL, FR 2
Surveys 2 IR, IT 2
LB 1
Column Totals 51 30 51 51
S. Minshall / A Review of Healthcare Information System Usability & Safety 153
CPOE themed papers in this review demonstrated that this specific IT implementation
caused the following: a positive effect on safety, particularly when coupled with a
decision support system (DSS) [5], increased compliance and improved legibility [6].
A review by Reckmann reported that between 1998 and 2007, the evidence that
CPOE interventions reduced errors is limited and not compelling [7]. Another review
by Weant [8] reported that errors increased during a paper to CPOE transition. Further,
inappropriate alerting, alert fatigue, inappropriate dose defaults, workflow disruption
and multitasking effects experienced and measured after CPOE implementation, all
contributed to a reduction of patient safety. Errors caused by features unique to the IT
implementation can be considered technologically induced errors. Many of the papers
call for further investigation and ask new research questions: What cognitive processes
contribute to alert overriding? Can heuristics be used to detect prescription errors?
Can frequently overridden alerts be safely disabled?
methods and a human factors approach to evaluate systems and processes that lead to
diagnostic and medication error. The resulting protocol describes methods adapted
from human factors and psychology to analyze the ways in which providers currently
use CPOE to communicate and identify barriers to effective electronic communication
[11]. Fieldwork by the foregoing researchers has provided valuable insight into the
nature of teamwork paper-use and the introduction of new technology.
Many more studies advocated and used simulation methods in their investigations.
Lilholt, in a paper responding to published concerns about applications being
unsuitable for clinical use, devised and tested a usability evaluation to investigate
problems in an EHR system by combining methods from laboratory tests and field
studies [12] and applying them in a realistic ward-simulator using an actor to play the
part of a patient. Lilholt concluded that some of the usability issues encountered only
manifested due to the realistic setting, suggesting that if the scenario were not realistic,
some usability issues would not have been revealed. Papers reporting high quality
simulations seemed to have fewer stated limitations seemed more effective and perhaps
left less room for doubt or criticism due to the effects of a simulated environment.
Although simulations may seem limited by their implicit lack of realism, their safety-
related use in other industries, notably aviation and nuclear power control, is
widespread.
Usability tests are performed to analyze a system that consists of three parts: hardware,
software and the users. Papers from studies of human factors – beyond the scope of this
review, but still relevant – show that older and younger adults use the same tools in
different ways, suggesting that life experience and risk-taking play a part in the
cognitive processes of decision-making [16]. People vary in ability, skill, familiarity,
competence, confidence and style. According to the reviewed papers, many clinical
tool devices and systems do not offer any adaptation to users’ variability. User
interfaces are criticized for inflexible login modes, poor medication models, rigid
controls, inflexible processes, ordering formats, rigid design and rigid workflows [17].
Examining a tool for usability can reveal these issues and potentially predict the
likelihood for and magnitude of technology-induced error.
The need for more granular usability studies is evident in the reviewed papers.
Buchel found “that devices which are already on the market differ drastically in their
usability [… and] that unsatisfying usability bears a high risk in health care.” [18].
That one device proves to be usable, does not mean that another similar device or its
software, is automatically similarly usable. One cannot state with confidence that
because a particular EHR app is deemed ‘usable’ on one tablet computer with a touch
screen as its main input device, will be just as usable on a similar tablet that uses a
stylus. Ideally, every device-software combination should be vetted for its error-
inducing potential as part of the procurement process. New technology arrives
regularly in the form of mobile hardware and cross-platform software. Improvements
and major changes to both are also frequent and can come with radical user interface
changes that fundamentally alters the way the device-software is used – rendering any
previous usability analysis, published or unpublished, as superfluous.
The variety of mobile devices is increasing. The need for either more usability
assessments or tighter restrictions on their use in the clinical setting will be needed to
measure and control patient safety issues.
4. Conclusion
New technology, such as CPOE, offers improvements over traditional paper tools and
it is shown to have a positive effect on patient safety. New technology also created the
opportunity for new errors to occur and led to the coining of the term technology-
induced error. The human factors of device and technology use, in particular
transitioning from the team-based use of traditional tools discussed in this review, show
that this area is under-investigated.
Fieldwork is essential to assess the validity of simulation studies and to gain
insight on the working practices of teams. Likewise, realistic simulations are useful to
demonstrate the minutiae of device use when an actor-patient is present. The bulk of
actual usability work is better done using simulation methods. The bulk of actual
simulations for usability research can be done using low-cost methods. This is
important because of the great volume of testing needed due to the ubiquity of mobile
devices and access to clinical and clinical-style software available to owners of these
devices. Ideally, each device-software pair should be assessed for its error-inducing
properties. To cope with the increasingly rapid growth of this area of technology more
usability testers and facilities will be required. Further research-methods studies and
156 S. Minshall / A Review of Healthcare Information System Usability & Safety
evidence-based metrics are needed to generate more efficient and simpler usability
protocols. The scope of this usability-testing problem is larger than it may seem given
the current proliferation and global distribution of new, sophisticated mobile devices
and un-vetted access to health-related software.
References
[1] Shortliffe E, Blois M. The computer meets medicine and biology: emergence of a discipline.
Biomedical Informatics, 2006; 1: 3–45.
[2] CHI Annual Report 2009-2010. Canada Health Infoway, May 2010: 1–48.
[3] Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimme SE, Strom BL. Role of computerized
physician order entry systems in facilitating medication errors. JAMA, 2005; 293(10): 1197–1203.
[4] Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability:
the relationship between usability problems and prescription errors when using a handheld application,
Int J Med Inform, 2005;74(7): 519–526.
[5] Kuperman GL, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, Classen DC, Bates DW.
Medication-related clinical decision support in computerized provider order entry systems: a review, J
Am Med Inform Assoc, 2007;14(1):29–40.
[6] Pirnejad H, Niazkhani Z, van der Sijs H, Berg M, Bal R. Evaluation of the impact of a CPOE system on
nurse-physician communication--a mixed method study, Methods Inf Med, 2009;48(4):350–360.
[7] Reckmann MH, Westbrook JI, Koh Y, Lo C, Day RO. Does computerized provider order entry reduce
prescribing errors for hospital inpatients? A systematic review., J Am Med Inform Assoc, 2009;16(5):
613–623.
[8] Weant KA, Cook AM, Armitstead JA. Medication-error reporting and pharmacy resident experience
during implementation of computerized prescriber order entry, American Journal of Health-System
Pharmacy, 2007;64(5):526–530.
[9] Saleem JJ, Russ AL, Justice CF, Hagg H, Ebright PR, Woodbridge PA, Doebbeling BN. Exploring the
persistence of paper with the electronic health record, Int J Med Inform, 2009;78(9):618–628.
[10] Miller A, Pilcher D, Mercaldo N, Leong T, Scheinkestel C, Schildcrout J. What can paper-based
clinical information systems tell us about the design of computerized clinical information systems (CIS)
in the ICU?, Aust Crit Care, 2010;23(3):130–140.
[11] Hysong SJ, Sawhney MK, Wilson L, Sittig DF, Esquivel A, Watford M, Davis T, Espadas D, Singh H.
Improving outpatient safety through effective electronic communication: a study protocol., Implement
Sci, 2009;4:62.
[12] Lilholt LH, Pedersen SS, Madsen I, Nielsen PH, Boye N, Andersen SK, Nøhr C. Development of
methods for usability evaluations of EHR systems, Stud Health Technol Inform, 2006;124:341–346.
[13] Kushniruk AW, Borycki EM, Anderson JG, Anderson MM. Preventing technology-induced errors in
healthcare: the role of simulation, Stud Health Technol Inform, 2009;143:273–276.
[14] Carvalho CJ, Borycki EM, Kushniruk AW. Using heuristic evaluations to assess the safety of health
information systems, Stud Health Technol Inform, 2009;143:297–301.
[15] Koppel R, Leonard CE, Localio AR, Cohen A, Auten R, Strom BL. Identifying and quantifying
medication errors: evaluation of rapidly discontinued medication orders submitted to a computerized
physician order entry system, J Am Med Inform Assoc, 2008;15(4):461–465.
[16] Ezer N, Fisk AD, Rogers WA. Age-related differences in reliance behavior attributable to costs within a
human-decision aid system, Human Factors: The Journal of the Human Factors and Ergonomics
Society, 2008;50(6):853–863.
[17] Borycki EM, Kushniruk AW, Keay E, Kuo A. A framework for diagnosing and identifying where
technology-induced errors come from. Stud Health Technol Inform, 2009;148:181.
[18] Büchel D, Baumann T, Matern U. Usability of radio-frequency devices in surgery, HCI and Usability
for Medicine and Health Care, 2007: 97–104.
Enabling Health and Healthcare through ICT 157
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-157
Abstract. The need for improved usability in healthcare IT has been widely
recognized. In addition, methods from usability engineering, including usability
testing and usability inspection have received greater attention. Many vendors of
healthcare software are now employing usability testing methods in the design and
development of their products. However, despite this, the usability of healthcare IT
is still considered to be problematic and many healthcare organizations that have
purchased systems that have been tested at vendor testing sites are still reporting a
range of usability and safety issues. In this paper we explore the distinction
between commercial usability testing (conducted at centralized vendor usability
laboratories and limited beta test sites) and usability testing that is carried out
locally within healthcare organizations that have purchased vendor systems and
products (i.e. public “in-situ” usability testing). In this paper it will be argued that
both types of testing (i.e. commercial vendor-based testing) and in-situ testing are
needed to ensure system usability and safety.
Introduction
The usability of healthcare information systems has been recognized as being critical to
the successful deployment, adoption and appropriate use of these technologies. Over
the past several decades a wide range of usability engineering projects, efforts and
laboratories have emerged to address these issues. However, despite these efforts, the
usability of healthcare IT (HIT) is still reported as being problematic, particularly when
commercial vendor systems, such as electronic health record (EHR) and electronic
medical record (EMR) systems are deployed in offices, clinics and hospital settings.
The reporting of a considerable number of usability problems and issues with these
systems is not just localized to Canada but is an international phenomena [1]. Many
vendors and developers of healthcare software are now applying usability engineering
methods, such as usability testing. However, despite this, the usability of HIT is still
considered to be problematic and many healthcare organizations that have purchased
systems that have been tested at vendor testing sites are continuing to report a range of
1
Corresponding author: Andre W. Kushniruk: [email protected]
158 A.W. Kushniruk et al. / Commercial Versus In-Situ Usability Testing
usability and safety issues [2]. In this paper we explore the distinction between
commercial usability testing (conducted at centralized vendor usability laboratories and
limited beta test sites) and usability testing that is carried out locally within healthcare
organizations that have purchased vendor systems and products (i.e. public “in-situ”
usability testing). It will be argued that both types of testing (i.e. commercial testing)
and in-situ testing are needed to ensure system usability and safety. It will further be
argued that on-site in-situ usability testing can provide organizations who have
purchased vendor HIT systems with critical feedback for: (a) customization of those
systems to improve usability, (b) improved user training, and (c) evidence to present
back to vendors in cases where usability is deemed to be extremely poor, leading to
potentially “unsafe” installations of HIT. It is also argued that there is a need for closer
collaboration between vendor usability efforts and efforts to improve usability that are
spearheaded at local healthcare institutions and which purchase HIT from vendors.
In this part of the paper we will describe the low-cost rapid usability engineering
approach we have developed and refined that can be applied within local healthcare
organizations (e.g. medical offices, clinics and hospitals) for doing usability testing.
We will refer to this approach as “in-situ”, as it is conducted in the real-life settings in
which healthcare IT is deployed (e.g. electronic health record deployment within the
hospital setting the system will be implemented in). Figure 1 shows a continuum from
vendor-based testing (conducted at vendor usability labs or beta test sites) to “public”
in-situ usability analyses carried out at the actual institutions where the healthcare IT is
being deployed. We have found that in-situ usability testing can be carried out
effectively by IT staff or health informaticians at local healthcare organizations without
requiring extensive training in the methodology, as the approach has been streamlined
for deployment for a variety of settings and locations both nationally and
internationally.
A.W. Kushniruk et al. / Commercial Versus In-Situ Usability Testing 159
The applications evaluated range from the evaluation of a medication order entry
system (using bar-coding technology) prior to its deployment in a hospital in Japan [3]
to the study of an introduction of an EMR at major American medical center (involving
in-situ testing both before and after system going live) [4].
Applying our approach to in-situ usability testing, our work is typically carried out
in real clinical settings (e.g. a physician’s office, in a clinic or even in an operating
room after hours). The approach typically involves asking between 8-10 subjects (e.g.
nurses or physicians) to interact with a system under study (e.g. an EMR) to carry out
tasks (e.g. to enter medications from a medication list). In addition, subjects may also
be asked to “think aloud” while carrying out the task, which is audio recorded. Overt
physical activities of the subjects (e.g. checking information in the computer against
paper records on their desk) are recorded using one or more low-cost portable digital
cameras. The computer screens are also recorded as a digital movie, using free screen
recording software installed on the computer running the system being tested (we are
currently using Hypercam©). When running this software (by simply plugging a
microphone into the computer) the result will be a movie of the user-computer
interaction with the audio track of the subject’s verbalizations (i.e. “thinking aloud”),
which can be played back for immediate informal review and analysis. Thus data
collected consists of digital movie files showing what users are doing on the computer
which are linked to audio recordings and which can also be linked to the external video
view of physical activities if needed. Using this approach we have carried out studies of
HIT in environments ranging from physician offices to patients’ homes. An extension
of the approach involves continuing to record users interacting with the system once it
is released for real use (obtaining of course, ethical approval).
In order to substantiate the benefits of this approach, recent work by Baylis,
Kushniruk and Borycki involved conducting a formal cost-benefit analysis of
conducting in-situ usability testing on a disease management software system to be
released throughout a healthcare system at a provincial level [5]. From this work, it was
found that in testing only a small subset of the system’s functionality the total cost
(including one-time cost of buying a laptop for testing purposes) was approximately
$8,000 with the most conservative estimate of benefit from this analysis being in the
range of $50,000 (and with the benefits considerably greater if the cost associated with
potential medical error is considered).
The analysis of the data collected from local in-situ usability testing can vary from
informal playback of recordings of user interactions (e.g. playing back digital movies
of user interactions with an EMR) at IT meetings, to more formal coding and analysis
of identified user problems and even statistical analysis. Most of our work involves an
analyst viewing the data and developing qualitative categories of problems (e.g.
navigational problems, problems users have in interpreting information presented and
other types of usability problems) and then quantifying the occurrence of the problems
(e.g. tabulating the frequency of the most common usability problems). The result of
this analysis can then fed directly back to the local system implementation team and
prioritized for remedial action (e.g. either through local customization, or feedback to
local user training). In cases where usability problems are deemed to be critical to fix at
160 A.W. Kushniruk et al. / Commercial Versus In-Situ Usability Testing
the local level, but the nature of the software (or the nature of the contract with the
vendor) precludes fixing at the local level, the data collected (and recommendations
made) need to be reported back to the commercial vendor for making more
fundamental changes to the software. Some of our work has involved applying low-
cost portable in-situ testing to identifying potential technology-induced errors that may
be caused by a system (e.g. inappropriate medication defaults in an order entry system),
or by poor user interface design [6]. It should be noted that by testing systems in their
real environment of use (as opposed to testing them in a vendor usability laboratory, or
only conducting usability testing at some beta test site), then problems and issues that
would only arise in the local setting (i.e. the hospital which purchased the system) can
be detected, analyzed and prioritized prior to system release. Along these lines, many
of the issues noted in the literature with the deployment of HIT are related to local
conditions, contexts and work environments, making this approach to usability testing
important to detect problems that would otherwise go unnoticed until a vendor system
is actually running live within a healthcare organization. This local testing is essential
for ensuring not just the usability but also for ensuring the safety of HIT deployed in
real-world contexts.
3. Experiences to Date
resultant high level of user uptake of the clinical guidelines (as determined from a
subsequent full scale trial that was conducted) [4].
4. Discussion
From our work we have found that many usability problems and errors that are
encountered in real clinical contexts are not detected or corrected by usability testing
conducted at centralized vendor usability laboratories prior to system release (or at only
selected beta test site healthcare organizations). We do feel that vendor-based testing
(including vendor usability laboratory testing, as well as regional centralized
conformance testing of vendor products) is necessary. However, it has been argued in
this paper that centralized private usability testing (such as that conducted by major
EHR vendors) needs to be complemented by “public” in-situ testing to ensure system
usability and safety. Current issues and challenges include getting permission to
conduct studies in a real environment and also obtaining rooms and locations after
hours for conducting in-situ usability testing. Furthermore, there is a need for improved
communication between healthcare institutions and vendors in order to allow for the
most serious usability and safety problems identified at the local level to be fed back
for software improvement at the vendor level. It is argued that if we are to ensure that
the systems we implement are both usable and safe, then these additional types of
studies (conducted in-situ at local healthcare organizations) are necessary and that they
can be carried out in a cost-effective and beneficial way to improve the safety and
efficiency of healthcare IT.
References
[1] Viitanen J, Hypponen H, Laaveri T, Vanska J, Reponen J, Winblad. National questionnaire study on
clinical ICT system. Int J Med Inform, 2011;80:708-725.
[2] Beuscart-Zephir M, Anceaux F, Meaux J, Degroisse M, Degoulet P. Impact of CPOE on doctor-nurse
cooperation for the medication ordering and administration process. Int J Med Inform, 2005;74(7):629-641.
[3] Borycki, E., Kushniruk, A., Kuwata, S., Kannry, J. Use of simulation in the study of clinician workflow.
AMIA Annual Symposium Proceedings, 2006, 61-65.
[4] Li, A., Kannry, J., Kushniruk, A., Chrimes, D., McGinn, T., Edonyabo, D., Mann, D. Integrating usability
testing and think-aloud protocol analysis with “near-live” clinical simulations in evaluating clinical
decision support. In press, Int J Med Inform, in press.
[5] Baylis T, Kushniruk A, Borycki E. Low-cost rapid usability testing for health information systems: Is it
worth the effort? Proceedings of MIE 2012, 363-367
[6] A.W. Kushniruk, M. Triola, E. Borycki, B. Stein, and J. Kannry, Technology induced error and usability.
Int J Med Inform, 2005, 74, 519-526.
[7] A.W. Kushniruk, V.L. Patel, J.J. Cimino, and R. Barrows, Cognitive evaluation of the user interface and
vocabulary of an outpatient information system. Proceedings of the 1996 Annual AMIA Conference, 1996,
22-26.
162 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-162
Abstract. This paper presents how a Human Factors (HF) engineering approach
has been applied to a European Project aiming at preventing Adverse Drug Events.
Methods used by HF experts and their main contributions are depicted. Main
lessons learned from HF involvement in this project are proposed in terms of
methods for ADE prevention interventions and project management.
Introduction
Human Factors (HF) activities are increasingly applied during the design of Health
Information Technology devices/software. Those activities consist in applying HF
expertise and methods all along the project and in involving representative end-users at
every step. Applying those methods supports a better consideration of HF issues in the
design, development and evaluation of devices. Ultimately it has a positive impact on
the acceptance of the product [1] and on the effectiveness of the product [2]. The HF
strategy is even recommended to comply with the European Medical Device Directive
[3] specifically to prevent the risk of use errors related to ergonomics features [4].
One of the most common HF strategies is the User-Centered Design process
(UCD) [5]. As described in Figure 1, this process includes four iterative HF activities
ranging from the analysis of the context of use and the specification of the users’
requirements to the design itself, the iterative evaluation of the solutions to ensure that
it meets users’ requirements and a long-term post-implementation monitoring. Besides,
to improve the performance of the user-centered part of the system development (e.g.
improving UCD process efficiency), a “process for usability” application is also
proposed [6]. This process is composed of 7 activities (Figure 2) starting with the
overall consideration of UCD process in the lifecycle (#1), then ensuring the
management of the UCD all along the process (#2), achieving all UCD tasks (#3 to #6)
and finally supporting the implementation of the product (#7).
1
Corresponding Author: [email protected]. Institut Hippocrate – CHRU de Lille, 2
avenue Oscar Lambret – 59037 Lille Cedex
R. Marcilly et al. / Integrating Human Factors in an International Research Project 163
Figure 1. The User-Centered Design process adapted from the ISO 9241-210 [5].
The paper at hand presents how the HF strategy, inspired by the HCL, has been
planned and conducted during a European Union project. Lessons learned from this
experience are presented.
The European Union project entitled "PSIP - Patient Safety through Intelligent
Procedures in Medication" aimed to (a) innovatively produce new knowledge on
Adverse Drug Events (ADE) (by data/semantic mining methods) and (b) investigate
different possibilities for reducing ADE (e.g. developing knowledge to be integrated
into Computerized clinical Decision Support System (CDSS) and solutions supporting
patients’ compliance) [7]. The entire project was considered as a patient-safety
intervention aiming at identifying the current barriers against ADE existing in hospital
settings and at reinforcing them through innovative technologies [8].
Table 1 describes the compliance of the PSIP project with HCL. Following the
requirements of the UCD process, HF experts worked tightly with researchers and
industrials in the shaping of the produced knowledge and in the design of the
applications displaying this knowledge to clinicians and patients.
Three teams of HF experts from three countries (Italy, Denmark, and France)
characterized by different background expertise worked together on the project in
cooperation with a Danish patient-safety unit.
Figure 2. Human-Centered Lifecycle (HCL) process description adapted from the ISO 18529 [6].
164 R. Marcilly et al. / Integrating Human Factors in an International Research Project
2. Methods
Main contributions of each HF team are presented in Table 2. Those methods were
applied all along the project for the HF patient-safety intervention and for the
development of innovative ADE prevention tools. Therefore representative end-users
and industrials have often been engaged (e.g. through focus groups, tests) all along the
project to optimize the intervention and the design process. Finally, HF experts worked
on a list of HF recommendations for the design of medication-related CDSS. Those
principles were proposed to industrials partners to assess their appropriateness and
technical feasibility.
Integrating HF expertise all along the project and at every level proved to be an
efficient approach. Participation in the management board is essential for the HF
approach and results to be actually integrated in the decision and development process.
3.1. ADE prevention requires complementary tools for the whole healthcare team
The multi-perspective approach allowed identifying that ADE prevention requires more
than a single tool addressed to a single healthcare professional. On the contrary, a set of
functions reinforcing the existing effective barriers developed by every professional is
necessary.
Table 2. Expertise fields and main contributions of each HF team. Tasks were often carried out in
cooperation.
HF teams Expertise Main contributions
Italy Italian’s expertise was oriented toward Analysis of the existing barriers of prevention
the error prevention through barriers against ADE in hospital setting based on
analysis and came from the aviation taxonomy of errors and work systems analysis
domain. [9].
Denmark Danish expertise was related to co- Design support and product evaluation based
design and user-driven design methods respectfully on co-design process [10] (e.g.
applied to medical devices and to the focus groups, prototyping) and simulation in a
evaluation within a full-scale medical full-scale simulator [11].
simulator.
Patient safety unit Validation of knowledge and of the
interventions in terms of patient safety risk.
France French team’s skills dealt with UCD process based on work system analysis,
usability, UCD process and cognitive end-users involvement, usability inspection and
ergonomics. test supported by usability goals [12].
R. Marcilly et al. / Integrating Human Factors in an International Research Project 165
Table 3. Functions required to reinforce the existing barriers and corresponding developed tools.
Required functions Developed tools
Raise the ADE awareness of every ADE-scorecards tool that allows clinicians getting an overview
involved clinician (physicians, of the incidence and quality of ADE in their own wards.
pharmacists and nurses)
Provide them with information at the CDSS integrated in Clinical Information System (CIS) or
point of care Computerized Prescription Order Entry (CPOE) informing
clinicians in real time of the risk of ADE related to ordered
meds.
Allow clinicians testing orders apart Standalone Intranet CDSS available to every clinicians apart
from their workflow from their workflow to simulate an order
Provide patients with medical Mobile patient component participating with clinicians to
information to support their patients’ awareness of ADE risks due to their medications.
compliance
This set is described in Table 3. Therefore, to optimally support ADE prevention,
designed tools should be clinicians’ partners, i.e. interacting with clinicians taking into
account their actions, and also team players, i.e. supporting the cooperation between
every healthcare professional involved.
Confronted to industrials’ technical proposals, those functions were developed
through four kinds of tools (Table 3). This set of tools was designed for every
healthcare professional involved in the medication use process (physicians, pharmacist,
nurses etc.). Hence, they are not workflow-dependent and may be implemented in
different kinds of work organizations.
Following a safety-oriented UCD process allows assessing the usability level of the
different tools and whether their usability features may cause use errors. Therefore
sufficiently usable tools without usability flaws that may cause use errors can be
identified and released.
During the project, the different tools did not achieve the same usability maturity.
At the end of the project, the patient component still faced remaining usability flaws
along with knowledge display issues. Those problems may have caused use errors.
Therefore, this tool was not released and researches about how to provide patients with
medical information had to go on. As for the integrated tools, they were more difficult
to evaluate than the standalone ones because integrated CDS functions inherit the
usability features from CPOE/CIS. Thus it was not obvious distinguishing the proper
CDSS usability features from the main system ones. At the end of the project,
improvements still had to be done on them to ensure that no usability flaws would
cause use errors. Finally, the standalone CDSS and the ADE-scorecards were assessed
as sufficiently usable and use error-prone-free and were released. The ADE-scorecards
were even installed in test wards of a northern France hospital for an impact study [13].
The tight cooperation between HF experts and industrials (knowledge and CIS/CPOE
vendors) in shaping of the knowledge and designing the tools convinced the industrials
of the usefulness of the HF approach: they are still working with PSIP HF experts for
the PSIP follow-up and they integrate now HF engineering in their technological
innovations projects. From the HF perspective, based on industrials’ collaboration, the
166 R. Marcilly et al. / Integrating Human Factors in an International Research Project
appropriateness and the feasibility of formulated principles for the design of CDSS
were validated [14] improving their usefulness.
4. Conclusion
The PSIP project ended-up successfully one year ago. The positive dynamic it created
amongst participants (industrials and health institutions) still continues. The ADE-
scorecards tool is well-received by users and health institutions. It is still installed in
the test wards to continue the impact evaluation in terms of patient safety on a longer
duration. Several hospitals are asking for roll out.
Acknowledgment
The research leading to these results has received funding from the European
Community’s Seventh Framework Program (FP7/2007-2013) under the Grant
Agreement n°216130-the PSIP project.
The authors would like to thank every Human Factors expert whose involvement
made completion of this project possible: C. Nøhr, M.B. Christiansen, A.M. Kanstrup,
K. Lawton, P. Skjødt, S. Jensen, C. Binzer, N. Leroy, S. Pelayo, S. Bernonville, C.
Riccioli & C. Cacciabue.
References
[1] K.A. Thursky, M. Mahemoff, User-centered design techniques for a computerised antibiotic decision
support system in an intensive care unit, Int J Med Inform 76 (2007), 760-768.
[2] J. Chan, K.G. Shojania,A.C. Easty, E.E. Etchells, Does user-centred design affect the efficiency, usability
and safety of CPOE order sets? J Am Med Inform Assoc 18 (2011), 276-281.
[3] Council Directive 2007/47/EC, Council Directive 2007/47/EC, (2007).
[4] International Electrotechnical Commission. Medical Devices - Application of usability engineering to
medical devices (Rep. No. ISO 62366), Geneva: International Electrotechnical Commission; 2008.
[5] International Standardization Organization, Ergonomics of human system interaction - Part 210: Human
centred design for interactive systems (Rep N°9241-210), International Standardization Organization,
Geneva, 2010.
[6] International Standardization Organization, Ergonomics - Ergonomics of human-system interaction -
Human-centred lifecycle process descriptions (ISO/TR 18529). International Standardization
Organization, Geneva, 2000.
[7] R. Beuscart, P. McNair, J. Brender, Patient safety through intelligent procedures in medication: the PSIP
project. Stud Health Technol Inform 148 (2009), 6-13.
[8] M.C. Beuscart-Zephir, N. Nohr, Human factors engineering for computer-supported identification and
prevention of adverse drug events. Stud Health Technol Inform 148 (2009), 14-24.
[9] P.C. Cacciabue, G. Vella, Human factors engineering in healthcare systems: the problem of human error
and accident management, Int J Med Inform 79 (2010), 1-17.
[10] A.M. Kanstrup, C. Nøhr, Gaming against medical errors: methods and results from a design game on
CPOE, Stud Health Technol Inform 148 (2009), 188-196.
[11] E. Ammenwerth, W.O. Hackl, K. Binzer, T.E. Christoffersen, S. Jensen, K. Lawton, P. Skjoet, C. Nohr,
Simulation Studies for the evaluation of health information technologies: experiences and results, HIM
Journal 41(2012), 14-21.
[12] R. Marcilly, S. Bernonville, C. Riccioli, M.C. Beuscart-Zephir, Patient safety-oriented usability testing:
a pilot study, Stud Health Technol Inform 180 (2012), 368-372.
R. Marcilly et al. / Integrating Human Factors in an International Research Project 167
[13] R. Marcilly, W.O. Hackl, M. Luyckx, E. Ammenwerth, Scorecards: a new method to prevent adverse
drug events? Preliminary results from a clinical field study, Stud Health Technol Inform 166 (2011),
234-245.
[14] S. Pelayo, R. Marcilly, S. Bernonville, N. Leroy, M.C. Beuscart-Zephir, Human factors based
recommendations for the design of medication related clinical decision support systems (CDSS), Stud
Health Technol Inform 169 (2011), 412-416.
168 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-168
Background
Medication errors are a significant source of actual and potential harm to patients [1].
Medication errors can include the wrong dose, frequency, route, and allergy details [2],
duplicate dosing [3], and other unintended interactions [4]. Community medication
records have the potential to reduce medication errors throughout the medication
management workflow [5, 6], such as: providing medication information at the point of
prescribing and dispensing [7], and supporting medication reconciliation [8]. However,
1
Corresponding Author. Box 3050 STN CSC Health Information Science, University of Victoria,
Victoria, BC V8W 3P5; Email: [email protected].
N.A. Kitson et al. / Usability Inspection to Improve an Electronic Provincial Medication Repository 169
Methods
PharmaNeet is a provinccially managed d electronic reepository that contains the rrecords for
communitty-based pharm macy-dispensed medication ns in British Columbia.
C PharmaNet is
accessed byb communityy pharmacistss and physiciaans through ceertified 3rd paarty vendor
applications [12]. Pharm maNet vendorrs were sent an a invitation letter via emaiil. Vendors
who agreeed to particippate completed d activity-drivven scenarios using five exxisting test
patients from
f the PhharmaNet traiining environ nment (not realr patient data). We
developed d standardized d user activitiees to highlighht aspects of medication
m management
workflow that were co overed by PharmaNet’s features (determ mined by reviewing the
PharmaNeet conformancce specificatio ons and mapp ping these to medication
m m
management
workflow)). (e.g., Figu ure 1, all annotated
a waalkthroughs area available freely at
ehealth.uvvic.ca).
We completed
c usaability inspectiions remotely y via screen shharing, whichh permitted
us to view w the vendorrs’ screens on n our local computers.
c W facilitated inspection
We
sessions while
w the venddors walked thhrough the sceenarios. We to ook extensive notes, and
the screenn was recordeed for analysiis. Three anallysts attended the walkthrooughs. One
analyst reeviewed the sessions in deetail using Jak kob Nielsen’s ten usabilityy heuristics
[13]. A seeverity scale was not used d; rather speciific issues weere mapped too focus on
broader design
d issues and improveements. The other team members revviewed the
inspectionn report to ennsure accuracyy and compleeteness. Confidential usabillity reports
were prov vided back to o the particippants to inforrm potential product imprrovements.
Aggregatee findings rellated to usability issues th hat had poten ntial safety isssues were
collected and summarrized. We compared c theese findings with the coonformance
specificatiions.
Figure 1: Example walkthrough for usability inspection. Simulated patient with allergy to penicillins.
170 N.A. Kitson et al. / Usability Inspection to Improve an Electronic Provincial Medication Repository
Results
Seven PharmaNet vendors, selling one or more PharmaNet applications, were invited
to participate; two consented. Usability inspection was conducted for four PharmaNet
applications (all participating vendor products). Inspection occurred in March 2011.
Ninety-six usability issues were identified across the four applications. The largest
proportion of observed usability issues (44%) was related to error prevention
(conditions that could lead to errors, e.g., being unable to confirm an action before
proceeding). Half (49%) of the observed usability issues had potential implications for
patient safety as determined by the analysts. Usability issues with implications for
safety were primarily related to: login and logout procedures (e.g., unclear procedures;
user name not clearly displayed); the display of patient name (e.g., patient name lost in
scrolling or covered by popup windows); the display of medications (e.g., no current
medication list, current medications not labeled, potential truncation of medication
administration directions, difficult to locate dose changes); the update and display of
alert information (e.g., timing of alerts, adverse reactions linked to a specific drug code
not the active ingredient or class); and the changing or discontinuation of medications
(e.g., unclear dosage selections, lengthy or incomplete picklists).
Discussion
For login and logout procedures, it is important to ensure that the correct authorized
user is accessing the system at any given time. A poorly visible login feature can
increase reliance on the previous user to logout; similarly, a poorly displayed logout
function can make it difficult for users to remember to logout [3]. We observed ways to
mitigate these challenges, including: clear login and logout buttons that do not scroll
off screen; straightforward, logical login and logout procedures; and screen timeout.
The display of patient name should be constant and consistent to help reduce the
likelihood that the wrong patient is being reviewed [3, 14]. There can be many
interruptions in practice; therefore, the patient name should never be obscured on
screen. We observed the patient name lost in scrolling, overlapping boxes, or not
appearing on each screen. It should be consistently and clearly displayed along with
key identifiers to ensure the right patient is being accessed at all times [15].
One of the key functions of a medication repository is to display medications to a
provider. A chronologic dispensing record can be very helpful to providers [16];
however, for many of the common medication related activities, it requires the user to
perform excess cognitive tasks. For example, to determine an expected list of current
medications from a dispensing record, the user must review each dispensing item and
calculate an expected end date and then compare it to today’s date. Similar mental
calculations must be made for medication use rates. Dose changes can be difficult to
locate in a long dispensing list. The user must remember current dose and then find
previous dispensings and mentally compare prescribing instructions.
In terms of the update and display of alerts, we found good instances of clearly
displayed alerts provided at the appropriate points. However, the reverse was also
identified, for instance, drug allergies appearing after the user dispensed the medication.
It appeared that adverse reactions are linked to a Drug Identification Number and did
not appear to be linked to the active ingredient or drug class. Identical alerts could be
duplicated when the vendor’s own pharmacy information system provided drug-to-drug
N.A. Kitson et al. / Usability Inspection to Improve an Electronic Provincial Medication Repository 171
interactions and maintained its own medication list in parallel to PharmaNet (all
pharmacy information systems are required to do this). This could lead to user alert
fatigue and alerts may be overlooked [17].
The patient’s medication profile should be up-to-date by ensuring medication
updates (changes or discontinuations) are in the system. This helps inform medication
decision-making and to generate the appropriate alerts (e.g., drug-to-drug interactions).
Limits on who can update a dispensing repository can impact accuracy of the data.
These features should fit the workflow of appropriate members of the care team. For
example, physicians often will cancel or change a medication dose without requiring a
new dispensing (e.g. “use the medication you have already, just take half a tablet
instead of a whole one”). In a dispensing repository like PharmaNet, only those who
dispensed a medication could “correct” that dispensing record. Prescribers and others
could not go in and change a previous dispense record to reflect the current activities of
the patient; this does not fit well with the task of reviewing and updating medications.
Recommendations
Figure 2: Example
E illustrattion of a potenttial current med
dication list, keyy information suuch as which
medications are current and when
w the current dispensed mediccations are expectted to end are proovided in one
list.
A meedication utiliization chart supports a review
r of meedication utillization by
providing a visual sum mmary of a specific disp pensed medicaation over tim me. When
reviewing g a dispensing record, it cann be difficult to
t draw out key
k informatioon [18] like
dosage ch hanges and addherence rates (e.g., pickup p frequency frrom the pharm macy) for a
single medication over time, especiaally when each h dispensing is
i represented separately
and mixed with all otther medicatiions. Figure 3 illustrates an a example m medication
n chart demon
utilization nstrating howw it can track medication usage
u over timme (in this
case, dailyy dispensings of methadonee). Again, thiss tool providess a better cognnitive fit to
the task of reviewing g medication utilization th han the simp ple reverse-chronologic
dispensing g record. In this
t chart, th
he provider caan see at a glance
g the freequency of
dispensing gs, dosage changes,
c d adherence. This provid
and des the userr with an
informativve snapshot too inform clinical decision-m making and reeduces cognitive load in
an alreadyy busy patient encounter. We
W observed on ne example off this in our innspections.
Referencees
[1] Kohn L.
L To err is humaan. National Acaddemy Press; 2000.
[2] Bates D, Teich J, Leee J, Seger D, Ku uperman GJ, Ma’Luf N, Boyle D, Leape L. Thhe Impact of
Compuuterized Physiciann Order Entry on
n Medication Prev vention. JAMIA 1999; 6: 313-321
[3] Koppel R, Metlay J, Cohen
C A, Abauck k B, Localio R, Kimmel
K SE, Strom BL. Role of CComputerized
Physician Order Entry Systems
S n Errors. JAMIA 2005;
in Facilittating Medication 2 293: 1197--1203.
N.A. Kitson et al. / Usability Inspection to Improve an Electronic Provincial Medication Repository 173
[4] Lau H, Florax C, Porsius A, DeBoer A. The completeness of medication histories in hospital medical
records of patients admitted to general internal medicine wards. Brit J Clin Pharmaco 2000; 49: 597-
603.
[5] Bell D, Cretin S, Marken R, Landman A. A Conceptual Framework for Evaluating Outpatient
Electronic Prescribing Systems Based on their Functional Capabilities. JAMIA 2004; 11: 60-70.
[6] Bassi J, Partridge C. Electronic Prescribing: Workflow Analysis Handbook. University of Victoria
eHealth Observatory: Victoria, BC; 2010.
[7] BC-Pharmacare. PharmaNet. [Internet] 2012. Available from:
http://www.health.gov.bc.ca/pharmacare/pharmanet/netindex.html
[8] Bassi J, Lau F, Bardal S. Use of Information Technology in Medication Reconciliation: A Scoping
Review. Ann Pharmacother 2010; 44: 885-897.
[9] Ash J, Berg M, Coiera E. Some Unintended Consequences of Information Technology in Health Care:
The Nature of Patient Care Information System-related Errors. JAMIA 2004; 11: 104-112.
[10] Vessey I. Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables Literature. Decision Sci
1991; 22: 219-240.
[11] Borycki EM, Kushniruk AW, Kuwata S, Watanabe H. Simulations to Assess Medication
Administration Systems, in Nursing and Clinical Informatics: Socio-Technical Approaches, B.
Staudinger, V. Höß, and H. Ostermann, Eds. Hershey PA: Information Science Reference; 2009: 144-
159.
[12] PharmaNet: Professional and Software Compliance Standards, Volume 3 – Technical Rules, Pharmacy.
BC Ministry of Health Services: Victoria, BC; 2010.
[13] Nielsen J. 10 Heuristics for User Interface Design. [Internet]. 2005. Available from:
http://www.useit.com/papers/heuristic/heuristic_list.html
[14] Khajouei, Jaspers M. The Impact of CPOE Medication Systems’ Design Aspects on Usability,
Workflow and Medication Orders: A Systematic Review. Methods Inf Med 2010; 49: 3-19.
[15] Guidance – Patient ID Number Input and Display: Microsoft Health Common User Interface. [Internet].
2010. Available from: http://mscui.net/DesignGuide/PatientNoDisplay.aspx
[16] Kent H. BC’s PharmaNet system proving convenient. CMAJ 2000; 162: 1192.
[17] Ash JA, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some Unintended Consequences of
Clinical Decision Support Systems, In: Proceedings of the 2007 American Medical Informatics
Association Annual Symposium; 2007: 26-30.
[18] Pelayo S, Leroy N, Guerlinger S, Degoulet P, Meaux J-J, Beuscart-Zéphir MC. Cognitive Analysis of
Physicians’ Medication Ordering Activity, In: Proceedings of ENMI: Connecting Medical Informatics
and Bio-Informatics; 2005.
174 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-174
Introduction
1
Corresponding author: Rasmus Rasmussen, Roskilde University, Universitetsvej 1, 4000 Roskilde,
Denmark; E-mail: [email protected]
R. Rasmussen and A. Kushniruk / The Long and Twisting Path: An Efficiency Evaluation of an EW 175
1. Methods
The usability study was performed as a longitudinal and naturalistic study of the ED
clinicians’ interactions with the EW system. The study involved continuous and long-
term screen recordings of the clinicians’ interactions with the EW system throughout
multiple five-hour periods during dayshifts at two EDs. The healthcare region and ED
management approved the study prior to it being conducted. Also, because the study
involved collection of live patient data the study had to be registered and approved by
the Danish Data Protection Agency. Clinicians on duty during the study were briefed
during morning meetings and throughout the study if questions or concerns arose.
User interactions with the EW were captured using the HyperCam 2 screen capture
software installed on 4 Gb flash drives. The resulting video files were stored on either
an external 2 Tb hard drive or a 16 Gb flash drive. User interactions were captured over
a period of five days between 10 AM and 4 PM each day at two EDs. This period was
specifically chosen because experience proved this was often the busiest time of the
day and should therefore produce the highest number of interactions with the EW.
Name
Description
Yes Number of steps to complete
Activity Complete task?
No Number of steps before aborting
Each video file was viewed and logged by the first-author using a predefined coding
scheme developed by both authors – see Figure 2. The initial viewing and logging was
carried out solely by the first-author due to restrictions imposed by the Danish Data
Protection Agency’s directives. Using the coding scheme, on-screen activity was
recorded and entered as entries in separate log files. Each entry contains a timestamp,
an activity indicator, a name for the activity and a description of the activity. In
instances of task oriented activity the entries indicate whether or not the task was
completed and the number of steps taken before completing or aborting the task. When
usability issues were discovered we marked them with an indicator and coded the
issues using one of the following categories: System bugs, efficiency problems, error
messages and work patterns. We also provided a description of the issue including
whether or not the user solved the issue.
Following the logging process both authors perused each log file and coded
activities of interest for further analysis. All codings were discussed to mitigate biases
in the analysis. The initial coding of the data was used to locate particular interactions
of interest that were further analyzed to identify efficiency issues. In this paper we
focus on the analysis of potential inefficiencies. This involved using the GOMS-KLM
method [3] to calculate how much time a theoretical expert user would spend on
completing a task with the EW system following a specific pathway determined by the
original system design.
Based on these analyses, our approach then involves proposing a redesign aimed at
improving the efficiency for that task, calculating how much time an expert user would
spend using this design before finally comparing this with the GOM-KLM calculations
for the original design.
We logged a total of 2863 entries and recorded 13 unique usability issues: 4 system
bugs (55 instances), 4 efficiency issues (141 instances), 3 error messages (38 instances)
and 2 inefficient working patterns (229 instances). The most common efficiency issues
found in the results concerned complicated and long pathways, which the EW system
forces the users to follow when using the system for specific tasks. This issue occurred
a total of 63 times (44.68 % of all instances of efficiency issues) in the results. The
complicated and long pathways become very apparent when new patients are added to
the EW or when certain information fields are updated. In logging the video files we
found that adding a new patient on average took 12.3 steps to complete. When users
completed this task error free e.g. without mistakes or interruptions and provide the
maximum amount of information the task required 19 steps to complete. These steps
are the following: 1) Open "Add row" dialog box 2) Open "SSN" dialog box 3) Input
SSN 4) Close "SSN" dialog box 5) Open “Note” dialog box 6) Type note 7) Close
R. Rasmussen and A. Kushniruk / The Long and Twisting Path: An Efficiency Evaluation of an EW 177
“Note” dialog box 8) Open “Problem” dialog box 9) Open problem selection dialog
box 10) Search/select problem 11) Close selection dialog box 12) Close “Problem”
dialog box 13) Open “Waiting for” dialog box 14) Select waiting for option 15) Close
“Waiting for” dialog box 16) Open “Location” dialog box” 17) Select location option
18) Close “Location” dialog box 19) Close “Add row” dialog box. As this indicates the
current design of the EW is based on individual dialog boxes for input into each
information field. In the following, we will use the add-patient task as an example to
demonstrate how the efficiency of the EW design could be improved via a simple
redesign. Using the GOMS-KLM method [3] we are able to calculate how much time a
theoretical expert user of the EW system would spend on completing the add-patient
task (see Eq. (1) where H = moving hands between mouse and keyboard, P = pointing
to a position on the display, K = tapping a key or button, M = mentally preparing for
next step – see [3] for definitions of the GOMS-KLM operators). Assuming that the
user starts the task with hands off the keyboard, that there is no system response time
and that the user inputs a 10-digit SSN and a 30-character note the calculations will be
as follows:
ܪ ܯ ܲ ܭ ܯ ܲ ܭ ܪ ܯ ሺͲͳ כ ܭሻ ܪ ܯ ܲ ܭ ܲ ܭ ܪ ܯ ሺͲ͵ כ ܭሻ
ܪܯܲܭܯܲܭܯܲܭܯܲܭܯܲܭܯܲܭܯܲܭ
ܯܲܭܯܲܭܯܲܭܯܲܭܯܲܭܯܲ ܭൌ
ݏ݀݊ܿ݁ݏ݊݅ݐ݊݁ݏ݁݉݅ݐ (1)
When replacing the operators in Eq. (1) with the times they represent (H= 0.4
seconds, P = 1.1 seconds, K = 0.2 seconds and M = 1.35 seconds) we find that a
theoretical expert user would spend 56.4 seconds on completing the add-patient task
when following the pathway that the system currently enforces. The amount of input
information needed to complete the task is independent of the pathway followed and
therefore cannot be reduced by redesigning the interface. However, by reducing the
number of steps needed to complete the task it is possible to make the interface more
efficient than the current. We will demonstrate this by proposing a theoretical interface
design where the input information is entered directly in text fields or by selection via
drop-down menus instead of opening new dialog boxes for each individual input. Once
again we assume that the user starts the task with their hands off the keyboard and
mouse, that there is no system response time and that the user inputs a 10-digit SSN
and a 30-character note. In this case we have the following steps for the task: 1) Open
"Add row" dialog box 2) Select "SSN" input field 3) Input SSN 4) Select "Note" input
field 5) Input note 6) Select "Problem" option from drop-down menu 7) Select
"Waiting for" option from drop-down menu 8) Select "Location" option from drop-
down menu 9) Close “Add row” dialog box. When applying the GOMS-KLM
calculations to the proposed redesign we arrive at Eq. (2):
Thus, it would take an expert user of the EW system 31.25 seconds to complete the
task of adding a new patient when using the proposed redesign of the systems interface.
This simple redesign of the EW interface thereby presents a reduction of the theoretical
task completion time by 25.15 seconds (44.6 %). Taking into consideration that each
ED receives approximately 40.000 – 45.000 patients each year a redesign of this
pathway could prove to be a significant time saving improvement over the current
178 R. Rasmussen and A. Kushniruk / The Long and Twisting Path: An Efficiency Evaluation of an EW
3. Conclusions
Through our usability evaluation of the EW system we found a wide range of usability
issues in the EW system. In this paper we chose to focus on long and complicated
pathways within the EW system. Using the GOMS-KLM we illustrated with an
example how the system could be redesigned to increase its efficiency. We found that
our proposed redesign could reduce the time needed to enter a new patient to the EW
by roughly 45 % and with the reservation that the clinicians do not always provide the
maximum amount of information we found that this could provide a time saving of
157.2 hours pr. year. Also, since the EW system can be accessed via multiple devices
the potential increase in efficiency could be far ranged and have a positive impact upon
the work practices at the ED. Furthermore, we argued that this redesign could have an
even greater impact than our results show since it could potentially affect a larger part
of the EW than we studied here. In conclusion we call for more and earlier usability
evaluations of healthcare information systems such as the EW studied here to ensure a
higher quality of the systems used by healthcare professionals.
References
[1] R. Rasmussen. Electronic whiteboards in emergency medicine: A systematic review, Proceedings of the
2nd ACM SIGHIT International Health Informatics Symposium (2012), USA, 483-492.
[2] E. Frøkjær, M. Hertzum, K. Hornbæk. Measuring usability: Are effectiveness, efficiency and,
satisfaction really correlated? Proceedings of the ACM CHI 2000 Conference on Human Factors in
Computing Systems (2000), 345-352.
[3] S.K. Card, T.P. Moran, A. Newell. The Keystroke-Level Model for User Performance Time with
Interactive Systems. Communications of the ACM 23(7) (1980), 396-410.
[4] W.D. Gray, B. John, M.E. Atwood. The precis of project ernestine – An overview of a validation of
GOMS, Proceedings of the CHI 92 Conference on Human Factors in Computing Systems (1992), 307-
312.
Enabling Health and Healthcare through ICT 179
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-179
Introduction
Personal Health Records (PHRs) have been lauded for their potential to engage health
consumers and provide them with governance over their own health information [1].
PHRs are used to access, manage and share personal health information, privately and
securely [2]. Moreover, if they are interoperable, PHRs may contain data from
different sources (e.g., pharmacies, labs, EHRs) [1]. However, health consumers act
as curators of their PHRs and thus, decide what health information is and is not
included. Ideally, PHRs provide comprehensive, longitudinal depictions of both static
(e.g., allergies) and dynamic (e.g., blood pressure, cholesterol levels) health data. In
addition to serving as a data repository, some PHRs offer interactive tools to foster
self-management, education and decision-making.
1
Corresponding Author: Helen Monkman, email: [email protected]
180 H. Monkman and A. Kushniruk / Applying Usability Methods to Identify Health Literacy Issues
PHRs are unique in that they are healthcare applications designed for consumers
who have limited or no healthcare expertise [3]. For PHRs to facilitate self-
management, it is imperative that they are designed to accommodate consumers with
limited health literacy to ensure users’ comprehension of the information [4]. Ratzan
and Parker [5] define health literacy as “the degree to which individuals have the
capacity to obtain, process, and understand basic health information and services
needed to make appropriate health decisions” (pg. iv). Health literacy extends beyond
reading and writing and encompasses a variety of other skills (e.g., listening, speaking,
arithmetic, problem-solving and decision-making), and is also influenced by culture
and society [6]. Health literacy skills are associated with a broad range of benefits
from increased health knowledge, improved self-reported health status, and reduced
healthcare costs (e.g., shorter hospitalizations, less frequent use of health care
services) [7].
Consumer health information (e.g., PHRs) systems can undoubtedly benefit from
the application user-centered design methods [8]. It is argued here that there is a close
interaction between usability and health literacy, particularly when we consider the
effectiveness, efficiency and satisfaction of end users (i.e., patients and lay people)
with PHRs. Specifically, the PHR user’s capacity for obtaining, processing and
understanding basic (as well as their own) health information will influence how
usable such systems are.
Health literacy levels and usability both have the potential to impact data
accuracy, quantity and quality of the information users enter into their PHRs. For
example, medical terms and definitions not understood by PHR users could result in
data entry errors or omissions. Whereas, if PHRs are challenging to use, health
consumers are less likely to consistently monitor symptoms, vitals and other statistics
into their PHRs. Thus, to optimize the value of PHRs, it is imperative that they are
usable and designed to mitigate the demands on health literacy, as well as develop
health literacy skills.
Currently, there is a dearth of research on methods for identifying and
ameliorating health literacy issues in consumer health information systems. Given the
relationship between usability and health literacy, usability evaluation methods may
prove to be effective in detecting potential health literacy issues in consumer health
information systems.
Three studies inadvertently identified health literacy issues during investigating
of PHRs for other purposes. Kim and Johnson [9] compared the utility and
functionality of 11 web-based PHRs and revealed a barrier to use that stemmed from
a health literacy issue. These authors reported the PHRs failed to provide guidance or
explanation to facilitate user data entry. Specifically, these authors found no resources
to help users interpret prescription labels or test results to increase the likelihood that
they were entered into the PHRs accurately. Cimino, Patel, and Kushniruk [10]
reported that although patients wanted to know their lab results, some had challenges
understanding and interpreting them. Segall and colleagues’ [3] also found that many
participants could not understand the content of their medical reports. Additionally,
these authors also suggested replacing jargon with the vernacular and expanding
acronyms. For example, Segall and colleagues proposed using “as needed” rather “prn”
and “by mouth” instead of “orally” (pg. 1239) and suggested other changes to.
In the previous studies, health literacy issues were revealed without an apriori
objective of their identification. The present study will demonstrate how usability
methods can be extended with to identify both usability and health literacy issues.
H. Monkman and A. Kushniruk / Applying Usability Methods to Identify Health Literacy Issues 181
1. Methods
At the time of testing, the PHR used in this study was Canadian based and offered at
select clinics as a tethered solution. The PHR allowed the storage of a variety of
personal health information (e.g., allergies, medications, health conditions).
Additionally, several tools were provided by the PHR to manage (e.g., Appointments,
Journal, Care Plan) and monitor (i.e., body dimensions, cardio, blood glucose, peak
flow) health, with specific modules (i.e., Asthma, Cardio, Diabetes) focused on
chronic illnesses.
In order to compare the results from the heuristic evaluation to the usability testing
method (and to explore the utility of this method to reveal health literacy problems)
one participant was used for pilot usability testing purposes. To do this, the PHR
account was pre-populated with health information to give the impression of an
existing record. A MacBook Pro laptop was used for this experiment. QuickTime
Player version 10.1 was used to simultaneously record the audio and the events that
occurred on-screen.
After the participant was seated in front of the laptop computer with the web
browser on the PHR login page, the audio and screen recording began. Once
QuickTime Player was running, the experimenter read the instructions to the task. The
participant was asked to “think aloud” or verbalize his thoughts as he attempted to
complete each task [8]. The participant completed the following three tasks
sequentially: 1) Enter a medication; 2) Interpret vital sign data; and 3) Determine
allergy documentation. After completing each task, the participant was asked to
evaluate and comment on various aspects of the screens they encountered during the
182 H. Monkman and A. Kushniruk / Applying Usability Methods to Identify Health Literacy Issues
previous task. Finally, when the participant completed all three tasks, he was asked to
make general comments about the PHR in a brief post-task interview.
2. Results
A total of 15 usability issues were revealed during this investigation of four tasks. The
heuristic that was violated most frequently was Health Literacy, followed by
Recognition rather than Recall, and Flexibility and Efficiency of Use (See Figure 1).
All of the Health Literacy violations occurred during the Medication and Allergy
tasks. As indexed by no violations for heuristic Error Prevention, the PHR was well
designed from this perspective. For example, the PHR clearly described any errors
that occurred while inputting data and the steps necessary to complete the data entry
successfully. Also, any data that was entered could be subsequently edited or deleted
very easily. The PHR was well organized; the categories and sub-sections were
logically grouped and all of the groups of items were kept within reasonable memory
limits.
One Health Literacy violation was rated as catastrophic and occurred in the
Allergies section. When adding a new allergy, the drop-down menu was confusing,
inconsistent and failed to list many common allergies (e.g., penicillin, peanut). For
example, “Penicillamine allergy”, “Penicillinase-resistant penicillins allergy” and
“Penicillinase-sensitive penicillins allergy” are allergy entries listed. The complexity
of these words and the nuances between the latter two options were unlikely to be
understood by a layperson without a healthcare background and users may be more
likely to select the incorrect option if they were trying to report a “Penicillin allergy”.
As an example of the list’s inconsistency, this menu listed pollen as a food allergy
rather than environmental. In order to remedy this issue, a detailed investigation of the
contents of the allergy options should be conducted. It is imperative that allergies can
be reported as simply as possible. If adding allergies is challenging, the probability
that errors in data entry will occur are increased and could have critical consequences
in an emergency situation.
The video of the participant interacting with the PHR was transcribed and annotated
in full, and then coded for usability issues. Of the three tasks completed by the
participant, the most usability problems (12) occurred when the participant was trying
to enter a medication. The PHR failed to facilitate entering a medication. For example,
the participant did not know what the strength and dosage of a medication values
were and repeatedly received error messages.
After completing all three tasks, the participant indicated that design changes
could improve the usability of the system. Specifically, the participant felt that the all
of the content font size should be adjustable, so it could be increased for easier
reading. Further, he expressed it would be helpful to have a link to more information
on medications and medication interactions. Lastly, the participant also wanted titles
on the blood pressure graphs and increased visibility of outlier values (e.g., red data
points).
A health literacy issue that was identified in both the heuristic evaluation and the
usability test was the medication entry dialogue box. This box was overly complex
and had fields for a lot of detailed information that users may not know. Further,
many of the categories of information may not be clear to users who do not have
medical expertise. The participant experienced difficulty differentiating between
strength and dosage. The PHR should facilitate understanding by adopting simpler
terms and providing examples. An approach to making prescription labels more
understandable and informative was found on the Women’s Health website
(http://www.womenshealth.gov/aging/drugs-alternative-medicine/how-to-read-drug-
labels.cfm). This website decomposed an image of a prescription to describe its
constituent parts with simple, easy to understand language. Making this information
available to users of PHRs is beneficial because the user has the opportunity to
improve their understanding of their prescription. Further, this strategy may also
increase the accuracy of the information entered into the PHR, which may be critical
in the case of an emergency. Thus, it is strongly recommended that the PHR adopts
strategies like this that increase user knowledge and simultaneously make information
easier to enter and more likely to be accurate.
3. Discussion
identified in this study through design changes, will result in a more usable PHR.
Moreover, modifying the content to accommodate its layperson user population will
increase the accuracy of the information entered and the PHR could serve as a useful
resource for health consumers.
Given the exploratory nature of this study, only a single usability specialist
conducted the heuristic evaluation. However, multiple evaluators have been shown to
reveal more usability issues than a single evaluator [12]. Thus, it is recommended that
for more detailed analysis, several evaluators should be included to identify a
comprehensive list of heuristic violations.
Some health literacy issues in PHRs may arise as a consequence of designers
adopting functional components from electronic health records (EHRs), which are
used by providers. Although this is not problematic for many areas, previous research
and the current study demonstrated that some health topics (e.g., medications, lab
results, allergies) require extra attention devoted to designing with health literacy
considerations in mind.
The utility of applying and extending usability methods to detect health literacy
issues in consumer health information systems was demonstrated in this study. Health
consumers might have limited medical knowledge, and design strategies should be
developed to assist users in understanding health information and inputting data
accurately. Some examples of techniques that could be adopted by consumer health
information systems to help improve the health literacy of their applications are: using
multi-modal (e.g., illustrations, movies) descriptions; linking to other resources; and
integrating definitions of term. To improve users’ health knowledge, the accuracy of
the information, and the adoption of consumer health information systems, it is
encouraged that usability methods be applied with an added focus on health literacy.
References
[1] P. C. Tang, J. S. Ash, D. W. Bates, J. M. Overhage, D. Z. Sands. Personal health records: definitions,
benefits, and strategies for overcoming barriers to adoption. Journal of the American Medical
Informatics Association : JAMIA 13(2) (2006), 121-126. doi: 10.1197/jamia.M2025
[2] D. A. Jones, J. P. Shipman, D. A. Plaut, C.R. Selden. Characteristics of personal health records:
findings of the medical library association/national library of medicine joint electronic personal health
record task force. Journal of the Medical Library Association: JMLA 98(3) (2010), 243-249. doi:
10.3163/1536-5050.98.3.013
[3] N. Segall, J. G. Saville, P. L'engle, B. Carlson, M. C. Wright, K. Schulman, J. E. Tcheng. Usability
evaluation of a personal health record. AMIA Annu Symp Proc (2011), 1233-42.
[4] L. Wuerdeman, L. Volk, L. Pizziferri, R. Tsurikova, et al. How accurate is information that patients
contribute to their electronic health record? AMIA Annu Symp Proc (2011), 834–838.
[5] S. C. Ratzan, R. M. Parker. Introduction. In C. R. Selden, M. Zorn, S. C. Ratzan, R. M. Parker, (Eds.),
National library of medicine current bibliographies in medicine: Health literacy. NLM Pub. No. CBM
2000-1. National Institutes of Health, U.S. Department of Health and Human Services, Bethesda,
2000.
[6] C. C. Cutilli. Health literacy: what you need to know. Orthopaedic nursing / National Association of
Orthopaedic Nurses 24(3) (2005) 227-233. doi: 10.1097/00006416-200505000-00014
[7] C. Speros. Health literacy: Concept analysis. Journal of Advanced Nursing 50(6) (2005), 633-640.
doi: 10.1111/j.1365-2648.2005.03448.x
[8] A. W. Kushniruk, V.L. Patel. Cognitive and usability engineering methods for the evaluation of clinical
information systems. Journal of Biomedical Informatics 37(1) (2004), 56-76. doi:
10.1016/j.jbi.2004.01.003
[9] M. I. Kim, K. B. Johnson. Personal health records: Evaluation of functionality and utility. Journal of
the American Medical Informatics Association: JAMIA 9(2) (2002), 171-180. doi:
10.1197/jamia.M0978
H. Monkman and A. Kushniruk / Applying Usability Methods to Identify Health Literacy Issues 185
[10] J. J. Cimino JJ, V. L. Patel, A. W. Kushniruk. The patient clinical information system (PatCIS):
Technical solutions for and experience with giving patients access to their electronic medical records.
International Journal of Medical Informatics 68(1-3) (2002), 113-127. doi: 10.1016/S1386-
5056(02)00070-9
[11] Usability.gov: Your guide for developing usable & useful Web sites. U.S. Department of Health &
Human Services. Accessed October 18, 2012 from
http://www.usability.gov/methods/test_refine/heuristic.html
[12] Nielsen, J. Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability Inspection Methods,
pp. 25-62. John Wiley & Sons, New York, 1994.
This page intentionally left blank
Patient Safety, Medical Errors &
Quality Management
This page intentionally left blank
Enabling Health and Healthcare through ICT 189
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-189
Abstract. This study examined how variations in the source and type of patient
health data affected health care providers’ perceptions of the trustworthiness and
usefulness of the data. Overall, respondents (n=107) reported moderate to high
levels of trust and usefulness of health status data from all sources. Technology
sources were rated as more trustworthy than traditional, non-technology sources
(paired t = -2.84, p <0.006). However, there was no significant difference between
technology sources and non-technology sources (paired t = -1.63, p <0.108) in
perceived usefulness for clinical decision making.
Introduction
Little is known about how health care providers value and subsequently use external or
remotely acquired data in their clinical decision making. Existing literature has focused
on the process of exchanging information across care settings, with little attention to
the perceived value of sharing data or the perceived value of data acquired from remote
sources [1]. Recent models in data provenance suggest that the lineage or tracing of the
source of the data reflects on the quality and trustworthiness of the data [2]. Indeed,
Simborg [3] notes that health care providers need to consider the source of the data in
order to evaluate its validity. While data quality can vary in any given instance [4, 5],
clinician values or generalized attitudes toward remotely acquired data could be just as
important as actual quality variance. Negative attitudes may lead clinicians to disregard
or discount non-traditional sources or unique types of health data.
With the dramatic expansion of technology use in health care, clinicians are
bombarded with clinical data, some of which may be perceived to be of limited or
unknown value and therefore contributing to the “noise” of clinical decision making [6].
Despite this overabundance, clinicians may decline to utilize a system that fails to
provide the “right” data [7]. New technological applications outside of traditional
health care settings are delivering novel types of data. Though these data are intriguing
and promising, neither researchers nor clinicians yet know their clinical value. For
example, motion sensor data gathered at TigerPlace, a smart home-enabled assistive
living facility, are being used to create activity density maps for residents [8]. While
the comparison of activity density maps with known clinical changes are intriguing,
the clinical utility of such data is still unknown.[8]. Uncertainty in intended use and
lack of experience with these new sources and types of data may heavily influence the
1
Corresponding author: Karen Courtney, email: [email protected]
190 K.L. Courtney et al. / Clinician Variations in Data Trust and Use
perceived value and subsequent adoption of novel data.[9]. Without understanding the
attitudes and behaviors of clinicians, we may design and implement clinical data
collection systems that health care providers neither trust nor use. In this study we
examined how variations in the source and type of data and prior experience with
health information technology affected health care providers’ perceptions of the
trustworthiness and usefulness of patient data.
1. Methods
2. Results
Exposure to individual types of data across data sources varied widely, ranging from
9.5% for exercise or social contact data from other monitor sources to 40.4% for
physiologic data captured by telehealth monitor sources. If a respondent indicated
having received any type of data within a technology data source in the past, we
considered the respondent to have prior exposure to that particular data source. In our
sample, 57.9% had been exposed to at least one type of data from any technology.
Prior exposure varied by data source. Overall, more respondents reported exposure to
data from telehealth monitor sources (38.3%) and wearable monitor sources (37.4%)
than other monitors (22.4%) or environmental monitors (18.7%). Prior exposure to a
particular technology source did not significantly influence respondents’ perceptions of
trustworthiness or clinical decision making usefulness of data from that source.
2.2. Trustworthiness
Respondents indicated a moderate to high level of trust in data from varying sources
(range M = 6.22-8.39). Among the traditional, non-technology data sources, the
specialist data source (M = 8.34 ± 1.41 SD) had the highest trustworthiness rating and
the wearable monitor data source (M = 8.39 ± 1.34 SD) had the highest trustworthiness
rating among the technology sources and overall. The community health screening
data source (M = 6.22 ± 1.93 SD) ranked lowest for trustworthiness overall and among
the non-technology sources. Among technology sources, other monitor data sources
(M = 7.08 ± 1.81 SD) held the lowest rating for trustworthiness. When data sources
were aggregated into two broad categories, technology sources (M = 7.75 ± 1.38 SD)
were rated as significantly more trustworthy than traditional, non-technology sources
(M =7.37 ± 1.24 SD) (paired t = -2.84, p <0.006), although the absolute difference in
ratings was relatively small. To explore whether this finding was still true without the
influence of community health screening data sources (the lowest rated source) on the
non-technology data sources, we compared technology sources against each of the
192 K.L. Courtney et al. / Clinician Variations in Data Trust and Use
three broad pairings (primary care provider-specialist, home health- nursing home, and
patient-family member). Technology sources were rated as more trustworthy than
the data sources in the home health-nursing home comparison (M =7.08 ± 1.54 SD)
(paired t = -4.25, p <0.000), the patient-family member comparison (M =7.41 ± 1.54
SD) (paired t = -2.09, p <0.039) but not the primary care provider- specialist
comparison (M =8.22 ± 1.30 SD) (paired t = 2.99, p <0.004).
2.3. Usefulness
With one exception, there was a significant, moderate positive relationship between
perceived trustworthiness of the data and perceived usefulness of the data in clinical
decision making across data sources (r = 0.363 – 0.646, p < 0.001). The weakest
association between perceived trustworthiness and perceived clinical decision
usefulness was for specialist-sourced data; whereas the strongest was for community
health screening-sourced data. A positive, moderate association was found between
perceived trustworthiness and usefulness when traditional, non-technology data sources
were aggregated (r = 0.494, p < 0.000). This association was considerably higher when
technology sources were aggregated (r = 0.702, p < 0.000).
3. Discussion
Of concern to engineers, informaticists and clinicians alike are factors that influence
the adoption and continued use of optimal health information technologies by health
care providers. Although anecdotal evidence has suggested that the provenance (data
source and process used to create the data) of health data might affect the clinician’s
perception of it, no studies to date have directly assessed clinicians’ perceptions of the
K.L. Courtney et al. / Clinician Variations in Data Trust and Use 193
trustworthiness and usefulness of data by source and type. As expected, we found that
perceived trustworthiness and usefulness are moderately correlated. Since our study
was conducted, other work has been published that also supports the relationship
between perceived usefulness and perceived information quality. [10]. More
importantly, our findings show that technology data sources were perceived at least as
favorably as data from traditional, non-technology, sources. This finding supports the
movement in health care towards more technology based sources of data. Ironically
this technology trend also includes data from outside traditional health care facilities
and may rely heavily on patients or family members as primary data providers.
This trend will ultimately raise other challenges. Some challenges may be cultural
such as how to integrate technology data into practitioner workflow. Other challenges
may be more technological in nature such as how to integrate new data sources into
existing health information systems. Inclusion of data provenance metadata alone
within electronic health records will require careful consideration of affected
terminologies and standards.
When developing the study, our team was concerned that clinicians may have had
little prior exposure to data from the technology sources. As a result, we added an
exposure question for each technology data source. Our results showed that prior
exposure to any of the technology data sources did not affect perceptions of
trustworthiness or clinical utility. Further research should attempt a more fine grained
assessment of prior experience. This study was limited by its small, non-representative
sample of health care providers. Some of the respondents may have also been pre-
disposed with positive attitudes towards technology in general as they were attending a
geriatric medicine update conference with an optional concurrent technology workshop.
This investigation is a critical first step in identifying differences that exist in
clinicians’ perceptions of data based on source. Understanding how information
technology as the delivery mechanism may change the perceived trustworthiness or
clinical decision making utility merits further inquiry.
References
[1] Crooks VA, Agarwal G. What are the roles involved in establishing and maintaining informational
continuity of care within family practice? A systematic review. BMC Fam Pract. 2008;9:65.
[2] de Lusignan S, Liaw ST, Krause P, Curcin V, Vicente MT, Michalakidis G, et al. Key Concepts to
Assess the Readiness of Data for International Research: Data Quality, Lineage and Provenance,
Extraction and Processing Errors, Traceability, and Curation. Contribution of the IMIA Primary Health
Care Informatics Working Group. Yearb Med Inform. 2011;6:112-20.
[3] Simborg DW. Consumer empowerment versus consumer populism in healthcare IT. J Am Med
Inform Assoc. 2010;17:370-2.
[4] Porter SC, Forbes P, Manzi S, Kalish LA. Patients providing the answers: narrowing the gap in
data quality for emergency care. Qual Saf Health Care. 2010;19:e34.
[5] Shaw A, Somerset M, Dunn L, Peters T, Faulkner A, Sharp D. Can we trust the quality of routine
hospital outpatient information in the UK? Validating outpatient data from the patient
administration system (PAS). J Health Serv Res Policy. 1998;3:203-6.
[6] Markman M. Information overload in oncology practice and its potential negative impact on the
delivery of optimal patient care. Curr Oncol Rep. 2011;13:249-51.
[7] Hincapie AL, Warholak TL, Murcko AC, Slack M, Malone DC. Physicians' opinions of a health
information exchange. J Am Med Inform Assoc. 2011;18:60-5.
[8] Rantz MJ, Skubic M, Miller SJ. Using sensor technology to augment traditional healthcare.
Annual International Conference of the IEEE Engineering in Medicine and Biology Society
20092009. p. 6159-62.
194 K.L. Courtney et al. / Clinician Variations in Data Trust and Use
[9] Shabati I, Leshno M, Blondheim O, Kornbluth J. The value of information for decision-making in the
healthcare environment. Stud Technol Inform. 2007;127:91-7.
[10] Hsiao JL, Chang HC, Chen RF. A study of factors affecting acceptance of hospital information
systems: A nursing perspective. J Nurs Res. 2011;19:150-60.
Enabling Health and Healthcare through ICT 195
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-195
Introduction
Synoptic operative reports are simply structured or discrete medical reports that
provide a snapshot of patient encounter with a clinician at the point of care.
Specifically, synoptic operative reports often have a direct and critical role in patient
care by providing the necessary information needed to understand, interpret, and
communicate a patient’s surgical history [1]. Synoptic operative reports have emerged
as the gold standard for capturing and displaying structured surgical data in clinical
settings [1-5]. Citing the work of Temple et al. [6], Paterson, Swain, Christie, Thibault-
Halman, and Behzadi [7] noted that the replacement of traditional narrative with a
synoptic operative report has proven to be an effective strategy in improving surgical
data for the knowledge transfer process. This approach enables the capture of discrete
data items and transforms a narrative operative report that is qualitative in nature to a
quantitative one that can be aggregated to generate information and knowledge [2].
Electronic synoptic operative reports have replaced dictated reports at many
healthcare institutions because users could demonstrate improvements in reliability,
1
Corresponding Author: Wilfred Bonney, Division of Neurosurgery, Capital District Health Authority,
Halifax, NS, Canada B3H 3A7; E-mail: [email protected]
196 W. Bonney et al. / Design and Implementation of Synoptic Operative Report Template
completeness, and timeliness with this alternative. Electronic synoptic operative reports
use “computer-based forms to describe the findings and important steps of an operation,
based on predefined templates for individual procedures” [1, p. 308]. In other words,
electronic synoptic operative reports are dependent upon predefined templates for
individual procedures. Such templates are based on synoptic methodology, which on its
own refers to the creation of a library of keywords and phrases to show a preferred way
to communicate a clinical concept in an operative report.
Electronic synoptic operative templates support the creation of clinical documents
that are of higher quality than those produced by the same individual using dictated
narrative reporting method [8]. The template serves as a prompt for pertinent
information, which could be reused to support the information needs of researchers and
administrators. In accordance with the clinical documentation standards set by the
Joint Commission Accreditation of Healthcare Organizations (JCAHO) and
Accreditation Association for Ambulatory Health Care (AAAHC), synoptic operative
report templates contain sufficient information to: identify the patient; support the
diagnosis; justify the treatment; document the postoperative course and results; and
promote continuity of care [1,9]. Thus, the continuing use of electronic templates in
medical practice promotes health policy informatics and secondary data use.
1. Related Studies
Several synoptic operative templates have been proposed and used by many vendors
and researchers in the healthcare domain. Noticeable amongst them is the WebSMR
template proposed and developed by the Alberta Cancer Surgery Working Group. The
Working Group developed and implemented a web-based synoptic surgical medical
record, known as the Alberta WebSMR, for rectal cancer resection [1]. Edhemovic,
Temple, de Gara, and Stuart [3] reported that following the implementation of a rectal
cancer template, the synoptic report captured 99% of the required data elements versus
45.9% captured via a dictated narrative report. This showed that the science of surgical
technique could be better measured using a synoptic reporting method. The synoptic
checklist approach has consistently shown superior results to narrative reports for
completeness [4]. The WebSMR did not, however, incorporate major interoperable
standards in the system design.
In a related study, Paterson and Soroka [10] developed and implemented Health
Level Seven (HL7) Version 3 Clinical Document Architecture (CDA) Release Two
(R2) template for capturing discharge summaries for chronic kidney disease patients.
The template was designed using Microsoft InfoPath and HL7 V3 CDA R2
specifications. Even though the study by Paterson and Soroka [10] utilized
interoperable standards in the system design and implementation, they did not focus on
synoptic operative reports. The proposed electronic Synoptic Operative Report-Spinal
Cord Injury (eSOR-SCI) template not only focuses on synoptic operative report but
also uses interoperable standards in its design and implementation.
2. Methods
The methods were primarily based on the principles of health interoperability using
HL7 V3 CDA R2, Systematized NOmenclature of MEDicine - Clinical Terms
W. Bonney et al. / Design and Implementation of Synoptic Operative Report Template 197
(SNOMED-CT), and Logical Observation Identifiers Names and Codes (LOINC) [12].
Drawing upon the work of Rector, Qamar, and Marley [11], Benson [12] asserted that
electronic information systems operate at two different levels: Model of Use and Model
of Meaning. Whereas Model of Use is defined as the human interface that describes
how the system is actually used, Model of Meaning is defined as the representation for
reporting and analysis that supports data processing and reasoning [12]. These two
distinct levels of system operation are necessary because each “model of use needs to
be convertible into a model of meaning to make it computable” [12, p. 221]. This study
extended the work of Rector et al. [11] by adding another level (i.e., Model of Design)
to the methodology to describe how electronic information systems are designed.
The methodology, therefore, involved the design and use of HL7 V3 CDA R2 and
SNOMED-CT together in the context of synoptic operative report for patients with
spinal cord injury. The eSOR-SCI template is the model of use in that it enables data to
be captured and displayed by clinicians after the completion of each surgical procedure.
The eSOR-SCI template was implemented using a model of meaning methodology to
ensure that data can be aggregated and analyzed for research purposes. The binding of
the SNOMED-CT with the HL7 V3 CDA R2 template structure depicts how the model
of meaning was achieved. In the case of the eSOR-SCI template, the clinical document
is the synoptic operative report generated by the system.
3. Results
The model of design represents the design architecture of the eSOR-SCI template. The
design architecture components of the eSOR-SCI template consisted of a template
based on the Rick Hansen Spinal Cord Injury Registry (RHSCIR) Forms, Interoperable
Standards; and Microsoft Infopath.
The RHSCIR forms for diagnoses and procedures provided the minimum core data
elements for the eSOR-SCI template. Both the diagnoses and procedures forms were
the source of the template sections. Based on secondary data use requirements, the data
elements provided the content to populate the diagnoses and the procedures as per
RHSCIR diagnosis and procedure forms. These data elements were represented using
HL7 V3 CDA R2 specifications to bind to the template hosted on the InfoPath platform.
these standards was facilitated by the use of published implementation guides. In the
context of eSOR-SCI template, the interoperable standards were used to capture,
represent, and store the minimum core datasets for the template and made available in
XML format that could be uploaded to proprietary document management systems.
The use of the interoperable standards was necessary to improve the data quality of the
template thereby facilitating secondary data use [13].
Microsoft InfoPath was used to build the eSOR-SCI template. The InfoPath platform
enables users of the system to enter patient’s surgical information and then view and
print the resulting report from the interface. The InfoPath platform also supported the
binding of the template with the modified XML schemas from the HL7 V3 CDA R2
specifications. InfoPath is ubiquitous and is already installed in many computers used
by the clinicians at Capital District Health Authority (CDHA) in Halifax, Nova Scotia.
The model of use comprises of the way that data are captured and displayed in the
eSOR-SCI template. The human interface of the template is made up of two views:
data entry view and report view. The template also consists of several sections
including Patient Information, Surgeon and Anaesthesiologist Information, Procedure
Date and Time, Pre-operative Diagnosis, Indications, Pre-operative Clinical Status (i.e.,
ASIA Scale, ASA Grade, and Glasgow Coma Scale), Co-morbidities, Post-operative
Diagnosis, Anaesthesia Information, Surgical Procedures, Implants, Surgery
Descriptions/Findings, Surgical Drains, Bone Graft, Fluid Intake/Output, Discharge
Disposition, and Author of Operative Report. A user manual has been developed to
assist clinicians and residents in understanding the appropriate use of the template.
In facilitating the model of meaning of the eSOR-SCI template, several concepts and
terminologies, utilized by the template, were encoded to SNOMED-CT expressions
with the uttermost goal of making the stored data reusable to support information needs
of researchers and administrators. Consequently, 391 out of the 402 clinical data
elements presented in the electronic template were encoded to SNOMED CT
expressions, representing 97.26% of all clinical data elements presented in the eSOR-
SCI template. Approximately 2.74% of the clinical data elements were considered to be
difficult to encode using the SNOMED-CT expressions. Table 1 shows the SNOMED-
CT compositional grammar for sample three procedures (i.e., Decompression
laminoplasty, Suboccipital craniectomy, and Facet Rhizotomy) adapted from the
RHSCIR form for procedures.
W. Bonney et al. / Design and Implementation of Synoptic Operative Report Template 199
Table 1. SNOMED-CT compositional grammar for three procedures adapted from the RHSCIR form for
procedures.
Procedures SNOMED-CT Compositional Grammar
Decompression laminoplasty 438374004|laminoplasty|+410756002|surgical decompression|
4. Conclusion
This research project aims to improve the quality of data for primary and secondary use
through implementing an electronic template for synoptic operative reports for spinal
cord injury patients. Success in this project would improve the quality of data, which is
already collected for other research projects, such as the RHSCIR, as well as for the
Canadian Institute for Health Information Discharge Abstract Database (CIHI DAD),
which collects health information from institutions across the country. The eSOR-SCI
template has been implemented and is being piloted by the residents at the Division of
Neurosurgery, CDHA in Halifax, Nova Scotia. The pilot study will gather data on
usefulness, usability, and use of the template for Neurosurgery operative reports.
It is expected that the synoptic reporting will enhance the transfer of information,
which will lead to improvements in the clinical data captured on spinal cord injury
operations for secondary data use (i.e., RHSCIR and CIHI DAD). Moreover, it is
anticipated that the eSOR-SCI template would assist surgeons in quickly completing
clinical documentation; improving data quality to support secondary data use; and
generating standardized reports that are easier to read and interpret. Success is achieved
if the implemented eSOR-SCI template is usable, useful, and used [14].
References
[1] Park J, Pillarisetty VG, Brennan MF, Jarnagin WR, D’Angelica MI, DeMatteo RP, Coit DG, Janakos M,
Allen PJ. Electronic synoptic operative reporting: Assessing the reliability and completeness of
synoptic reports for pancreatic resection. J Am Coll Surg. 2010; 211: 308-315.
[2] Urquhart R, Grunfeld E, Porter GA. Synoptic reporting and the quality of cancer care: A review of
evidence and Canadian initiatives. Oncol Exch. 2009; 8(1): 28-31.
[3] Edhemovic I, Temple WJ, de Gara CJ, Stuart GCE. The computer synoptic operative report: A leap
forward in the science of surgery. Ann Surg Oncol. 2004 Oct; 11(10): 941-947.
[4] Mack LA, Dabbs K, Temple WJ. Synoptic operative record for point of care outcomes: A leap forward
in knowledge translation. Eur J Surg Oncol. 2010 Sep; 36 (Suppl 1): S44-S49.
[5] Vergis A, Gillman L, Minor S, Taylor M, Park J. Structured assessment format for evaluating operative
reports in general surgery. Am J Surg. 2008 Jan; 195(1): 24-29.
[6] Temple WJ, Francis WP, Tamano E, Dabbs K, Mack LA, Fields A. Synoptic surgical reporting for
breast cancer surgery: An innovation in knowledge translation. Am J Surg. 2010; 199(6): 770-775.
[7] Paterson G, Swain D, Christie S, Thibault-Halman G, Behzadi F. Evaluation of SNOMED CT for the
spinal cord injury domain. In: proceedings of AHIC 2012: Advances in Health Informatics Conference.
Toronto, Ontario, Canada. 2012.
[8] Paterson G. Boundary infostructures for chronic disease: Constructing infostructures to bridge
communities of practice. Saarbrücken, Germany: VDM Verlag; 2008.
[9] Abdelhak M, Grostick S, Hanken MA, Jacobs EB. Health information: Management of a strategic
resource. 4th ed. St. Louis, Missouri: Saunders; 2007.
200 W. Bonney et al. / Design and Implementation of Synoptic Operative Report Template
[10] Paterson GI, Soroka SD. Formative evaluation of the clinical pragmatic attributes of components
chosen for a boundary infostructure. In: Abidi R, Bath P, Keselj V, editors. Proceedings of iSHIMR
2006: The 11th International Symposium on Health Information Management Research; Halifax, NS,
Canada: Faculty of Computer Science, Dalhousie University, 2006, p. 375-385.
[11] Rector AL, Qamar R, Marley T. Binding ontologies and coding systems to electronic health records
and messages. Appl. Ontol. 2009 Jan; 4(1): 51-69.
[12] Benson T. Principles of health interoperability HL7 and SNOMED. London, United Kingdom:
Springer-Verlag; 2010.
[13] Bonney W. Improving the quality of primary care data with interoperable standards. In: Traver V, Fred
A, Filipe J, Gamboa H, editors. Proceedings of HEALTHINF 2011: International Conference on Health
Informatics; Rome, Italy: SciTePress; 2011. p. 446-450.
[14] Grant AM, Moshyk AM, Kushniruk A, Moehr JR. Reflections on an arranged marriage between
Bioinformatics and Health Informatics. Methods Inf Med. 2003; 42: 116–120.
Enabling Health and Healthcare through ICT 201
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-201
Introduction
In the last years complex computational systems have been developed in the health
domain. The integration of information is one of the major challenges in health
informatics. Medical reports are often made at several institutions, which use
proprietary vocabularies and, if this information is not properly prepared and presented,
can cause important failures of interpretation. To solve this problem one can resort to
the use of ontologies, which allow to design and model concepts and relations correctly
[1]. Through the languages of the Semantic Web ontologies can be represented by
adding semantic annotations. To facilitate the population of the ontology, automated
processes or semi-automatic ones can be applied. Text mining techniques are applied,
using several techniques, such as natural language processing and machine learning [2].
Thus, through the creation of ontologies and its population, it is possible to represent
medical information, allowing institutions to improve patient health care systems.
The main objective of this work is to create an ontology that allows us to represent
information expressed in medical reports and to create automatic mechanisms that ex-
tract information and carry out a population of this ontology. Therefore, we will obtain
a medical knowledge base, making possible to access it through the use of inference
engines and to develop more powerful health information systems.
1
Paulo Quaresma, [email protected]
202 L. Borrego and P. Quaresma / Processing Medical Reports to Automatically Populate Ontologies
1. Methodology
In the context of this work, we used a set of 590 histopathological reports, which were
supplied by the Veterinary Hospital of the University of Évora. The reports were
analyzed and all the relevant fields identified.
From the analysis of documents it was possible to create an ontology covering the do-
main of histopathological reports. Among the various existing languages to represent
ontologies, the OWL language was chosen because it is more efficient on the Web than
XML, RDF and RDFS, it is a W3C recommendation and, also, because it has a formal
semantics definition.
As methodology to create the ontology we followed the proposal of Noy and
McGuiness [3], from the Stanford University, which defines the sequence of steps
needed to create the ontology and to model the knowledge.
The ontology was built from scratch due to the shortage of work developed in this
area, most likely due to the great complexity of the domain and the fact that Semantic
Web languages are a relatively new area. First, we analyzed all medical reports and
extracted all terms. Then, with the help of the health professionals classes, their
properties and restrictions were created. We normalized all concepts and, when
possible, we used the UMLS terminology (http://www.nlm.nih.gov/research/umls/),
aiming to facilitate the interoperability of the system. The ontology was developed
using the Protege tool (http://protege.stanford.edu/). The overall scheme of the domain
is composed of twelve classes related to each other, as shown in Figure 1.
L. Borrego and P. Quaresma / Processing Medical Reports to Automatically Populate Ontologies 203
The last phase of the work concerns the extraction of information from the XML
labeled medical reports, creating class instances in the ontology. For such, we
developed a Java application that extracts information from reports using the DOM
parser xml (http://www.w3schools.com/dom/dom_parser.asp), processing the
information contained in labels and creating the instances in the ontology using the
Jena framework (http://jena.sourceforge.net). All the information extracted from
“simple” fields, such as, “Report”, “Place”, “Technician”, “Responsible”, “Doctor”,
“Proprietary” and “Animal”, could be directly instantiated in the ontology. Regarding
the content of examinations and of the conclusion, the information needs to be
processed because it corresponds to a text with a set of anatomical entities, lesions and
observations related to each other. Thus, the text of each label is extracted and created a
vector.
For example, for the text “Foi enviado para ana´ lise um fragmento cutaˆ neo com
um no´ dulo cutaˆ neo ulcerado. Ao corte apresentava colorac¸a˜ o acinzentada./It
was sent to be analysed a cutaneous fragment with an ulcerated cutaneous nodule.
When cutting it showed a grayish colour”, we obtain the vector represented by Figure
2.
We developed a 4-step algorithm that processes this initial vector and transforms
it: 1) remove tags within tags; 2) create anatomical entity(ies) that will eliminate some
labels of the vector simplifying it; 3) create lesion(s) that will eliminate other labels of
the vector (“lesionName”, “type”, “size”, “color”, “degreeMalignancy”) simplifying it;
and 4) create relations between anatomical entities, lesions and observations.
The first phase of the transformation of the vector is fundamental because it is
important that instances with labels don’t appear in the ontology. For the previous
204 L. Borrego and P. Quaresma / Processing Medical Reports to Automatically Populate Ontologies
vector, the label ”type” within the label “lesionName” is removed adding a new
position of the vector, as shown in Figure 3.
The second and third phase of the proposed transformation of the vector is mainly to
facilitate, in first place the relationship between the labels occurring in the fourth phase,
and later the creation of instances in the ontology. Figure 4 corresponds to the
application of these two phases to the previous vector.
L. Borrego and P. Quaresma / Processing Medical Reports to Automatically Populate Ontologies 205
After the creation of instances in the ontology, we used once again the Protege´
framework to evaluate the results and perform some tests. In total, for all reports of the
corpus, 7824 instances were created in the ontology. These are correctly structured
according with the texts. We also observed that all information labeled by Minorthird is
instantiated in the ontology and errors were not detected. The results are very positive
and achieve the objectives originally proposed.
Using the Minorthird plugin DL Query, it is possible to make queries and obtain results
directly from the created medical knowledge base. For instance, applying the query
“Which are the medical reports that have exams that have the lesion with the name
‘Furunculosis’?”, the system quickly provides the resulting instances, saving the time
from a manual search and allowing the results to be used in new searches.
3. Conclusion
This work aims to create and to populate ontologies as a way to acquire knowledge
from medical reports. We have used histopathological reports, written in a semi-
structured form and having textual fields. An ontology was created for this domain of
histopathological reports and heuristic and machine learning techniques were applied,
allowing to initially automatically label the reports and, at a later stage, to relate entities
and automatically populate the ontology. The results show that this is a promising
approach, allowing the automatic extraction of relevant information.
The use of information systems in the health domain is nowadays indispensable,
aiming to help professionals in health or patient care. Our work helps to support the
claim that the use of ontologies in medical reports can represent an important step in
this area, providing new and more powerful tools and supporting either investigators or
health professionals in their work.
References
[1] Quaresma P, Bratsas CH, Maglaveras N. A framework to describe problems and algorithms in medical
informatics via ontologies. In 4th European Symposium on Biomedical Engineering, Patras, Greece.
University of Patras, 2004.
[2] Feldman R. Sanger J. The Text Mining Handbook Advanced Approaches in Analyzing Unstructured
Data. Cambridge: Cambridge University Press, 2007.
[3] Noy N, McGuinness D. Ontology Development 101: A Guide to Creating Your First Ontology.
Stanford Knowledge Systems Laboratory Technical Report KSL-01-05 and Stanford Medical
Informatics Technical Report SMI-2001-0880, March 2001.
[4] Cohen W. Minorthird: Methods for Identifying Names and Ontological Relations in Text using
Heuristics for Inducing Regularities from Data. 2004. Available from: http://minorthird.sourceforge.net
This page intentionally left blank
Health Modeling
This page intentionally left blank
Enabling Health and Healthcare through ICT 209
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-209
Introduction
1
Corresponding Author.
210 K. Keshavjee et al. / A CAS Perspective of Health Information Technology Implementation
implementation over the continuum of providers who provide care, the patients who
receive care, the organizational context in which care is provided, the tasks that are
associated with a process, the environment within which the organization exists and
other relevant processes and constraints [7].
Complex adaptive systems (CAS) are a means of looking at the complexity and
interacting issues that define HIT implementation [8]. CAS can help reframe the
thinking about HIT implementation and acknowledge the non-linearity of processes
and how behaviors and tasks emerge as a result of non-linearity [9].
Surprisingly, a search for complexity theory and CAS within the health informatics
literature yields a paucity of articles [8,9]. While these studies have suggested that HIT
implementation issues are characteristic of CAS, they have not specifically looked at
HIT implementation through the CAS lens.
The medication prescribing, dispensing and administration process is one of the
most common sources of medical errors due to the complexity of the process and the
significant potential for harm from potent medical interventions [11, 12]. Therefore it is
an excellent process to study from the perspective of CAS. In this paper we analyze
and link the prescribing process to CAS theory. This allows us to leverage the prolific
research being done in other disciplines for gaining insight into developing a
framework for HIT implementations.
1. Methods
We developed and used a use case of prescribing and dispensing in the community to
identify CAS elements. Figure 1 displays the use case.
We analyzed the prescribing and dispensing process using the lens of CAS. We
listed the features of CAS and identified instances of prescribing and dispensing which
fit those features. Our goal was to identify a mechanism or method to predict
unintended consequences and to predict areas of higher risk in HIT implementation.
Figure 1. Simplified medication prescribing, dispensing and administration process in the outpatient setting
K. Keshavjee et al. / A CAS Perspective of Health Information Technology Implementation 211
2. Results
Non-linear processes Changes in one area affect things in another area, but in non-linear and non-
intuitive ways. E.g., decreasing drug-drug interactions leads to increased
work for physicians. Some errors in prescribing are easily solved by
pharmacists; others require a phone call.
Emergent behavior Unintended consequences of HIT [2,3]. Workload shifts from pharmacist to
physician when e-prescribing is implemented.
Requisite Variety Each network of prescribers, dispensers and patients has its own unique
characteristics. There are no uniform methods for prescribing and dispensing.
Changes that work in one network do not work in other networks.
Simple Rules Prescriptions have a very small number of elements, yet they can have far-
reaching implications. The act of prescribing is made of several simple rules
(the contents and process of a prescription) but the end-result is an
information-rich document with far more complex interpretations and
implications for patient health.
Self-organization Pharmacists can monitor prescription problems and work with patients and
physicians to correct them. E.g., during the recent large scale switching from
one opioid to another in Ontario.
A review of Table 1 quickly demonstrates the futility of trying to predict the path
and outcomes of an HIT implementation when there are large numbers of interacting
variables with non-linear relationships between them. Even a downturn in the
economy can cause a perturbation in an HIT implementation if strategic plans change
because of financial issues.
The key insight to emerge from our analysis is that HIT implementations are
inherently unpredictable and that unintended consequences are to be expected. Rather
than trying to predict and control an implementation, CAS theory points to an approach
that is more organic and iterative in nature: implement a small, minimally viable part of
the HIT and observe the results (non-linear processes, feedback loops). Measure the
impact of the implementation on key goals (i.e., develop new feedback mechanisms for
the new system, watch out for emergent behaviours). Listen to nay-sayers and
objectors, they may have useful advice to give and may be your best predictors of what
could go wrong (self-organization, feedback loops). Make course corrections and
adjustments (feedback, self-organization). Watch out for boundary issues such as
regulations impinging on workflow (non-discrete boundaries). Don’t expect that what
worked in one setting will necessarily work in another (requisite variety). Develop
new mechanisms for communication, especially for meta-communication
(connectivity). Implement some more. This approach is reminiscent of quality
improvement approaches such as the Plan-Do-Study-Act cycle which has been a
successful model for generating change in complex clinical situations and is also
reminiscent of the lean start-ups movement which promotes the idea of continuous
quality improvement with small, incremental changes.
3. Discussion
In this paper we advocate for using complex adaptive systems theory to study HIT
implementation issues and to make recommendations on how we can improve the HIT
implementation process. The real strength in complex adaptive systems is in the word
system. Healthcare processes are not disparate entities that exist in isolation. Rather
they are complex systems with multiple actors and parts that are not linear but are
defined by variation and dynamic evolution. Complex adaptive systems acknowledge
that complexity exists in a system because of the range of actors and the multiple ways
that they integrate with one another while engaging in system processes. Modeling
healthcare processes from the perspective of this complexity allows us to understand
the impact that implementing HIT will have a on a process such as how the burden of a
task may shift from one provider to another or how the process may evolve over time.
The end point of our analysis is that CAS does not provide prediction, but forces us
to rethink our HIT implementation paradigms and question what we think we know.
CAS provides us new ways to conceptualize HIT implementation and suggests new
approaches to increasing HIT implementation successes.
References
[1] Institute of Medicine. Crossing the quality chasm: a new health system for the twenty-first century.
2001. Washington.
K. Keshavjee et al. / A CAS Perspective of Health Information Technology Implementation 213
[2] Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care:
the nature of patient care information system-related error. Journal of the American Medical
Informatics Association 2004 (11); 104–112.
[3] Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of
unintended consequences related to computerized provider order entry. Journal of the America Medical
Informatics Association 2007 (14); 415– 423.
[4] Koppel R, Wetterneck T, Telles JL, Karsh BT. Workarounds to barcode medication administration
systems: their occurrences, causes, and threats to patient safety. Journal of the American Medical
Informatics Association 2008 15 (4); 408-423.
[5] Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability:
the relationship between usability problems and prescription errors when using a handheld application.
International Journal of Medical Informatics 2005 (74); 519-526.
[6] Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health
care – an interactive sociotechnical analysis. Journal of the American Medical Informatics Association
2007 14 (5); 542-549.
[7] Sittig DF, Classen D. Safe electronic health record use requires a comprehensive monitoring and
evaluation framework. Journal of the American Medical Aassociation 2010 303(5); 450-451.
[8] Westbrook JI, Braithwaite J, Georgiou A, Ampt A, Creswick N, Coiera E, Iedema R. Multimethod
evaluation of information and communication technologies in health in the context of wicked problems
and sociotechnical theory. Journal of the American Medical Informatics Association 2007 14; 746–755.
[9] Ellis B, Hebert SI. Complex adaptive systems (CAS): an overview of key elements, characteristics and
application to management theory. Informatics in Primary Care 2011 19; 33–37.
[10] Koppel R, Leonard C, Localio AR, Cohen A, Auten R, Strom BL. Identifying and quantifying
medication errors: evaluation of rapidly discontinued medication orders submitted to a computerized
physician order entry system. Journal of the American Medical Informatics Association 2008 15(4);
461-465.
[11] Phillips DP, Bredder CC. Morbidity and mortality from medical errors: an increasingly serious public
health problem. Annual Reviews of Public Health 2002 23; 135–150.
214 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-214
Abstract. Jurisdictions in Canada, the US, the EU and Australia are struggling
with regulation of ever evolving software in medicine. Recently this discussion has
had a focus on electronic medical records (EMRs). There is a mountain of evi-
dence that EMRs have actualized potential to lead to the injury of patients through
the information they offer to facilitate care. We are undertaking a systematic re-
view of relevant literature in the field to uncover some of the latent hazards. We
hypothesize that this exploration, using a variation on Leveson's system theoretic
accidents models and processes (STAMP) model as a classification tool, will pro-
vide two benefits. First, the model will be sufficient to capture the complexity of
the domain and its hazards, thus providing a holistic perspective on the problem.
Second, the classification process will provide insight as to what steps might be
taken to mitigate the risk that medical errors associated with these software tools
will arise in health care systems which employ them. In this continuation of our
study we still have not been able to produce evidence which contradicts either hy-
pothesis.
Introduction
1
Corresponding Author.
F. Mason-Blakley and J. Weber / A Systems Theory Classification of EMR Hazards 215
and tools we used to extract the results we present in section four. In section five will
provide our conclusions and a discussion of the work that remains.
1. Background
We have motivated this topic a number of times in our EMR hazards research [1, 2,
3].We will provide a cursory overview of Leveson's STAMP model, an overview of
previous work, and we will review and revise a number of definitions.
Leveson [4] describes a theory she calls the System Theoretic Accidents Models and
Processes (STAMP). A fundamental underlying concept of this theory is that when
accidents occur, it is not because of human error or even any single particular weakness
in the system of participants and interactions from which that accident arose. Much like
the "Swiss cheese" model for which Reason [5] advocates, Leveson proposes that in-
stead, accidents occur as a consequence of the alignment of a number of weaknesses in
the control structure that is intended to prevent them. That is to say that any system,
whether it be the aeronautics industry, the nuclear power industry or the health care
industry can be modeled as a web of control loops, and each loop and control node in
that loop has its own weaknesses. Leveson describes this control web as being com-
prised of interactions and controllers that rely on process models to affect their control
of processes in the system. Weaknesses in control structures can come in two primary
forms, weak or missing controllers, or weak or missing control interactions.
There are a number of notable authors who have provided critical perspectives on the
benefits of EMRs. These tools are popularly touted as the cure to the ills of the health
care system. The media has at times implied that with the use of EMRs medical error
and economic waste in health care will be a thing of the past. Authors like Bates [7],
Ash [8], Koppel [9], van der Sijs [10], Saleem [11] and others however have put this
notion in doubt.
Saleem [11] identified a sense of resentment in professional practice against EMR
tools. This resentment is a reaction to the failure of the studied tools to satisfy the
workflow of their users. He describes how the elements of the EMR system operate in
unison to provide care. The studied care providers used the EMR tools, but also used a
variety of paper workarounds when the EMR tools did not support their workflow. In
order to understand the safety of the EMR system which Saleem studied, one would
need to understand all of its components and could not inspect the software in isolation.
Safety is an emergent property.
In [1] we described the first iteration of the study we now present. In [3] we pro-
vide a STAMP-centric hazard analysis of an e-iatrogenesis case study by Horsky [6]. In
[2] we provide a formal methods focused evaluation of that study.
216 F. Mason-Blakley and J. Weber / A Systems Theory Classification of EMR Hazards
1.3. Definitions
We provide here our definitions which will constrain our problem. We will define the
term EMR and we will also contrast it to two other terms: EHR, and PHR. These are
provided to clarify the inclusion and exclusion criteria which were used for the corpus
selection. We will refine and refactor the definitions based on out work in [1]. Finally,
we will refine the definition of e-iatrogenisis [6], to clarify our intended sense of the
term safety.
Electronic Medical Record: An EMR is a unit/suite of software that is used
for collecting, storing, manipulating or making available clinical information
about a patient's health care delivery process to facilitate the progression of
that patient's care.
Electronic Health Record: An EHR is a unit or suite of software that is de-
signed or used for extracting data about a patient's process of care from an
EMR for the purpose of maintaining an epidemiological understanding of a
population.
Personal Health Record: A PHR is much like an electronic medical record
except for one fundamental difference. A PHR is used by the patient to record
their health care process rather than by their physician.
E-Iatrogenesis: E-Iatrogenesis can be defined as medical error which is
caused (in part) by IT.
2. Corpus Identification
The corpus of work which has been inspected to date in this iteration of our literature
review was extracted strictly from the Engineering Village indexing service. This ser-
vice indexes a breadth of technical literature including IEEE and ACM, but does not
cover medical literature. Upon completion of this study we intend on including Pub-
Med as a source and seeking additional expertise in identifying other medical sources.
In order to capture the initial corpus of papers we executed the same query as in
[1]. This initial query yielded 116 results of which 21 were finally included in the re-
view. Inclusion and exclusion criteria were applied to determine acceptance in the
study; however, as the degree of agreement between the authors as to how the criteria
are to be applied remains low at this time, the criteria will only be summarized here. At
the completion of this study it is the aim of the authors to include an analysis of the
agreement on the application of the criteria including the use of a metric such as
Cronbach's alpha [13]. Here, we will only state that the papers were filtered through
three phases that inspected titles, abstracts and finally full papers. This filtering process
will by the end of this study be supplemented by including papers that are highly cited
in the primary corpus in a secondary corpus, and similarly a tertiary corpus will be in-
cluded based on references in the primary and secondary corpora.
The Atlas.TI version 6.2.27[13] application was used to record observations about the
papers in the corpus. This tool allows annotation and quotation of the text of pdfs. Dur-
F. Mason-Blakley and J. Weber / A Systems Theory Classification of EMR Hazards 217
ing the study, only when the information about the reported e-iatrogenesis hazards and
or instances were sufficient to populate an error report template based on the nomencla-
ture of Phillips and Gong [15] was information extracted. The information extraction
template consists of two primary elements: an error state and a precipitating event.
The error state is comprised of three subcomponents: the error element, the error
condition, and the error context. The error element describes the precise element which
was in error. The error condition describes how the error element was in error. Finally,
the error context describes how the error element fits into the broader context of the
software system.
The precipitating event is comprised of three elements as well: the event agent, the
event task, and the event context. Event agents are users who participated in the initiat-
ing or propagating the error state. The event task is the activity in which the event agent
was engaged when they initiated or propagated the error state. A challenge of the
methodology is that though much could be reported about context of e-iatrogenesis
instances and hazard, typically little is reported.
4. Results
The initial query returned 116 results. Of those 116 results, 30 were duplicate listings, 5
referred to non-articles/books, and 1 had been retracted. 29 papers of the initial 116
were excluded based on title, 26 were excluded based abstract, and 5 papers were ex-
cluded based on full content. 21 papers remained for inclusion in the primary corpus.
We uncovered 200 reported e-iatrogenesis instances or hazards. In order to classify
these discoveries, we employed an EMR specific instance of the STAMP model. We
present the annotated model in Figure 1. The annotations in this figure represent the
number of reported e-iatrogenesis instances or hazard and. the number of parent con-
cepts respectively. The arrows represent the control channels between controllers and
processes.
In analyzing the distribution of hazards across the model we present in Figure 1, it
is clear that two primary themes emerge: tool interaction issues and weaknesses in pro-
cess control. The tool interaction weaknesses are expressed across the nodes and edges
to which the EMR and physician process models are attached. Upon inspection of the
hazard descriptions it becomes clear that these hazards can be broken down into two,
non-exclusive subcategories: tool/work flow and display/control issues.
Figure 1. Representation of the STAMP model used to classify hazards and e-iatrogenesis instances.
218 F. Mason-Blakley and J. Weber / A Systems Theory Classification of EMR Hazards
The most immediate conclusion which can be drawn from the results is that our hy-
potheses were not disproved. It is no surprise that the issues with tool/work flow mis-
match and user interface issues were highlighted as major hazard and e-iatrogenesis
sources. Experts in the field have been writing about these topics since the inception of
the EMR. The third theme which arose from this study, the theme of problems with
system controls, is more interesting in that it has been less explored in this domain.
If there were a single lesson to be learned from the preliminary results of this
study, it would be that additional effort in quality assurance, both in its enforcement
and execution may be key to reducing the frequency and severity of e-iatrogenesis. By
employing quality assurance controls which are used in developing EMRs including
user acceptance testing and usability assessment we may be able to simultaneously
mitigate tool/workflow mismatch issues as well as user interface problems.
References
[1] F. Mason-Blakley, J. Weber, CIS system hazards derived from literature using systems and human
factors perspectives. IHI 2012 (2012), 419-428.
[2] J. Weber, and F. Mason-Blakley, On the Safety of Electronic Medical Records, FHIES 2011 (2011),
177-194.
[3] J. Weber, F. Mason-Blakley, The Safety of Electronic Medical Record (EMR) Systems: What Does
EMR Safety Mean and How Can We Engineer Safer Systems? ACM SIGHIT Record 1(2) (2011), 13-22.
[4] N. Leveson, Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press, 2011.
[5] J. Reason, Human Error: Models and Management, BMJ 320(7237) (2000), 768-770.
[6] J. Horsky, G. Kuperman, V. Patel, Comprehensive Analysis of a Medication Dosing Error Related to
CPOE, JAMIA 12(4) (2005), 377-382.
[7] D. Bates, et al. Improving EHR Accuracy and Increasing Compliance with Health Maintenance Clinical
Guidelines Through Patient Access and Input, IJMI, 75(10-11) (2006), 693- 700.
[8] J. Ash, M. Berg, E. Coiera, Some Unintended Consequences of Information Technology in Health
Care: The Nature of Patient Care Information System-Related Errors, JAMIA 11 (2004), 104-112.
[9] R. Koppel, J. Metlay, A. Cohen, B. Abaluck, A. Localio, S. Kimmel, B. Strom, Role of Computerized
Physician Order Entry Systems in Facilitating Medication Errors, JAMA 293 (3/2005), 1197-1203.
[10] H. van der Sijs, T. van Gelder, A. Vulto, M. Berg, J. Aarts, Understanding Handling of Drug Safety
Alerts: A Simulation Study, IJMI 79(5) (2010), 361-369.
F. Mason-Blakley and J. Weber / A Systems Theory Classification of EMR Hazards 219
[11] J. Saleem, A. Russ, C. Justice, H. Hagg, P. Ebright, P. Woodbridge, B. Doebbeling, Exploring the Per-
sistence of Paper with the Electronic Health Record, IJMI 78(9) (2009), 618-628.
[12] R. Holden, Cognitive Performance-Altering Effects of Electronic Medical Records: An Application of
the Human Factors Paradigm for Patient Safety, Cognition, Technology and Work 13(1) (2011), 11-29.
[13] M. Allen, W. Yen, Statistics notes: Cronbach's Alpha, BMJ (1997) 314-572.
[14] Scientific Software. Atlas.ti. [cited Sep. 2012], Available from: http://www.atlasti.com.
[15] W. Phillips, Y. Gong, Developing a Nomenclature for EMR Errors. HCI: Interacting in Various Appli-
cation Domains 13th International Conference. HCI International 2009 5613 (2009), 587-596.
220 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-220
Introduction
Clinical practice guidelines (CPG) have been used more frequently during the last
years [1]. Continuity of care programs containing CPG aimed at planning and decision
support for healthcare professionals are therefore being developed [2]. The Capital
Region of Denmark is exploring the potential benefits of an information system
supporting the planning and coordination of chronic patient across sectors [3]. Patients
with Chronic Obstructive Pulmonary Disease (COPD) and Diabetes Mellitus Type 2
(DM2) are selected to establish a proof of concept project. Currently there are no
information system supporting the coordination and planning across community
nursing, general practitioners and hospitals in Denmark. The consequence is limited
planning and reduced coordination across the three sectors followed by decreased
quality and compliance of CPG. International experiences indicate that IT-systems can
enhance compliance as well as quality of care [4;5].
The Capital Region in Denmark has launched a project: “Chronic 5” that aims to
demonstrate the potential benefits of a Planning and Coordination Module (PCM). The
project analyzes and specifies requirements for such a system and builds and tests a
PCM prototype. Clinician end-users, clinical managers, quality managers, IT-architects
and health informaticians performed the analysis and the specification.
The PCM is basically designed to establish and maintain a cross organizational
overview and virtualized management of all health services in individual patient cases
among all relevant health actors – including the patient. All health services in an
individualized patient plan are mapped to relevant CPGs
1
Sanne Jensen: [email protected]
S. Jensen et al. / Benefits of a Clinical Planning and Coordination Module: A Simulation Study 221
The purpose of PCM is to support the coordination across sectors, concerning the
status and planning for patients with COPD and DM2 according to the CPG, and
handling of derived activities and services. This digital support will be groundbreaking
in Denmark, and will offer new opportunities for coherence and continuity in the care
activities. Moreover it will possibly ensure a higher compliance to the existing
continuity programs and CPG.
To realize the intended benefits of a PCM usability of the system is pivotal [6].
Usability may be defined as “extent to which a system, product or service can be used
by specified users to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use” [7]. When using simulation it is possible to
assess the effect of an information system in different contexts as well as evaluating
efficiency, satisfaction and effectiveness [8]. The objective of the simulation study was
to assess the potential benefits of a PCM for health care professionals involved in
planning and coordination of patients with COPD and DM2, primarily focusing on the
efficiency of the PCM, and secondary on satisfaction. Efficacy and effectiveness has
not been assessed.
This paper presents the test of the PCM-prototype and the results from the test, and
discusses the potential benefits and concerns of a PCM. Furthermore the use of
simulation as a method for testing potential use of clinical information systems is
discussed.
1. Method
The test was conducted as a controlled full-scale simulation study. The concept of
simulation has been used for training medical skills during the last 40 years [9] and has
during the last decade been used to assess health information systems [10]. A
simulation study makes it possible to assess the use of a prototype in a realistic
environment [11], and is well suited for assessing potential impact [12] as well as
cognitive processes and usability [13].
The test encompassed 18 simulation runs including six general practitioners (GP),
six community nurses, six hospital physicians and two simulation patients. The
simulation runs were bundled into six tests. In each test healthcare professionals from
each of the three end-user groups were participating. 10 scenarios were composed; five
about a patient with COPD and five about a patient with DM2. The scenarios covered
planning of therapy and further diagnosing concerning a recently diagnosed patient at
the GP, visitation at the community nurse, rehabilitation at the community nurse,
treatment of a patient at an outpatient clinic due to exacerbation of the condition, and
assignment of responsibility from the hospital physicians to the GP. The scenarios
resembled different points of impact focusing on core functionalities and the
assignments from one healthcare professional to another. Interface issues such as colors,
buttons and minor functionalities were not part of the assessment. The scenarios were
composed to assess nine hypotheses i.e. the first nine questions in Figure 1.
Before the simulation took place the testers were introduced to the concept and the
functionalities of the PCM and they could get hands-on in order to get acquainted with
the information system. During the test the same general tasks were performed. In
cooperation with the “patient” and on the basis of the existing findings and plans, the
healthcare professionals were asked to revise and modify the plans for the patient. The
prototype had simulated integrations to other information system in order to replicate
222 S. Jensen et al. / Benefits of a Clinical Planning and Coordination Module: A Simulation Study
2. Results
The results from the questionnaire are shown in Figure 1. The horizontal scale depicts
the median of the respondents answer on a five point likert agree/disagree scale to the
24 questions on the vertical axis. The hypotheses tested in the first nine questions were
verified. Among the remaining questions only one obtained a lower score than 3 i.e. the
question concerning whether the PCM would release more time to be spend with the
patients. From the interviews, however, the general opinion was that the PCM would
reduce the time spend on the planning and coordination, but it remain unresolved,
whether the time would be spend with the patients. This result was the only one with
discrepancy between interview and questionnaire.
The core concept of the PCM was assessed as being very useful and creating many
benefits. New ideas were brought up during the interviews – eg. the PCM could be a
coaching tool for senior doctors and an instrument for communication among
colleagues or between other groups of healthcare professionals. Primary care nurses
were not part of the original scope, but were spotted as new potential users by a GP
who saw the PCM as a very valuable tool for them. Also quality management was
perceived to be enhanced, and the content of referrals and discharge letters could
possibly be reduced, since information concerning the patient would be known by all
parts.
Most of the healthcare professionals had difficulties understanding the concept of a
PCM in the beginning. The concept was innovative and forced them to see planning
and coordination in a new way. The simulation and observation of the others using the
S. Jensen et al. / Benefits of a Clinical Planning and Coordination Module: A Simulation Study 223
3. Discussion
The healthcare professionals found potential clinical benefits in using the PCM, which
would improve quality and patient safety. Furthermore new future users were
discovered and new potential ways of using the PCM were revealed. Only simulated
patients were used during the test, but several potential benefits for the patients were
detected. A supplementing simulation with genuine patients would therefore be
recommended to test the sustainability of these observations. The healthcare
professionals were quite satisfied with the realism in the simulation, and it helped them
to gain insight in the possibilities of PCM.
The scenarios did not cover all possible applications of the PCM but were
composed to enable assessment of the nine hypotheses. A simulation test does not fully
224 S. Jensen et al. / Benefits of a Clinical Planning and Coordination Module: A Simulation Study
resemble the use of an information system in the clinic, but offers a high degree of
realism, depending on the degree of fidelity. A simulation test should therefore not be a
substitution for a pilot implementation, but regarded as a complementary test without
risk of injuring real patients.
Several issues that were brought up e.g. terminology and responsibility had not
been visible before the simulation test, but are very relevant and needed to be addressed
prior to implementation. Furthermore it was discovered how the simulation was a
powerful learning tool for the new users in spe.
The results from this simulation study conclude that GPs, community nurses and
the hospital physicians and patients will benefit from a PCM. The benefits include
improvements in communication, planning and coordination, work practice, and
quality management. Several organizational issues have to be addressed including use
of terminology and delegation of responsibilities before an information system as PCM
can be implemented. Furthermore the results show that full-scale simulation studies are
a useful method for testing the feasibility of information systems.
References
[1] Hanson D, Hoss BL, et al., Evaluating the evidence: guidelines, Aorn Journal 16 (2008), 388-393.
[2] National Board of Health in Denmark. Clinical Practice Guidelines concerning patients with chronicle
deceases; recommandations from the Danish National Board of Health in Denmark.
http://www.regionh.dk/menu/sundhedOghospitaler/Til+fagfolk/Kronisksygdom/Forloebsprogrammer/I
ndhold_forloebsprogrammer/Forloebsprogrammer+for+kronisk+sygdom/Kronisk+sygdom+programme
r.htm (in Danish) (accesses March the 22nd, 2012).
[3] Program for projects concerning chronicle patients in the Capital Region of Denmark 2011
http://www.regionh.dk/NR/rdonlyres/F783D082-5E74-469D-87A9-88755A39A9A3/0/ Kronikerpro-
grammets_projektkatalog_sep_2011.pdf (accesses June the 15th, 2012).
[4] Hogan TP, Wakefield B, et al., Promoting access through complementary eHealth technologies:
rcommandations for VA's Home Telehealth and personal health record programs, Journal of General
International Medicin 26 (2011), 628-635.
[5] Jackson GL, Krein SL, et al., Core Issues in Utilizing Information Technology to Improve Access:
Evaluation and Reseach Agenda, Journal of General Internal Medicine 26 (2011), 623-627.
[6] Peute LW, Spithoven R, et al., Usability studies on interactive health information systems; where do we
stand? Stud Health Technol Inform 136 (2008), 327-332.
[7] ISO 9241-11, Ergonomic requirements for office work with visual display terminals (VDT's). Part 11:
Guidance on usability, 2012.
[8] Dahl Y, Alsos OA, Svanæs D. Fidelity Considerations for Simulation-Based Usability Assessments of
Mobile ICT for Hospitals. Journal of Human-Computer Interaction, 26(5) (2010), 445-476.
[9] Lippert A, Dieckmann PG, et al., Simulation in medicine, Notfall Rettungsmed. 12 (2009), 49-52.
[10] Borycki E, Kushniruk A, et al.. Designing and Integrating Clinical and Computer-based Simulations in
Health Informatics: From Real-World to Virtual Reality. Modeling, Simulation and Optimization –
Focus on Applications (2010), 31-52.
[11] Rossnagel H, Zibuschaka J. Using Mobile Media for emergency Management - A Design Approach
Proceedings of the 8th International ISCRAM Conference (2011).
[12] Ammenwerth E, Hackl WO, et al. Simulation studies for the evaluation of health information
technologies: experiences and results. Health Information Management Journal, 41(2) (2012), 14-21.
[13] Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical
information systems. J Biomed Inform 37(1) (2004), 56-76.
[14] Jaspers MW, Steen T, van den Bos C, Greenen M. The think aloud method: a guide to user interface
design. International Journal of Medical Informatics 73 (2004), 781-795.
[15] Kragelund L. Obser-view seen as a data generating method and a learning space. 2009.
http://static.sdu.dk/mediafiles/Files/Om_SDU/Institutter/Ifpr/Konferencer/Adult_Learning/Abstracts/3
%20LindaKragelund.pdf ( accesses June the 15th 2012).
[16] Kjeldskov J, Skov MB, Stage J. Instant data analysis: conducting usability evaluations in a day.
NordiCHI '04 Proceedings of the third Nordic conference on Human-computer interaction (2004).
Mobile Technologies and Telehealth
This page intentionally left blank
Enabling Health and Healthcare through ICT 227
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-227
Abstract. There is a large and ever increasing number of mobile phone health,
wellness, and medical applications on the market. However, there is little guidance
or quality assurance available for consumers. This paper provides a health
consumer focused framework for considering a range of issues associated with
selecting and using mobile phone applications downloaded from the Internet.
Keywords. Mobile phone apps, M-health, data storage issues, software issues
Introduction
1. Background
Mobile phone applications are small software programs that provide a specific
functionality and can be downloaded onto a phone via the Internet. Most frequently
they are associated with smart phones, which use a proprietary operating system (OS)
1
Elizabeth Cummings, School of Computing & Information Systems, University of Tasmania, Private
Bag 87, 7001, Hobart, Tasmania. E-mail: [email protected]
228 E. Cummings et al. / Issues and Considerations for Healthcare Consumers
and allow the user to download specifically designed applications that run only on that
OS, although cross platform mobile apps are now beginning to be developed. Many
mobile phone applications can only be run using the Internet to provide real-time
information and data exchange whilst others are independent from the Internet and
developed as standalone applications, which do not require Internet access. Other apps
can be operated in either mode, with restricted functionality in the offline mode.
It has been postulated that smart phone applications can transform mobile phones
into assistive devices for people with disabilities [4]. As smart phones become more
affordable and the applications remain relatively low cost there is a rapidly increasing
acceptance and use of assistive applications by consumers. Consumer use of mobile
applications is varied. Some consumers use mobile phones to obtain relevant health
information over the World Wide Web (WWW). Other consumers have taken
advantage of the growing availability of mobile applications that are specifically
targeted towards assisting individuals with the promotion of their own health and
wellness as well as the self-management of chronic illnesses such as chronic
obstructive pulmonary disease, diabetes and cardiovascular disease. Other mobile
applications are provide tools that improve communication with health care providers
or facilitate the delivery of patient care, for example: animated videos to educate
patients about the human body, wellness, prevention activities, disorders and diseases.
There are currently more than 13,500 medical, healthcare and fitness applications
in the apple app store alone. The majority of these applications are aimed at consumers
rather than healthcare professionals. Although many are potentially good applications,
researchers and health professionals are increasingly advocating that consumers
consider the types of applications they are downloading or purchasing with some
skepticism, particularly when selecting and using health focused applications [5]. Some
of these considerations form the focus of this paper.
2. Method
The authors of this paper had each independently examined and used a range of mobile
applications for health and wellbeing. This examination had led to considerable
concern regarding the security and quality of mobile phone applications and the data
they collect. In this context we developed a “Consumer Perspective Framework” (see
Table 1) within which we consider the use of mobile applications from a consumer
perspective. The framework emphasizes some key aspects of mobile phones and their
associated applications that need to be assessed by consumers when considering their
use. Key elements of the framework will be discussed in the following section.
3. Discussion
Two major areas of concern relating to mobile phone software applications and data
storage were identified: (1) data issues and (2) software issues.
There are several data issues that consumers should concern themselves with,
including: (1) storage and privacy, (2) ownership, (3) corporate use and (4) location.
Storage and Privacy: One challenge is keeping personally identifiable health
information secure. Essentially, mobile health requires the use of strong encryption and
authentication processes to ensure the security of these data. However, as patients are
increasingly recognising the potential benefits of digitally recording their health
information this issue is becoming more discussed in the patient population.
Researchers such as Walker et al [6] have identified that patients are less concerned
about privacy than health care professionals. However, Gilbert et al [7] emphasise that
ensuring privacy is not violated where sensitive information is concerned is a difficult
problem to overcome. There is minimal evidence of the use of secure networks or
encryption (as in eHealth applications).
Ownership: There is minimal information available about these mobile phone
applications at the point of purchase regarding data storage, use (primary and
secondary), durability and ownership. These issues need to be considered when
entering personal data, including health data, into a mobile phone application. This
information may not be available to the consumer via the app store prior to purchase.
There is a requirement that all applications cover these issues in their licensing
agreement, which the consumer accepts at the time of initial use. However, some users
do not understand the importance of these licensing agreements (which by requirement
are very long). One application’s licensing agreement, containing standard information,
was 26 screen pages long and difficult to read on a mobile phone due to the small
screen size. Thus the important elements of the agreement may be difficult to read.
Corporate Use: Some low cost mobile phone applications are sponsored by private
enterprises, which can have implications for data privacy and security. One of the key
issues is the secondary use of data as application sponsors often provide support to
developers in exchange for access to end user personal data. Although, through
licensing agreements, this is agreed to by end users it is unlikely that the implications
of such agreements have been understood by consumers. Third parties do not have to
disclose their actual use of the data and application developers have little or no control
over secondary data use. There is limited information about the duration of data storage,
or its subsequent use. In the event that the consumer is aware of secondary data
transmission, there is the element of consent to the frequency of data transmission.
There is concern about the timing of data transmission and to what extent consumers
expect their information to be transmitted automatically, and what information is
transmitted?
Location: The applications available through the Apple iTunes store are required
to meet certification requirements regarding the content and development of the
application [8]. This certification does not extend to when and where personal
information may be sent once the application’s disclaimer has been accepted by the
user. No certification process is required for apps available through the android market
[8] and so there is little evidence of where the data is processed or stored. The location
230 E. Cummings et al. / Issues and Considerations for Healthcare Consumers
There are several software issues that also need to be considered: (1) accessibility, (2)
clinical effectiveness, (3) credibility, (4) information quality, and (5) consumer usage.
Accessibility: One of the first issues confronting consumers of health and
wellbeing applications is the issue of how to locate them. Each of the different phone
operating systems (e.g. android) has its own propriatary application stores (app stores)
where applications can be purchased and downloaded. The sheer number of
applications currently avaliable has the potential to make the search and selection of a
useful health or wellness app overwhleming. The majority of app stores have relatively
unsophisticated search mechanisms and so for individuals who are uncertain of what
they want may find identifying and selecting an app time consuming and cumbersome.
Clinical Effectivness: There are a large number of medical, health and wellbeing
applications. Most of these applications have a limited description of the application’s
purpose and few reviews documenting the application’s clinical quality (thereby
making selection of mobile applications on the basis of clinical effectiveness more
difficult for consumers). Thus, mobile applications may or may not even function as
expected, let alone be of use to the individual in supporting a wellness behaviour or
helping with the self-management of a consumers’ disease. Where applications are
calculating health or wellbeing measures from data inputs it is important to know and
understand the means, or algorithms, by which these calculations are being made.
Inaccurate calculations, or non-standard ones, may be in conflict with current medical
trends and advice and may lead to medical error [10]. It is also possible that results may
be ambiguous or misinterpreted by consumers with low health literacy.
Credibility: Anyone can create an application so it is difficult to differentiate
credible, safe applications from those which are not. There is little information
available in an app store to indicate credibility and if there are reviews these can also be
biased or based on unhelpful metrics. As a result of this, there is a need for
undestanding of the possible issues for both consumers and health care practitioners.
There is potentially a role for policy development which will assist people in
classifying these applications. For example inn the USA the Food and Drug
Administration recently released draft guidelines for Mobile Medical Applications [9].
Information Quality: As with health related websites, there is concern about the
quality and accuracy of information and advice provided through mobile phone apps.
For the scope of this paper the concept of information quality is made up of several
elements: (1) quality of the information in the application program incorporated by
developers, (2) quality of the information entered into the application by consumers,
and (3) quality of information received by the user through their interaction with the
application.
Consumer Usage: The motivations behind the use of mobile apps have the
potential to impact on the quality of the information recorded. Is the health consumer
E. Cummings et al. / Issues and Considerations for Healthcare Consumers 231
after real behavioural change or are they after information to show a healthcare
provider to illustrate compliance with a care program? Information quality has the
potential to impact on healthcare consumers overall program of care, depending on the
credibility that the application holds with the healthcare provider and the methods by
which it is used by the consumer.
Having devised and described this framework the authors are intending to move to
the next stage and conduct research with consumers to test the reliability of the
framework. This work is intended to provide a consumer focused understanding of the
most appropriate method by which the framework can be implemented for use by
individual consumers to assist them in selecting the most reliable applications. This
work will include a comprehensive exploration of literature relating to the framework.
4. Conclusion
There are many facets to consider when selecting a mobile app for use with health and
wellbeing in mind. Consumers are encouraged to consider the security and secondary
use of the data prior to engaging with these applications. App stores and developers
should be encouraged to provide more reputable information about the privacy and
security of the data as well as the efficacy and source of the programs. Healthcare
professionals as well as consumers, providers and developers have a role to play in
ensuring that only high quality efficacious apps are developed into the future. In this
paper we describe a framework that can be used by consumers and health professionals
when considering mobile application for healthcare use.
References
[1] International Telecommunications Union (ITU), ICT data and statistics. 2012. [cited 2012 Jan 20]
Available from: http://www.itu.int/ITU-D/ict/statistics/.
[2] Mobithinking. Global mobile statistics 2012 Part A: Mobile subscribers; handset market share; mobile
operators. 2011[cited 2012 Aug 21] Available from: http://mobithinking.com/mobile-marketing-
tools/latest-mobile-stats/a#subscribers.
[3] Jones J, Hook S, Park S, Scott L. Privacy, security and interoperability of mobile health applications.
Proceddings of the 6th international conference on universal access in human-computer interaction.
Orlando, FL. Springer-Verlag; 2011.
[4] Turisco F, Garzone M. mHealth for your organization, CSC, 2011. [cited 2011 Oct 10]. Available from:
http://assets1.csc.com/health_services/downloads/CSC_Harnessing_the_Value_of_mHealth.pdf,.
[5] Doughty K. SPAs (smart phone applications) – a new form of assistive technology. Journal of Assistive
Technologies. 2011; 5(2): 88-94
[6] Walker J, Ahern DK, Le LX, Delbanco T. Insights to Internists: I Want the Computer to Know Who I
Am. J Gen Intern Med. 2009; 24(6): 727-732
[7] Gilbert P, Chun BG, Cox L, Jung J. Automating privacy testing of smartphone applications. Technical
Report CS-2011-02, 2011. Duke University
[8] Rose C. Smart Phone, Dumb Security. Review of Business Information Systems, 2012; 16(1): 21-
262012
[9] FDA, Draft Guidance for Industry and Food and Drug Administration Staff: Mobile Medical
Applications, July 21, 2011.
[10] Baylis TB, Kushniruk AW, Borycki EM.. Low-cost rapid usability testing for health information
systems: Is it work the effort? Stud Health Technol Inform. 2012; 180: 363-7.
232 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-232
Introduction
Searching about the clinical use of mobile devices produces impressive results about
their increasingly ubiquitous use by care providers. Anecdotes about Apple iPads
having improved teaching and medical practice are common. However, is this
technology hyperbole or are there tangible benefits?
1
Corresponding Author. Jonn Wu, BCCA, 600 West 10th Avenue, Vancouver, BC V5Z 4E6, Canada;
E-mail: [email protected].
J. Wu et al. / The Introduction and Evaluation of Mobile Devices to Improve Access 233
1. Statement of Problem
The BC Cancer Agency (BCCA) uses an electronic health record named CAIS;
however, it lacks adequate physical space for clinicians to access records via desktop
computers. Also, clinicians do not have access to computers in the patient rooms,
during clinical conferences or grand rounds.
2. Purpose
3. Methodology
The project followed a two-phased approach. Phase I gave users the ability to
download clinical reference applications and access the Cancer Agency Information
System (CAIS) using a generic viewer (Citrix). Phase II incorporated usability
feedback from Phase I to deploy a highly configurable health record viewer delivered
as a native iPad application.
3.1.2. Instrumentation
Participants were emailed with a link to complete two confidential surveys. This
involved a survey before the iPads were distributed and a survey after twelve weeks of
use. Participants rated items on a Likert scale.
3.2.2. Instrumentation
Communication was addressed with an intranet website. Participants were emailed with
a link to the intranet site and listed the four steps required to enroll in the pilot initiative.
Step 1 provided information about the purpose of pilot; Step 2 incorporated privacy and
security requirements by having participants sign a project privacy agreement; Step 3
provided a baseline survey that aligned with Phase I; and Step 4 provided instructions
on how to launch the patient record viewer application on the iPad.
4. Results
Pre Usage
In the pre-usage survey, the user group rated themselves as very comfortable with
mobile devices, with a large majority (86%) of the group using mobile devices daily
and 78% already finding mobile devices useful in their professional practice. A
majority (67%) of the users envisioned that having a mobile tablet would enhance their
clinical workflow, with 43% of users already having an idea of what functions they
could employ, and close to all users (91%) looking forward to adding a mobile tablet to
their clinical practice.
Post Usage
Upon completion of the three-month pilot project, a large majority of the users (83%)
reported having an overall positive experience using the mobile tablet, with only 3% of
users summarizing the pilot project as a negative experience.
The mobile tablet was reported as being easy to carry around (70%), had
satisfactory screen resolution (63%) and an appropriate screen size (50%). Users found
using the mobile tablet an efficient way to check and write email (62%) and review
electronic records (55%), with a majority (63%) of users noting a positive impact on
their workflow through the use of a mobile tablet; highlighted by 76% of users
explaining that CAIS through their mobile tablet was more up to date than the patient’s
paper chart. In addition, once users became comfortable with the mobile tablet, over
60 percent of users began searching out other medical application to incorporate into
their clinical workflows.
Initially, a number of users had minor complaints regarding Internet connectivity
(38%), and login issues to PHSA Network (16%). In most cases, the help of on-site
support addressed these issues. Another suggestion was to increase the font size of the
CAIS viewer (55%).
In summary, users found that the use of a mobile tablet was a useful tool in their
clinical practice (73%), with the tablet primarily being used for email (96% of users),
review of imaging and pathology (90% of users), and review of labs and appointment
notes (87% of users). For 75% of users, the use of a mobile tablet met or exceeded
their expectation in terms of functionality in their practice.
Pre Usage
In general, the results show that clinicians at BCCA are among the early adopters of
mobile computing technology according Rogers’ Diffusion Theory (with more than
90% of the respondents are using mobile computing technology in their clinical
practice). The majority of the respondents (> 80%) are either very comfortable or
comfortable with using mobile computing technology.
BCCA clinicians highly appreciate (98%) the in-time availability of information, a
privilege peculiar to mobile technology that subsequently offer clinicians the ability to
respond instantly to patients questions and enquiries. Other highly ranked and expected
functionalities include: using the iPad for patient education, communication purposes,
and for clinical decision-making. Moreover, almost all respondents (98%) expect that
236 J. Wu et al. / The Introduction and Evaluation of Mobile Devices to Improve Access
the iPad and other mobile computing technology to be become larger part of the cancer
care experience for both clinicians and patients.
Post Usage
The majority of the post-implementation survey respondents (92%) are either highly or
moderately satisfied with the iPad and VH Chart application. BCCA clinicians rated
the iPad project between 4 and 5 with 5 being extremely positive. The iPad application
visual interface, multiple functionalities, and ease of use were the main drivers for
clinicians’ positive feedback. Figures 1, 2 and 3 illustrate early utilization.
Based on the results of the post-pilot survey results, most of the post-
implementation survey respondents (92%) found that the response time, security and
reliability of the iPad and the mobile application to be either moderately or highly
acceptable. In terms of the information quality, most of respondents (92%) found the
information within mobile application to be either highly or moderately acceptable.
Similarly, most participants (90%) agree that there were sufficient technical support
and training resources for the iPad Mobility project. in general, most of the respondents
(95%) rated the project implementation process either moderately or high acceptable.
In terms of productivity and efficiency, clinicians reported reduction in the number
of interruptions during patient visits and a reduction in the need for printing (reports,
conference documents). Most of the survey respondents (> 80%) believe that the iPad
improved their workflow as it facilitated the retrieval of information from CAIS. This
allowed the clinicians to involve the patients in care planning, increase collaboration
with other healthcare providers and enhance support in decision.
5. Conclusion
The majority of clinicians supported the use of a mobile device to improve access to
patient health records, but recognized limitations of a generic viewer to simulate a
desktop view of CAIS. Clinicians preferred the iPad mobile application because of its
ergonomic features and ease of use. Also, clinicians provided feedback to enhance the
mobile application features. Notable is that most clinicians viewed the iPad application
as complementary to their desktop computer and improved their workflow.
Figure 1. Phase II – Usage Statistics for First Month - # Logins per day.
J. Wu et al. / The Introduction and Evaluation of Mobile Devices to Improve Access 237
Figure 2. Phase II – Usage Statistics for First Month - # Unique Logins per day.
Figure 3. Phase II – Usage Statistics for First Month – Total Logons by top 20 users.
238 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-238
Introduction
Over the past thirty years, personal computer use has traditionally tethered users
to a fixed point due to both the physical size of devices, the requirement of an
electrical outlet as a power source, and the use of hardwired network cabling.
Recently, personal computer technology has seen an immense change through the
decrease in component size, increase in microprocessor performance, advancements
in battery technology and the rise of cellular communications and mobile Internet.
Since the introduction of the Apple iPad in the spring of 2010, an immediate shift
in the form of touch-based tablet computer devices has been observed. Touch
enabled technology is becoming more and more prevalent today and its influence
can be observed in the interfaces of the latest computer operating systems such as
Microsoft Windows 8’s ‘metro’ interface and the Launchpad in Apple’s Mac OS
10.8 Mountain Lion.
Computer devices have become extremely portable with fingers and voice
replacing the mouse and keyboard as the primary means of control. Although the
introduction of such technology has been exciting, especially to many in the health
care industry, it has not been quickly embraced by all health care organizations [1].
1
Corresponding Author: Joseph Lindsay, Room A2102, Parkwood Hospital, 801 Commissioners Rd E,
London CA; Email: [email protected]
J. Lindsay et al. / Moving Mobile: Using an Open-Sourced Framework 239
A major issue with touch-based technology is the fact that previous software,
which was designed with a traditional mouse and keyboard interface in mind, was
not intended for use with a touch-based interface. Regardless of how accurate the
touch screen at recognizing the user’s touch, the accuracy is restricted by the size
and position of a user’s finger [2, 3]. Although these software programs may
function flawlessly on a touch-based device, many are not practical due to the
usage of small buttons, fonts and very complex menu systems. This paper
describes the approach taken to overcome this problem when encountered by a
team of students at the School of Health Information, University of Victoria
(UVic).
1. Background
The project began as a team assignment in a graduate level database design course
offered at the School of Health Information Science, UVic. This course is one
requirement of the MSc degree in Health Informatics, a unique program in Canada.
The four student team members came from a variety of backgrounds and
experiences, all with strong clinical backgrounds but only one with technical
expertise. None of the team members had experience programming in SQL though
skills were developed throughout the course with the guidance of the instructor,
Dr. M.H. Alex Kuo. This course ran for just over a three-month period, from
January until the beginning of April
2012.
In the class, teams were instructed to design and build an electronic medical
record (EMR) application using Oracle 10g/XE and the application builder in Oracle
Application Express (APEX). For this project, it was decided to design the
application to be used in the Aging Brain and Memory Clinic (ABMC) at Parkwood
Hospital, in London Ontario Canada. This clinic’s primary function is to assess
referred geriatric patients who have complaints of memory impairment.
Through a needs-assessment conducted in the ABMC, tablet computers were
identified as the preferred primary clinical data-capturing device, with a desktop
version also being required. Currently, the data recorded in ABMC clinical
assessments is completed on paper as no electronic tool has been found which
meets the needs of such a specialized geriatric clinic. The preference for staff to
use a tablet device was not surprising, as mobile computing is fast becoming one
of the main methods in which electronic information is utilized today. Some of the
newest and most popular computer tools on the market are not provided or
supported by hospitals, but yet are still being carried by the physicians, residents,
fellows, and nurses [4].
The omnipresence of powerful handheld communication devices, such as
smartphones and tablets, are fundamentally changing both the quantity and quality of
information available to clinicians at the point of care [5]. These mobile devices
are far more than storage devices; they are sophisticated tools with beautiful full-
color screens, high-speed internet access, and offer users the ability to
communicate, via voice or text, to anyone in the world. Tablet computers have
become noticeably popular as physician adoption for professional purposes nearly
doubled in the United States between 2011 and 2012, reaching 62% according to
the Taking the Pulse survey [5]. In addition, the study stated that one-half of
240 J. Lindsay et al. / Moving Mobile: Using an Open-Sourced Framework
tablet-owning physicians, who participated in the national survey, have used their
device at the point-of-care.
Due to the increasing demand for mobile information, the team began to
research options for enabling an Oracle APEX application to be used conveniently
on a tablet device with a touch-based interface. Team members first came across
the Internet blog of Marc Sewtz, a senior software development manager at Oracle
in APEX [6]. The blog post was entitled Getting started with mobile in apex and
familiarized team members with the jQuery Mobile open-source user interface
framework. As summarized in the jQuery Mobile documentation, jQuery Mobile is
“a unified user interface system that works seamlessly across all popular mobile
device platforms”[7]. Through careful examination, the jQuery Mobile framework
was selected as the user interface for the mobile version of the EMR application.
2. Methods
Dr. Michael Borrie, D irector of the ABMC, provided the team with the paper based
assessment tools and clinic forms which were used to create the dataset for the EMR
system. The dataset was normalized and then each table was programmed into
Oracle 10g/XE in the SQL Workshop in APEX. The latest version of APEX
(version 4.1) for this project, however version 3 and above would have met the basic
requirements to complete the project while allowing for the use of the jQuery
Mobile framework.
The core application, entitled mEMRy, was created for use as a desktop
version of the EMR and was able to perform the features that were required by
the clinic staff. These features included a modular design, being able to capture
required data fields, the ability to database the images of completed cognitive tests,
and the generation of summary reports based on the data collected in clinic visits.
Using the desktop version of mEMRy as a framework, the team began to create
new APEX templates, which allowed the standard APEX user interface to be replaced
by the jQuery Mobile interface.
In APEX, the way in which data is displayed on the screen is determined by the
selected template. For jQuery Mobile to be enabled and used as the user interface in
the application, mobile templates needed to be created for every data element chosen
for display on a page. Figure 1 shows a storyboard illustration of the initial
search page and is labeled to indicate the type of APEX element along with the name
of the template used to enable the element for use with the jQuery Mobile interface.
Templates were created with the awareness of the ergonomics of touch screen
use and then tested using a trial and error process [2, 3, 8]. Team members tested
and took observations of the mobile and desktop versions of the mEMRy
application on the iPad 3, Samsung Galaxy Tab, Windows XP desktop computer
(1024x768 resolution display), Windows 8 Tablet (Acer Iconia T ab W500), and an
Apple Mac Mini (1920x1080 resolution display). As the development of these
applications was completed for academic purposes, no formal evaluation or usability
assessment was conducted on either version of the application.
J. Lindsay et al. / Moving Mobile: Using an Open-Sourced Framework 241
3. Results
Once the final prototype version of the mEMRy mobile EMR had been
completed, 41 new mobile APEX templates had been created. Of those, 22
templates were used in the application with 2 being button templates, 5 page
templates, 8 region templates and 7 report templates. The desktop and mobile
versions of mEMRy are featured in figure 2 respectively. When viewing both the
mobile and desktop versions of mEMRy on a tablet device, a number of interesting
points were observed. In the desktop version on both the iPad 3 and Samsung
Galaxy Tab, text and visual elements overlapped one another consistently. Screen
orientation needed to be locked in order to prevent the page from being reoriented
when the tablet was rotated. Many buttons were difficult to activate by touch and
were found to be much smaller than the 20mm size recommended by ergonomists [8].
Text based links were also found to be difficult to select and activate.
Figure 2. The desktop (right) and mobile (left) versions of the mEMRy application.
242 J. Lindsay et al. / Moving Mobile: Using an Open-Sourced Framework
In using the mobile application, it became quickly apparent that there was a
conflict between the jQuery Mobile framework and APEX. If page transitions were
enabled, page links seemed to act erratically with some links functioning correctly,
others becoming inactive, and others pulling information from linked pages and
displaying it on top of the previous page content. Through troubleshooting and
consulting the jQuery Mobile source documents, it was found that page transitions
operated using Asynchronous JavaScript and XML (Ajax) could be disabled with a
simple command in the header of the page template. In the upcoming release of
Oracle APEX 4.2, jQuery Mobile is planned on being fully supported, with a base
mobile theme included in the package, and full support for Ajax [9]. Once Ajax was
disabled, the jQuerly Mobile framework was successfully incorporated into APEX as
the user interface. All pages displayed properly on every device and no limitations in
performance were observed.
4. Discussion
References
[1] Gold J. For Hospitals, There’s No App For That, Kaiser Health News. [Internet]. 26 Dec 2011. [cited 8
J. Lindsay et al. / Moving Mobile: Using an Open-Sourced Framework 243
Abstract. The purpose of this review paper is to explore the impacts of social
media on healthcare organizations, clinicians, and patients. This study found that
healthcare organizations, clinicians and patients can benefit from the use of social
media. For healthcare organizations, social media can be used primarily for
community engagement activities such as fundraising, customer service and
support, the provision of news and information, patient education, and advertising
new services. The study also found that the most widely used social media venues
for physicians were online communities where physicians can read news articles,
listen to experts, research new medical developments, network, and communicate
with colleagues regarding patient issues. Patients can benefit from the use of social
media through education, obtaining information, networking, performing research,
receiving support, goal setting, and tracking personal progress. Future research
should further examine other financial, technological, informational, ethical, legal,
and privacy issues surrounding the use of social media in healthcare.
Introduction
The use of social media by healthcare institutions, clinicians, and the public has
increased over the past few years. This use of social media is part of a growing trend
and is due to a realization that healthcare institutions and clinicians need to be more
engaged with their patients. Social media provides an online platform for interactions
to occur around various health topics relating to patient education, health promotion,
community outreach, public relations, and crisis communication [1]. Social media
includes various approaches such as blogs, microblogging (e.g., Twitter), social
networking (e.g., Facebook and Patients Like Me), video and file sharing (e.g.,
YouTube), e-games, and wikis. Despite the use of social media related to healthcare,
the effects of social media on patient care have not been well documented, and with
1
Corresponding Author: Dr. Mowafa Househ, King Saud Bin Abdul Aziz University for Health
Sciences, College of Public Health and Health Informatics, Riyadh, Kingdom of Saudi Arabia; Email:
[email protected]
M. Househ / The Use of Social Media in Healthcare 245
over 80% of Americans seeking health information online [1], the impacts of social
media on health cannot be ignored.
With the widespread use of the internet and the relatively low cost of bandwidth,
social media, especially social networking, is being adopted by healthcare professionals
and patients. There has been growth in the use of social media, especially online social
networking, related to healthcare. Perhaps one of the earliest studies carried out to
assess the use of social media in healthcare focused the use of the Bulletin Board
System for the education of nurses [2]. That study found that the major impediments to
implementing such a system were cost, maintenance, and the lack of training of nurses.
In a commentary published by the Journal of the American Medical Association,
Shachak and Jadad argue that the use of social networks will eventually lead to a more
patient-centered healthcare system that will improve communication and information
flow between patients, providers, and administrators [3]. Other studies have shown that
Facebook users seek health information [4], and social networking websites, such as
Patients Like Me, are beginning to cater to the need for social networks devoted to
providing health information [5, 6]. With the growing use of social media in healthcare,
the purpose of this paper is to review the impacts of social media on healthcare
organizations, clinicians and patients.
1. Organizational perspective
2. Physician perspective
Physicians have also been users of social media for both professional and personal
interests. It is estimated that 65% percent of physicians use social media for
professional purposes [10]. The most widely used social media venues for physicians
are online communities where physicians can read news articles, listen to experts,
research new medical developments, network, and communicate with colleagues
regarding patient issues [10]. Physicians rarely use social media sites to communicate
with patients. In fact, the American Medical Association has advised physicians to
maintain “appropriate boundaries of the patient-physician relationship” and has
recommended that physicians should “consider separating personal and professional
content online.” A study documenting patient privacy violations on the Facebook
profiles of medical students and residents found that medical students were more likely
than residents to violate privacy, especially when working in medical missions in
developing countries [11]. The study concluded that medical professionals have a
responsibility to use social networking sites in an ethical manner that does not violate
patients’ privacy rights. However, physicians are beginning to develop an interest in
interacting with patients online. Approximately 60% of physicians favor interactions
with patients through social media, showing strong support for patient education,
followed by monitoring patients’ health, behavior and drug adherence and giving care
advice to patient groups that would lead to “better education, increased compliance,
and better outcomes” [10]. The challenge in the future will be for physicians to develop
guidelines regarding how to interact with patients through social media websites in a
manner that is ethical and that does not violate patients’ privacy rights.
3. Patient perspective
Patient groups have benefited the most from the use of social media for health
purposes. It is estimated that in the United States, 74% of all internet users utilize social
media [12], and searching for health information online has become one of the most
popular online activities [1]. Patients can use social media for a variety of reasons that
include education, obtaining information, networking, research, support, goal setting,
and tracking personal progress. Patients have the ability to express themselves, share
their stories, learn from others and spread health knowledge [10]. Social media creates
a forum for patient participation that extends beyond the reach of the hospital or the
local clinic. These forums can help empower and uplift patients when they read the
experiences of other patients [10]. Some physicians believe that social media can be
beneficial for patients with chronic diseases, cancer, rare diseases, and depression and
for patients with questions or goals related to maternal and infant care, wellness and
prevention, and weight management [10].
The introduction of social media sites such as Facebook, Twitter, and YouTube
has revolutionized the way individuals seek, share and use information, specifically
health information. A recent study on the sharing of sensitive health information
through Facebook found that Facebook users openly sought and shared behavioral,
mental, and genetic information [6]. An interesting finding from this study was that
most Facebook users included in the study publicly identified themselves by divulging
their name, photo, and location when seeking sensitive health information through
online postings. The author of the study suggested that the growing use of social media
M. Househ / The Use of Social Media in Healthcare 247
4. Discussion
5. Conclusions
References
[1] H. Korda, Z. Itani, Harnessing Social Media for Health Promotion and Behavior Change, Health
Promotion Practice (2011): 1-9.
[2] M.M. Russin, J.H. Davis, Continuing education electronic bulletin board system: provider readiness and
interest, J Contin Educ Nurs 21(1) (1990),23-7.
[3] A. Shachack, A. Jadad, Electronic Health Records in the Age of Social Networks and Global
Telecommunications, JAMA 303 (5) (2010), 452-453.
[4] M. Househ, Sharing Sensitive Personal Health Information through Facebook: the Unintended
Consequences, Stud Health Technol Inform 169 (2011), 616-20.
[5] M. Househ, Re-examining Perceptions on Healthcare Privacy - Moving from a Punitive Model to an
Awareness Model, HEALTHINF 2012 - Proceedings of the International Conference on Health
Informatics, Vilamoura, Algarve, Portugal, 1 - 4 February, 2012. SciTePress, 2012, 287-291.
[6] M. Househ, Mobile Social Networking Health (MSNet-Health): Beyond the mHealth Frontier, Stud
Health Technol Inform 180 (2012), 808-12.
[7] Social media use in healthcare for quality improvement, [cited Jun 21, 2011], [updated Jan 15, 2012],
Available from: http://tcf.innovationcell.com/book/archive/2010/10/general-use-social-media-
healthcare-organizations.
[8] Greystone Net, [cited Jan 25, 2012], [updated Jan 18, 2012], Available from:
http://www.greystone.net/wtn/Page.asp?PageID=WTN000018.
[9] N. Hum, et al. A picture is worth a thousand words: A content analysis of Facebook profile photographs,
Computers in Human Behavior (2011) 1828-1833.
[10] M. Modahl, L. Tompsett, T. Moorhead, Doctors, Patients and Social Media [cited Sep, 2011], Available
from: http://www.quantiamd.com/q-qcp/doctorspatientsocialmedia.pdf.
[11] L. Thompson, et al, Protected Health Information on Social Networking Sites: Ethical and Legal
Considerations, Journal of Medical Internet Research (2011): e8.
[12] P. Eckler, G. Worsowicz, W. Rayburn, Social Media and Health Care: An Overview, Practice
Management (2010), 1046-1049.
Evaluation
This page intentionally left blank
Enabling Health and Healthcare through ICT 251
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-251
Introduction
1
Corresponding author
252 Y. Quintana et al. / POND4Kids: A Global Web-Based Database
2004-05
2005-01
2005-07
2006-01
2006-07
2007-02
2007-08
2008-02
2008-08
2009-02
2009-08
2010-02
2010-08
2011-02
2011-08
2012-02
2012-08
Figure 1.Total Number of Logins (all sites) Figure 2. Records in POND Guatemala
3. Provide educational support for data collection, quality control, and analysis.
4. Provide a multi-lingual system that supports data sharing and comparison in a
universal format while allowing data entry and reporting in the user’s native
language.
This paper presents a technical overview of the system, reviews global usage of
POND, identifies some of the barriers and facilitators for adoption and usage of POND,
and provides an example of successful POND use at the National Pediatric Oncology
Unit of Guatemala, and highlight on-going challenges.
POND4Kids allows uniform data collection so that clinical data can be shared and
meaningfully compared between centers. POND4Kids uses a centralized web server
(Linux, MySQL, and PHP) that can host multiple virtual sites. Each site has an
administrator that can add users and control the level of access of each user. Data can
be shared with individuals at another POND site, and sharing can be deactivated at any
time. By design, personal information such as names cannot be shared between sites,
such that shared data always complies with HIPAA privacy rules in the US, which are
similar to privacy rules in many other countries. The intent is to allow data about a
particular disease to be shared with a disease expert to assist with analysis and
development of strategies to improve outcomes. Users can create Quick Data Entry
forms that can be used within a POND site or shared with other sites via a common
library. Protocols templates can also be created and used locally, or shared with other
sites via a common library. Using the online meeting capabilities of the Cure4Kids
website (www.Cure4Kids.org), training is provided on best practices for clinical data
collection and clinical quality improvement programs. Additional details of the POND
system can be found in [4-6]
Y. Quintana et al. / POND4Kids: A Global Web-Based Database 253
2. Outcomes
Since its launch in 2003, POND use has increased steadily, with use spreading only by
word of mouth in the pediatric hematology/oncology community. As of September 12,
2012, POND has 1,222 users from 233 sites in 66 countries. There are over 50,000
records and 800 collaborative data shares and 4,000 logins each month (Figure 1).
2000 900
800
1500 700
600
500
1000
400
300
500 200
100
0 0
2005-12
2006-06
2006-12
2007-06
2007-12
2008-06
2008-12
2009-06
2009-12
2010-06
2010-12
2011-06
2011-12
2012-06
2005-12
2006-06
2006-12
2007-06
2007-12
2008-06
2008-12
2009-06
2009-12
2010-06
2010-12
2011-06
2011-12
2012-06
3. Discussion
POND provides clinical tools that fill an unmet need at many pediatric oncology
hospitals, particularly those that have common protocols and wish to share information.
There are several reasons for the success of POND at some sites, and lack of use at
other sites. Challenges include training, adequate data manager staffing, and quality
management. Availability of online training and support is one of the factors that has
helped the deployment of POND but at times it can be difficult due to limited
bandwidth and lack of Internet reliability. POND is sufficiently easy to use with
minimal training for most users. Online help and training are available in English and
Spanish. However, data managers need to have computer experience, and some may
need additional supervision and guidance. A regional network of POND data managers
was formed in Central America that allows for continued training and mutual support
and encouragement. The UNOP site in Guatemala has provided regional training
support for other Latin American centers both online and by hosting data managers
from other Spanish-speaking sites when they are starting their data management
programs.
The availability of appropriately trained data mangers and funding for salaries is a
major challenge for many sites. Lack of consistent funding for data managers at some
sites has led to inconsistent data entry. Since 2004 a grant from the Pediatric Oncology
Group of Ontario (Canada) and financial support from St. Jude Children’s Research
Hospital (Memphis, TN) have funded full-time data managers in several Central
America countries, and these tend to be the sites with most active use. Other sites do
not have sufficient funding to hire data managers, and the clinical staff does not have
the time to enter records. The UNOP site now has 5 data managers that facilitated the
widespread use of POND by doctors, nurses, nutritionists, and psychosocial support
staff.
High-quality data is the cornerstone on which quality improvement rests, so POND
has specific quality reports that identify the most common data entry errors, including
unlikely and impossible values [4]. However, these reports need to be regularly run and
acted upon to improve areas of missing or incomplete data. There may also be a lack of
standard operating procedures, inconsistent data collection methods, or missing paper
records. More training and on-site audits could facilitate additional improvement in
data quality and efficiency. Recent additions to POND include error-checking tools that
show missing or possible out of range data. UNOP also has implemented a daily review
all data entered in POND in the previous day that has greatly improved data quality.
Security is an important part of any medical records system. The POND software
resides in a dedicated server with a security firewall. Users with administrator
credentials manage access to data by users of the site and control sharing with external
collaborators. Although the data are encrypted and password protected, some
administrators are reluctant to store data outside their institution. The location of the
POND server in the United States has provided some concern to some sites. Future
development of a local POND system would facilitate use by those sites that do not
wish their data to reside on foreign servers.
To obtain benefit from the data-sharing capabilities of POND, access to clinical
experts is essential. Common treatment protocols used by pediatric oncology units in
Central America were created in POND. Principal investigators of shared protocols
have regular virtual meetings to review individual patients with complicated medical
problems, protocol data, and administrative issues, and to develop strategies to improve
Y. Quintana et al. / POND4Kids: A Global Web-Based Database 255
4. Conclusions
Acknowledgments
We thank support from the American Lebanese Syrian Associated Charities (ALSAC)
Cancer Center Support (CORE) grant CA-21765 from the National Cancer Institute,
the Pediatric Oncology Group of Ontario, and a grant from Johnson and Johnson Inc.
(Centocor and Janssen divisions).
References
[1] S. Howard, M. Metzger, Y. Quintana, CH Pui, L. Robison, R. Ribeiro, Childhood cancer epidemiology
in low-income countries, Cancer 112 (2008), 461-472.
[2] S.C. Howard, M. Marinoni, L. Castillo, M. Bonilla, G. Tognoni, F. Antillon, M.G. Valsecchi, C.H. Pui,
R.C. Ribeiro, A. Sala, R.D. Barr, G. Masera, MISPHO Consortium. Improving outcomes for children
with cancer in low income countries in Latin America, Pediatric Blood Cancer 48 (2007), 364-369.
[3] M. Valsecchi, E. Steliarova-Foucher, Cancer Registration in developing countries: luxury or necessity?
The Lancet Oncology 9 (2008),159-167.
256 Y. Quintana et al. / POND4Kids: A Global Web-Based Database
[4] L. Ayoub, L. Fu, A. Pena, J.M. Sierra, P.C. Dominguez, C.H. Pui, Y. Quintana, A, Rodriguez, R.D. Barr,
R.C. Ribeiro, M.L. Metzger, J.A. Wilimas, S.C. Howard. Implementation of a data management
program in a pediatric cancer unit in a low income country, Pediatric Blood Cancer 49 (2007), 23-27.
[5] Y. Quintana, A.N. Patel, P.E. Naidu, S.C. Howard, F.A. Antillon, R.C. Ribeiro, POND4Kids: A Web-
based Pediatric Cancer Database for Hospital-based Cancer Registration and Clinical Collaboration,
Studies in Health Technology and Informatics 164 (2011), 227-31.
[6] Y. Quintana, R. O’Brien, A. Patel, J. Becksfort, A. Shuler, A. Nambayan, D. Ogdon, G. Chantada, S.C.
Howard, R.C. Ribeiro, Cure4Kids: Research challenges in the design of a website for global education
and collaboration, Information Design Journal 16 (2008), 243-249.
[7] E. Supriyadi, P.H. Widjajanto, I.Purwanto, J. Cloos, A.J. Veerman, S. Sutaryo, Incidence of childhood
leukemia in Yogyakarta, Indonesia, 1998-2009. Pediatr Blood Cancer. 57(4) (2011), 588-93.
Enabling Health and Healthcare through ICT 257
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-257
Introduction
1
Corresponding Author.
258 J. Bassi et al. / Knowledge Translation in eHealth: Building a Virtual Community
increase capacity in conducting eHealth benefits evaluation. This paper describes the
theory behind KT and describes how it has been applied in the context of eHealth to
create a KT strategy.
strategies, expertise, and resources. Based on these guides, we describe our KT activity
below in terms of:
Goal: the purpose of sharing knowledge and its expected action
Message: the knowledge being shared
Audience: individuals and/or groups that should receive the knowledge
Strategy: approaches used to translate the knowledge
Expected Impact: effect of the KT activity
2. The KT Plan
2.1. Goal
2.2. Message
In terms of knowledge creation, the UVic eHealth Observatory was created in 2009 to
monitor the effects of HIS deployment in Canada [14]. Researchers at the eHealth
Observatory have been collaborating with Canada Health Infoway to explore and
synthesize evidence around HIS evaluation. Five types of knowledge products have
been created: (1) conceptual eHealth evaluation models; (2) a rapid evaluation toolkit;
(3) systematic literature reviews; (4) educational materials, and (5) the results of
eHealth evaluation studies. For example, the Benefits Evaluation Framework specifies
a set of dimensions and categories for factors which influence HIS adoption [15]. A
knowledge product stemming from the Benefits Evaluation Framework is the updated
Infoway Benefits Evaluation Indicators Technical Report [16]. We are also aware of
other sources of knowledge that would be valuable to those working in this area. For
instance, the Agency for Healthcare Research and Quality in the United States has
developed the Health IT Evaluation Tookit [17]. This particular KT activity focuses on
the HIS domains of EMRs in physician offices, prescribing, and consumer health
systems such patient health portals, with data quality as an overarching area. Emphasis
is placed on the exchange portion of KT as we are stressing the need for individuals to
260 J. Bassi et al. / Knowledge Translation in eHealth: Building a Virtual Community
contribute their own knowledge and experiences as well. In this way, knowledge
creation and sharing is the message at the core of this KT activity.
2.3. Audience
2.4. Strategy
KT activities should expect to elicit change or have a positive impact. For example, the
impact can be a direct change in practice or indirect such as informed decision making
[9]. According to McCartney et al. [19] there is no consensus on measures of success
for electronic communities but some potential areas are effectiveness in meeting
objectives and health of the community in terms of member satisfaction and level of
activity. Barwick et al. [10] list examples of reach, quality, and use indicators and
potential output and process measures. For this KT Community, our impact measures
include statistics on participation in live sessions and discussion forums, contributions
to the repository and case studies. The majority of our impact evidence will be
collected through user feedback surveys completed upon joining the community, after
each live session, and at the conclusion of the project. The KT Community currently
has attracted over 70 health care professionals, researchers and developers,
demonstrating a great interest and need. Although we may not be able to measure it
directly, the larger impact will be in terms of the frequency and quality of benefits
evaluations conducted in the field using the knowledge shared through the KT
Community as well as the new knowledge and ideas generated from interactions
between community members. Evaluation is often not included within the scope of HIS
projects, and is sometimes considered the exclusive domain of HIS researchers. By
bringing together a diverse audience, the community hopes to challenge perceptions
around roles of evaluators and to encourage broader participation in evaluation.
3. Conclusion
Acknowledgement
Funding support for this work was provided by a Canadian Institutes of Health
Research Knowledge Translation Grant.
References
[1] R. Scott, C. Saunders, M. Palacios, D. Nguyen, S. Ali. Technology-Enabled Knowledge Translation and
Our Environment, In: K Ho, S Jarvis-Selinger, HN Lauscher, J Cordeiro, R Scott (Eds). Technology
Enabled Knowledge Translation for eHealth Principles and Practice. New York, Springer, 2012: 325-
337.
[2] S.E. Straus, J. Tetroe, I. Graham. Defining knowledge translation. CMAJ. 181(3-4) (2009), 165-168.
[3] National Physician Survey. 2007 results. 2007. [cited Aug 20, 2012]. Available from:
http://www.nationalphysiciansurvey.ca/nps/2007_Survey/2007nps-e.asp.
[4] National Physician Survey. 2010 results. 2010. [cited Aug 20, 2012]. Available from:
http://www.nationalphysiciansurvey.ca/nps/2010_Survey/2010nps-e.asp.
[5] Ontario Hospital Association (OHA), eHealth Adoption Findings and Comparisons: Levels of eHealth
Adoption across Ontario, Canada and US Hospitals, 2011. [cited Aug 23, 2012]. Available from:
http://www.oha.com/CurrentIssues/Issues/eHealth/Pages/eHealthAdoptionFindingsandComparisons.as
px.
[6] F. Lau, M. Price, J. Boyd, C. Partridge, H. Bell, R. Raworth. Impact of electronic medical record on
physician practice in office settings: a systematic review. BMC Med Inform and Decis Mak 12 (2012),
10.
[7] Canadian Institutes of Health Research (CIHR). PowerPoint Presentation: About Knowledge Translation
at CIHR, 2009. [cited Aug 20, 2012]. Available from: http://www.cihr-irsc.gc.ca/e/39158.html.
[8] I.D. Graham, J. Logan, M.B. Harrison, S.E. Straus, J. Tetroe, W. Caswell, N. Robinson. Lost in
knowledge translation: time for a map? J Contin Educ Health Prof. 26 (2006), 13-24.
[9] R. Reardon, J. Lavis, J. Gibson. From Research to Practice: A Knowledge Transfer Planning Guide.
Toronto: Institute for Work & Health, 2006.
[10] M. Barwick, D. Butterill, D.M. Lockett, L. Buckley, P. Goering. Scientist Knowledge Translation
Training Manual. Toronto, The Hospital for Sick Children / Centre for Addiction and Mental Health,
2005.
[11] Canadian Institutes of Health Research (CIHR). Knowledge to Action: An End-of-Grant Knowledge
Translation Casebook. Ottawa, Her Majesty the Queen in Right of Canada, 2010.
[12] D.M. Berwick. Disseminating innovations in health care. JAMA. 289(15) (2003), 1969-1975.
[13] H.A. Heathfield, D. Pitty. Evaluation as a tool to increase knowledge in healthcare informatics.
MEDINFO 1998. (1998), 879-883.
[14] eHealth Observatory. eHealth Observatory to Monitor the Effects of Health Information System
Deployment in Canada. [cited Aug 20, 2012] Available from: http://www.ehealth.uvic.ca.
[15] F. Lau, M. Price, K. Keshavjee. From benefits evaluation to clinical adoption: making sense of health
information system success in Canada. Healthc Q 2011 14(1) (2011), 39-45.
[16] Canada Health Infoway. Benefits Evaluation Indicators Technical Report Version 2.0. [cited Aug 23,
2012]. Available from: https://www.infoway-inforoute.ca/index.php/resources/reports/benefits-
evaluation/doc_download/544-benefits-evaluation-indicators-technical-report-version-2-0.
[17] Agency for Healthcare Research and Quality (AHRQ). Health IT Evaluation Toolkit, 2011. [cited Aug
20, 2012]. Available from:
http://healthit.ahrq.gov/portal/server.pt/community/health_it_tools_and_resources/919/health_it_evalua
tion_toolkit/27872.
[18] F. Alvino, R. Agrifoglio, C. Metallo, L. Lepore. Learning and Knowledge Sharing in Virtual
Communities of Practice: A Case Study, In: A. D’Atri, M. Ferrara, J.F. George, P. Spagnoletti (Eds.),
Information Technology and Innovation Trends in Organizations. Berlin, Springer, 2011.
[19] K. McCartney, D. Hooker, J. Cordiero, H.N. Lauscher, K. Ho. Practical Considerations in Building an
Electronic Community of Practice for Health Care: Lessons from the Literature, In: K. Ho, S. Jarvis-
Selinger, H.N. Lauscher, J. Cordeiro, R. Scott (Eds). Technology Enabled Knowledge Translation for
eHealth Principles and Practice, New York, Springer, 2012, 325-337.
Enabling Health and Healthcare through ICT 263
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-263
Introduction
1
Corresponding Author.
264 J. Kim / Complex Interventions in Healthcare and Health Informatics: A Scoping Review
1. Methodology
A scoping review addresses broader topics than a systematic review and determines
what literature exists on a topic to summarize what is known and to identify gaps. It
does not synthesize the evidence the same way that a systematic review does. It is a
literature review that examines the extent, range and nature of activities in a domain,
then summarizes and disseminates the findings [4].
The search strategy involved the identification of literature through MEDLINE. This
was supplemented with select articles found in a previous search on complex health
informatics interventions, and a prior collection. Search parameters used were
‘complex interventions’ in the title, English language publications, and publication
between the year 2000 and June 2012.
1.2. Selection
As is common with scoping reviews, some exclusion criteria were devised post hoc
based on increasing familiarity with the literature [4]. Articles were excluded if they
were not available online. Interventions on dental care, alternative or Chinese medicine,
or sports medicine, interventions for research and physical interventions such as stents
were excluded to limit the review to health service interventions in traditional
healthcare settings. Literature was selected for final inclusion on the basis of a full text
review.
1.3. Charting the Data, Collating, Summarizing and Reporting the Results
Charting the data is equivalent to data extraction as carried out in systematic reviews. It
is “a technique for synthesizing and interpreting qualitative data by sifting, charting and
sorting material according to key issues and themes” [4, p. 15]. For this scoping review,
a key theme was the attributes of complex interventions, which emerged from data
charting iteratively in a ground-up approach. Data about complex interventions
logistics was also charted and analyzed. The scoping review involves no assessment of
the quality of literature.
2. Results
The MEDLINE search yielded 154 articles; 99 were selected for the review. An
additional 19 were added from the previous search and prior collection. This led to 118
articles included in the study. Of these, forty-six articles were identified for the
synthesis. They included frameworks, commentary on a broad range of complex
interventions topics, editorials and reporting criteria.
J. Kim / Complex Interventions in Healthcare and Health Informatics: A Scoping Review 265
Synthesis from the review led to the identification of five common attributes that would
identify interventions as complex interventions and the extent of their complexity.
These attributes support a multi-dimensional definition of the term ‘complex
interventions’, and a view that complexity should be seen as existing along a
continuum. These attributes are described below.
Complex interventions are comprised of multiple components [3]. The lack of
specificity of those components is problematic in conducting and reporting studies as
some view complex interventions as an overall ‘black box’. This hinders the ability to
examine effects of each ‘unpacked’ component of complex interventions and determine
effectiveness [5]. Specific types of intervention components have been proposed for
health service interventions [6] and health informatics interventions [7]. One or more
components may be ‘active ingredients’ within the intervention, and it is important to
understand how they exert their effects [3]. It is possible that it can only be speculated
which components are (most) active [8].
Interactions, whether among components, the behaviors of those delivering the
intervention or the behaviors of those receiving the intervention [3] are another
attribute of complex interventions. Where people and organizational levels are involved,
the intervention can be viewed through multiple levels of abstractions and
sociotechnical lenses [9].
The more complex an intervention, the more it is necessary to have a theoretical
foundation [10]. The theory underlying the intervention should provide an
understanding of how the intervention causes change so that weak links in the causal
chain can then be identified and strengthened [3]. It should also ideally articulate
distinct groups affected by the intervention and how they differ in this regard, as well
as the mechanisms by which this occurs [11]. A number of theories, frameworks, and
models were found in the complex interventions literature [3][12].
Complexity can also lie in context [8], which includes the environment in which
complex interventions exist. Context may be the most significant aspect of complexity
[13]. The investigation of how intervention effects are influenced or modified by
context is a recently emerging focus for community intervention trial research [14].
The components of a health informatics intervention can contain many elements of
context. They are: technical components, organizational factors, logistical factors,
behavioral factors and informational components. Some of these may facilitate delivery
of the intervention and others may be barriers to implementation. These components
and factors should be pre-identified and accounted for when conducting studies of
complex health informatics evaluations [7].
Measurement is the final attribute of complex interventions. Complex
interventions themselves must be measureable in order to be studied, and there are
multiple ways in which each component can be measured such as frequency and
intensity. The effects of complex interventions must be measured, whether on people,
groups and organizations [3]. One way to distinguish between simple and complex
interventions is to consider the simplicity or complexity of causal chains and processes
[15]. It may be that the use and interpretation of 'outcomes' as they are traditionally
266 J. Kim / Complex Interventions in Healthcare and Health Informatics: A Scoping Review
3. Discussion
In this review we have identified five common attributes that can be used to define and
operationalize complex interventions. Some of these attributes, such as behaviors of
those receiving the intervention and of those delivering the intervention, are mentioned
in the literature without definition or suggestion of how they would be implemented in
practice. Even for those attributes that have been discussed in the literature, such as
intervention components and theoretical foundations, there are different ways to
interpret and define complexity, based on the discretion of the researcher. Examples
include the importance of the context and environment and whether to base the
theoretical foundation at the individual or broader level.
Few specific references to, or distinctions made for, complex health informatics
interventions were found. It could be that they are not different than complex
J. Kim / Complex Interventions in Healthcare and Health Informatics: A Scoping Review 267
References
[1] C. Paterson, C. Baarts, L. Launsø, M.J. Verhoef, Evaluating complex health interventions: a critical
analysis of the 'outcomes' concept, BMC Complementary and Alternative Medicine 9 (18) (2009), 1-11.
[2] N.C. Campbell, E. Murray, J. Darbyshire, J. Emery, A. Farmer, F. Griffiths, et al, Designing and
Evaluating Complex Interventions to Improve Health Care, British Medical Journal (2007) 455-459.
[3] P. Craig, P. Dieppe, S. Macintyre, S. Michie, I. Nazareth, M. Petticrew, Developing and evaluating
complex interventions: new guidance, Medical Research Council, 2008.
[4] H. Arskey, L. O'Malley, Scoping studies: towards a methodological framework. International Journal of
Social Research Methodology 8 (1) (2005), 19-32.
[5] M. Petticrew, When are complex interventions 'complex'? When are simple interventions 'simple'?
European Journal of Public Health 21 (4) (2011), 397-399.
[6] N. Wolff, Randomised trials of socially complex interventions: promise or peril? Journal of Health
Services Research & Policy 6 (2) (2001), 123-126.
[7] I. Shcherbatykh, A. Holbrook, L. Thabane, L. Dolovich, Methodologic Issues in Health Informatics
Trials: The Complexities of Complex Interventions. Journal of the American Medical Informatics
Association (2008) 575-580.
[8] P. Hawe, A. Shiell, T. Riley, Theorising Interventions as Events in Systems. American Journal of
Community Psychology 43 (2009), 267-276.
[9] B. Kaplan, P. Flatley Brennan, A.F. Dowling, C.P. Friedman, V. Peel, Towards an Informatics Research
Agenda: Key People and Organizational Issues. Journal of the American Medical Informatics
Association (2001), 235-241.
[10] P. Hawe, A. Shiell, T. Riley. In response to Spillane V., Byrne M.C., Byrne M., Leathem C.S.,
O’Malley [11] M. & Cupples M.E. Monitoring treatment fidelity in a randomized trial of a complex
intervention. Journal of Advanced Nursing 60 (3) (2007), 343–352.
[11] M. Mackenzie, C. O’Donnell, E. Halliday, S. Sridharan, S. Platt, Evaluating complex interventions: one
size does not fit all. British Medical Journal 340 (c185) (2010), 401-403.
[12] E. Murray, S. Treweek, C. Pope, A. MacFarlane, L. Ballini, C. Dowrick, et al, Normalisation process
theory: a framework for developing, evaluating and implementing complex interventions, BMC
Medicine (2010), 1-11.
[13] A. Shiell, P. Hawe, L. Gold, Complex interventions or complex systems? Implications for health
economic evaluation. British Medical Journal 336 (2008), 1281-1283.
[14] P. Hawe, A. Shiell, T. Riley, L. Gold, Methods for exploring implementation variation and local
context within a cluster randomised community intervention trial, J Epidemiol Community Health 58
(2004), 788-793.
[15] D. Rickles, Causality in complex interventions, Med Health Care and Philos 12 (2009), 77–90.
[16] S. Treweek, Complex Interventions and the Chamber of Secrets: Understanding Why They Work and
Why They Do Not. Journal of the Royal Society of Medicine 98 (2005), 553.
268 J. Kim / Complex Interventions in Healthcare and Health Informatics: A Scoping Review
[17] S.E. Lamb, C. Becker, L.D. Gillespie, J.L. Smith, S. Finnegan, R. Potter, et al, Reporting of complex
interventions in clinical trials: development of a taxonomy to classify and describe fall-prevention
interventions, Trials 12 (125) (2011), 1-8.
[18] K. Seers, Evaluating Complex Interventions. Worldviews on Evidence-Based Nursing, Second Quarter,
2007.
[19] R. Emsley, G. Dunn, I.R. White, Mediation and moderation of treatment effects in randomised
controlled trials of complex interventions. Statistical Methods in Medical Research 19 (2010), 237-270.
[20] B. Blackwood, Methodological issues in evaluating complex healthcare interventions. 2006.
[21] W.H. Delone, The DeLone and McLean model of information systems success: A ten-year update.
Journal of Management Information Systems 19 (4) (2003), 9-30.
[22] E. Rogers, Diffusion of Innovations. New York: Simon and Schuster, 2008.
Consumer Informatics
This page intentionally left blank
Enabling Health and Healthcare through ICT 271
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-271
1
Corresponding author. Hospital Clinic de Barcelona, Villarroel 170, 08031Barcelona, Spain.
[email protected], +34 93 2275690
272 I. Grau et al. / Forumclínic: The Shaping of VC to Assist Patients with Chronic Diseases
Introduction
1. Objectives
2. Methods
The following data were used to analyze the presence or absence of a virtual
community:
Google Analytics’ downloadable data.
Sociodemographic indicators reported during user registration. (This allowed
crossreferencing to server logs, in order to evaluate general behaviour; such
as the number of times a user logged into the system, posted, etc.)
Forum postings.
A virtual community was defined to exist when the number of user visits to a
chronic disease forum was greater than those for the portal’s main page. Each portal
has information classified according to the following categories: 1) general summary of
I. Grau et al. / Forumclínic: The Shaping of VC to Assist Patients with Chronic Diseases 273
the disease; 2) weekly news, where recent research articles are transcribed into lay
language by our board-certified health professionals; 3) specialist-moderated forums;
and 4) patient, health provider, and 3D biological animation videos which explain in
simple terms the pathophysiology, treatment, and rehabilitation of disease. This
structure facilitates a dynamic relationship between patients and health professionals,
and aims to foster autonomy. The proceeding results correspond to data from the 2008
calendar year. More recent data was not used due to a server upgrade problem. The
necessary ethics approval was acquired and data was de-identified in accordance with
[6].
3. Results
Table 1 displays overall web site activity. As can be seen, the data indicate that nearly
half of visitors accessed information from the Americas (primarily South America).
Schizophrenia and Breast Cancer were the most-accessed portals. It is also intriguing to
note that cardiovascular risk was not as popular as the other forums, despite its
prevalence in Hispanic communities.
Table 2 displays the activity of forum users in the five portals that were
evaluated for the status of virtual community along with their corresponding user
profiles. In total, a third of Forumclínic’s visitors accessed the forums. Proportionately,
this was as high as 62% for the Breast Cancer portal. The overall percentage of forum
visits when compared portal visits was 23.25%. Chronic Obstructive Pulmonary
Disease (COPD), with a percentage of 20.75%, is considered a VC but does not reach
this average.
It can be seen that users are in their late thirties or early forties and, are primarily
female, except for cardiovascular disease. Family members, rather than patients
suffering the condition, are increasingly present in the Schizophrenia and COPD
communities (self-reported in the forum). Italics and the colour red were used in the
table to show forums which did not meet the definition for a virtual community.
4. Discussion
The data support the Forumclínic portal model as an effective and social approach for
chronic disease information consumer engagement.
Table 1. Activity at the Forumclínic web in 2008
Total Users Visits Pages Americas Pages Seen
/visit (% Visits) forum
Schizophrenia 44,488 63,622 5.11 40.4 122,106 37.6
Diabetes 51,345 59,490 4.21 42.1 21,192 8.5
COPD 40,039 49,299 4.32 41.0 44,172 20.8
Cardiac Ischemia 47,005 54,773 3.62 48.2 20,554 10.4
Breast Cancer 41,874 59,715 4.86 36.1 180,028 62.0
Depression 35,195 45,717 3.95 37.7 58,329 32.3
Cardiovascular 14,291 18,910 2.41 46.7 4,208 9.2
risk factors
Home page 99,229 103,563 5.00 56.0 11,479
Forumclínic 373,466 455,089 4.37 44.6 462,068 23.3
274 I. Grau et al. / Forumclínic: The Shaping of VC to Assist Patients with Chronic Diseases
5. Conclusions
The development of four virtual communities confirms that patients need participative
role health-related web sites. Patients appreciate an increased degree of interaction
with experts and fellow patients which allows for more collaborative self-
management.
The portals that were successfully characterized as virtual communities in this
study continue to grow and be populated by a high proportion of female users. The
Depression and Schizophrenia portals were particularly useful in providing access for
communities that are commonly stigmatized.
Forumclínic is a living project that is growing both in activity (e.g. visits) and
participation (e.g. number of registered users engaging in forums). We have been
successful in creating functional virtual communities by providing health information
access through a model of patient education resources and user interaction. Future
research will increase our understanding on the contextual determinants of success for
user engagement in the Spanish and Catalan-speaking chronic disease community.
References
[1] G. Demiris, L. Afrin, S. Speedie, K. Courtney, M. Sondhi, V. Vimarlund, C. Lovis, W. Goossen, and
C. Lynch. Patient-centered applications: use of information technology to promote disease
management and wellness. A white paper by the AMIA knowledge in motion working group.
Journal of the American Medical Informatics Association 15.1 (2008), 8-13.
[2] M. Castells, La galaxia internet. Reflexiones sobre internet, empresa y sociedad, Editorial in Plaza
y Janes, Barcelona, 2002.
[3] G. Eysenbach, J. Powell, M. Englesakis, C. Rizo, A. Stern, Health related virtual communities and
electronic support groups: systematic review of the effects online peer to peer interactions, British
Medical Journal 328 (2004), 1166.
[4] O’Reilly T. What is Web 2.0? Design patterns and business models for the next generation of
software. O’Reilly Media, 2005.
[5] B. Wellma, J. Boase, W. Chen, The networked nature of community: online and offline, It & Society
1 (2002), 151-165.
[6] I. Hrynaszkiewicz, Me. Norton, A. Vickers, D. Altman. Preparing raw clinical data for
publication: guidance for journal editors, authors, and peer reviewers. British Medical Journal 340
(2010), 181.
[7] J. Frost and P. Massagli. Social uses of personal health information within PatientsLikeMe, an
online patient community: what can happen when patients have access to one another's data.
Journal of Medical Internet Research 10.3 (2008), e15.
276 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-276
Abstract: The near-pervasive introduction of ehealth systems, and the more recent
implementation of systems intended for patient use offer patients the opportunity
to participate in their own care. Unfortunately the design of these systems means
that they may work better for “People Like Us’ rather than for those on the wrong
side of the ‘digital divide’. This paper looks at the professional, practical and
ethical implications of this conundrum.
Introduction
This paper argues that the current approach to the design of personal ehealth systems
may serve to accentuate the gap between privileged and disadvantaged end users and
healthcare recipients, rather than improve equity of access to health care services.
The problems facing healthcare services in the developed world are well
documented and understood. They include financial challenges from increasing
treatment costs [1], resource scarcity with an aging workforce, and an increasing
burden of chronic disease [2]. Where healthcare is publicly funded, Governments also
face the politically sensitive challenge of determining appropriate healthcare service
levels in an environment of increasing citizen demands and expectations for care.
Health reform remains a critical area for policy debate and has led to a range of
solutions to address the problems faced. In Australia, ehealth systems are seen as one
set of solutions to these problems. Unfortunately many of these ehealth systems have
delivered mixed results or have not generated the predicted savings as a result of
designs that under-estimate the complexity of healthcare practice.
One response to these design challenges has been to focus on personal ehealth
systems tailored and customised to the needs of individual health users. One example is
Australia’s personally controlled electronic health record (PCEHR) which was
launched in July 2012. However, differences in individual knowledge, skills and
inclination to use such systems may be resulting in system designs primarily suitable
for a privileged group of literate and motivated end users, rather than for those most
urgently in need of improved health service delivery.
1
[email protected]
C. Showell and P. Turner / The PLU Problem: Are We Designing Personal eHealth for PLUs? 277
1. Methods
To explore the argument that the design of contemporary personal ehealth systems
maybe problematic, a literature review was conducted to better understand research on
the relationship between literacy and personal ehealth. Using a combination of search
terms ehealth/personal ehealth and literacy/health literacy 100 papers available through
PUBMED were identified and reviewed. 17 of these papers were also identified as
providing insights on the characteristics of healthcare recipients.
2. Analysis
be able to summarise text) [8]. Health literacy is also a problem. The US Agency for
Healthcare Research and Quality found that poor health literacy was “...associated with
increased hospitalizations, greater emergency care use…and, among seniors, poorer
overall health status and higher mortality.” [9] The disadvantaged are less likely to
have home internet access, and there is evidence that living in a rural area and having a
medical condition will make home internet use even less likely. [10] The inter-
relationship between all of these factors is complex, and it can be difficult to separate
cause from effect. However, it is apparent that there is an association between low
income, poor literacy (textual, technical and health), chronic disease, and poor health
outcomes.
For better or worse, healthcare systems are developed predominantly by white,
educated middle class professionals (PLUs) who design systems and processes which
they see as being appropriate and user-friendly. This tendency to design for ourselves
(the PLU problem) results in patient instructions using complex language; patients
receiving complex verbal descriptions of health issues using clinical terminology; and
the explicit assumption that online tools are an important and appropriate option for
healthcare service delivery.
Catwell and Sheikh [11] considered the evaluation of ehealth systems, and argued
that the evaluation should be continuous and systemic. They cautioned that “[l]arge
investments in eHealth may, by diverting resources result in a shortfall in funding for
basic infrastructure, equipment, and staffing elsewhere in the system. …investing in
developments such as telemedicine, which are only likely to be accessible to a minority,
would exacerbate the digital divide and existing health inequities.” [11]
Figure 1: Risk of unintended consequences – the disadvantaged may end up worse off
Any diversion of health budgets towards generic personal ehealth systems is likely
to deliver benefits to PLUs, while leaving the ‘quality’ of healthcare for DDDs
unchanged. In fact, there is a risk that diversion of health budgets to ehealth will be
used to justify and facilitate a reduction in funding from other, more conventional
healthcare services. The potential for these unintended adverse consequences from
ehealth investment for DDDs is illustrated in Figure 1.
3. Results
tech savvy and street smart, but may be less helpful in areas of healthcare where the
individual’s needs are great, and the cost of meeting those needs is high.
PLU systems will place additional resources at the disposal of those who are
adaptable and capable — those who probably do well with existing systems and
services. Making reasonably healthy people a little bit healthier is not necessarily the
most effective use of scarce health resources. Better cost benefits in terms of health
outcomes are likely to be achieved by focusing on the DDDs whose health is worst,
rather than designing and implementing for the PLUs.
A report from the European Union [12] comments on the introduction of Ambient
Intelligence (AmI) in healthcare. The report notes:
“AmI claims to be particularly people-oriented, implying that it will also be
inclusive - providing, of course, it lives up to its promises of being user-friendly,
unobtrusive and controllable. But…the fundamental question remains…[will] AmI …
include the majority of people or…benefit mostly young, urban and mobile techno-
freaks. In the latter case, AmI could become an additional source of exclusion in
society.” The report cautions that “[t]he relation between social exclusion and health
status is well known. Digital divides…can negatively affect health.” [12]
The greatest challenge (arguably) is providing healthcare services to those who
lack the motivation and capacity to make positive changes in their lives. They show
little interest in adopting a healthy diet; regular exercise doesn’t interest them; if they
smoke, they’ll probably continue to do so. Their poor literacy will make it hard for
them to adapt to a routine of regular medication, and they will struggle to monitor their
symptoms effectively. They may benefit from some form of mentoring, possibly with a
technology component, but it will need to be carefully targeted at their capability, skills
and worldview. The solution to the challenge of poor uptake (by DDDs) is often
framed as requiring better solutions, including eHealth, Internet access, or training. But
many of the processes and systems designed for supportive care and self-care are
tailored for the designers and PLUs. This paper argues that there is a greater need in the
design of personal ehealth systems to recognize the constraints imposed by a class of
potential users who face very high barriers to technology adoption and effective use.
4. Discussion
Solutions to healthcare problems should be targeted at those with the greatest need.
However, a PLU-designed health service, or a PLU designed ehealth system is unlikely
to deliver equitable benefit to all groups of patients and has the potential to further
disadvantage some groups. The needs of the DDDs should be considered as a special
case, and explored in some detail. Once those needs are clearly understood, they should
either be incorporated into the overall design, or provided for through a focused
alternative, that may not initially require citizen/patient use of an eHealth system.
It may be more appropriate to design and implement solutions tailored for the
capabilities of DDDs, and to then extend implementation progressively to other groups
until marginal benefits become too small to fund. Designers should focus on user
centred design, as many do, but in doing so should take care to identify and specify the
class of user to which the initiative is targeted. The sense of disempowerment among
DDDs will make them reluctant to participate in design activities. They may struggle to
express themselves, or to see their opinions as having merit. Because of the many
categories of limited capability which interfere with the use of technology by DDDs,
280 C. Showell and P. Turner / The PLU Problem: Are We Designing Personal eHealth for PLUs?
this approach is likely to be more difficult to implement. These solutions may also be
less interesting for designers, developers and academic researchers due to the lack of
feature and functional complexity. Indeed the design activities may produce systems
that their creators and implementers might not want to use. It is argued however, that in
terms of overall cost benefit, approaches like these could be potentially very rewarding
as illustrated in Figure 2.
5. References
[1] Holman H (2005) Chronic disease and the healthcare crisis. Chronic Illn. 1, 265–274.
[2] ABC News, Healthcare crisis highlighted, Last updated August 17, 2009, Accessed on August 17,
2009.
[3] Benefits of a PCEHR - nehta.gov.au, Last updated November 20, 2011, Accessed on November 20,
2011.
[4] Pagliari C, Detmer D, Singleton P (2007) Potential of electronic personal health records. BMJ 335,
330–333.
[5] Bourdieu P (1987) Distinction: A Social Critique of the Judgement of Taste, Harvard University Press.
[6] Clarke P, Leigh A (2011) Death, Dollars and Degrees: Socio - economic Status and Longevity in
Australia. Economic Papers: A journal of applied economics and policy 30, 348–355.
[7] Chronic Diseases and Associated Risk Factors in Australia, 2006, Australian Institute of Health and
Welfare.
[8] ABS (2008) Australian Social Trends 2008, Australian Bureau of Statistics, Canberra.
[9] Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Viera A, Crotty K, Holland A, Brasure M, Lohr
KN, Harden E, Tant E, Wallace I, Viswanathan M Health Literacy Interventions and Outcomes: An
Updated Systematic Review, AHRQ, Rockville, MD.
[10] Wang J-Y, Bennett K, Probst J (2011) Subdividing the Digital Divide: Differences in Internet Access
and Use among Rural Residents with Medical Limitations. Journal of Medical Internet Research 13,.
[11] Catwell L, Sheikh A (2009) Evaluating eHealth Interventions: The Need for Continuous Systemic
Evaluation. PLoS Med 6, e1000126.
[12] Cabrera M, Burgelman J-C, Boden M, da Costa O, Rodriguez C (2004) eHealth in 2010: Realising a
Knowledge-based Approach to Healthcare in the EU.
Enabling Health and Healthcare through ICT 281
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-281
Introduction
Information and communication technologies play a pervasive role in our lives. They
impact how we communicate and interact with our families, loved ones, colleagues,
friends, government, healthcare, and private institutions. In the healthcare domain, the
use of information and communication technologies (ICTs) to improve clinical care
and administrative decision-making has been referred to as e-health [1]. The primary
goal of e-health is the use of the Internet and related applications to improve access to,
efficiency of, and quality of care within healthcare with the goal of improving the
health status of patients [2]. The discipline of e-health has existed for many years and
includes an increasing number of multifaceted interventions that are aimed at
improving health status through the use of information technology. One relatively
unknown area of research within this domain is referred to as spiritual e-health.
Spiritual e-health is the use of the internet and related technologies to improve the
spiritual well-being of a patient [3]. Several studies have focused on the concept of
spiritual e-health, but none have researched the role of e-health applications within the
context of the Islamic spiritual belief system [3].
Spirituality plays a large part in Muslim daily life. Muslims are required to pray
five times a day, fast during the holy month Ramadan, give to charity, and make a
pilgrimage to the holy city of Mecca. Each of these religious obligations is a
1
Corresponding Author: Dr. Mowafa Househ, King Saud Bin Abdul Aziz University for Health
Sciences, College of Public Health and Health Informatics, Riyadh, Kingdom of Saudi Arabia; Email:
[email protected]
282 M. Househ / Islamic E-Health: Definitions, Applications, and Challenges
manifestation of the spiritual Muslim belief system. As part of the healing process,
Muslims believe that prayer, supplication, recitation from the holy book, the Quran,
and other religious rituals can help in improving health. Muslims are also taught
through prophetic teachings that fasting during Ramadan can help to improve health.
There are many public health issues during the month of Hajj, when millions of
Muslims from around the world gather in Mecca to fulfill their religious obligations.
The country of Saudi Arabia, the birth place of the Islamic faith, has significant
concerns about Muslim daily life. For example, all public hospitals in Saudi Arabia
have Islamic religious affairs departments that help to provide religious guidance,
education, and spiritual healing to patients. In addition to establishing various places of
worship, these religious departments use technologies such as web pages, SMS
messaging and web-based information to assist in the patient care process. Furthermore,
users on social networking sites such as Facebook have developed spiritual healing
groups for Muslims.
The holy month of Ramadan and the Hajj pilgrimage are two important annual
Islamic events that affect the spiritual and physical health of Muslims. During the holy
month of Ramadan, it is obligatory for Muslims to fast. Patients who decide to fast are
likely to be monitored by their primary care physicians using technologies such as
electronic medical records to monitor blood sugar levels. During the Hajj season, when
more than two million worshipers converge on Mecca from more than 180 countries,
public health issues related to communicable diseases have become a significant
concern, and various public health information systems are employed to monitor
disease outbreaks.
Given the uses of information and communication technology to support Islamic
spiritual practices, the purpose of this exploratory paper is to define Islamic e-health
and to describe its applications and future development.
2. Methodology
Data collection, including interviews and internet searches, began in November 2011
and ended in January 2012. The interviews included one interview with hospital
religious affairs personnel in Saudi Arabia that lasted 120 minutes, a brief 10-minute
interview with one family care physician, and two separate 10-minute interviews with
one public health expert and one health informatics expert in Saudi Arabia.
M. Househ / Islamic E-Health: Definitions, Applications, and Challenges 283
Internet and social media sites, such as Facebook and the Apple iTunes Store, were
searched using terms such as Islamic healing, ruqya (spiritual healing), fatwa (Islamic
Law), Hajj, Ramadan, informatics, electronic medical record, public and health
informatics. Various results were returned through Facebook groups, through the
Google Search engine and through the Apple iTunes store.
Most of the Islamic e-health interventions that were identified were related to education
and information on spiritual healing. In Islam, spiritual healing is referred to as Ruqya
and is a form of supplication that is supposed to protect and heal. A Muslim can
perform Ruqya on himself or on others [4]. Ruqya was used by the Muslim Prophet
Muhammad, Peace Be Upon Him (This is a supplication that a Muslim performs when
mentioning the name of the Muslim Prophet), and Ibn Alqayyim, a 13th-century
Muslim scholar, said, “Ruqyah is one of the greatest remedies that the believer should
use regularly.” [4] A prophetic tradition collected by Muslim, a famous collector of
authentic Muslim prophetic sayings, narrates the following story [4]:
Uthman complained to the Messenger of Allaah (Peace be Upon
Him) about pain that he had felt in his body from the time he had
become Muslim. The Messenger of Allaah said to him: “Put your
hand on the part of your body where you feel pain and say
‘Bismillaah (in the name of Allaah) three times, then say seven times,
‘I seek refuge in the glory and power of Allaah from the evil of what I
feel and worry about’.”
Google web searches identified various sources of information in English on how
to perform Ruqya, such as Islam Q&A, which provides detailed descriptions of how to
correctly perform Ruqya, its benefits, and some common misconceptions. Several
Facebook groups on Islamic spiritual healing were found, such as the Islamic Spiritual
Healing group. Various iPad applications, such as Ayat Ruqya, list religious
supplications and how to perform them. Furthermore, within Saudi Arabia, religious
affairs departments perform Ruqya on patients who request these services. However,
patients are informed that the service is complementary to the medical treatment they
receive.
3.2. Hajj
Hajj is a religious obligation for Muslims that was prescribed by God at the time of the
Prophet. Muslims believe that its roots date to the time of the Prophet Abraham. Every
Muslim who is financially and physically able to perform Hajj is required to do so once
in his or her lifetime. Every year, approximately 3 million Muslims converge in Mecca,
Saudi Arabia, to fulfill this important religious obligation. In the Quran, God says,
“And Hajj (pilgrimage to Makkah) to the House (Ka‘bah) is a duty that mankind owes
to Allaah, those who can afford the expenses (for one’s conveyance, provision and
residence) [5].”
Much of the information related to Hajj was found through Google and through
interviews conducted with various public health and informatics experts within the
284 M. Househ / Islamic E-Health: Definitions, Applications, and Challenges
field. The results show that health information systems are used intensively during the
Hajj period to control the flow of pilgrims, to track movement, and for health
surveillance. Yamin discusses a framework for the use of information systems to
improve Hajj management [6]. The author suggests the use of information technology
for 1) collecting information on pilgrims and providing them with RFID tags; 2) processing
pilgrims and collecting additional matching information; 3) installing wireless remote
sensors; and 4) processors and display screens. The author argues that these methods will
help to reduce wait times at airports, track the movements of individuals, contain the spread
of disease, improve future planning and assist in tracking illegal immigrants.
3.3. Ramadan
Ramadan is a holy Muslim month in which it is obligatory for every healthy Muslim to
avoid eating from sunrise to sunset. God revealed the following in the Quran [7]:
“The month of Ramadan [is that] in which was revealed the Qur'an,
a guidance for the people and clear proofs of guidance and criterion.
So whoever sights [the new moon of] the month, let him fast it; and
whoever is ill or on a journey - then an equal number of other days.
Allah intends for you ease and does not intend for you hardship and
[wants] for you to complete the period and to glorify Allah for that
[to] which He has guided you; and perhaps you will be grateful.”
Participants in this study noted that Muslims living with diabetes are significantly
affected by fasting during the month of Ramadan. The participants noted that diabetic
Muslim patients are encouraged to fast and follow a prescribed regimen in which the
patient takes a full dose of medication prior to breaking the fast and one half of a dose
prior to beginning the fast before sunrise. During this month, diabetic patients are
monitored, and their blood test results are transferred to an electronic medical record
that helps their family physicians track fasting patients’ blood glucose levels. The
interviewed participants did not mention any other benefits to patients from the use of
information technology during Ramadan.
4. Discussion
This exploratory study defined the concept of Islamic e-health and surveyed some of
the existing applications that support this concept. This study revealed that the primary
focus of Islamic e-health has been spiritual health, followed by Hajj systems and
Ramadan. Information technology may provide significant benefits for Muslims’
connections for spiritual guidance, improvements to Hajj systems and the use of
technologies such as the EMR to monitor Muslim patients who fast during Ramadan.
This research is exploratory and examines some of the relevant issues related to Islamic
e-health.
The study has several limitations worth mentioning. The numbers of interviewees were
small and could have been expanded to include other stakeholders, such as patients and
M. Househ / Islamic E-Health: Definitions, Applications, and Challenges 285
Acknowledgements
I would like to thank Sheikh Dr. Ahmad Naji Ali, NGHA Religious Affairs, who spent
several hours helping me develop this idea. I would also like to thank Mr. Osama
Alghanmi and Drs. Wesley Rohrer, Munir Abuhelaleh, and Abdulrahman Alsughayr
for their valuable input. Furthermore, I would like to thank the King Abdullah Institute
for Medical Research for help in editing this document.
References
[1] H. Oh, C. Rizo, M. Enkin, A. Jadad. What is eHealth (3): A systematic review of published definitions.
JMIR, 7(1) (2005), e1.
[2] HIMSS. HIMSS E-Health SIG white paper. [cited Jan 10, 2012]. Available from:
http://www.himss.org/content/files/ehealth_whitepaper.pdf .
[3] J. Van Nooten, H. Oh, B. Pierce, F.J. Koning, A.R. Jadad. J Pastoral Care Counsel. 60 (4) (2006), 387-
94.
[4] S. Al Munajid. The Virtues of ruqyah and du’aa’s to be recited therein. [cited Jan 10, 2012]. Available
from: http://islamqa.info/en/ref/3476 .
[5] The Holy Quran. 3, 97.
[6] M. Yamin. A framework for improved hajj management and future research. School of Information
Sciences and engineering, University of Canberra, Australia. [cited Jan 7, 2012]. Available from:
http://hajjmanagement.com/attachments/documents/Yamin_Jedda_Conf.pdf .
[7] The Holy Quran. 2, 185
286 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-286
a
Electrical & Computer Engineering, University of British Columbia, Vancouver,
Canada
Introduction
Leading a healthy lifestyle and making positive health behaviour changes have been
found to be key to preventing illness and managing chronic diseases. In fact, self-
management of ones health has been shown to be of key significance in achieving
positive health outcomes for all people, healthy and sick [1, 2, 3]. Furthermore, we
intuitively understand that our life choices are heavily influenced by family, friends,
colleagues and other connections we have. A significant factor in health behaviour is
one’s close and distant social networks, which have been found to be a contributing
factor to health outcomes, where ones social networks can be used to improve health
behaviour through facilitating social integration and social support [4].
There is recent interest in the field of Human-Computer Interaction (HCI) to
explore online social networks and health behaviour change [5, 6]. Further, HCI
researchers have looked at designing technologies to promote a more active lifestyle [7,
8] and a more nutritious diet [9].
1
Corresponding Author: PhD Candidate, Electrical & Computer Engineering, Vancouver, BC, Canada;
E- mail: [email protected]
N. Kamal and S. Fels / A User-Centred Methodology for Designing an Online Social Network 287
Existing research has not looked specifically at designing online social network
and online social games to motivate health behaviour change. For this reason, we
explore the design space for an online social network system for positive health
behaviour change through a user-centred design process. User-centred design is a
methodology used in Human-Computer Interaction (HCI) as a process of interaction
design [10]. We base the user-centred design process on existing theoretical models on
health behaviour change and participation in online social network. These theoretical
models provide specific behavioural determinants that yield the Appeal Belonging
Commitment (ABC) Framework. The determinants from the ABC Framework provide
the points of inquiry throughout the user-centred design methodology. We use this
methodology by first completing an initial user inquiry and developing prototypes with
increasing fidelity for an online social network system that we call VivoSpace. The
prototypes that were developed included a low-fidelity paper prototype, an interactive
medium-fidelity prototype and fully functional high-fidelity prototype.
The ABC Framework was developed through the distillation of 13 theoretical models.
The theoretical models for health behaviour change included in the ABC Framework
are Health Belief Model, Social Cognitive Theory, Theory of Reasoned Action, Theory
of Planned Behaviour, Common Sense Model, and The Transtheoretical Model. The
theoretical models for participation on online social networks included in the ABC
Framework are Uses and Gratification Theory, Common Identity Theory, Common
Bond Theory, Social Identity Theory, Organizational Commitment Theory, Behaviour
Chain for Online Participation and social network threshold. These theories yield a
framework that provides: individually-based determinants (appeal) that include self-
efficacy, knowledge, social enhancement and expectations about outcomes; socially-
based determinants (belonging) that include sense-of-belonging, subjective norms and
social categorization; and temporal stages (commitment) [11, 12].
288 N. Kamal and S. Fels / A User-Centred Methodology for Designing an Online Social Network
Figure 1. User-centred design method by using the ABC Conceptual Framework to form the inquiry through
the iterative evaluation techniques.
1.2. User-Inquiry
The initial user inquiry for this methodology used online and paper questionnaires. The
questionnaires inquired about demographic information such a gender, age and ethnic
identity. The inquiry then used a 5-point Likert Scale to inquire each respondent’s
agreement to the determinants from the ABC Framework. The results yielded good
agreement to the ABC Framework. Generally respondents felt that they were living a
healthy lifestyle, understanding how to live healthy, eating healthy food and exercising
regularly. However, interestingly, the majority of respondents felt that they are capable
of living a healthier lifestyle. Most respondents also recognized the social influences
on their health. Furthermore, older respondents (over 65 years old) felt that they ate
healthier food more than young adults. Furthermore, Chinese and South Asians used
online social networks for social enhancement (a determinant from the ABC
Framework) more than Canadians. Generally, respondents felt stronger about
connecting with similar individuals than belonging to a group or community, but both
showed strong motivation [13, 11].
Based on the determinants from that ABC Framework, an initial design for
VivoSpace was developed. The paper prototypes were developed using Adobe
Illustrator. There were 14 pages in total. The paper prototype was evaluated through
individual in-person interviews with 11 participants. They were shown each page of
the prototype and described the key functionality of the design. Participants felt that
they did not want to have their health data in the same place as other personal digital
assets. However, they liked the idea of social interaction with their health information
and the dashboard. They also wanted to see a greater focus on goal setting and
gamification [13].
N. Kamal and S. Fels / A User-Centred Methodology for Designing an Online Social Network 289
From the results of the interviews conducted on the paper prototype, an interactive
medium fidelity prototype was developed for VivoSpace. The medium fidelity
prototype was developed using HTML, CSS, Javascript and jQuery with the vision to
develop a realistic interactive representation of our system. In total there were 32
HTML pages, 1 CSS file, and 2 javascript files. The medium fidelity prototype was
evaluated in a laboratory with 36 participants. There were 6 groups of tasks that
participants were asked to complete; after each task group, they were asked to complete
a questionnaire that was based on the ABC Framework. Belonging was evaluated
using indirect inquiry through the adaption of the Helping Game Experiment from
behavioural economics. Similarly, Commitment was evaluated through the In-Group
Experiment from social psychology. The results showed good agreement with the ABC
Framework; however, the design should provide greater motivation for users to
provide information [14].
Based on the previous designs and evaluation, a fully functional high fidelity prototype
has been developed for VivoSpace. The system architecture for the high-fidelity
prototype is shown in Figure 2. The online social network system was developed in
PHP using the CodeIgniter [15] web application framework. The CodeIgniter web
application framework enforces a Model-View-Controller (MVC) development pattern,
which separates the application logic from the presentation: the model represents the
data structures; the view contains the code for presenting the information to the user;
and the controller contains the bulk of the application logic and processes. The
database used was MySQL, and nutritional information for food was obtained from an
external database, Wolfram Alpha [16], through an Application Programming Interface
(API). The high fidelity prototype is then used to allow people to use VivoSpace in
290 N. Kamal and S. Fels / A User-Centred Methodology for Designing an Online Social Network
2. Conclusions
References
[1] E.B. Fisher, C.A. Brownson, M.L. O’Toole, G. Shetty, V.V. Anwauri and R.E. Glasgow, Ecological
approaches to self-management: the case of diabetes, American Journal of Public Health 95(9) (2005),
1523–1535.
[2] T. Bodenheimer, K. Lorig, H. Holman and K. Grumbach, Patient self-management of chronic disease in
primary care, The Journal of the American Medical Association 288(19) (2002), 2469–2475.
[3] E.H. Wagner and T. Groves, Care for chronic diseases, The British Medical Journal 325(7370) (2002),
913–914.
[4] S. Cohen, Social relationships and health, American Psychologiest November (2004), 676-684.
[5] M.E. Morris, S. Consolvo, S. Munson, K. Patrick, J. Tsai and A.D.I. Kramer, Facebook for health:
opportunities and challenges for driving behavior change, Proceedings of the 2011 annual conference
extended abstracts on Human factors in computing systems, CHI EA’11, (2011), 443–446.
[6] M.W. Newman, D. Lauterbach, S. Munson, P. Resnick and M.E. Morris, It’s not that i don’t have
problems, i’m just not putting them on facebook: challenges and opportunities in using online social
networks for health, Proceedings of the ACM 2011 conference on Computer supported cooperative
work, CSCW ’11, (2011), 341–350.
[7] S. Consolvo, D.W. MadDonald and J.A. Landay, Theory-driven design strategies for technologies that
support behavior change in everyday life, Proceedings of the 27th international conference on Human
factors in computing systems, CHI ’09, (2009), 405–414.
[8] J.J. Lin, L. Mamykina, S. Lindtner, G. Delajoux and H.B. Strub, Fish’n’Steps: encouraging physical
activity with an interactive computer game, Proceedings of the 8th international conference on
Ubiquitous Computing, UbiComp ’06, (2006), 261–278.
[9] A. Grimes, V. Kantroo and R.E. Grinter, Let’s play!: mobile health games for adults, Proceedings of the
12th ACM international conference on Ubiquitous computing, Ubicomp ’10, (2010), 241–250.
[10] H. Sharp, Y. Rogers and J. Preece, Interaction Design: Beyond Human-Computer Interaction, Wiley,
West Sussex England, 2007.
[11] N. Kamal, S. Fels, M. Blackstock and K. Ho, The ABCs of Designing Social Networks for Health
Behaviour Change: The VivoSpace Social Network, In E. Kranakis (ed.) Advances in Network Analysis
and its Applications, (2012) Springer Berlin / Heidelberg. 307–337.
[12] N. Kamal, S. Fels and K. Ho, Online social networks for personal informatics to promote positive
health behavior, Proceedings of second ACM SIGMM workshop on Social media, WSM ’10, (2010),
47–52.
[13] N. Kamal, S. Fels, M. Blackstock and K. Ho, VivoSpace: Towards Health Behavior Change using
Social Gaming, In J. C. Anacleto, S. Fels, N. Graham, B. Kapralos, M. S. El Nasr, and K. Stanley
(Eds.), Entertainment Computing ICEC 2011, Lecture Notes in Computer Science 6972 (2011), 319-
330.
[14] N. Kamal and S. Fels, Determining the Determinants of Health Behaviour Change Through Online
Social Networks. In M. Bang, E.L. Ragnemalm (Eds), Persuasive 2012, Lecture Notes in Computer
Science 7284 (2012), 1-12.
[15] CodeIgniter, [cited Sep 14, 2012]. Available from: http://codeigniter.com .
[16] Wolfram Alpha, [cited Sep 14, 2012]. Available from: http://www.wolframalpha.com.
Enabling Health and Healthcare through ICT 291
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-291
Abstract. This paper compares two social media sites that aim to support patients
to enhance self-management. The first site, PatientsLikeMe is a well established
global site designed to allow peer-to-peer communication between people with
similar conditions. The second, HealthShare, is a recently developed site for
Australians described as “Australia’s Social Health Network”. The comparison
conducted examines the purpose, ownership, and design of both sites as well as
how the data they collect is used. Analysis highlights that PatientsLikeMe actively
facilitates patient self-management, while HealthShare is revealed to be a
professionally moderated health information portal presented as a social
networking site. While the impetus for the development of PatientsLikeMe is
clear, the motives underpinning HealthShare are less obvious. With increasing
patient interest in connecting with, and sharing information with one another,
awareness of the nature and motivations underpinning sites that provide these
services is of increasing relevance.
Introduction
Rising costs of health care and an aging population are placing strains on most health
care systems. Alternative mechanisms are being sought to provide support for patients,
especially those with chronic disease.
Self-management has been described as “…a dynamic process incorporating
an individual’s capability and confidence to engage in activities which enable them to
deal with the impact of living with a chronic condition on all aspects of their life” [1].
Opportunities for self-management have recently increased with the development
of Web 2.0 [2]. Social Media provide opportunities for patients to support each other
outside the formal medically controlled environment by sharing personal health
information and experiences. In some instances, these opportunities are changing the
traditional ‘Patient-Physician’ relationship to one where patients interact with one
another to acquire mutual support, learning and self-management [3]. Technology now
provides a mechanism to create person-to-person support groups online, providing
patients with peer support from others with similar diseases, along with ‘expert’ advice
from the patient perspective [4].
1
[email protected].
292 L. Ellis et al. / Social Media and Patient Self-Management: Not All Sites Are Created Equal
In order to better understand how these identified trends are working in practice,
two social media sites offering support for patient self-management were identified for
comparison. The two sites were: PatientsLikeMe [http://www.patientslikeme.com/] and
HealthShare [http://www.healthshare.com.au/].
1. Method
2. PatientsLikeMe
PatientsLikeMe was co-founded in 2004 by three MIT engineers, Ben Heywood, Jamie
Heywood and Jeff Cole, as a means of providing support for Stephen Heywood, the
brother of Ben and Jamie, who had been diagnosed with amyotrophic lateral sclerosis
[ALS]. Their aim was to use Web 2.0 technology to provide a mechanism for people
like Stephen to connect and share information with other ALS patients [6].
The aim of the site is for patients to use it in conjunction with other patients who
have the same disease. Patients report their own health information which is then
presented graphically in their profile and shared with other patients, providing
opportunities for active dialogue amongst patients [3]. The goal of the PatientsLikeMe
website is to help patients answer the question: “Given my status, what is the best
outcome I can hope to achieve, and how do I get there?” [7].
Used responsibly PatientsLikeMe can also be an important tool for medical
research [6]. Through member participation information is gathered to better
understand disease trajectories, medical treatments and medications. Data provided by
patients is used to discover trends and evaluate new possible treatments [8].
PatientsLikeMe conducts research by investigating the patient data. Researchers
administer surveys to groups on the site to gather additional patient self reported data.
This has resulted in numerous publications that have contributed useful information to
the broader medical community [7, 9, 10].
The PatientsLikeMe site provides clear information about the founders, and how
and why the site was originally developed, including a declaration that the company is
‘for-profit’ but not one with a ‘just for profit’ mission. It states “We follow four core
values: putting patients first, promoting transparency [“no surprises”], fostering
openness and creating ‘wow” [11].
While the site offers privacy settings and indicates that patients can volunteer their
"personally identifiable information," PatientsLikeMe retains the right to share the de-
identified data with their partners and with other patients [12].
L. Ellis et al. / Social Media and Patient Self-Management: Not All Sites Are Created Equal 293
3. HealthShare
HealthShare, was established in 2009 by Darryl Jackson and Gavin Solsky, it is owned
by Adin Holdings Pty Ltd. Jackson has expressed the view that “…technology [can]
change and improve the 'old world' way of sharing healthcare experiences and
recommendations” [13]. The site was founded to help Australians with life saving
medical advice. The death of a 25 year old fitness instructor by overdosing with
paracetamol was given as an example of the reason for establishing the site by helping
people by providing information to them [14]. HealthShare’s vision is to become the
leading online presence in Australia for health consumers, professionals, community
organisations and service providers to collaborate. The co-founder Darryl Jackson
previously worked for the Australian Self-management Industry [ASMI], the peak
body representing companies involved in the manufacture and distribution of consumer
healthcare products in Australia. The site claims that “HealthShare’s program delivers
enhanced public perception of your brand” and “trust and an emotional connection with
health consumers and practitioners” [15].
The site is designed as a set of moderated communities, each dealing with a single
disease. The site includes a range of health communities covering over 350 health
topics. HealthShare has the support of health organisations such as the Heart
Foundation, Diabetes Australia and Alzheimers Australia. HealthShare identifies itself
as “Australia’s Social Health Network” where people can find other people with similar
health experiences. The structure of HealthShare is similar to Wikipedia, but with
credible moderated information, a type of ‘Facebook’ for health. The site was designed
to provide information for a range of communities commencing with the major health
issues affecting many Australians and then subsequently growing to added those issues
affecting smaller numbers of Australians. Overall the site displays a strong focus on
chronic diseases and topics related to healthy living.
Although the site is privately funded, the founders have a sponsorship program that
involves content partners working with HealthShare to have the opportunity to sponsor
particular health communities through the site [16]. HealthShare also provides the
opportunity for patients to recommend healthcare professionals. This is not a rating
system, and there is a need to explain why you would recommend that particular
healthcare professional. The CEO states that the average Australian needs more
information when choosing a health practitioner. HealthShare has a statement
identifying that the owners take privacy seriously by providing complete privacy
protection and unparalleled security. Personal information will not be released to any
other party.
4. Results
Web 2.0 has provided opportunities to support communication and information sharing
in a way that has not been available before. PatientsLikeMe has taken advantage of
these new opportunities by providing a social networking site focused on the patient
and her individual health. From initial registration onwards the focus is the patient as
an individual (your health, your current status, and the medication you are taking and
how this is affecting you). The site then offers individuals the opportunity to see other
people with similar health issues, taking similar medication or having similar reactions.
PatientsLikeMe also allows a patient to identify multiple concurrent conditions for
294 L. Ellis et al. / Social Media and Patient Self-Management: Not All Sites Are Created Equal
example registering for breast cancer and menopause together. The site is moderated
and professional healthcare information is available. Forums are grouped into higher
level categories such as Women’s Health and Pregnancy thereby facilitating broader
discussion of cross-related issues.
In contrast, although HealthShare has expressed its aim as being “Australia’s
Social Health Network” the site’s functions and features mean it is actually a
conventional web-portal that is tightly moderated to primarily provide health
information from healthcare professionals. When registering for HealthShare the focus
is on the particular medical condition, not the patient. Users are unable to link medical
conditions together into one community and instead, individual conditions are
presented in medically defined condition specific silos. For example, a patient with
breast cancer and menopause would be required to access different parts of the site to
acquire information on these conditions and would not be able to easily integrate the
separate streams of information and advice provided, even though both conditions
affect the same individual patient. HealthShare primarily operates with an ‘information
push’ model such that information is sent out to the user. Most of the content on the
site appears to be provided by numerous healthcare provider moderators. Where a
question is posted on the site it is usually answered by the healthcare moderator, rarely
by another patient user.
5. Discussion
The US based, PatientsLikeMe is a global site for all to use and share. HealthShare is a
site expressly designed for Australians. The HealthShare perspective is that a localised
service is necessary to avoid a situation of information not being relevant to Australians
(such as the unavailability in Australia of a particular drug being prescribed for a
particular disease). This perspective highlights the different underlying assumptions in
the two sites about the role of social media in supporting patient self-management and
how the advice, experiences and support available from patients for patients is valued.
Significantly, while both sites are owned and operated by private companies and
both purport to be about supporting patients to enhance self-management, there appears
to be a strong contrast in the nature, motivations and interests these sites and their
owners display in relation to patients.
PatientsLikeMe transparently declares that the business is a profit making
enterprise. The site uses the data collected from the site to support research and clinical
trials. PatientsLikeMe have their own research team and a brief scan of the current
literature soon reveals that these industry researchers also work in conjunction with
academics to investigate the data. While some recent discussions have begun to
question the use of this data [17] to date, PatientsLikeMe have continued to be
‘transparent’ in declaring how data is used and by whom.
In contrast HealthShare provides no clear information about whether or how any
de-identified data is used. There is no clear information relating to the profit status of
the company, or its business models and prospective business plans. Is HealthShare an
altruistic initiative motivated by a concern for Australian patients, or is it an example of
Astroturfing – a commercially driven activity leveraging patient interest in social
networking to generate profit from health profiling and data usage of its user base?
From the information available through the site it is difficult to tell definitively!
L. Ellis et al. / Social Media and Patient Self-Management: Not All Sites Are Created Equal 295
The rapid emergence of health focused social media has leveraged the strong
interest and appetite amongst many ‘internet empowered’ patients to connect with one
another and to openly and interactively share knowledge, experiences and attitudes
about their health and well-being. Simultaneously, on-going debates on data protection
and the security of electronic patient records as well as risks associated with personal
health profiling and business ethics related to harvesting patient data for profit have
emerged.
As the comparison of the two sites above illustrates, different health focused social
media sites support and value patients in very different ways. These differences
however are most evident, not in the health information provided by each site per se,
but rather in how patients are able to interact about their conditions with other patients.
Understanding and identifying the nature and motivations underpinning health related
social media sites is another dimension for patients to consider when seeking internet
support to self-manage.
References
[1] H. Cameron-Tucker, A study of supervised exercise with a chronic disease self-management programme
for people with COPD, University of Tasmania, PhD Thesis 2009, Australia
[2] T. O'Reilly, What is Web2.0- Design Patterns and Business Models for the Next Generation of Software.
2005 [cited 2012 January], Available from: http://oreilly.com/web2/archive/what-is-web.html.
[3] J.H. Frost, M.P. Massagli, Social Uses of Personal Health Information Within PatientsLikeMe, an Online
Patient Community: What Can Happen When Patients Have Access to One Another’s Data. Journal of
Medical Internet Research 10(3) (2008).
[4] D. Hoch, T. Ferguson, What I've Learned from E-Patients. PLoS Med 2(8) (2005),e206.
[5] Health on the Net Foundation (HON Code) [cited October 2012]; Available from:
http://www.hon.ch/HONcode/Patients/Visitor/visitor.html.
[6] L.J. Trevena, PatientsLikeMe and the tale of three brothers. Medical journal of Australia 195(5) (2011),
258-9.
[7] P. Wicks, M. Massagli, J. Frost, C. Brownstein, S. Okun, T. Vaughan, et al, Sharing Health Data for
Better Outcomes on PatientsLikeMe. Journal of Medical Internet Research 12(2) (2010).
[8] Anonymous. UCB to Mine Social Media Website for Data. Pharmaceutical Representative 39(8) (2009),
8.
[9] J. Frost, S. Okun, T. Vaughan, J. Heywood, P. Wicks, Patient-reported outcomes as a source of evidence
in off-Label prescribing: Analysis of data from PatientsLikeMe, Journal of Medical Internet Research 13(1)
(2011).
[10] G. Burnett, M.L.A. Lustria, J. Cortese, M. Kazmer, J. Frost, J.H. Kim, et al. PatientsLikeMe: Social
norms and information values in an online health support group. Proceedings of the American Society for
Information Science and Technology 46(1) (2009), 1-4.
[11] J. Heywood. PatientsLikeMe. 2005 [cited Oct 2012]; Available from:
http://www.patientslikeme.com/about.
[12] Anonymous. PatientsLikeMe. 2005 [cited January 2012]; Available from: www.patientslikeme.com.
[13] Darryl Jackson HC. HealthShare. 2011 [cited Jan 2012]; Available from:
http://www.healthshare.com.au/docs/DOC-6685.
[14] Channel 9 Television Australia, Kerri-Anne Morning Show Interview 13th October 2011. [updated 2011;
cited Oct 2012] Available from: http://www.youtube.com/watch?v=YKm_3qtSU7k.
[15] Darryl Jackson HC. HealthShare Pty Ltd [updated 2012; cited Oct 2012]; Available from:
http://www.healthshare.com.au/questions/44932-how-can-our-company-sponsor-a-healthshare-community.
[16] HealthShare Pty Ltd [cited Oct 2012]; Available from:
http://www.healthshare.com.au/sponsor-profile-benefits.
[17]BiolArtist2012 Science-Based Medicine: Exploring issues and controversies in the relationship between
science and medicine [cited Oct 2012] Available from:
http://www.sciencebasedmedicine.org/index.php/the-perils-of-patient-driven-clinical-research/comment-
page-1/.
296 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-296
Abstract. Little information is known about how Saudi health websites compare
internationally. The purpose of this paper is to compare two leading Saudi health
websites with leading international health websites. The study was conducted as a
convenience sample at a graduate health college in Saudi Arabia. A total of 42
students participated in the study. The study found that, in general, English
websites have higher levels of performance with regard to quality of information,
authority and objectivity, coverage and currency, and design. However, the
respondents considered Saudi health websites to be superior with regard to
maintaining privacy and security. The results indicate that much more work is
needed in designing Saudi Health to make them more trustworthy and credible.
The limitations of this work and future research directions are also discussed.
Introduction
For years, the development of websites with trustworthy health information within the
Arab world has been neglected. Much of the available Arabic health information has
focused on a particular healthcare organization’s goals, services, and policies [1]. A
study conducted by Al-Tuwaijri determined that, of the 122 Arab health websites
evaluated in terms of providing high quality health information, only five provided
trustworthy health information to the Arab consumer [1]. As a result of this research,
various healthcare organizations within the Arab world began to develop health
information websites that are trustworthy and credible to improve health literacy and
empower patients. A recent study on how Saudis use online health information found
that 58% of those surveyed used the internet to search for health information [2]. In the
study, most patients discussed the health information found online with their physician
and believed that sharing such information improved patient-physician relationships
[2].
1
Corresponding Author: Dr. Mowafa Househ, King Saud Bin Abdul Aziz University for Health
Sciences, College of Public Health and Health Informatics, Riyadh, Kingdom of Saudi Arabia; Email:
[email protected]
M. Househ et al. / Empowering Saudi Patients 297
With the growing need for trustworthy and credible health information in Saudi
Arabia, various healthcare organizations have developed health information websites
for the Saudi public. This paper evaluates two leading healthcare information websites
sponsored by public healthcare organizations within the country. It compares two
leading Saudi health websites to leading English websites in Canada, England, and
Australia. The study evaluates various dimensions related to providing trustworthy
health information online. The King Abdullah Bin Abdulaziz Arabic Health
Encyclopedia (KAAHE), one of the largest initiatives for providing credible online
health information within the Arab world, was not included in this study because it was
not available online when the study was conducted.
1. Methodology
A convenient sample survey was conducted using male and female students at the
College of Public Health and Health Informatics at King Saud Bin Abdul Aziz
University for Health Sciences, Riyadh, Kingdom of Saudi Arabia from November
2011 to January 2012. The College of Public Health and Health Informatics is the first
college in Saudi Arabia to offer a graduate program in Public Health and Health
Informatics within the country of Saudi Arabia. The program accepts applications from
the various regions within the country as well as the Gulf region. The sample represents
some of the most highly educated groups within the country of Saudi Arabia.
The survey used by Ali et al. to evaluate Egyptian health websites was used in the
present study to evaluate two leading Saudi health websites [3]. The tool consists of
one technical category and one ethical category. The technical category evaluates six
items related to authority, objectivity, coverage, currency, design, privacy and security.
The ethical category evaluates the quality of information, informed consent, and
professionalism.
The administered questionnaire included 49 questions and took approximately 3
hours to complete. The questionnaire was e-mailed to each participant with a Microsoft
Excel attachment. The Microsoft Excel attachment included survey questions and clear
instructions on how to input data and return the survey. The participants were provided
one week to complete the survey. Participation in the study was voluntary, although
completing the survey constituted 20% of the student’s final grade.
The participants were asked to evaluate two Saudi websites using the survey.
Through random selection, two students were asked to evaluate a leading English-
language health website. A total of 21 English websites were included in the
evaluation: 9 from the U.S.A, 5 from the United Kingdom, 3 from Australia, and 3
from Canada. In addition, after they completed the survey, the participants were asked
to rank all 21 English health websites from 1 to 10, with 1 being the least trustworthy
and usable and 10 being the most trustworthy and usable. The researchers then added
the scores and divided them by 21 to give a total score between 1 and 10. Any number
between 1 and 4 was classified as having low trustworthiness and usability; health
websites that scored 5-7 were classified as having medium trustworthiness and
usability; and websites with scores of 8-10 were classified as having high
298 M. Househ et al. / Empowering Saudi Patients
trustworthiness and usability. In this study, the Saudi health sites were compared to the
four highly rated and trustable English websites (as ranked by the respondents).
1.2. Analysis
Data was analyzed by descriptive data analysis and graphs using Microsoft Excel.
2. Results
All 42 students returned their survey within one week of receiving it. The participants
in the convenience sample included 31 females and 10 males. Their ages ranged
between 24 and 55 years. Most of the students were in health sciences programs
encompassing various professions such as health informatics, medicine, nursing,
pharmacy, medical sciences (laboratory technologists, x-ray techs, etc.), and dentistry.
Every respondent noted that all highly rated English sites provided a higher quality of
information than Arabic websites and mid-to-low quality English websites. Each
respondent also noted that medical professionals provided the health information
available on top rated English websites, while nearly half the respondents noted that
Saudi health websites contained information that was written by medical professionals
(52% and 55% for the two Saudi websites considered). Furthermore, every respondent
noted that the information provided by highly rated English websites included scientific
studies, while this was the case for only 43% and 36% of the two Saudi websites. All
respondents indicated that the language was clear for the top rated English websites; in
this regard, high ratings were also given to the Saudi websites (98% and 95%.) Only
63% of respondents found that the highest ranked English websites provided a
publication date, while a much lower percentage respondents found that the Saudi
websites did so (33% and 26%).
With regard to the use of references, 88% of respondents noted that the highly
ranked English websites provided information on their sources, while much lower rates
for Saudi websites were reported (24% and 7%). With respect to HONCODE
certification, which certifies credible and trustworthy websites, 75% of respondents
reported that highly rated English websites have HONCODE certification, while none
of the Saudi websites were found to have HONCODE certification.
This study also considered the authority and objectivity of each website. When
considering the highly ranked English websites, 88% of respondents reported that
authors were identified. Much lower rates of author identification were reported for the
Saudi websites (31% and 26%). Three-quarters (75%) of respondents found that the
highly ranked English websites clearly stated institutional affiliations on their website;
on the other hand, most respondents noted a lack of clarity in this regard on the Saudi
health websites considered (67% and 57%). Respondents also found that most of the
highly rated English health websites contained the authors’ contact information (75%),
M. Househ et al. / Empowering Saudi Patients 299
while much lower rates were indicated for the Saudi health websites considered (33%
and 24%).
Regarding the websites’ purpose(s) and objectives, all highly ranked English
websites were found to clearly state their objectives. Most respondents reported that the
purposes of the Saudi health websites were clearly stated (71% for both sites).
Respondents also noted that, for all highly rated English websites, the material
presented was in line with the objectives. Most respondents also noted that the material
presented on Saudi health websites corresponded with the objectives (75% and 62%).
All respondents noted that the top English websites satisfied their information needs.
Similarly, almost all respondents (95%) noted that one Saudi health website satisfied
their information needs, but the other Saudi website had a much lower rate (63%) for
satisfying the informational needs of the respondents.
The majority (88%) of respondents found the highly ranked English websites to be
current and up to date. This measure was lower for Saudi websites, 79% and 33%. All
respondents noted that all highly rated English websites were up to date, while only
86% considered one Saudi website to be up to date and only 48% considered the other
to be current.
2.4. Design
All respondents noted that the highly rated English websites could be accessed and
navigated easily. The respondents also indicated that both Saudi websites were easily
accessed and navigated (86% and 83%). Almost all respondents (88%) noted that the
highly ranked English websites used visuals to convey information. Students reported
that one Saudi website was better than the English websites in using visuals to convey
information (95%). For the other Saudi website, only 74% of respondents reported the
use of visuals to convey health information.
All respondents reported that high quality graphics were used in the highly ranked
English websites, while the respondents indicated lower satisfaction with Saudi
websites (81% and 62%). The respondents also found that all highly ranked English
websites were useful, while a much lower number of respondents (76% and 52%)
found the Saudi health websites to be useful. Most of the respondents (88%) noted that
the highly ranked English websites had interactive features that were easy to use, while
the respondents reported lower satisfaction with the Saudi health websites in this regard
(74% and 64%).
Regarding software updates, all respondents reported that the highly ranked
English websites could be accessed without additional viewers or plug-ins. Only 86%
and 81% of the respondents noted that the Saudi websites could be accessed without
software plug-ins. With regard to informing or providing support for downloading the
software required, both Saudi health websites ranked higher (52% and 29%) than the
high-ranking English language websites; only 25% of the respondents noted that the
highly ranked English websites provided information regarding where to download the
plug-ins required.
300 M. Househ et al. / Empowering Saudi Patients
2.5. Privacy
Regarding privacy, almost all of the respondents (95% and 79%) considered the Saudi
websites to be more secure than the highly rated English websites (which had a support
rating of 75%). Regarding unauthorized data access, most of the respondents (79% and
75%) considered both Saudi websites to be better than the highly rated English
websites (which had a support rating of 75%).
3. Discussion
This study assessed and compared two highly ranked Saudi websites with highly
ranked English websites. Overall, the study found that English websites performed
better with regard to quality of information, authority and objectivity, coverage and
currency, and design. However, the respondents also indicated that the Saudi health
websites were better in maintaining privacy and security. The results indicate that much
work is needed in designing Saudi Health websites to be more trustworthy and credible
compared to English sites.
A current initiative is attempting to address these shortcomings. The King
Abdullah Bin Abdulaziz Arabic Health Encyclopedia (KAAHE) was launched on the
Internet in Saudi Arabia in April 2012. This initiative, led by the Saudi National Guard
Health Affairs organization, aims to provide credible and reliable health information to
the Arab world [4]. It is, therefore, different from the websites evaluated in this study,
because the websites in this study are more local and focus on specific populations or
disease groups. By providing a credible and trusted source of health information,
KAAHE is expected to improve health literacy and to indirectly improve patient safety
in Saudi Arabia and throughout the Arab world.
The present study compared two highly ranked Saudi websites with highly ranked
international English websites. This research was limited by its use of a convenience
sample, which may limit the results’ generalizability and validity. The sample used in
this study was composed of students who were highly educated and familiar with the
use of technology and health websites. Future research should be conducted in
hospitals using a wider and more diverse population and should assess the performance
of the KAAHE project within Saudi Arabia and the Arab world. Another limitation of
this study is the use of descriptive statistics; future plans include providing more
detailed statistical analyses.
References
[1] E.M. Borycki, M.S. Househ, A.W. Kushniruk, C. Nohr, H. Takeda, Empowering Patients: Making
Health Information and Systems Safer for Patients and the Public. Contribution of the IMIA Health
Informatics for Patient Safety Working Group, Yearb Med Inform 7(1) (2012), 56-64.
[2] K. Alghamdi, N. Moussa, Internet use by the public to search for health-related information.
International Journal of Medical Informatics 81(6) (2012), 363-373.
M. Househ et al. / Empowering Saudi Patients 301
[3] N.M. Ali, A. Gamal, A.A. Mohamed, A Pilot Study For Evaluating Egyptian Health Websites: Technical
And Ethical Perspectives, Bangladesh Journal of Bioethics 1(3) (2010).
[4] M.M. Altuwaijri, Empowering patients and health professionals in the Arab World. The King Abdullah
bin Abdulaziz Arabic Health Encyclopedia on the Web. Yearb Med Inform (2011), 125-30.
302 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-302
Abstract. Providing tailored and easily accessible health information for mental
health clinicians and patients can be enabled through Information Technology and
Communications (ITC). The literature is mixed regarding the quality, utility and
accessibility of health information in the popular press for this purpose. However,
there is consensus that mental health information in the popular press is readily
available, easily comprehended by patients, and is continually updated. We report
the process by which mental–health-focused articles in the popular press are
identified, screened, and disseminated to a large network of doctoral level
psychologists (the PsyUSA network). We analyze 4-year article distribution and
access data, and conclude that the distribution of mental–health-related popular
press articles prompted article access. We leverage this experience to formulate a
model for direct access to clinician-vetted mental–health-related popular press
through a curated web based archive.
Introduction
As early as 2003, 80% of American consumers searched for health information on the
web, with 21% searching mental health-related topics [1]. A vast array of health
information is readily available online via newspapers, magazines, television news
outlets, and press releases; i.e., “the popular press.” Health information may be
provided via such outlets through reported research results, review articles, news
articles, exposés, expert interviews, book reviews, tips and advice. This information
educates and entertains, increasing awareness of mental health-related issues.
A major advantage of popular press psychological resources is that they are written
in laymen’s terms and can be understood by most. Further, articles in the popular press
are often available for free or nominal charge. However, health information delivered
by popular media can be compromised by quality, accessibility, and utility issues [2-7].
Relevant information needed to evaluate article quality often is missing [3, 8], and
articles can “disappear” or be sequestered behind pay walls.
The increasing focus upon patient-centered care warrants increasing the
availability of applicable, reliable, and valid information related to patient needs.
R.T. Zozus Jr. et al. / Taming Mental–Health-Focused Popular Literature: A Crazy Idea? 303
1. Background
The PsyUSA network initiated operations on October 28, 1995, with a goal of
providing a forum for doctoral-level psychologists to exchange information, interact,
broaden exposure to psychological topics and stay current. Toward the lattermost goals,
mental–health-related articles are disseminated by listserv (push technology) via email
“posts,” wherein brief portions of the article (as well as relevant links connecting to the
article in its entirety) are sent to psychologists by email each day. From May 1, 2006
through September 1, 2012, a total of 10,603 article snippets and links have been
distributed though the PsyUSA listserv—an average of 4.64 articles posted per day.
Volunteer doctoral-level clinical psychologists, referred to as “Newshounds,”
identify and disseminate posts to the network; i.e., upstream versus downstream
filtering [9]. Today, three Newshounds (first three authors) work for the network in this
capacity on a rotating weekly basis and spend an estimated 1.5 to 2 hours to prepare
weekly article postings. This time is spent in identifying, reading, and discerning which
popular psychology articles should be distributed, in formatting the information sent to
comply with copyright and listserv requirements, and in obtaining shortened universal
resource locators (URLs) to access the full article.
Identifying articles focused on popular psychology for dissemination to the
PsyUSA network is done manually, using different article identification strategies.
They include utilizing website aggregators of psychology-focused articles, Rich Site
Summary (RSS) or general website aggregators to track websites with frequent fresh
mental health related content; and setting up alerts to notify Newshounds of new
articles.
Screening articles for PsyUSA network distribution is based on Newshound
experience, educational background, and personal interest. Articles are screened in a
two-part process. Initially, titles and brief portions of articles are scanned. Based on
perceived interest to PsyUSA network members, Newshound interest, and perceived
quality of the article source; articles are selected for further review. The second phase
of the screening process begins as articles judged potentially worthy of dissemination
are read in full. Criteria considered include presentation of novel information or ideas
likely to stimulate discussion and thought, completeness of topic coverage,
transparency of information sources and assumptions (or full disclosure of primary data
or evidence), methodological rigor and relationship of the article’s discussion of
relevant research. Language use, general clarity, and organization are also considered
when judging article quality.
Selected articles are then prepared for dissemination. Fair Use provisions of U.S.
copyright law dictate that only the title, attribution, and short snippet from copyrighted
articles be distributed in a post. A web link to the original article is always provided.
304 R.T. Zozus Jr. et al. / Taming Mental–Health-Focused Popular Literature: A Crazy Idea?
2. Methods
This observational study reports analysis of data pertaining to one PsyUSA network
Newshound (the first author). All articles posted by this Newshound Since July 12,
2009 were assigned links using the Bit.ly URL-shortening service (www.bitly.com) for
all non-press release articles; full press releases are intended for public dissemination
and thus were distributed directly in the post. Because Bit.ly has been reported as the
most popular URL-shortening service since May of 2009 [10, 11], it offers the best
indirect means of estimating article popularity for the general public.
The Bit.ly service provides one Bit.ly link for each article. Standard Bit.ly metrics
break down the total number of clicks by referrer (in this case, the Newshound), as well
as the total number of clicks on Bit.ly to the article from any referrer. The Bit.ly
metrics were analyzed to explore full article access by PsyUSA network members.
These analyses were conducted on aggregate web statistics, from which individual
behavior cannot be determined. Additionally, the first author kept records of article
screening, e.g., articles that he read through fully but did not disseminate because of
quality concerns (using a service called Instapaper <http://www.instapaper.com/>).
This research was submitted as Pro00041763 to the Duke University Institutional
Review Board and was granted exemption.
3. Results
From July 12, 2009 to August 21, 2012, the first author sent 1,170 posts to PsyUSA
members (approximately 324 per year). Bit.ly observational data during this period
shows that each post to the PsyUSA network generated on average 7 clicks (median of
3) to the full article. During the same period, PsyUSA members logged an average of
215 clicks per month on these posted articles. In the general population of non-PsyUSA
members, each of the Bit.ly links generated an average 320 clicks per article (median of
18) and 9,843 clicks on average per month.
Figure 1 shows the average number of clicks per month on posted articles
disseminated by the first author, by year. The drop in Bit.ly referenced articles from
2010 to 2011 corresponds to an increase in press release distribution starting in 2011.
R.T. Zozus Jr. et al. / Taming Mental–Health-Focused Popular Literature: A Crazy Idea? 305
In 2009 and 2010 an average of 6.5 and 4.9 press releases were distributed each month
respectively; whereas in 2011 and 2012, the Newshound distributed 25 and 29 press
releases each month respectively. Recall that the full press releases text is distributed;
obviating the need to use Bit.ly and therefore lowering the average Bit.ly clicks per
month. A drop in PsyUSA network membership from 2007 to 2012 of about 75
members may have also contributed to the decline in Bit.ly accessed articles.
4. Discussion
Full access to psychologically focused articles as indicated by Bit.ly metrics was robust,
especially considering that articles were popular press, and “pushed” to psychologists
without regard to individual need or interest. We interpret clicks on Bit.ly links
disseminated by Newshounds as an indication that the recipient found the article of
sufficient interest to follow the link to the full article. We leave the reader to form
opinion as to whether or not exposure to psychological information of interest to a
practicing psychologist is of use to them in their practice. Network clinicians have
noted using these articles in patient education, as well as for increasing their
psychological breadth of knowledge and staying current in the field. There has been no
formal assessment of the uses and perceived helpfulness of distributed articles to
network members. Future survey research with PsyUSA network members could
provide such insight.
Given the article access with the “push” model of information dissemination in the
PsyUSA network, we posit that a “pull” model could be even more useful- e.g., a
searchable archive of vetted mental health focused articles accessible by patients with a
clinician-gated recommendation, or even the general population on demand. A recent
report that consumers receive a majority of their health information from “traditional
media and interpersonal sources” [12] underscores this need.
For such a “pull” model to work, a comprehensive, organized, easily searched web
archive of high quality popular press information must be created and maintained.
Finding relevant articles for inclusion in the database can be accomplished by
augmenting current methods with intelligent-search web crawlers to automate article
identification, application of heuristic algorithms to identify articles most likely to be
topically pertinent; and application of existing heuristics for health information quality
on the internet. While Newshounds currently operate informally to assess article
quality (described above), heuristics exist for assessing the quality of health
information on the web [13] that can also be applied and semi-automated. Further
other indications of article quality such as user feedback and rating may be employed.
Article tagging to facilitate information retrieval could be accomplished by
leveraging the Library of Medicine’s Medical Subject Headings (MeSH) database. It
uses common language, has natural language definitions, and provides a hierarchical
structure that could aid organization and search. Additional research is needed to
clarify patient and caregivers search and navigation strategies, and to ascertain the
usefulness of including a brief abstracted article summary along with the basic
information now provided of article an article snippet, title, attribution, and link to full
article.
Copyright considerations figure prominently in constructing such a database.
Because of the non-profit, educational nature of this work; inclusion of the article title,
306 R.T. Zozus Jr. et al. / Taming Mental–Health-Focused Popular Literature: A Crazy Idea?
as well as one or two article paragraphs should be allowed under Fair Use provisions of
U.S. copyright law, and Fair Dealing Doctrine of the Commonwealth Nations.
Maintenance of such a web-based archive would be needed. Such maintenance
would include ensuring that articles continue to be available at their cited web location,
and that they still are available freely (or at nominal fee).
As with all web based health information, users of this data base should be clear
that although psychology-focused popular media articles may offer advice and tips
(often by experts in the field), the information they provide should not be construed as
professional advice suited to an individual’s psychological health issues and needs.
Further research should include content analysis of high-use popular press to
discern which mental health articles generate the most consumer interest- and why.
Additional research may also inform how popular media psychology focused articles
can best be used for education purposes.
5. Conclusion
6. Limitations
Due to the observational nature of this work, we cannot attribute cause. And we cannot
directly link clinician access to usefulness. Further, generalizability of our conclusions
is limited because PsyUSA network membership have self-selected based on perceived
value of receipt of popular-press articles. These findings are further limited by the
retrospective nature of data collection that relied on aggregate counts from Bit.ly as an
indicator of article use and popularity.
References
informatics: Bridging the gap between access to information and information understanding. Biomed
Inform Insights 2 (2009), 1–10.
[9] G. Eysenbach, “Credibility of Health Information and Digital Media: New Perspectives and
Implications for Youth." In: Digital Media, Youth, and Credibility. Ed. M.J. Metzger, A.J. Flanagin.
The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge,
MA: The MIT Press, 2008. 123–154.
[10] J. Wortham, Bit.ly eclipses TinyUrl on Twitter, New York Times–Technology. Bits. [cited Sep 15, 2012],
[updated May, 2009], Available from: http://bits.blogs.nytimes.com/2009/05/07/bitly-eclipses-tinyurl-
on-twitter/.
[11] SiteOnn. Top 3 URL shorteners you should use. Available from http://siteonn.com/top-3-url-shorteners-
you-should-use, accessed on September 15, 2012.
[12] E.W. Maibach, D. Weber, H. Massett, G.R. Hancock, S. Price, (2006). Understanding consumers’
health information preferences: Development and validation of a brief screening instrument. J Health
Commun 11 (2006), 717–736.
[13] A. Jadad, R. Alejandro, A. Gagliardi, Rating health information on the Internet: Navigating to
knowledge or to Babel? JAMA 279 (1998), 611–614.
308 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-308
Abstract. Patients with chronic illnesses require tools and resources to facilitate
self-management. Personal Health Records (PHRs) are a promising option for
delivering these tools and resources to patients with chronic illnesses. As such,
many organizations are becoming interested in PHR procurement. However,
traditional procurement methods may not ensure the system success and adoption.
In this study a group of subject matter experts discussed the possibility of
converting a paper-based PHR into an electronic tool. These discussions resulted
in generation of several important criteria for assessing commercially available
PHR solutions and other considerations related to PHR procurement. These
considerations should be contemplated and discussed with stakeholders prior to
PHR procurement. In order to realize the benefits PHRs, it is imperative that the
appropriate selection is made. Prior to purchase commitment, a trial period can
prove extremely useful for performing usability analyses and ensuring
interoperability. Supplementing traditional procurement methods with these
preliminary user evaluations will increase the likelihood that the selected system
best matches the needs of users and purchasers. Moreover, the risk of system
failure and the risk of limited adoption of the PHR by the public will be reduced as
a result of adopting these methods.
Introduction
1
Corresponding Author: Helen Monkman, email: [email protected]
H. Monkman and A. Kushniruk / Considerations for Personal Health Record Procurement 309
1. Methods
A chronic disease interest group was developing a paper-based PHR and was interested
in also offering their PHR to patients in an electronic format. The purpose of study was
to generate a set of criteria to assess commercial-off-the-shelf (COTS) PHR products as
potential options for converting the existing paper-based PHR into an electronic tool.
310 H. Monkman and A. Kushniruk / Considerations for Personal Health Record Procurement
The principal investigator coordinated and chaired the meetings. Two subject matter
experts from different arthritis groups provided insight on the paper-based PHR,
clinical and patient requirements. During the development of the paper-based PHR
patients were questioned and consulted about their needs and preferences for an
electronic PHR; however, given that this study was only preliminary, patients were not
included on the team. Two other representatives were subject matter experts in
information technology and provided insight on functional and technical requirements.
1.2. Procedure
The group met on multiple occasions to discuss and come to consensus on a set of
criteria to compare commercial-off-the-shelf (COTS) PHRs with the paper-based PHR.
Additionally, criteria were developed to assess issues unique to electronic tools (e.g.,
privacy and security). Criteria ranged from user focused aspects of PHRs (e.g.,
functionality, usability) to criteria important from the perspective of the dispensing
agency (e.g., vendor stability, pricing).
In conjunction with the identification of criteria, questions were generated to
assess these criteria. Some questions were adopted from PHR evaluation questions
developed by the former California Regional Health Information Organization
(CalRHIO). However, many of these questions were not well suited to the Canadian
context so they were supplemented with questions created by the project team.
To improve the likelihood that an appropriate PHR selection was made, limitations
of conventional procurement processes were considered through discussion to arrive at
extensions and new approaches.
2. Results
In addition to the criteria that were developed (summarized in Table 1), other important
considerations for PHR procurement discussed by the group are outlined here.
services. Nurses are available 24 hours a day by phone and dieticians and pharmacists
are available during specific times. By integrating existing resources such as
HealthLinkBC with a PHR, users could be presented with credible, personalized
information according to a multitude of variables (e.g., gender, weight, illnesses,
location). Not surprisingly, personalized GFWECVKQPCNKPHQTOCVKQPJCUDGGPUJQYPVQDG
RQRWNCTYKVJRCVKGPVU=?
It was imperative that any COTS PHR considered was generic enough to support the
self-management of any chronic illness, but could be tailored according to the specific
condition(s) of individual patients. This strategy would be most effective for
integrating the entirety of a patient’s health information as well as streamlining the data
entry process for users. That is, users would likely be deterred from using different
PHRs for their different conditions. Thus, a common foundation of tools for all
patients (healthy and ill) with complementary modules for specific illnesses would be
the ideal solution. This would bolster the appeal for a variety of chronic illness special
interest groups. Additionally, the core components could be used by all health
consumers not only those affected by chronic illness.
Before committing to purchase a PHR, the dispensing agency was encouraged and
advised to negotiate a trial period (e.g., 6 months). This trial period allows for the PHR
to be evaluated with usability methods and tested for interoperability with other health
information systems (e.g., e-prescriptions, test results, electronic health records). This
would involve testing with a wide range of participants from the public. Given that
usability is a key factor in system adoption and interoperability plays a fundamental
role in the usefulness of the PHR, it is imperative that the selected PHR demonstrate its
capabilities in these areas before a purchase commitment has been made. This is
following a framework described by Kushniruk and colleagues for reducing risk of
system failure and adoption by carrying out usability testing of candidate systems prior
to final purchase of regional health IT systems [5].This trial period also provides an
opportunity for users to provide feedback on their likelihood of system adoption and
the advantages and disadvantages of the product itself.
3. Discussion
would be required to interface with a single system. Given that the core functionality
was developed, the costs of creating specialized modules would be minimized.
Additionally, this strategy would ensure the dispensing agency had the most bargaining
power regardless of whether a COTS or custom developed PHR solution was selected.
A provincial PHR is also likely to gain favour from Canadians who are waiting for a
PHR endorsed and dispensed by the government. Earlier this year, the Province of
Alberta adopted this provincial PHR strategy and it will be interesting to watch the
progress and outcomes of this approach [10]. Until all of the provinces have adopted a
PHR solution approach, individual chronic disease groups are likely to continue
pursuing PHRs procurement on their own.
Although the demands of chronic illness care are likely to motivate the deployment
and adoption of PHRs, people without chronic illnesses could also benefit from their
use. Furthermore, PHRs becoming more common and users demanding digitized
information may be the driving force behind ubiquitous EHR adoption. However, in
order to achieve these benefits, it is imperative that new and improved procurement
methods are adopted to ensure the selected systems best match the needs of their users
and purchasers.
References
[1] Health Council of Canada. Helping patients help themselves: Are Canadians with chronic conditions
getting the support they need to manage their health? Canadian Health Care Matters, bulletin 2
[Internet], 2010. [cited 2010 Sep 3] Available from www.healthcouncilcanada.ca/tree/2.17-
AR1_HCC_Jan2010.pdf
[2] Solomon MR.. Information technology to support self-management in chronic care: A systematic
review. Dis Manage Health Outcomes, 2008;16(6): 391-401. doi: 10.2165/0115677-200816060-00004
[3] Holman H, Lorig K. Patient self-management: A Key to effectiveness and efficiency in care of chronic
disease. Public Health Reports 2004;119(3): 239-243. doi: 10.1016/j.phr.2004.04.002
[4] Jones DA, Shipman JP, Plaut DA, Selden CR. Characteristics of personal health records: findings of the
medical library association/national library of medicine joint electronic personal health record task
force. Journal of the Medical Library Association: JMLA 2010;98(3):243-249. doi: 10.3163/1536-
5050.98.3.013
[5] Shortliffe EH, Cimino JJ. Biomedical informatics computer applications in health care and
biomedicine. 3rd ed. Springer, New York, 2006.
[6] Deloitte Consulting, Deloitte Center for Health Solutions. 2009 Canadian health care consumer survey
report: Treating patients as consumers. Washington DC,[Internet] 2009. [cited 2012 Sep 3] Available
from www.deloitte.com/assets/Dcom-Canada/Local%20Assets/Documents/Public%20Sector/
ca_en_healthcare_consumersurvey_oct09.pdf
[7] Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits,
and strategies for overcoming barriers to adoption. J Am Med Inform Assoc 2006;13(2):121-126. doi:
10.1197/jamia.M2025
[8] Jones R, Pearson J, McGregor S, et al. Randomised trial of personalised computer based information
for cancer patients. BrMed J 1999;319(7219):241-1247.
[9] Kushniruk A, Beuscart-Zephir MC, Grzes A, Borycki E, Watbled L, Kannry J. Increasing the safety of
healthcare information systems through improved procurement: Toward a framework for selection of
safe healthcare systems. Healthcare Quart 2010;13:53-58.
[10] Alberta awards PHR contract to TELUS-led group. Canadian Healthcare Technology. January 26, 9(2)
[Internet](2012). [cited 2012 Sep 5] Available from http://www.canhealth.com/News1878.html
314 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-314
*
}
³*
³
³
|
< *
*!
` !
!
*
*
**
*
<
*
}<
}*
*
}!
|
|
`
<
*
} <#*
} "`
^
! }`}"}^
! }^³ ¿*
<<
M. Clayton et al. / An Initial, Qualitative Investigation of Patient-Centered Education in Dentistry 315
}
!
}*
}`
*
!
`
<|`}*
|`
!
<
|
*
`
!
<
"¦ } <¾
*
`
|
`
}
\
*
| |}<
|
!
*
<
*
*
*
`
*
¾ `< ¾ <
\ ¾}
¾ `
¾ |
*
¾`
¾<^
|
*`
¾}
*
¾<
"}}
{
¾ *
*
*
`
*
<
**
`
*
*
|
*¾}
<
!
*
|\|
} |
`
!
`|
|
*
}
!
`
*
*
<
|
`*
Z} *
**
*
<
\
*
|
!
| |
|
< ^
*
!
}
}
|
*
*
`
< ^
`|
}|
*
}
` *
<
} | !
` *
`
<
!
`
*
<<
}
}
`
}
*
< !
`
| !
*
!
**
*
<} } <
* *
!
`
`
!
`
<
`|*
*
`
* *`
*
^
!
"`
^
! #
! <
316 M. Clayton et al. / An Initial, Qualitative Investigation of Patient-Centered Education in Dentistry
`|
|
*`
*
}
} *
}
|
*
`|
|
|
*
<#
`|
``
*
<
!
`
}|!
`
}
*`
*
*
`
`
<
*
|
Z
`
!
`
*
<
Z\
`
*
`
!
`
``|<*
|
|
*
`|}*
À^*
!
`
*
`*
^
`| *
#
#
^
*
^
*
}
!
`
^`
#
}
} *
`
}
*
`
|
|
< *
* !
!#}\*}
}!
¤ <
|
*
!
`! *
!
< | *
` *
|<
**
`
*
} `
!
* **
< |
*
*
}
\
}
<
} *
}
}
<
^
`|
`
*
**
*
<
*
**
} !
**
} ¨ `
}
< ^
* *
!
*
*
}*
*
}
*
*
<¤*
!
*
|
M. Clayton et al. / An Initial, Qualitative Investigation of Patient-Centered Education in Dentistry 317
*
!
<
!
*
|
<
|
**
} !
}
<
\ !
*
}
*
|
*
*
*
!
}
}
<
!
`*
**
*
*
}| |
`
`
¾
**
} ` <
`
¾
**
} ` <
<
}|
*` ¾
*
*
¾<
/<^
`
`<
Z %
QX \ '
**
`
`
!
*
*
<!
`
}
|
*
!
`<
}
`! *
*
}
|
*
**
}
*
|}
*
*
*
**
|
|
*
*
< |`} *
|
*
| *
*
`
<
} |
}
` !
318 M. Clayton et al. / An Initial, Qualitative Investigation of Patient-Centered Education in Dentistry
*
*
}
!
^
! <
Z<}|
`
<|`}
`
!
`
**|
\
|
!
!
<
*
|
| *
``
| *
!
*
<*
*
\|
|!
`
`
<
} *
!
|\|
!
}
!
*
}
Z
<
"
<
¥ }
} `< #
|
%
(
<³<
\ } } ! } ¤ £} }
¦<
* *
<%(
<£³<
`
\ } ¦
£<
**
`
*
*
<|(
%<£³<
£
} |
Z } ¤! £} ¦! <
\
Z
<%(
<!³<
Enabling Health and Healthcare through ICT 319
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-319
Introduction
1
Corresponding Author
320 J.P. Ronan / Tailored Care Management with Patient-Centered WBP in Primary Health Care
1. Relational Context
health care settings [16]. Additionally, static tailored health messaging had negative
outcomes when initiated in a worksite setting [17]. Tailored health communication
derived through interpersonal communications has the potential for high effectiveness
when coupled with relational approaches. In a recent meta-analysis, Wanyonyi,
Themessl-Huber, Humphris and Freeman reported that “a significant and positive
effective of face-to-face tailored health messaging upon participants behaviors” [18].
For primary health care practice, relational tailored health communications are
understood to be most appropriate when accounting for communication reception and
acceptance coupled with motivational behavioral change pathways [19-20].
Clinical information systems are those dynamic repositories of information that are
created, stored, collated, and protected for the purpose of creating historical records of
patient encounters. In the 21st Century these clinical information systems are evolving
from static paper-based structures to electronic databases where records can be securely
produced, archived, shared, evaluated, and coordinated via complex electronic input,
throughput and output technologies. These health informatics technologies have
created an opportunity for expanding capabilities for the delivery of primary health care.
There are four primary interconnected platforms that frame clinical information
systems at the primary health care level. Presented here by reducing level of magnitude,
they are Electronic Health Record (EHR), Electronic Medical Record (EMR), Personal
Health Record (PHR), and Web-Based Portals (WBP). The EHR is a computer-based
collection of health information that has been gathered by and is managed by an
enterprise, typically a physician or hospital. EMR is a computerized platform for
managing medical information collected by a hospital or physician practice and is
typically understood as less in scope and functionality than an EHR. EMRs are
transitioning into EHRs as sophistication requirements increase over time. PHRs are
person-centered systems designed to track and support health activities across one’s
entire life experience, not limited to a single organization or provider. WBPs are static
or dynamic web portals that house secure point-in-time health information for the
purpose of education, communication, assessment, or management related to health or
illness conditions. These WBPs are interconnected to EHRs, and PHRs simultaneously
or separately. Following is a description of WBP implementation for primary health
care practice.
Within a context of patient centered design, primary health care practices establish
scalable EHR and a secure clinic practice WBP for creating expedient communication
between health care providers and patients regarding tailored health/illness
management strategies. WBPs adjacent to EHRs are desirable because of their
flexibility in communication, scalability for PHRs, as well as fire-wall security and
integrity protection of EHR content by designing customized specific interfaces.
Not only can patients access the practice web-based portal and log-on to secure
modules for access to standardized screening tools, they may also free-text symptoms
or concerns such as medication issues or sub-optimal treatment experiences directly to
the health care team for immediate action. These WBP communications are routed to
the practice EHR and imported to cue for action in the form of a pop-up notice for the
case manager and or the provider of record. An assessment and action is required by a
member of the health care team in response to the patient inquiry. Immediacy and
322 J.P. Ronan / Tailored Care Management with Patient-Centered WBP in Primary Health Care
safety is enhanced by designing the pop-up to remain on the EHR screen until action is
taken. A tailored action plan response by a member of the health care team is
forwarded back to the patient at the web-based portal address. A text/email is
automatically forwarded to the patient for notification of a pending action plan.
Additionally, the action plan is routed to the patient’s EHR permanent record. A notice
is forwarded to the initiator of the action plan as soon as it is opened by the patient in
the web-based portal. If the action plan includes clinical follow-up it is coordinated
through the web-based portal with the office practice via electronic routing. This design
allows for a range of complexity to be managed appropriately, enhancing clinical
outcome and quality. Accommodating disadvantaged patients by developing alternative
communication pathways is mandatory. For example, use of telephone, proxy care
respondents, such as parents, children, siblings, and designated care givers are
acceptable alternatives.
Health information technology has increasingly become more accessible to the
health care team and now is one of the most important interfaces for communicating
health information. Second only to interpersonal networks, the internet is the primary
source for health information for the health conscious consumer with print media,
television, and radio becoming more passive sources [21].
Targeted designs for web-based portals is extensive with deployment in areas of
provider-patient relationship [22], health coaching for primary care patients with
chronic conditions [23], as an adjunct to personal health records [24], screening for
chronic conditions [25], promoting patient-centered preventive care [26], patient
satisfaction and the quality of office visits [27], and disseminating of health reminders
[28].
In a systematic review of web-based portals to improve diabetes outcomes, [29] it
was reported that usability studies have found that many patients are open to the use of
technology in their disease management, regardless of age. However, greater attempts
to assist disadvantaged, older, or less computer-literate populations must be designed
into any web-based portal initiative. While disadvantaged populations have shown less
inclination to enroll in patient portals programs, once enrolled, participation was no
different than all other participants [30]. Ancker, Barrón, Rockoff, Hauser, Pichardo,
Szerencsy & Calman [31] reported that during the first two years of a patient web-
based portal deployment, adoption rates for low-income populations with chronic
disease experience only low rates of disparity-related participation. It is essential to
recognize that web-based portal technologies focused on wellness/illness cannot be
dismissed because of disparate or disadvantaged circumstances for any population. A
usability strategy that focuses on an emphasis on deploying solution to include all
segments of the population must be attained that reinforces and sustains the relational
context.
Currently, the majority of designs for patient web-based portals can be classified as
static, offering information, education, questionnaires, or providing contact conduits to
the health care team. A more dynamic method is describable and would include an
approach that is more dialogical, relationship-focused for the patient with the health
care team. Such a system may improve both acute and chronic care outcomes.
Report of a recent random control trial of a dynamic WBP for chronic asthma
control is an example of the use of a nurse case manager for patient interaction and
management [32]. However, on careful examination this model design contains too
much content with excessive demands on the patient to complete daily ‘check in’ for
metric reporting, education, status assessment, hence usability will diminish long term
J.P. Ronan / Tailored Care Management with Patient-Centered WBP in Primary Health Care 323
sustainability. This model positions the nurse case manager adjacent to the point of
care instead of being integrated within the primary health care team. Langstrup [33]
reported on an asthma monitoring web-based portal implementation that failed
integration and long term sustainability wherein technology did not become a durable
portion of the primary health care practice. What was cogent in the Langstrup [33]
experience was that the primary care providers were distant to the project (delegated
responsibility) instead of engaged as partners in care with their patients.
4. Conclusion
Based on review of what is known from existing models, the author proposes that
by critically analyzing and incorporating experiences from the literature regarding
relational contexts, tailored health communications coupled with patient-centered web-
based portals present an integrated schema that will enhance effective health
communications and optimized health outcomes in primary health care practices in the
21st century.
References
[1] Hess JD. Gadow's relational narrative: an elaboration. Nursing Philosophy. 2003 Jul;4(2):137-48.
[2] Gadow SG. Relational Narrative: The Postmodern Turn in Nursing Ethics. Scholarly inquiry for
nursing practice. 1999;13(1):57.
[3] Newman MA. Theory for nursing practice. Nurs Sci Q. 1994 Winter;7(4):153-7.
[4] Duggan A. Understanding interpersonal communication processes across health contexts: Advances in
the last decade and challenges for the next decade. J Health Commun. [Review]. 2006 Jan-
Feb;11(1):93-108.
[5] Newman MA, Sime AM, Corcoran-Perry SA. The focus of the discipline of nursing. Adv Nurs Sci.
1991 Sep;14(1):1-6.
[6] Newman MA. Newman's theory of health as praxis. Nursing Science Quarterly. 1990 Spring;3(1):37-41.
[7] Rogers ME. The science of unitary human beings: current perspectives. Nurs Sci Q. 1994
Spring;7(1):33-5.
[8] Hartrick GD, Colleen V. Relational Practice and Nursing Obligations. Adv Nurs Sci. 2007;30(3):192.
[9] Loescher LJ, Ronan JP. Toward a holistic view of genetic technology as a way of knowing. Holist Nurs
Pract. 1998 Apr;12(3):1-8.
[10] Institute of Medicine (U.S.). Committee on Quality of Health Care in America. Crossing the quality
chasm : a new health system for the 21st century. Washington, D.C.: National Academy Press; 2001.
[11] Haggerty J, Burge F, Levesque JF, Gass D, Pineault R, Beaulieu MD, et al. Operational definitions of
attributes of primary health care: Consensus among Canadian experts. Ann Fam Med. [Article]. 2007
Jul-Aug;5(4):336-44.
[12] Epping-Jordan JE, Pruitt SD, Bengoa R, Wagner EH. Improving the quality of health care for chronic
conditions. Quality Safety Health Care. 2004;13:299 - 305.
[13] World Health Organization. The World Health Report 2003: Shaping the future. Geneva: World Helath
Organization; 2003.
[14] World Health Organization. Current and future long term needs. Geneva: World Helath Organization;
2002.
[15] Noar SM, Grant Harrington N, Van Stee SK, Shemanski Aldrich R. Tailored Health Communication to
Change Lifestyle Behaviors. Am J Lifestyle Med. 2011 March/April 2011;5(2):112-22.
[16] Ruffin MT, Nease DE, Sen A, Pace WD, Wang C, Acheson LS, et al. Effect of Preventive Messages
Tailored to Family History on Health Behaviors: The Family Healthware Impact Trial. Ann Fam Med.
2011 January 1, 2011;9(1):3-11.
[17] McHugh J, Suggs LS. Online Tailored Weight Management in the Worksite: Does It Make a Difference
in Biennial Health Risk Assessment Data? J Health Commun. [doi: 10.1080/10810730.2011.626496].
2011 2012/03/01;17(3):278-93.
324 J.P. Ronan / Tailored Care Management with Patient-Centered WBP in Primary Health Care
[18] Wanyonyi KL, Themessl-Huber M, Humphris G, Freeman R. A systematic review and meta-analysis of
face-to-face communication of tailored health messages: implications for practice. Patient educ couns.
2011;85(3):348-55.
[19] Rimer BK, Kreuter MW. Advancing Tailored Health Communication: A Persuasion and Message
Effects Perspective. Journal of Communication. 2006;56:S184-S201.
[20] Bandura A. Health promotion by social cognitive means. Health Education & Behavior.
2004;31(2):143-64.
[21] Dutta-Bergman MJ. Primary sources of health information: comparisons in the domain of health
attitudes, health cognitions, and health behaviors. Health Communications. 2004;16(3):273 - 88.
[22] Zickmund SL, Hess R, Bryce CL, McTigue K, Olshansky E, Fitzgerald K, et al. Interest in the use of
computerized patient portals: role of the provider-patient relationship. J Gen Intern Med.
2007;23(Supplement 1):20 - 6.
[23] Leveille SG, Huang A, Tsai SB, Allen M, Weingart SN, Iezzoni LI. Health coaching via an internet
portal for primary care patients with chronic conditions: a randomized controlled trial. Med Care.
2009;47(1):41 - 7.
[24] Clarke JL, Meiris DC. Electronic personal health records come of age. American Journal of Medical
Quality. 2006;Supplement to Volume 21(3):5S-15S.
[25] Leveille SG, Huang A, Tsai SB, Weingart SN, Iezzoni LI. Screening for chronic conditions using a
patient internet portal: recruitment for an internet-based primary care intervention. J Gen Intern Med.
2007;23(4):472 - 5.
[26] Chou AF, Nagykaldi Z, Aspy CB, Mold JW. Promoting patient-centered preventive care using a
wellness portal: preliminary findings. J Prim Care Community Health. 2010;I(2):88 - 92.
[27] Mooney BL, Boyle AM. Power to the patient; using portals can improve patients' satisfaction and your
practice's efficiency. Med Econ. 2011:S12 - S4.
[28] Koonce TY, Giuse DA, Beauregard JM, Giuse NB. Toward a more informed patient: bridging health
care information through an interactive comminication portal. Journal of the Medical Library
Association. 2007;95(1):77 - 81.
[29] Osborn CY, Mayberry LS, Mulvaney SA, Hess R. Patient web portals to improve diabetes outcomes: A
systematic review. Current Diabetes Report. 2010:422 - 35.
[30] Goel MS, Brown TL, Williams A, Hasnain-Wynia R, Thompson JA, Baker DW. Disparities in
enrollment and use of an electronic patient portal. J Gen Intern Med. 2011;26(10):1112 - 6.
[31] Ancker JS, Barron Y, Rockoff M, Hauser D, Pichardo M, Szerencsy A, et al. Use of an electronic
patient portal among disadvantaged populations. J Gen Intern Med. 2011;26(10):1117 - 23.
[32] Ahmed S, Bartlett SJ, Ernst P, Pare G, Kanter M, Perreault R, et al. Effect of a web-based chronic
disease management system on asthma control and health-related quality of life: protocol for a
randomized controlled trial. Trials. 2011;12(260):1 - 17.
[33] Langstrup H. Making connections through online asthma monitoring. Chronic Ilness. 2008;4:118 - 26.
Ontologies, Trust and Standards
This page intentionally left blank
Enabling Health and Healthcare through ICT 327
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-327
1. Prescribing Medicines
Prescribing medicine is a social act [1]. Through prescriptions physicians show their
patients that they recognize their complaints and are trying to help them. Where
medication is seen as the essence of medical practice, prescribing is the main thing
expected from a physician. A non-prescribing physician is seen as a contradiction.
Prescribing medications represents two sides of the same coin. On the one hand it
shows the authority of the physician by being able to solve a problem of the patient, on
the other hand the patient demands an instantaneous solution of his complaint.
Prescribing a medicine comes to the rescue of offering an immediate therapy, even its
efficacy is doubtful. A prescription functions as a legitimation of the patient’s sickness.
In some cultures a patient never goes home without a prescription, in other both
physicians and patients feel restraints [2]. It may explain why medication consumption
varies wildly in different Western countries. However, the overall consumption of
medications has grown exponentially and has become an important cost factor of health
care [3]. Health authorities and administrators have sought to contain the cost growth
1
E-mail: [email protected]
328 J. Aarts / The Social Act of Electronic Medication Prescribing
and one attempt was to make physicians aware of the notion of rational prescribing.
Not only would doctors prescribe drugs that really would work for a disease, but also
they would be encouraged to look at what costs the effect was achieved and whether
prescriptions could be filled with cheaper medicines. A similar role was attributed to
pharmacists and they were given the right to alter prescriptions to substitute with
cheaper ones. Often it would mean that generic drugs would replace brand drugs.
Rational prescribing was seen as one of the outcomes of evidence based medicine.
Based on the results of clinical trials and meta-studies guidelines and protocols would
be established to guide physicians in making diagnoses, selecting the proper course of
action and assess treatment outcomes. It would be hoped that doctors would adopt
these insights and put them to use in practice. In hospitals the costs of physician
prescribing would be borne by the organization. The cost increase prompted one
hospital in the early 70s to design and implement an electronic prescribing system for
physicians and this new class of information systems became also known as
computerized physician order entry (CPOE) systems [4]. Much later focus shifted to
patient safety. In his seminal paper in 1994 Lucian Leape wrote that the error rate in the
practice of medicine was high and that more than half of them could be attributed to
medication errors [5]. Already in the late 90s studies reported a positive effect of CPOE
on the decrease of medication errors [6, 7]. The landmark Institute of Medicine studies
on human errors in medicine and quality of care advocated the use of electronic
prescribing systems in health care [8, 9]. The adoption however has shown to be a very
slow process. Aarts and Koppel reported nine years later that in seven Western
countries they studied the adoption rate was nowhere higher than 20% of the
hospitals[10]. Another study reported that ten years after the publication of the IOM
reports efforts to reduce errors in medicine had limited success [11]. There is clearly a
gap between the expectations of professional and organizations adopting scientific
evidence and technology to improve practice and reality.
2. Electronic Prescribing
Electronic prescribing is a social act. Though electronic information systems are often
seen as “neutral” tools to help a job done, they represent in fact social norms embedded
in technology. The original Eclipsys system allowed physicians only to pick medicines
from an approved hospital formulary. The hospital administrators expected that
physicians would only select from this formulary and thus help save medication costs.
The hospital did indeed report savings and improved efficiency of nursing
documentation [12]. But embedded norms do form a clash with reality. Another norm
is the authority to prescribe, which rests with physicians. Only physicians are
authorized to enter medication orders. In many hospitals it was (and perhaps still is)
practice that nurses in night and weekend shifts could order pain medications to relieve
their patients, because of limited availability of physicians. For that purpose nursing
stations would have a pile of pre-signed prescription notes that nurses could fill out.
Physicians trusted this work practice because they knew that the nurses were
knowledgeable and experienced. The advent of CPOE made this way working
impossible and put the burden of entering orders completely on physicians. In one
instance it caused physicians to protest the increased workload [13]. In other instances,
users created workarounds to make their work doable. In a study of the implementation
of CPOE in a Dutch hospital Goorman and Berg found how the problem of only
J. Aarts / The Social Act of Electronic Medication Prescribing 329
3. The Future
In September 2010 the director-general of the Dutch health inspectorate announced that
electronic prescribing would become mandatory on January 1, 2012. Most likely, it will
be less of a problem in hospitals. Most hospitals in the Netherlands are currently
implementing electronic prescribing technologies. The shortage of physicians have led
to the introduction of nurse practitioners and physician assistants, who have received
prescribing authority, supervised by physicians. In a way the problem of the informal
work practice of nurses filling out orders has been removed by reconsidering
prescribing authority [20]. Clinical wards are in a way a kind of a micro-cosmos, where
professionals influence each other’s behaviors. The wide-scale implementation of
electronic prescribing will be much more difficult in primary care. More often, patients
visit a primary care doctor with vague complaints. In such situations emphasis on
330 J. Aarts / The Social Act of Electronic Medication Prescribing
References
[1] van der Geest S, Whyte SR, Hardon A. The anthropology of pharmaceuticals: a biographical approach.
Annu Rev Anthropol. 1996;25:153-78.
[2] Payer L. Medicine & culture: varieties of treatment in the United States, England, West Germany, and
France. 1st ed. New York: Henry Holt; 1988.
[3] Lied TR, Gonzalez J, Taparanskas W, Shukla T. Trends and current drug utilization patterns of Medicaid
beneficiaries. Health Care Financ Rev. 2006 Spring;27(3):123-32.
[4] Hodge MH. History of the TDS medical information system. In: Blum BI, Duncan K, editors. A history
of medical informatics. Reading (MA): Addison-Wesley Publishing Company; 1990. p. 328-44.
[5] Leape LL. Error in medicine. JAMA. 1994 Dec 21;272(23):1851-7.
[6] Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to
prevent errors of omission. J Am Med Inform Assoc. 1997 Sep-Oct;4(5):364-75.
[7] Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al. Effect of computerized
physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998
Oct 21;280(15):1311-6.
[8] Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human, building a safer health system.
Washington, D.C.: National Academy Press; 2000.
[9] Committee on Quality of Health Care in America. Crossing the quality chasm, a new health system for
the 21st century. Washington, D.C.: National Academy Press; 2001.
[10] Aarts J, Koppel R. Implementation of computerized physician order entry in seven countries. Health Aff
(Millwood). 2009 Mar-Apr;28(2):404-14.
J. Aarts / The Social Act of Electronic Medication Prescribing 331
[11] Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA, Sharek PJ. Temporal trends in rates
of patient harm resulting from medical care. N Engl J Med. 2010 Nov 25;363(22):2124-34.
[12] Watson RJ. A large-scale professionally oriented medical information system - five years later. J Med
Syst. 1977;1(1):3-21.
[13] Massaro TA. Introducing physician order entry at a major academic medical center: I. Impact on
organizational culture and behavior. Acad Med. 1993 Jan;68(1):20-5.
[14] Goorman E, Berg M. Modelling nursing activities: electronic patient records and their discontents. Nurs
Inq. 2000 Mar;7(1):3-9.
[15] van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician
order entry. J Am Med Inform Assoc. 2006 Mar-Apr;13(2):138-47.
[16] van der Sijs H, Aarts J, van Gelder T, Berg M, Vulto A. Turning off frequently overridden drug alerts:
limited opportunities for doing it safely. J Am Med Inform Assoc. 2008 Jul-Aug;15(4):439-48.
[17] van der Sijs H, Bouamar R, van Gelder T, Aarts J, Berg M, Vulto A. Functionality test for drug safety
alerting in computerized physician order entry systems. Int J Med Inform. 2010 Apr;79(4):243-51.
[18] Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized
physician order entry systems in facilitating medication errors. JAMA. 2005 Mar 9;293(10):1197-203.
[19] Berg M, Aarts J, Van Der Lei J. ICT in health care: sociotechnical approaches. Methods Inf Med.
2003;42(4):297-301.
[20] Kroezen M, van Dijk L, Groenewegen PP, Francke AL. Nurse prescribing of medicines in Western
European and Anglo-Saxon countries: a systematic review of the literature. BMC Health Serv Res.
2011;11:127.
[21] Hulscher ME, Grol RP, van der Meer JW. Antibiotic prescribing in hospitals: a social and behavioural
scientific approach. Lancet Infect Dis. 2010 Mar;10(3):167-75.
[22] Butler CC, Rollnick S, Pill R, Maggs-Rapport F, Stott N. Understanding the culture of prescribing:
qualitative study of general practitioners' and patients' perceptions of antibiotics for sore throats. BMJ.
1998 Sep 5;317(7159):637-42.
[23] Aarts J. The future of electronic prescribing. Stud Health Technol Inform. 2011;166:13-7.
[24] Bosk CL, Dixon-Woods M, Goeschel CA, Pronovost PJ. Reality check for checklists. Lancet. 2009 Aug
8;374(9688):444-5.
332 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-332
Introduction
According to an IOM report, somewhere between 44,000 and 98,000 people die in
different American hospitals every year because of “errors committed by medical
professional” [3]. The errors caused by technology-induced errors (i.e. errors that result
from use of health information technology) are related to human factors can come in
many forms, such as slips and lapses [2]. Although some active medical errors may be
detected, current approaches may not contribute to the problem of dealing with latent
errors. Even though work has been done in preventing technology-induced error in
healthcare, latent errors are still difficult to prevent due to the complexity of healthcare
processes and work activities. Some errors may arise from conscious, preconscious,
and unconscious human factors. For the purpose of analysis, errors are conceptualized
as an iceberg consisting of both active and latent errors. Active errors are associated
with conscious factors, while the latent errors include preconscious and unconscious
factors (see Figure 1).
1
Corresponding author: David T. S. Chio, School of Health Information Science, University of Victoria
Phone: +250 721 8575 E-mail: [email protected]
D.T.S. Chio et al. / Improving Decision Quality in Healthcare with an Error Prevention Model 333
1. Related Work
Provision of safe patient care is a major challenge confronting today’s health care
system [1]. Well-designed patient safety initiatives based on systematic interventions
may produce the best results in enhancing the quality of health care processes [5]. In
addition, Berner and Lande [1] summarize current data on the use and impact of
clinical decision support systems.
In order to bridge the gap between process and decision modeling, a formalized
linkage model has been proposed to facilitate the integration of strategic, decision and
process objectives within a single framework [7]. Furthermore, a number of
frameworks and models for considering technology-induced error have been compared
to show how they are practically employed in the prediction and prevention of such
error [2] (see Table 1).
Figure 3. An extension framework to improve the decision quality by reducing the errors
D.T.S. Chio et al. / Improving Decision Quality in Healthcare with an Error Prevention Model 335
In complex systems, when one component of a system which serves multiple functions
fails, all of the dependent functions may fail as well. As the dependency on technology
in complex systems increases and systems become more tightly coupled in time and
sequence, so does the likelihood of accidents [3]. By using this EPM model, error
prevention can be integrated in one systematic approach to deal with the human errors
when technology changes human tasks, shifts workloads, and tends to reduce or
eliminate human decision making.
This model provides guidelines for the development of cognitive and systematic
interventions to decrease medical errors to not only identify errors, but also transform
problems into a warning system. In table 2, some available methods, tools and
techniques (T&T) can be embedded within this model for some applications.
Discussion
In this paper, frameworks for considering error prevention were considered. Some
papers related to human errors and decision making were reviewed for their relevance
to error prevention and decision outcome improvement. Upon review of literature on
human errors in healthcare decision quality, the limitations of human beings are seen as
336 D.T.S. Chio et al. / Improving Decision Quality in Healthcare with an Error Prevention Model
a major factor contributing to this type of error in medical decision processes. Even
though considerable work has been done in preventing decision error in healthcare,
latent errors are still difficult to prevent due to the complexity of healthcare systems.
An Error Prevention Model (EPM) was proposed to provide potential solutions to
avoid preventable adverse events for CDSS to reduce human errors in the process of
decision making. Error lifecycle modeling could assist a healthcare authority to achieve
strategic objectives by providing methodologies and tools to develop integrated
healthcare process models, however further work is needed on how to identify the
methods and analytic tools to fit the specific error pattern.
Such a model has potential application to have a hybrid decision support
framework and EPM model built on best practice in healthcare. Validating this model
empirically will be next step, and this will involve performing systematic experimental
studies. Further study is needed to develop and test error mechanisms and the error
prevention engines proposed in this paper.
References
[1] Berner ES, La Lande TJ. Overview of Clinical Decision Support Systems. New York, NY: Springer New
York; 2007:3-22.
[2] Borycki EM, Kushniruk AW, Brender J. Theories, models and frameworks for diagnosing technology-
induced error. Studies in health technology and informatics. 2010; 160:714-718.
[3] Kopec D, Kabir MH, Reinharth D, Rothschild O, Castiglione JA. Human errors in medical practice:
systematic classification and reduction with automated information systems. Journal of medical systems.
2003:297-313.
[4] Elwyn G, O'Connor A, Stacey D, Volk R, Edwards A, Coulter A, et al. Developing a quality criteria
framework for patient decision aids: online international Delphi consensus process. BMJ: British
Medical Journal. 2006; 333(7565):417-419.
[5] Kalra J. Medical errors: overcoming the challenges. Clinical biochemistry. 2004; 37(12):1063-1071.
[6] Marquard JL, Robinson SM. Reducing Perceptual and Cognitive Challenges in Making Decisions with
Models . New York, NY: Springer New York; 2008:33-55.
[7] Neiger D, Churilov L, Flitman A. Business Objectives Modelling. Boston, MA: Springer US; 2009; 19:1-
26.
[8] Scott C, Bellala G, Willett R. Generalization Error Analysis for FDR Controlled Classification, In:
Proceedings of the 2007 IEEE/SP 14th Workshop on Statistical Signal Processing; 2007:792-796.
Enabling Health and Healthcare through ICT 337
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-337
Abstract. The revised Medical Device Directive has been adopted by the EU in
2010. A major change is that software for certain purposes is now considered a
medical device. This entails that a new view needs to be developed on the design,
development, evaluation and post-market surveillance of medical software that
meets the definition of a medical device. This paper identifies some issues at stake
and discusses them.
Introduction
In 2010 the EU adopted a revised Medical Device Directive (MDD) [1]. A rather small
change in the text – the insertion of the word “software” in the definition of a medical
device – may have significant consequences for the health IT community. Now some
standalone software are considered Medical Device (MD): a MD is “any instrument,
apparatus, appliance, software, material or other article, whether used alone or in
combination, including software intended by its manufacturer to be used specifically
for diagnostic and/or therapeutic purposes and necessary for its proper application”.
Such software are subject to the same regulation as MDs: to introduce them on the
market, it is necessary to demonstrate their safety and efficacy. The manufacturer must
carry out a conformity assessment, set up a technical file and sign a European
declaration of conformity. The MDD explicitly mentions the role of evaluation
(technical, clinical and usability evaluations) for proving that the device is safe and
meets the stated objectives.
Since its adoption, the new definition raised many questions and has been
discussed in various contexts. The aim of this paper is to explore some consequences of
the revised MDD on design, development, evaluation and post-deployment of medical
software and to discuss the challenges for the health IT community. A brief case study
1
Sylvia Pelayo – CIC IT Lille-EVALAB, CHRU de Lille, Bâtiment Hippocrate, 2 Avenue Oscar
Lambret, Lille Cedex, F-59037 FRANCE – Tel: 0033 3 20 44 48 83 – [email protected]
338 S. Pelayo et al. / Software as a Medical Device: Regulatory Critical Issues
Most of the discussions about the MDD focused on which software meets the
definition for a MD and which software does not. According to IEC 62304 [2], MD
software is “a software system that has been developed for the purpose of being
incorporated into the medical device being developed or that is intended for use as a
medical device in its own right”. Parallel to this definition, the manufacturer is anyway
the one determining if his software is or not a MD depending on the specification of its
intended use.
But, the further specification that “software for general purposes when used in a
healthcare setting is not a medical device” has introduced a certain level of ambiguity
on the interpretation of software to be classified as MD, especially when standalone
and/or software modules are used for the transmission and storage of patient data for
medical purposes [3]. For illustration, Electronic Health Records (EHR) are considered
an electronic view of the paper file that stores data and documents on a certain patient.
Given that some modules of EHR systems are the main sources for clinicians to make
diagnosis and establish patient treatment, they could be considered as MD software.
They should also follow the rules applied to MDs and have to be evaluated considering
the risk posed to patients and users. However, the European (and Canadian) Directive
tends to exclude EHR from the definition of MD software. Conversely, the US
regulation diffuses a detailed list of MDs and seems to suggest that EHR are MD
software. At first glance, the guidance document to qualify software as MD [4]
provides clarity, but with a closer look it can become a battlefield of interpretations. In
any case, in the end, the manufacturer is the one choosing. It is left to him to argue
whether his software is a MD or not. It is also left to him to draw the boundaries and to
argue that some modules are no MDs and others are.
The major requirement of the MDD is that MDs should be designed and manufactured
so that they will not compromise the safety of patients and users of the device. This can
be considered the first challenge for the developers of medical software. The MDD
states that a device should be designed so that it adheres to patient safety principles and
that state of the art methods are used. Currently, such a systematic patient safety based
design is not common practice and state of the art methods are not articulated. The
question arises whether the ISO standards should be considered as state of the art or
whether additional research based recommendations are needed [5]. Some initiatives tend to
provide this kind of supports such as the Standards Australia who published a handbook
with requirements for the presentation of health data [6]. The Microsoft and the National
Health Service have also developed guidelines for a common user interface for electronic
patient record systems [7]. But these initiatives are perceived by manufacturers as
“killers” of innovation because they would lead to the standardization of the design.
Moreover, manufacturers are supposed to publish sufficient information for an
objective analysis about the trade-offs between safety benefits and harms of their MD.
But manufacturers consider as confidential the knowledge about the design of their
S. Pelayo et al. / Software as a Medical Device: Regulatory Critical Issues 339
MD, especially drawbacks and glitches. Not sharing the software problems is an
industry implicit norm to keep competitors at bay. Manufacturers feel crashed with this
“aggressive government regulations of health IT” [8] and constantly claim that
regulations will cause an exodus of IT manufacturers from the field. In honesty, one
can acknowledge that there is a high number of regulation requirements usually
associated with onerous conformity assessment procedures.
There is also the important question of whether software regulation is even feasible.
Apart from evaluation, the MDD mentions explicitly the ergonomic factors that should be
considered and integrated in the design and development cycle following a standard user-
centred approach. The aim is to secure patient safety (as well as the safety of the users of
the device) by preventing usage errors. But such approaches are still not universally
applied in the development of medical software and it is an arduous challenge to
overcome.
First of all, many methodological issues are raised by the required conformity
assessment procedure. On the one hand, manufacturers should use published data on
other devices that are sufficiently similar to the device at stake. But it is very difficult
to see this applied to medical software. Let’s take the example of Computerized
Physician Order Entry (CPOE). Are different CPOE sufficiently alike that studies of
benefits and potential harm can serve as proof that a particular CPOE is safe? On the
other hand, software implementation is massively complex. It may imply many users
with different goals and may lead to resistance, work processes reorganizations and/or
frustration. It is difficult to predict all the scenarios in which software failures could
result in patient harm and liability [9]. Finally, the medical specificities often request
specific configurations and customizations. Medical rules and order sets, for instance,
are almost always locally developed and extensively modified. These adaptations may
improve the fit between software and the work processes and enhance patient safety.
But, even an “innocent” modification may have negative repercussions on the use of
the software. How to track and control all these post-hoc changes? Does one need to
prove that the order sets are safe? And what if changes are made over time?
To conclude on usability methodological issues, the standard proving conformity
with the ergonomic Essential Requirement [10] raises various problems and needs to be
improved in its current version. It is supposed to guide manufacturers for the usability
engineering process but it is almost impossible to understand and apply for non-
usability experts. The major risk is that manufacturers interpret the standard in a wrong
way. The usability process is a subpart of the risk management process (risk analysis
focused on errors of use) that manufacturers often perfectly handle. They are also
persuaded to have all the necessary information to document the usability file. But they
often identify the risks related to technical problems of the MD and not the risks of
errors of use.
to manage their insulin treatment determining the right dose of insulin by analyzing
ongoing treatments, glycemic index goals, previous and current days’ results, and
personal variables. A telecare platform enables physicians and nurses in the
telemonitoring to support patients using physician-approved protocols. Usability
specialists have accompanied the company in the validation phase of the usability
process (i.e. evaluation of the achievement of the safety criteria for usability) and the
documentation of the usability file, subpart of the technical file for CE marking. The
usability study in still in progress, only partial results are presented in the paper.
The demonstration of the safety and efficacy of their product was a lengthy and
onerous process for the company. But, the revised regulation brought the company a
potential big benefit. CE marking allows the introduction on the market. The further
step with additional assessments may allow their software to be registered in the list of
products and services qualifying for reimbursement (LPPR) to be reimbursed by the
National Health Insurance. The inclusion would give a competitive advantage
compared to the others editors of this kind of software. Aware of this, the company had
approached a large pharmaceutical manufacturer and a research team on diabetes to
create a consortium enabling to face all the challenges to attain CE marking, and the
reimbursement of their software MD.
First, a clinical study was conducted to show that the tool significantly improves
metabolic control and effectively helps patients manage their care and maintain their
desired lifestyle. Significant improvement in glycated hemoglobin levels was achieved
in six months using the app. A second clinical study was planned to confirm these
results and measure the medico-economic impact of the app on large-scale care
management. The usability intervention was decided by the in-house person in charge
of the regulatory affairs, because he realized that the validation phase for usability
needed to be performed by usability specialists.
As it suited the manufacturer that his app was a MD, the MEDDEV guiding the
qualification of standalone software [4] was interpreted in this way: the app (i) was a
computer program, (ii) performing actions on data different form storage, archival,
lossless compression, communication or simple search, (iii) its actions were for the
benefit of individual patients, and (iv) its actions were for monitoring and treatment.
The precise MD boundary raised different points of view. For the manufacturer, the
MD was only the standalone software enabling the patient to manage his insulin
treatment in his Smartphone. As the interface between the app and the telecare platform
was not yet completely finalized, the manufacturer preferred that the usability
evaluation focused on the app only. But for usability specialists, it was not acceptable
in terms of patient safety: the app cannot be used by the patient safely without the
telecare platform used by professionals to manage the monitoring. Professionals were
mandatory to help patients in their decisions to adapt or not their treatment regarding
the app suggestions. The software MD was also defined as the app integrated in the
Smartphone and linked to the telecare platform to support interactions between
professionals and patients.
S. Pelayo et al. / Software as a Medical Device: Regulatory Critical Issues 341
A standard user-centred approach was performed: some interviews with patients and
professionals, a literature review of incidents related to errors of use reported with this type
of apps, a usability inspection and usability tests. The results show that while the
manufacturer was convinced he had identified and documented all risks related to the
software MD, risks of wrong usage were obviously missing. Due to superficial
usability problems, the main risk for patients is that they register wrong data without
realizing. Another problem more related to a “loss of opportunity” is the feeling for
patients of not being followed by a real professional (because the interface with
professionals was not yet finalized). Several patients stressed the importance of not
feeling isolated but feeling “there is someone on the other side, a real person”.
5. Discussion - Conclusion
This study shows that cooperation with manufacturers within the regulation procedures
is possible, provided that (i) there is a benefit to the manufacturer and (ii) for “simple”
software MDs. The manufacturer here has a particular interest in following correctly all
the procedures to be included in the LPPR. His software MD is not a complex one. It
remains today a real challenge to test systems as they are implemented in highly
networked and interfaced systems. Concerning the qualification of software as MD, the
manufacturer is the one deciding regarding his interests. But it is also to market actors
to require for instance that software has a CE marking to be sure that it was developed
according to certain standard. One could imagine that hospitals, for instance, may buy
only software MD with CE marking to reduce the risk of compromised patient safety.
Another major challenge lies in maintaining innovation during the implementation of
regulations. We have to think about a potential set of recommendations for how
regulation could be implemented with minimal impact on innovation. Future
innovations should not be stifled due to poorly thought out safety regulations.
References
[1] Council directive 93/42/EEC, [cited Sep 8, 2012], Available from: http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=CONSLEG:1993L0042:20071011:en:PDF.
[2] International Electrotechnical Commission, Medical Device - Software Life Cycle (Rep. No. ISO
62304) (2006), International Electrotechnical Commission, Geneva, 2006.
[3] D. Luzi, F. Pecoraro, Medical Device Software: A new Challenge, Proceedings of the XXIV Conference
of the European Federation for Medical Informatics, 2012.
[4] J. Talmon, S. Pelayo, The medical device directive and software: the challenges for the HFE community.
Communication presented to the 5th International Symposium on Human Factors Engineering in
Health Informatics, 2011.
[5] European Commission, [cited Sep 8, 2012], Available from: http://ec.europa.eu/health/medical-
devices/files/meddev/2_1_6_ol_en.pdf.
[6] User Interface requirements for the presentation of health data. HB 306-2007. Standards Australia,
2007.
[7] National Health Services, Common User Interface, [cited Sep 8, 2012], Available
from: http://www.connectingforhealth.nhs.uk/systemsandservices/data/cui.
[8] C.A. Longhurst, M.L. Howard, Health information technology and patient safety: rigorous transparent
342 S. Pelayo et al. / Software as a Medical Device: Regulatory Critical Issues
evaluation of software is crucial, but regulation may stifle innovation, BMJ 344:e1096 (2012), 1-2.
[9] R. Koppel, D. Kreda, Health Care Information Technology Vendors’ “Hold Harmless” Clause:
implications for patients and clinicians, JAMA 301 (2009), 1276-1278.
[10] International Electrotechnical Commission, Medical Devices- Application of usability engineering to
medical devices (Rep. No. ISO 62366), International Electrotechnical Commission, Geneva, 2007.
Nursing Informatics
This page intentionally left blank
Enabling Health and Healthcare through ICT 345
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-345
Introduction
1
Corresponding author: Elizabeth Borycki, email: [email protected]
346 E.M. Borycki et al. / Developing National Level Informatics Competencies
There are few opportunities for nurses in their undergraduate studies to engage
with EMRs, EPRs and EHRs [1]. Many countries around the world: (1) have not fully
described the nursing informatics competencies that will be expected of nurses upon
graduating from their undergraduate nursing programs, (2) have not developed a clear
strategy for integrating these competencies into undergraduate nursing curriculums, and
(3) have not invested in developing tools representative of real-world clinical settings
that would help student nurses to understand the underlying theory and principles
embedded within HISs in order to be able to use these technologies [1]. As a result,
many student nurses are graduating without sufficient knowledge of nursing
informatics to be able to work effectively and efficiently in clinical practice settings
such as hospitals and clinics [1]. National governments, nursing educational
organizations and nursing informaticians around the world have recognized this is an
issue [2-4,6,8-11]. Two countries that are currently engaged in the process of
developing and integrating nursing informatics into undergraduate nursing programs
are Canada and Australia. In this paper we describe and compare the methods used to
develop these competencies, and the strengths and limitations of the approaches in
terms of work involving the science of developing national level competencies.
Australia Canada
Process of Development
1. Review of literature 1. Development of a Task Force and sub-task
2. Survey all Australian Nurses using an online forces.
survey. 2. Review of the literature.
3. Develop draft competencies. 3. Draft a list of national nursing informatics
4. All Australian nurses review a draft list of the competencies by the sub-task force.
competencies. Nurse informaticians validate 4. Presentation to the full task force for review
the competencies. and refinement.
5. National focus groups are conducted for further 5. Presentation to stakeholders (nursing
validation of feedback informatics experts and a diverse group of
6. Changes are made based on feedback and a interested nurses and students from across the
final draft is sent out to all Australian nurses country) for review and refinement.
and nurse informaticians 6. Presentation for review to Deans and Directors
7. Further changes were made. of the School’s of Nursing for review and
8. Final competencies are developed and validated survey completion.
by nurse informaticians. 7. Refinement by the sub-task force.
9. Competencies are submitted for publication and 8. Presentation to the full task force and
acceptance into the National Registered Nurses refinement.
Competencies Framework
Background Work
- Extensive literature review was conducted to Review of the literature that included:
ascertain existing competencies outlined in the -A review of the national and international literature
research grey and academic literatures.
- Draft evidence based competencies were -A review of nursing competencies at both the
formulated from the analysis of the literature in provincial and regulator levels.
consultation with the Project Advisory Committee,
key stake holders and expert nurse informaticians
- An on line questionnaire (34 questions) of all
competencies was constructed from the literature
review, and sent to all Australian nurses to identify
NI priorities for the profession.
- Draft competencies were developed from the
outcomes of the questionnaire
Type of Stakeholder Involvement
Project Advisory Committee A taskforce of nursing informatics experts was
- Oversaw all of project brought together to develop the methods and oversee
Key stake holders (Australian Nurses) and expert the process: The findings were:
nurse informaticians -presented at a stakeholder forum to 50 nursing
- were consulted about the competencies (from informatics experts and others
draft through to completion) -stakeholders reviewed and 20 competencies
- Focus groups were conducted with nurses (8 developed
questions) Draft competencies were also presented to Deans
and Directors of Schools of Nursing, stakeholders
and the Canadian Association of Schools of Nursing
committee. This group of stakeholders was asked to
complete an online questionnaire with the
competencies.
Validation Processes
Following draft competency list approval: Following draft competency list approval:
- Competencies were sent to all Australian - Competencies were presented at a stakeholder
Nurses and nurse informaticians for validation forum for 50 nursing informatics experts and
- Focus groups were conducted other nursing groups.
- Changes were made to the competencies from - Changes were made based on feedback
feedback - Competencies were presented to Deans and
- A final draft was sent out to all Australian Directors of Schools of Nursing, stakeholders
Nurses and nurse informaticians for final and the CASN committee. This group of
validation stakeholders was asked to complete an online
- Changes were made based on feedback [3] questionnaire with the competencies. [2]
348 E.M. Borycki et al. / Developing National Level Informatics Competencies
References
[1] Borycki EM, et al. The University of Victoria interdisciplinary electronic health record educational
portal. Stud Health Technol Inform, 2005;143:49-54.
[2] Nagle L, et al., Nursing informatics: Entry to practice competencies for registered nurses. Canadian
Association of Schools of Nursing: Ottawa. 2012.
[3] Kenny A. Online learning: enhancing nurse education? J Adv Nurs, 2002;38(2):127-135. DOI:
10.1046/j.1365-2648.2002.02156.x
[4] Goudreau J, Pepin J, Dubois S, Boyer L, Larue C, Legault A. A second generation of the competency-
based approach to nursing education. Int J Nurs Educ Scholarsh 2009;6:1-15.
E.M. Borycki et al. / Developing National Level Informatics Competencies 349
[5] Kushniruk AW, Borycki EM. Human, social and organizational aspects of health information systems.
IGI Global: Hershey, New York. 2008.
[6] Conrick M, et. al. A Framework for Nursing Informatics in Australia: A Strategic Paper.[Internet] 2004
Available from
http://www.hisa.org.au/system/files/u2615/Nursing%20Informatics%20in%20Australia_final%20Versio
n1%5B1%5D.pdf
[7] Garde S, Harrison D, Hovenga E. Skill needs for nurses in their role as health informatics professionals:
A survey in the context of global health informatics education. Int J Med Inform 2005;74(11-12):899-
907
[8] Coalition of National Nursing Organisations. Position statement: Nursing and E-health, [Internet] 2008.
Available from http://www.conno.org.au/pdf/CoNNO_Position_Statement_Ehealth.pdf
[9] Carter BE, Axford RL. Assessment of computer learning needs and priorities of registered nurses
practicing in hospitals. Comput Nurs 1993;11(3):122-126.
[10] Hegney D, Buikstra E, Eley R, Fallon T, Gilmore V, Soar J. Nurses and Information Technology: Final
Report. [Internet] 2007 Available from: http://www.anf.org.au/it_project/PDF/IT_Project.pdf
[11] Foster J. Development of Australian Nursing Informatics Competencies: Final Report, 2011
Unpublished.
[12] Staggers N, Gassert CA, Curran C. A Delphi study to determine informatics competencies for nurses at
four practice levels. Nurs Res 2002;51(6):383-97.
350 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-350
Introduction
1
[email protected]
K. Abrahamson et al. / What Nurses Are Talking About 351
1. Communities of Practice
The word community is used to mean a wide range of relationships based on such
categories as: geography (neighborhoods, villages, towns, cities), culture (ethnic groups,
religious groups), organizations (families, kinships, associations), identity (craft, trade),
and physical/biological similarities (scientific organisms) to name a few. One particular
type of community, connected through shared knowledge, is called a Community of
Practice (CoP). A Community of Practice, in a basic sense and for the purposes of this
study, can be thought of as a group of individuals who share a body of knowledge and
work together to solve a common problem. Organizational and rhetorical scholars have
argued that CoPs demand scholarly attention for their ability to increase effectiveness
and reduce organizational costs [7,8,9]. In addition, CoPs offer possibilities to save
lives (e.g., in high risk industries), and alter oppressive conditions by giving voice to
personal and collective narratives of its members. For example, CoPs have developed
in nursing and midwifery, firefighting, and the Armed Forces to serve as forums for
practitioners who share a concern for what they do, and wish to improve effectiveness
through regular interaction.
Although the theoretical parameters of CoPs are still being constructed [10],
Community of Practice is traditionally viewed as a combination of three fundamental
elements: domain, community, and practice [9]. Domain refers to the set of issues that
bring individuals together. These domains may be based on a profession (cardiologists,
teachers, nurses, etc.) or because members face similar problems in different contexts
(handling paperwork, workplace motivation, etc.). The domain represents the topic
with which CoP members are passionately interested. Community refers to the
individuals who participate in the CoPs and the social context they create. Membership
in CoPs is dependent on the idea of reciprocation; individuals trust that membership
will be beneficial and mutual respect encourages willingness to share, expose, and
provide empathetic feedback [9]. Practice is concerned with the exchange of
knowledge that improves effectiveness in whatever domain the community shares, and
creates a shared repertoire of tools, cases, and ideas. A defining characteristic of CoPs
is that, although they utilize a variety of formats, members may come from diverse
backgrounds a long as the interest in the domain is shared [9,10].
Tacit knowledge refers to knowledge that individuals have but cannot easily
communicate. Unlike explicit knowledge, which is formally communicated through
organizational processes such as training or orientation, tacit knowledge represents the
unspoken but important knowledge that does not get formally relayed, or participants’
intuitive knowledge of the domain [9,11]. CoPs have potential to give voice to
members of a community who may not otherwise be heard within formal
organizational structures, challenge oppressive ideologies through the expression of
narratives that tell the story of their practice, and construct new worldviews.
Nursing work has historically relied upon both explicit and tacit knowledge to
inform its practice [12]. Interestingly, the emergence of online asynchronous forums
may provide a visual text of such knowledge transfer between participants. We argue
that some online forums are an example of CoPs, and that these forums are worthy of
our scholarly attention. Online forums offer a unique space for nurse dialogue because
a) the internet provides a certain level of anonymity that may allow openness, b)
personal story-telling in these forums often encourages others to share narratives and
exchange tacit knowledge with an interest in the domain, and c) internet forums are
becoming more common and further exploration is needed to investigate the
352 K. Abrahamson et al. / What Nurses Are Talking About
communication occurring in this relatively novel setting. We explored the scope and
content of a nursing-focused community of practice, specifically a nursing
asynchronous online forum, to more fully understand how and why it is being used.
Results have potential to inform health care organizations regarding the domain and
practice issues that are relevant to CoPs participants.
2. Methods
Data was collected from the General Nursing Discussion forum found on the website
allnurse.com. This website was developed in 2003 as a resource for nurses, and
provides information about job opportunities, nursing education, financial aid, and
travel opportunities in addition to the asynchronous online forums addressing a number
of respondent-initiated topic streams. As such, this site represents the everyday
discourse of a community of practice in the domain of nursing. As a fairly new source
for mining texts, online forums require several a priori decisions to be made before
analysis. For this project, the particular site for study was chosen from among many
nursing forums for three reasons; (1) the forum is public and does not require
membership approval, (2) the forums are searchable, and (3) there is no warning to
researchers that the site is off limits. Once the site was chosen, on-line forums were
searched. We categorized respondent initiated threads within the online forum in terms
of content and characteristics. The study was designed specifically to be exploratory in
nature, and thus an open-coding method was employed to discover the topics addressed
by participants. Categories were first developed by the first author, then reviewed,
discussed, and refined through collaboration between authors. Each threaded post was
assigned to a single category deemed to best fit the overall content of the thread. To set
realistic parameters for this study, one month of thread topics (May of 2012) were
examined. Thus, a limitation of this work is the potential emphasis on issues that
occurred temporally close to the time of data collection. Future research would benefit
from a more time representative design.
3. Findings
A total of 294 discussion threads were initiated during the time period of May 1-May
31, 2012. The most frequently noted category of discussion was advice regarding
career planning within the domain of nursing. Examples of threads in this category
include questions regarding employment (Should I send my resume if I haven’t passed
the NCLEX yet?), pay expectations (Can I live this lifestyle as a nurse?), interview tips
(Should I speak of another offer during an interview?), and advice regarding career
strategy (How did you know your specialty and when?). The second most frequent
category included posts seeking answers to specific clinical questions or concerns.
Interestingly, many of these questions requested information that is readily available in
policy manuals or nursing reference materials (How do I flush a PICC line?), or cited
actions taken in the clinical setting then asked, essentially, ‘did I do it right?’ Nearly all
threads remained tightly linked to nursing-specific topics. The emergence of
discussions regarding unjust treatment (21%), job-related emotional difficulties (4%),
and sharing stories (4%) highlight the potential for internet forums to allow community
narratives to develop. Additionally, practice tools in terms of technical advice and
K. Abrahamson et al. / What Nurses Are Talking About 353
resource sharing accounted for approximately 25% of posts, reflecting the potential of
internet forums to allow for exchange of ideas applicable to practice change. Categories
of forum thread content are noted in Table 1.
Because privacy concerns were an emergent theme in the existing literature
addressing medical professional use of online forums [6], we also examined threads for
information that would identify patients, identify a specific facility, describe a patient
situation with enough detail that the patient could potentially identify themselves in the
thread, and/or endorse specific products. Of the 294 posts none named or otherwise
identified a patient, and only 4 posts named a facility (3 were in reference to potential
employment or an employment interview). Specific patient situations (without
identifying information) were highlighted in 11 posts; these 11 posts were contained
within the categories of dissatisfaction with nursing, concerns/commentary regarding
poor or dangerous care, technical questions, sharing stories, and ethic concerns. No
thread overtly endorsed a product.
4. Discussion
The goal of this study was to analyze the content of a nursing focused internet forum to
1) better understand the topics and content of nursing discussions that occurred outside
of the nursing workplace but within the nursing community and 2) discuss the potential
of internet forums to constitute a Community of Practice (CoPs) among nurses. We
discovered that within a single nursing focused internet forum the majority of posts
dealt directly with the domain of nursing, nurses were willing to share information that
included emotion-based content, and nearly half of the posts were contained within the
two most frequent categories (advice regarding career planning and technical
questions). Additionally, apart from the two most dominant categories, thread content
varied considerably over a wide range of categories, indicating that nurses were
engaging the forum membership for both professional interaction and information
needs. Surprisingly, nurses frequently used the forum to ask information regarding the
safety or procedural steps of clinical tasks such as IV line flushing and medication
administration, information that should be readily available on a nursing unit or in an
established nursing reference. The finding that nurses are seeking such information
from anonymous online sources (there is no way for members to verify the identity,
experience, clinical capability or qualifications of fellow members) is of concern, and
may reflect poorly on the safety culture of health care institutions. Despite recent
efforts to encourage nurses to openly question procedures, admit when further
information is needed, and discuss situations where patient safety may have been
comprised within their organizational settings, some nurses are instead seeking this
type of support from internet forums.
The nature of a community of practice (CoPs) is one that encourages individuals to
share their experiences in order to solve common problems. Some threads appeared in
the form of questions, wherein nurses seek advice from other community members,
while other threads were declarative in nature and made assertions regarding personal
experiences and professional knowledge. The CoP framework draws our attention to
knowledge “as communicatively constituted in practice” [10,p.193]. We recognize that
not all nurses who participate in online nursing forums represent CoPs, but believe
there is valuable information about how knowledge, especially tacit knowledge, is
rescued and shared through this type of interaction. As theorists continue to examine
354 K. Abrahamson et al. / What Nurses Are Talking About
CoPs, they are realizing that the “enactment of CoPs can differ greatly as regards
mutual engagement, shared repertoire, and negotiation of a joint enterprise” [10 ,p.
195]. If the benefits of CoPs are to be realized, we must also take the changing nature
of community into account in order to include the explicit and tacit knowledge shared
in online communities.
Table 1. Categories of Forum Thread Content
Category n %
Advice re: career planning (hiring, pay, interview, job choice, career 75 26%
strategy)
Technical questions, clinical questions 51 17%
Advice re: formal education (course of study, coursework, schools) 24 8%
Perception of unjust treatment by administration or leadership 21 7%
Issues related to scheduling or shift work 14 5%
Handling job-related emotional difficulties 13 4%
Sharing stories (telling a story and/or asking others for their stories) 13 4%
Issues specific to nursing management 12 4%
Professional development or training/related readings/ instructional videos 11 4%
Licensure issues 9 3%
Interpersonal conflict with co-worker or colleague 9 3%
Dissatisfaction with nursing career in general 8 3%
General discussions of issues affecting nursing care 5 2%
General discussions of issues affecting personal health 5 2%
Discussion comparing practice settings without requesting career advice 4 1%
Concerns/commentary of poor or dangerous care 4 1%
Discussion regarding of public image of nursing 4 1%
Comments on forum use or the forum itself 4 1%
Questioning the ethics of a specific situation 3 1%
Using forum to gain employment or recruit nurses 3 1%
Event announcements 3 1%
Missing/unable to categorize 3 1%
Interpersonal conflict with patient 2 <1%
Using forum to recruit nurses for research studies 1 <1%
Asking where to find a specific resource 1 <1%
References
Introduction
There have been several efforts over the years to establish nursing informatics
competencies for nurses, beginning with Staggers and colleagues work in 2001 and
2002 [1,2]. This work was followed in the U.S. more recently by the TIGER initiative
(Technology Informatics Guiding Educational Reform) [3-5] and work in Canada
supported by Canada Health Infoway and the Canadian Association of Schools of
Nursing [6]. The resulting competency documents provide statements describing the
knowledge, skills and attitudes needed for modern nursing practice and suggest that
nurse educators must incorporate nursing informatics content into their curricula. In
addition, the national accrediting bodies for nursing education in both countries suggest
that graduating students should be prepared with basic nursing informatics
competencies (6,7). Although these efforts indicate agreement that informatics content
is required in undergraduate curricula, faculty has little guidance on teaching such
1
Corresponding Author
N. Frisch and E. Borycki / A Framework for Leveling Informatics Content Across Four Years 357
content. For example, the TIGER initiative provides broad goal statements such as a
beginning nurse should “access data and perform nursing documentation using
computerized patient records” and should “recognize the role of informatics in nursing”
(8, p 58] but does not provide concrete examples about how or where to bring a wide
range of nursing informatics competencies into undergraduate nursing curricula that are
already challenged to cover a wide range of required content. In addition, prior work
on leveling nursing informatics content has addressed competencies and outcomes
across levels of nursing education (i.e. the undergraduate, masters and doctoral
level)[1], not leveling content within undergraduate programs. The current paper
presents a contribution to the literature as it describes how nursing informatics
competencies can be integrated across BSN curricula.
The purpose of this paper is to present a useable framework, developed by the
authors, suggesting ways that faculty can incorporate much of the needed material
into content areas already being addressed in virtually all nursing programs within the
four years of a baccalaureate nursing program. In addition, the authors present a
practice appraisal tool that faculty and students may use to track exposure and
achievement of nursing informatics competencies over the four years of a program.
In baccalaureate programs, nursing curricula are designed to move the learner through
course content and clinical experiences that ensure the graduate is ready for beginning
professional practice. This means that in a four-year university program, the learner
begins as a novice nurse (i.e., one without nursing knowledge and experience) and ends
as a generalist ready for practice in any of nursing specialty areas. Nursing curricula
have been widely influenced by the work of Benner [9] who has documented the
development of nurses throughout their careers in her publication “From Novice to
Expert”. Though there are program and regional variations, nursing curricula take an
approach of thinking about student learning and development over the four-year
experience. Nursing curricula begin with content and experiences that are more
elementary or foundational to the discipline and move the learner through experiences
that build on that foundation to increasing levels of complexity. Thus, it is relatively
common that students provide care first to individual adults and care for families and
communities later; they build care management skills by taking responsibility for one
patient, moving to take responsibility for a group of patients, and only then take on unit
management/leadership experiences.
In order to incorporate nursing informatics awareness and competencies into
nursing curricula, the authors have selected four content areas that are addressed in one
way or another in all programs. These are: 1) professional responsibility, 2) care
delivery, 3) community nursing and care of populations, and 4)
leadership/management. These content areas are present in a BSN curriculum as
illustrated in Figure 1 as four squares. Nursing informatics represented by the arrows
moving from the centre out, depicting the integration of such competencies from
elementary to advanced.
358 N. Frisch and E. Borycki / A Framework for Leveling Informatics Content Across Four Years
Professional
Care Delivery
Responsibility
Community/
Leadership
Population- and
based Management
Nursing
Students provide basic care in beginning nursing courses and most often learn clinical
skills in a nursing laboratory before carrying out care in actual clinical situations. There
are two areas of care delivery that require the learner to understand care in a
technology-enabled practice setting: 1) documentation of nursing care and 2)
medication administration.
In learning care documentation, the beginning nurse needs an understanding of the
legal status of nursing notes, signatures (paper or electronic) that indicate personal
knowledge/observation of the material for which the signature is given, and -- in digital
records -- the differences between narrative language and use of standardized
languages. Introduction to nursing and other standardized languages (such as the
ICNP®, NANDA-I, C-HOBIC, ICD-10, DSM-IV-R) is essential content. The
beginning nurse need not be an expert in all of these language systems, but needs to
understand their development and use in recording in electronic health records systems.
Providing a foundation early in a nursing curriculum with reinforcement of knowledge
throughout the program gives students an opportunity to grow in their sophistication of
use, critique, and, ultimately, improvement of digital documentation systems.
Medication administration cannot be taught without attention to paper systems,
hybrid systems and electronic order entry systems. Nursing laboratories that simulate
all three are best suited to teach the learner how to practice in current and future
systems. Paper systems provide the basics for the learner to understand how a
medication order is processed from initial writing to administration and recording.
Electronic systems involving physician order entry, pharmacist and nurse receipt,
pharmacy dispensing, and nurse administration are complicated and, when explained,
help the student understand care processes and the nursing and interdisciplinary roles in
safe practice. Decision supports related to medication dose and interactions between
medications provide opportunity for learning related to system benefits, possible errors,
need for professional judgment and vigilance in professional behaviours.
Addressing issues such as nursing documentation, standardized languages,
decision supports, accountability for professional decisions and actions, and
understanding of systems processes address the TIGER competency area of “clinical
information management” and its related skills of maintain records, capturing data,
360 N. Frisch and E. Borycki / A Framework for Leveling Informatics Content Across Four Years
producing summary records of care and managing medication orders and care [10]and
the CASN informatics competencies [6] under the domain of “information and
knowledge management” with indicators related to use of nursing data using
standardized languages, recording nursing data, use of hybrid and homogenous record
systems, and under the domain of “information and communication technologies” with
indicators related to understanding various components of health information systems.
Table 2 provides examples of performance appraisal items of these competencies.
Community nursing and care of populations most frequently comes later in a BSN
curriculum – often in years three or four. The student must build on knowledge and
practice experience with care of individuals to care of client groups to expand practice
to the community/population level. Areas that the student must learn have to do with
retrieval and interpretation of population-based data to determine needs, risks and
surveillance activities as well as need to consider basic privacy/confidentiality issues in
terms of transferal of data over wireless networks (from client homes to clinical record
systems), protection of client/patient data when data are held on hand-held laptops or
portable devices such as computers used in the home. The student will need to become
acquainted with population datasets, interpretation of population-based statistics and
new considerations for privacy and confidentiality. In addition, when providing care to
clients in their homes, the student needs to gain an awareness of how clients obtain
access to their personal health data and make use of personal health records. When this
new content is delivered through classroom and clinical experiences, the faculty are
addressing the TIGER competency of “management of patient groups or populations”
and related skills of use of information to support epidemiological investigations and
population health issues, and the CASN informatics competencies [6] under the
domain of “information and communication technologies” with indicators related to
care delivery to diverse populations, assisting patients and their families to use
information and technologies to manage their own health, and critical evaluation of
data from a variety of sources. Table 3 provides examples of performance appraisal
items of these competencies.
N. Frisch and E. Borycki / A Framework for Leveling Informatics Content Across Four Years 361
Courses in leadership and management come near the end of BSN curricula. Students
need to understand approaches to leadership and to think about their own personal
leadership styles. While a new graduate is rarely hired directly into a management
position, BSN graduates will be in positions of leading teams and directing and
supervising care soon after graduation. Further, many will move into supervisory
positions without returning to school for additional formal education. There are two
important informatics content areas that directly impact the nurses’ ability to serve in
supervisory and management positions: 1) access to data for assessments of quality and
safety, and 2) evaluation of technology and its impact on workflow. Quality and
safety have become essential parameters for evaluation of institutional performance.
While each healthcare system will have related data identified and stored in varying
ways, the BSN graduate nurse must have enough background to understand data
collection, retrieval and interpretation for these purposes. Use of aggregate data to
identify trends, ‘change points’, and progress toward institutional goals can be taught at
the undergraduate level. Again the BSN graduate will not be the expert, but must have
a basic understanding of the processes used to ensure quality and safety. Also, a nurse
supervisor needs an understanding of the well-known fact that technology impacts
workflow and behaviours [11]. Adding hand-held devices, personal paging systems,
bar code identification, or new documentation systems will impact (positively or
negatively) on the work of the nursing team. To fully participate in technology roll-
outs, a supervisor must be in a position to ask questions related to workflow, time
management and the results of pilot-tests of new technology. Courses can include this
content with case-studies from practice experiences to help the graduate learn. Use of
data for quality and safety and an understanding of technological impacts on workflow
address TIGER competency of “clinical information management” and related skills
such as data capture, information management in relation to clinical workflow tasks
[10] and the CASN informatics competencies [6] under the domain of “professional
and regulatory accountability” with indicators related to advocacy for use of
information systems to support safe, quality care; and under the domain of
“information and communication technologies” with indicators related to
understanding the ability of technology to improve health systems and quality care.
Table 4 provides examples of performance appraisal indicators.
Practice Appraisals
Documenting student learning and tracking student learning outcomes are integral tasks
for professional nursing programs. Practice appraisal forms provide both faculty and
students the ability to document exposure to content and learning of content across the
entire four years of a program. Such forms can be used as part of end of term student
evaluations, incorporated into student portfolios of individual learning, and as measures
contributing to overall program evaluation. The authors have developed a nursing
informatics practice appraisal tool (see Appendix A) that is a form that could be used
for each of these purposes. This tool, titled the University of Victoria Informatics
Practice Appraisal - BSN is organized according to the four content areas addressed
above and provides indicators of student learning. The authors present this tool for
faculty or students who may wish to incorporate it into their own programs.
Discussion
Nursing faculty is often faced with the challenge of being asked to add content to their
courses and curricula without being able to omit content already present. This need to
add new and changing content is even more difficult when faculty themselves are not
expert in the new content. Existing frameworks for leveling nursing informatics
competencies provide information about how to integrate these technology-based
competencies across differing levels of nursing education (i.e. the undergraduate,
masters and graduate). In this work we outline a framework that can support faculty
decision making when integrating nursing informatics competencies across a four year
undergraduate baccalaureate in nursing program. The framework we outline illustrates
a place in curricula where content and learning experiences can be incorporated into
existing teaching plans and activities and provides a way forward for faculty addressing
the informatics content required for practice in our future. We also present a practice
appraisal tool for aiding faculty in their process of documenting and tracking student
learning. Our future work will include testing the framework to determine whether it
can be used to support faculty decision making in integrating such context in an
undergraduate program context. In carrying out this work (i.e. integrating nursing
informatics content into four year baccalaureate programs), faculty will be able to
demonstrate that their curricula meet current published standards for informatics in
BSN curricula and will be able to retrieve student learning of this content in practice
appraisals.
References
[1] Staggers N, Gassert CA, Curran C. Informatics competencies for nursing at four levels of practice. J of
Nurs Education, 2001;40(7);303-316.
[2] Staggers N, Gassert CA, Curran C. A Delphi study to determine informatics competences for nurses at
four levels of practice, Nursing Research, 2002;51(6); 383-97.
[3] Skiba DJ, Dulong D. Using the TIGER vision to move your agenda forward. Nurs Management,
2008;2,14-16.
[4] Sensmeier J. 2006 TIGER Summit – Evidence and informatics transforming nursing. CIN, 2007;1;55-
56.
[5] Dulong D. Informatics: the TIGER project. Online J of Issues in Nurs, 2009;13(2);(2p).
N. Frisch and E. Borycki / A Framework for Leveling Informatics Content Across Four Years 363
Appendix A
______________________________________________________________________
Introduction
In 1999, the Institute of Medicine (IOM) released a report, To Err is Human, which
predominantly focused on health care errors and revealed that between 44,000 and
98,000 Americans die from medical errors each year [1]. The report stressed the
importance of using reporting systems to collect standardized information about
adverse events for the purpose of improving patient safety. The report also
recommended the use of voluntary reporting systems for systemic failures and near-
misses to identify potential problems before harm occurs [1].
Almost 12 years later, in 2012, the Institute of Medicine produced a report on the
impact of health information technology on patient safety [2]. The IOM reported that
health information technology products are expected to improve patient safety only if
the products are well-designed and strategically implemented. Incidences in which
poorly designed health information systems caused medical errors were a concern,
particularly due to the impact on patient safety. The report encourages collaboration
between health information technology vendors, physicians, nurses, and hospitals to
improve patient safety.
According to IOM reports, Computerized Physician Order Entry (CPOE) systems
were identified as a means of improving patient safety by reducing hospital medication
errors. CPOE systems are broadly considered an essential component for health
information systems that helps improve patient safety by reducing medical errors.
CPOE provides passive and active reminders, alerts and alarms that notify medical
1
Corresponding Author
368 M. Househ et al. / Patient Safety Perspectives: The Impact of CPOE on Nursing Workflow
staff (such as nurses) of incorrect orders, and information about possible drug-drug
interactions and missed medication doses that allow medical staff to correct, modify
and rewrite incorrect entries[3]. A large number of serious medication errors occur
every year and include prescribing the wrong drug, drug over-dosage or unobserved
drug interaction and allergy. CPOE offers accurate drug information, dose auto-
calculations and support for making the appropriate decision at the point of care.
CPOE application helps decrease medication errors in multiple ways. For example,
when a nurse inputs a physician’s order into a CPOE system and there is a mistake on
the order, the CPOE system will give him/her a warning alarm. As a result, the nurse is
given the opportunity to review the prescribed medication with the physician and to
correct and re-enter the order.
A study conducted by Bates et al. showed that CPOE application reduced the
medication error rate by 55% [4]. A systematic review of 10 studies was performed:
five of the studies indicated that the use of CPOE significantly decreased the number
of medication errors, four of the studies concluded that the resulting decrease in
medication errors was not significant and one of the studies showed no change in the
number of medication errors with the use of CPOE [5]. Some studies have shown that
the application of the CPOE system has a negative impact on medication errors. A
study conducted by Koppel et al. reported that the CPOE system facilitates medication
errors and identified 22 types of medication errors that can result from CPOE
application [6].
The use of CPOE in hospitals is facilitating many changes to the processes of
prescribing, dispensing and administering medications. These changes are becoming
more apparent in the tasks of nursing staff, as more physicians are requesting nurses to
enter orders into CPOE systems [7]. In 2009, Fields et al. reported that the total impact
of CPOE on nursing can be determined by evaluating three aspects of nursing: process,
relationships and operations. Moreover, nurses’ concerns and difficulties with using the
CPOE system may potentially affect nursing workflow [8].
Little work has determined nursing perspectives on the use of CPOE and its
potential impact on nursing workflow and patient safety. In this paper, we discuss the
importance of the patient safety culture within the nursing profession, the nursing
perception of CPOE, and the impact of CPOE on nursing workflow.
Patient safety is defined by the IOM as "the prevention of harm to patients"[9]. There
exists much concern regarding the healthcare system’s ability to prevent errors and to
facilitate learning from errors, which requires the involvement of the hospital
administrator and medical staff [10]. Improving quality within hospitals has long been
a goal of medical professionals, especially nursing professionals. For example, in 1855,
Florence Nightingale played a major role in creating a patient safety culture when she
gathered and analyzed the mortality data of British troops and then promoted hygienic
practice [11]. In 1859, she was credited with initiating the implementation of the
world’s first hospital performance measures. Today, nurses play a major role in
improving patient safety within hospitals. The impact of nurses on patient safety is
greater than the impact of medication administration or injury prevention in health care
settings .
M. Househ et al. / Patient Safety Perspectives: The Impact of CPOE on Nursing Workflow 369
One of the benefits of using CPOE in nursing practice is that CPOE supports safe drug
administration by promoting “the five rights of drug administration”; this can provide
nurses more time to focus on the professional steps of medication administration, such
as the monitoring of adverse events and the continuous assessment of patients [14].
However, nurses have resisted the introduction of information technology into hospitals.
In 2003, Trimmons reported on a study that evaluated nurse resistance to the use of
information technology [15]. The study revealed that nurses found that the use of
information technology increased their dependence on technology and therefore
undermined their critical thinking skills. The nurses in the study were also concerned
with the reliability and security of health information systems. Additionally, nurses
were concerned that they would be unable to access patient information during a
system failure.
However, Ash et al. reported that nurses were comfortable in using CPOE systems
that were well-designed. Nurses reported that easy-to-use systems improved their
workflow and that badly designed systems actually hindered workflow, especially
when they were not customized or implemented accordingly. The authors also reported
that one major success of the project resulted from strong support and leadership from
management [16].
The ways in which nurses interact with CPOE are also changing. CPOE systems
were originally designed for physicians to enter medication and lab orders. However, a
study conducted by Kazemi et al. (2010) showed that most of the CPOE systems
implemented in hospitals are not used by physicians, but rather by nurses and other
medical staff [7]. Physicians were found to be resistant to using CPOE systems and
relied on nurses and other medical staff to input CPOE orders.
The studies conducted by Trimmons, Kazemi and Ash indicate that physicians are
relying on nurses to enter CPOE orders and that, as long as the technology helps
improve patient care and is designed to support workflow, nurses are more willing to
use the technology.
The full impact of CPOE on nursing tasks and patient safety has not been examined
and, therefore, remains poorly understood. Small studies are available on this subject,
but as more health information systems are implemented and used by nurses, the need
to understand the impact of these systems on nursing workflow and patient safety is
increasing. Ash et al. reported that nurses and other clinicians were willing to use
CPOE systems if they were well-designed and implemented [14]. The study noted that
CPOE implementation had a major impact on nursing, but it was unclear whether
CPOE systems would decrease or increase the duration, number or type of nursing
tasks, such as order processing and other patient care duties.
Another study, conducted by Bleich et al., reported that the implementation of
CPOE increased the time nurses spent performing tasks on the system while reducing
time spent performing patient care activities [16]. The authors recommended achieving
a greater and more thorough understanding of the CPOE system and the effects on
CPOE users, including nurses who interact with the system daily to retrieve and
manage patient orders and improve patient care and safety.
370 M. Househ et al. / Patient Safety Perspectives: The Impact of CPOE on Nursing Workflow
When a CPOE system is implemented and physicians enter orders from other
places within the hospital, nurses become dissatisfied with CPOE. Nurses must log into
the system many times to check for new orders, which increases the time spent using
the system rather than visiting bedsides. Therefore, this additional clerical task is
imposed on already-burdened professionals and is also economically inefficient [15]. A
study examining nursing time expenditure determined that nurses spend approximately
20% of their time on direct patient care, 9% on activities related to the unit, 25% on
indirect patient care, 30% on documentation, and 15% on personal activities [17]. In
fact, there exists a difference in the amount of time nurses spent on documentation
before and after using the CPOE system, indicating that more time will be available for
nurses to spend on direct patient care and improving the quality of work documentation
[18].
Successful implementation and adoption of CPOE are more likely when the
developers and implementers understand the complexities and unpredictability of the
nurses' workflow. For instance, an inability to ensure the compatibility of the system
with nursing workflow may lead to increased encounters of unintended consequences.
Clearer guidance is required from hospitals on the use of CPOE systems by physicians,
nurses, and other medical staff. Without clear policies, well-developed CPOE systems
that are well-implemented and -designed may hinder nursing workflow and impact
patient safety and patient care.
CPOE with Computer Decision Support Systems (CDSS) can help health care
providers achieve optimum care for their patients. Raebel et al. conducted a study to
assess CPOE abilities in alerting prescribers of contraindicated medications during
pregnancy. The trial included more than 11,000 women who were enrolled in urban
Kaiser Permanente Centers. The study findings were statistically significant and
indicated that CPOE with CDSS reduced potential medication errors [19].
Although CPOE systems can improve patient care, they can negatively impact
organizational culture, workflow processes, and medical errors due to poor design. In
2006, Campbell et al. identified unintended consequences that resulted from CPOE
implementation. The study made use of an expert panel by using an iterative process in
which the list of CPOE adverse consequences was divided into categories. This study
revealed that CPOE can generate new types of errors, which include, but are not limited
to, organizational culture, workflow process, and order errors that occur when users
accidentally select an item that appears beside the intended choice. This is a result of
poor design [20].
As an increasing number of nurses enter physician orders into CPOE and
administer such orders, more work is required to understand the impact of CPOE on
nursing workflow and, in particular, the impact of these changes on patient safety and
patient care. Kazemi et al. suggest that the involvement of nurses in entering data into
CPOE serves as a form of collaborative effort between physicians and nurses that may
improve patient safety and patient care [7]. This further highlights the need to study the
impact of CPOE on nurse workflow and the direct impact on patient safety.
M. Househ et al. / Patient Safety Perspectives: The Impact of CPOE on Nursing Workflow 371
References
[1] Institute of Medicine. To Err is Human. Building A Safer Health System. November 1999. Web Access
[http://www.iom.edu/~/media/Files/Report%20Files/1999/To-Err-is-
Human/To%20Err%20is%20Human%201999%20%20report%20brief.pdf]
[2] Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. November
2011. Web Access [http://www.iom.edu/Reports/2011/Health-IT-and-Patient-Safety-Building-Safer-
Systems-for-Better-Care.aspx]
[3] Ash, J. S., Sittig, D. F., Dykstra, R., Wright, A., Mcmullen, C., & Richardson, J. (2010). Identifying Best
Practices for Clinical Decision Support and Knowledge Management in the Field, 806-810.
doi:10.3233/978-1-60750-588-4-806
[4] Bates et al (2001). Reducing the Frequency of Errors in Medicine Using Information Technology, in
Journal of the American Medical Informatics Association Volume 8 Number 4 Jul / Aug 2001
[5] Jesse I. Wolfstadt I, Jerry H. Gurwitz. The Effect of Computerized Physician Order Entry with Clinical
Decision Support on the Rates of Adverse Drug Events: A Systematic Review. University of
Toronto.Canada.2008
[6] Koppel R, Metley JP, Cohen A, Abaluck B, Localia AR, Dimmel SE, Strom BL. Role of computerized
physician order entry systems in facilitating medication errors. JAMA. 2005;293, 1197-1203
[7] Kazemi A, Fors UGH, Tofighi S, Tessma M, Ellenius J Physician Order Entry Or Nurse Order Entry?
Comparison of Two Implementation Strategies for a Computerized Order Entry System Aimed at
Reducing Dosing Medication Errors. JMedInternetRes2010;12(1):e5
[8] Khajouei,R., P. C.Wierenga, et al.(2011)." clinicians satisfaction with cpoe ease of use and effect on
clinicians workflow, efficiency and medication safety."Int J Med Inform Oh H, Rizo C, Enkin M, Jadad
A (2005). What is eHealth (3): A systematic review of published definitions. JMIR, 24;7(1):e1
[9] Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report
of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC:
National Academy Press; 2000.
[10] Maddox P, Wakefield M, Bull J. Patient safety and the need for professional and educational change.
Nurs Outlook, 2001;49(1):8-13.
[11]Poissant L, Pereira J, Tamblyn R, et al. The impact of electronic health records on time efficiency of
physicians and nurses: a systematic review. J Am Med Inform Assoc 2005;12:505-16.
[12] Smith J, Crawford L. The link between perceived adequacy of preparation to practice, nursing error,
and perceived difficulty of entry-level practice. JONAS Healthc Law Ethics Regul 2003;5(4):100-3
[13] Keohane, C., Bane, A., Featherstone, E., Hayes, J., Woolf, S., Hurley, A., Bates, D., Gandhi, T., & Poon,
E. (2008). Quantifying Nursing workflow in medication administration. Journal of Nursing
administration, 38(1), p. 19-26.
[14] qtd in. Teasdale C. 2008. Nursing Perceptions of a Computerized Physician Order Entry System.
Northern Kentucky University. Web Access:
[http://books.google.com.sa/books?id=FgrRNzl65AIC&pg=PA19&lpg=PA19&dq=trimmons+nursing&
source=bl&ots=Xi1dXSWdr0&sig=kqnfvCasIxKGcHFGOER1wgliEIM&hl=en&redir_esc=y#v=onepa
ge&q=trimmons%20nursing&f=false]
[15] Ash, J., Gorman, P., Seshadri, V., & Hersh, W. (2004). Computerized physician order entry in U.S.
hospitals: Results of a 2002 survey. Journal of American Medical Informatics Association, 11(2), 95-99.
[16] Bleich, M. R., Hewlett, P. O., Santos, S. R., Rice, R. B., Cox, K. S., & Richmeier, S.(2003). Analysis of
the nursing workforce crisis: A call to action. American Journal of Nursing, 103(4), 66-74.
[17] Lee TT, Mills ME, Lu MH. Impact of a nursing information system on practice patterns in Taiwan
Comput Inform Nurs. 2008 Jul-Aug;26(4):207-14.
[18] Mohamed N., and Gabr H., (2010): Quality improvement techniques to control medication errors in
surgical intensive care units at emergency hospital. Journal of Medicine and Biomedical Sciences,(5): 24
-35.
[19] Raebel MA, Carrlo NM, Kelleher JA, Chester EA, Berga S, Magid DJ. Randomized trial to improve
prescribing safety during pregnancy. JAMIA.
[20] Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related
to computerized provider order entry. JAMIA. 2006;13, 547-556.
This page intentionally left blank
Healthcare Modeling and Simulation
This page intentionally left blank
Enabling Health and Healthcare through ICT 375
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-375
Introduction
Despite increased adoption of the electronic medical record (EMR) in primary health
care reported in recent years, its impact on provider performance and patient outcomes
has been mixed [1,2]. For instance, a systematic review by Lau et al. [3] of 43 EMR
evaluation studies has shown only 51.2% had positive impact, while 30.2% had no
effect. Given the recent attention to transform primary health care in Canada and the
great expectations on EMR as the enabler [4], the lack of positive evidence to date is
cause for concern. That said there are successful primary health care practices that have
shown improved care thru innovative use of their EMR. Examples are the use of
individualized decision support and reminder tools by Holbrook et al [5] to improve
diabetes care in a Canadian community, and an EMR supported quality improvement
project by Nemeth et al [6] in 99 primary care practices in the United States. Both
studies had multiple interventions in which EMR was one component. In this paper we
present a multivariate EMR integration model that is used to define, quantify and
explain factors that may affect the successful integration of EMR systems to improve
1
Corresponding Author;: Francis Lau, School of Health Information Science, University of Victoria,
P.O. Box 3050 STN CSC, Victoria BC, Canada V8W3P5
376 F. Lau et al. / Developing a Multivariate EMR Integration Model for Primary Health Care
1. Purpose
2. Methods
3. Results
First we define EMR adoption in terms of an adoption score based on the EMR
adoption framework. Second we define EMR integration in terms of the EMR adoption
score and the contextual factors that influence how well the EMR can be integrated.
Third we define PHC quality index in terms of process and outcome quality indicators,
which are derived from the PHC indicators from CIHI [9] and the Summary of QUality
InDex (SQUID) algorithm by Nietert et al [10]. Fourth we model the relationship
between EMR integration and PHC quality index. These definitions are as follows:
F. Lau et al. / Developing a Multivariate EMR Integration Model for Primary Health Care 377
For each of the parameters in the model we searched for relevant literature to find out
how others have defined them. We have included three such parameters to illustrate the
source of evidence used. For example, our Practice organization definition is based on
five published Canadian PHC organizational performance studies [4,10-13]. Our
Practice size definition is based on five published studies from Canada, United States
and Australia [1,14,16,17]. Table 2 shows the suggested PHC indicators for our model,
which are drawn from PHC content standard development work by CIHI [9].
For testing, we need to collect data for the parameters identified in the model. We then
conduct statistical analysis to test the model. These steps are described below:
Data Collection – We need to collect data for the parameters from multiple sources
through environment scan, survey and data extraction methods. The variables,
sources and tools are listed below. The tools are available via the UVic eHealth
Observatory website (http://ehealth.uvic.ca/resources/tools/tools.php).
a) EMR adoption score – from care providers and support staff through face-to- face
interviews using the EMR adoption survey tool
b) EMR products and configurations – from EMR vendor, jurisdictional and Infoway
websites and published reports using the environment scan tool
c) User prior EMR knowledge and user satisfaction – from care providers and support
staff through face-to-face interviews using the user assessment tool
378 F. Lau et al. / Developing a Multivariate EMR Integration Model for Primary Health Care
ܲ ݔ݁݀݊݅ݕݐ݈݅ܽݑݍܥܪൌ
݁ݎܿݏ݊݅ݐ݀ܣ ሺ ݐܿݑ݀ݎܴܯܧ ݊݅ݐܽݎݑ݂ܴ݃݅݊ܿܯܧ
ݕݐ݈݅ܽݑݍܽݐܴܽ݀ܯܧ ܶ݅݉݁ ݊݅ݐܽݐ݈݊݁݉݁݉݅݁ܿ݊݅ݏ ܲ ݁ݕݐݎ݁݀݅ݒݎ
ܷ ݈݁݃݀݁ݓܴ݊݇ܯܧݎ݅ݎݎ݁ݏ ܷ ݊݅ݐ݂ܿܽݏ݅ݐܽݏݎ݁ݏ ܲ ݊݅ݐܽݖ݅݊ܽ݃ݎ݁ܿ݅ݐܿܽݎ
ܲ ݁ݖ݅ݏ݁ܿ݅ݐܿܽݎ ܲ ݐ݊݁݉݁ݒݎ݉݅݁ܿ݅ݐܿܽݎ ݁ݒ݅ݐ݈݊݁ܿ݊݅ܽ݅ܿ݊ܽ݊݅ܨሻ (5)
c) Prediction – Use the multivariable model from (b) to do prediction
4. Next Steps
HIMSS Analytics has recently released an US Ambulatory EMR Adoption Model that
measures adoption stages but it is focused on EMR functionality without any linkage to
contexts [18]. Conversely, two recent PHC evaluation studies from Ontario and Quebec
have identified many factors that influenced PHC performance but neither examined
their relationship with EMR adoption in detail [19,20].
Table 2. Example PHC indicators from CIHI.
Legends: PHC# indicator reference number; + reviewed by author.[MP] as feasible to extract from EMR, *not part of
prioritized CIHI list but reviewed as feasible
References
[1] C. Schoen, R. Osborn, M.M. Doty, D. Squires, J. Peugh, S. Applebaum. A survey of primary care
physicians in eleven countries, 2009: perspectives on care, costs and experiences. Health Affairs 28
(2009), w1171-w1183.
[2] CMA. National Physician Survey 2010. [cited March 25, 2012]. Available from:
http://www.nationalphysiciansurvey.ca..
[3] F. Lau, M. Price, J. Boyd, C. Partridge, H. Bell, R. Raworth. Impact of electronic medical record on
physician practice in office settings – systematic review. BMC Med Inform Decision Making 12
(2012):1-10.
[4] B. Hutchison, J. Levesque, E. Strumpf , N. Coyle. Primary health care in Canada: Systems in motion.
Milbank Quarterly 89 (2011), 256-88.
[5] A. Holbrook, L. Thabane, K. Keshavjee, L. Dolovich, B. Bernstein, D. Chan, S. Troyan, G. Foster, H.
Gerstein, for the COMPETE II Investigators. Individualized electronic decision support and reminders
to improve diabetes care in the community: COMPETE II randomized trial. CMAJ 181 (2009), 37-44.
[6] L.S. Nemeth, A.M. Wessell, R.G. Jenkins, P.J. Nietert, H.A. Liszka, S.M. Ornstein. Strategies to
accelerate translation of research into primary care within practices using electronic medical records.
Journal of Nurs Care Qual 22 (2008), 343-9.
[7] I. Shcherbatykh, A. Holbrook, L. Thabane, L. Dolovich. Methodologic issues in health informatics trials:
The complexities of complex interventions. JAMIA 15 (2008), 575-80.
[8] M. Price, F. Lau, J. Lai. Measuring EMR adoption: A framework and case study. ElectronicHealthcare
10 (2011), e25-e30.
[9] Canadian Institute for Health Information. Draft Pan-Canadian Primary Health Care Electronic
Medical Record Content Standard, Version 2.0 – Business View. Ottawa, Ontario, 2011.
[10] S. Dahrouge, W. Hogg, G. Russell, R. Geneau, E. Kristjansson, L. Muldoon, S. Johnston. The
comparison of models of primary care in Ontario (COMP-PC) study: methodology of a multi-faceted
cross-sectional practice-based study. Open Medicine 3 (2009), 149-64.
[11] J.L. Haggerty, R. Pineault, M.D. Beaulieu, Y. Brunelle, J. Gauthier, F. Goulet, J. Rodrigue. Practice
features associated with patient-reported accessibility, continuity and coordination of primary health
care. Annals of Family Medicine 6 (2008), 116-23.
[12] J.F. Levesque, R. Pineault, P. Tousignant, A. Couture, R. B. De Silva, M. Breton. Assessing the
evolution of primary healthcare organizations and their performance (2005-2010) in two regions of
Quebec province: Montreal and Montergie. BMC Family Practice 11 (2010), 95-108.
[13] G.M. Russell, S. Dabrouge, W. Hogg, R. Geneau, L. Muldoon, M. Tuna. Managing chronic disease in
Ontario primary care: The impact of organizational factors. Annals of Family Medicine 7 (2009),309-
18.
[14] C.M. DesRoches, E.G. Campbell, S.R. Rao, et al. Electronic health records in ambulatory care- A
national survey of physicians. New England Journal of Medicine 1 (2008), 50-60.
[15] D. Gans, J. Kralewski, T. Hammons, B. Dowd. Medical groups’ adoption of electronic health records
and information systems. Health Affairs 5 (2005), 1323-33.
[16] R. MacGregor, P. Hyland, C. Harvie, B.C. Lee, A. Dalley, S. Ramu. Benefits of ICT adoption and use
F. Lau et al. / Developing a Multivariate EMR Integration Model for Primary Health Care 381
in regional general medical practices: A pilot study. Health Information Management Journal 35
(2006), 23-35.
[17] S.R. Simon, R. Kaushal, P.D. Cleary, et al. Correlates of electronic health record adoption in office
practices: A statewide survey. Journal of American Medical Informatics Association 14 (2009), 110-7.
[18] HIMSS Analytics. Health Information Management Systems Society - Ambulatory EMR Adoption
Model. [cited October 24, 2012]. Available from:
http://www.himssanalytics.org/emram/AEMRAM.aspx .
[19] E. Strumpf, J.F. Levesque, N. Coyle, B. Hutchison, M. Banes, R.J. Wedel. Innovative and diverse
strategies toward primary health care reform: Lessons learned from the Canadian experience. Journal of
American Board Family Medicine 25 (2012), S27-33.
[20] R. Pineault, P. Tousignant, D. Roberge, P. Lambarche, D. Reinharz, D. Larouche, G. Beaulne, D.
Lesage. Research Collective on the Organization of Primary Care Services in Quebec – Detailed
Report. [cited October 24, 2012]. Available from:
http://www.greas.ca/english/publication/pdfreport/detailed_report_english.pdf.
382 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
1
Corresponding Author
Enabling Health and Healthcare through ICT 383
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-383
Introduction
There are many benefits that are expected from an electronic prescribing (e-
prescribing) system for Canadian healthcare. These benefits include greater efficiency
and a possible reduction in medication errors and adverse drug events (ADEs) [1].
Canada Health Infoway (Infoway) has found some benefits from e-prescribing
implementations in Canada, such as a reduction in ADEs, greater patient compliance,
and greater productivity from providers [2]. However, since there currently is no
functioning e-prescribing system in Canada that is completely electronic, the ability of
the system to function at both a provincial and national level is not clear. As we are
unable to evaluate the performance of a live e-prescribing system, a reasonable
alternative would be to consider the use of simulation modeling for evaluation.
Simulation modeling can be used to model a proposed system, making it possible to
1
Corresponding Author
384 A. Ghany et al. / The Use of Discrete-Event Simulation Modeling
design and measure the performance of a system before it is actually built [3].
Simulation modeling of e-prescribing may be of benefit in identifying transition issues,
key productivity bottlenecks and patient safety concerns that may disrupt or delay
clinical adoption. Simulation modeling might also be able to demonstrate the potential
impact that e-prescribing would have to Canadian healthcare. Others have developed
simulation models of e-prescribing systems or closely related systems, such as
computerized provider order entry systems (CPOE). Bell et al., with the RAND
Corporation, developed and used a simulation model to predict how the
implementation of different aspects of an e-prescribing system would affect variables
such as process times and workflow efficiency [4]. Anderson et al. developed a
simulation model to test how changing certain aspects of a CPOE system would impact
variables such as medication errors, ADEs, and costs [5].
We set out to develop two discrete-event simulation models, one of the
handwritten prescribing system currently used in the out-patient setting in Canada and
one of a proposed electronic prescribing system for Canada, and then compare the
performance of these systems to each other.
1. Methods
The modeling software Arena was selected for building our discrete-event simulation
models. Arena has been selected by other researchers to develop e-prescribing
simulation models, such as Bell, who were able to successfully develop their
simulation models using this software [4]. Drug prescribing and dispensing processes
for both handwritten prescribing and electronic prescribing were first mapped out in
Arena using a standard modeling notation, Business Process Modeling Notation
(BPMN). Both models included processes involved in drug prescribing and dispensing.
The handwritten prescribing simulation model was built based on the clinical
experiences of a member of the research team (KK). The electronic prescribing
simulation model was inspired by a proposed e-prescribing system for Canada
developed by Canada Health Infoway [6]. The models were reviewed by a pharmacist
and two data modelers for completeness and accuracy.
Data values for both the handwritten prescribing and e-prescribing simulation
models were estimated based on the clinical experiences of a member of the research
team (KK). Data values were estimated for process times, number of prescriptions, and
decision points. Minimum, maximum, and average values for each process were also
estimated.
Once the simulation models were built and populated with data, test scenarios were
developed to test the models. Test scenarios were designed to test the completeness of
the models and to check for any bugs present in the models. Sensitivity analysis was
also performed.
A. Ghany et al. / The Use of Discrete-Event Simulation Modeling 385
After the models were tested, the handwritten prescribing simulation model was
compared to the e-prescribing simulation model. This was done by running each
simulation model and then comparing the output reports for the variables of interest.
2. Results
Both the handwritten prescribing simulation model and the e-prescribing simulation
model were successfully built in Arena and were determined to be complete and
accurate by the pharmacist and data modelers who reviewed the models. Figure 1
displays a condensed version of the handwritten prescribing simulation model. Figure
2 displays a condensed version of the e-prescribing simulation model.
Running test scenarios through each model revealed that the models were running
properly and were complete. Small bugs in both models were corrected. We were
able to compare the handwritten prescribing simulation model to the e-prescribing
simulation model on several levels. We compared the number of processes involved in
each model and were able to determine that the e-prescribing model clearly contained
more processes than the handwritten prescribing model. The handwritten prescribing
model contained 34 processes, while the e-prescribing model contained 60 processes.
We found that we could compare workflow efficiency between the two models by
comparing similar processes in both models for average process times. Finally, we
were able to compare how the initial input of patients or prescriptions were distributed
throughout the handwritten system and the e-prescribing system once the simulations
were run. The results of the comparison between the two simulation models are
displayed in Table 1.
3. Discussion
it is not the ideal choice for modeling feedback loops [7]. Finally, we found that while
Arena was suitable for modeling both systems with a sufficient level of detail, it was
not suitable for modeling the interactions between entities in both models.
In future, we plan to use a complex adaptive system approach to simulating the
handwritten prescribing system and electronic prescribing system by using agent-based
or systems-based simulations.
References
[1] The Institute of Medicine. Preventing Medication Errors: Quality Chasm Series. Washington: National
Academy of Sciences; 2006.
[2] Canada Health Infoway. National impacts of generation 2 drug information systems. Technical Report.
Deloitte; 2010.
[3] Mo, JPT & Mahmoudi, M. Optimisation and simulation of high speed production system. Journal of
Achievements in Materials and Manufacturing Engineering. 2008;31(2): 794-802.
[4] Bell D, Schueth A, Crosson J, Guinan J, Wu S, Pevnick J, Wang CJ, Neuman S, Patel M, Park H,
Tysinger B, Malakar CL, Schoeff D, Bradley M, and Newberry S. Pilot testing of electronic prescribing
standards. (Report). Funded by the Agency for Healthcare Research and Quality. 2007.
[5] Anderson J, Jay S, Anderson M, and Hunt T. Evaluating the capability of information technology to
prevent adverse drug events: a computer simulation approach. Journal of the American Medical
Informatics Association. 2002; 9(5): 479-490.
[6] Canada Health Infoway. HL7 v3 pan-Canadian messaging standards: Implementation guide volume 8 –
Pharmacy. 2009.
[7] Sweetser A. A comparison of system dynamics and discrete event simulation. In: Proceedings of 17th
International Conference of the System Dynamics Society and 5th Australian & New Zealand Systems
Conference, ed. Cavana RY, Vennix JAM, Rouette EAJA, Stevenson-Wright M and Candlish J,
Wellington, New Zealand: System Dynamics Society; 1999.
388 Enabling Health and Healthcare through ICT
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
doi:10.3233/978-1-61499-203-5-388
Introduction
Collaborative care delivery is challenging due to the fact it requires the integration of
multiple people, processes and information and communication technologies. Failure to
properly integrate these different entities results in communication issues, medical
errors, and poor information management [1].
Information and communication technologies (ICTs) can facilitate collaborative
care delivery by supporting different types of interactions [2]. To date there has been
limited research on ICT design to support collaboration with the more prominent focus
being on individual ICT usage [3]. However before we can designing ICTs to support
collaboration we need to fully understand the various interactions that exist as part of
collaboration.
Co-agency, described as humans, technology and processes working together in
the pursuit of jointly held goals [4], has been used to understand collaborative practices.
A shortcoming with existing studies is that they have not studied the multiple ways in
which collaborative agents, processes and technology integrate. It is these dynamic
integrative processes that we need to focus on because that is where problems most
often occur. Coiera’s interaction design theory states that we need to look beyond
1
Corresponding Author. Dr. Craig Kuziemsky, Telfer School of Management, University of Ottawa,
55 Laurier Avenue East,Ottawa, ON K1N 6N5, e-mail: [email protected]
C.E. Kuziemsky and J.A. Cornett / A Model of Collaborative Agency and Common Ground 389
1. Study Design
The authors studied collaborative care delivery in two different areas of medicine. First
was a study of palliative sedation on an inpatient palliative care unit. Second was a
study across three hospitals of a surgical information management system (SIMS) that
is used in all areas of the perioperative process. Both studies had specific objectives
related to studying aspects of collaboration. The findings from both studies were
combined and reanalyzed to provide insight on collaborative agency and common
ground
1.2. Methods
Qualitative content analysis [7] was used to re-analyze the data from both studies. The
analysis was not done with any preconceived themes other than wanting to identify
components of collaborative agency and common ground.
2. Results
Our results are presented in two sections. First we develop a model of collaborative
agency. Second we identify four types of common ground and describe how they
influence the development of collaborative agency.
390 C.E. Kuziemsky and J.A. Cornett / A Model of Collaborative Agency and Common Ground
Fig.1 shows our model of collaborative agency. There are two main aspects to the
model. First are the collaborative factors and second is the coordination of the factors
as part of collaborative care delivery.
Agents
Information
Collaborative Action
Event Processes and
workflows
Completed actions
facilitate subsequent
events If ineffective
Channels Try another
iteration
Figure 1. Model of collaborative agency (dotted oval is the collaborative cycle with four collaborative
factors within it)
Collaborative agency is initiated with an event. The event then facilitates the
integration of four collaborative factors: agents, information, processes and workflows,
and channels. The integration of the different factors leads to an action. There are two
aspects to the model that illustrates the complexity of collaborative agency. First is that
collaborative agency is a dynamic process with multiple potential iterations. Although
a collaborative event will use each of the four collaborative factors in the collaborative
cycle (dotted oval in fig.1) the factors can be integrated in many different ways. There
are also different variations of each factor. For example, channels that are used to
communicate can include paper, electronic or oral means, while agents can be intra or
inter disciplinary. Second, collaborative agency is not a one-time action but rather is
an ongoing cycle of collaborative events and different integrations of the collaborative
factors. A collaborative event may cycle back to form a new iteration of the
collaborative factors if the action was ineffective and did not achieve the desired
outcome. For example, if a communication action did not reach the desired recipient a
new channel or different agent may be integrated into the event. Successful
collaborative events will drive subsequent events as part of the continuum of care.
Common ground represents the shared meaning that the collaborative factors need to
have in order for collaboration cycle to work effectively and facilitates the coordinative
aspect of collaborative agency. If there is weak common ground amongst any of the
collaborative factors it will adversely impact the development of collaborative agency.
C.E. Kuziemsky and J.A. Cornett / A Model of Collaborative Agency and Common Ground 391
3. Discussion
This study combined co-agency and common ground to introduce the concept of
collaborative agency. We developed a model of collaborative agency and identified
four collaborative factors that comprise agency. We have also expanded existing work
on common ground by identifying four specific types of common ground and
describing their role in the development of collaborative agency. Although the
importance of co-agency and common ground has been previously described this is the
392 C.E. Kuziemsky and J.A. Cornett / A Model of Collaborative Agency and Common Ground
first study that has combined the two and identified specific collaborative factors and
types of common ground.
The collaborative factors and common ground types can be used to design and
evaluate ICTs to support collaboration. For example, while technology is often blamed
for workflow problems, the description of technology and process common ground in
section 2.2 revealed that people do not always use systems in an appropriate way.
Although SIMS was blamed for workflow and redundant data entry issues it was
actually a lack of technology or process common ground that caused the issues.
Conducting in-situ evaluation studies, better training on system usage, and longitudinal
evaluation studies would help the formation of process and technology common ground.
Collaborative agency and common ground can also help process redesign to
support collaborative care delivery. One of the biggest barriers to common ground
development was informal communication practices or processes. Informal processes
(i.e. workarounds) make it difficult for collaborative agents to coordinate actions and
may be an ineffective or even unsafe way of doing the process. Informal information
reduces standardization which can adversely impact information exchange. Protocols
that facilitate collaborative agency and support common ground development (i.e.
formalized processes and information) should be developed as part of collaborative
care delivery.
This study has illustrated the relationship between collaborative agency and
common ground and their roles in supporting collaborative care delivery. While
collaborative agency represents the integration of agents, processes, information and
technology it is common ground that ensures that the collaborative factors are actually
interoperable with each other.
Acknowledgements
This work was supported by a Discovery Grant from the Natural Sciences and
Engineering Research Council of Canada and a School of Management Research Fund
from the Telfer School, University of Ottawa.
References
[1] Ash JS, Berg M, Coiera E. Some Unintended Consequences of Information Technology in Health Care:
The Nature of Patient Care Information System-related Error. J Am Med Inform Assoc 2004;11:104–
112
[2] Reddy MC, Gorman P, Bardram J. Special issue on Supporting Collaboration in Healthcare Settings:
The Role of Informatics. Int J Med Inform, 2011;80(8):541–543
[3] Reddy MC, Shabot MM, Bradner E. Evaluating collaborative features of critical care systems: A
methodological study of information technology in surgical intensive care units. J Biomed Inform,
2008;41:479–487
[4] Thraen I, Byron B, Mullin S, Weir CR. Characterizing “information transfer” by using a Joint
Cognitive Systems model to improve continuity of care in the aged. Int J Med Inform, 2012;81:435–441
[5] Coiera E. Interaction design theory Int J Med Inform, 2003;69:205-222
[6] Kuziemsky CE, Varpio L. Describing the clinical communication space through a model of common
ground: ‘you don’t know what you don’t know’, AMIA 2010 Symposium Proceedings,2010; 407-411.
[7] Hseih HF, Shannon SE. Three approaches to qualitative content analysis. Qual Hlth Res
2005;15(9):1277-1288.
Enabling Health and Healthcare through ICT 393
K.L. Courtney et al. (Eds.)
IOS Press, 2013
© 2013 ITCH 2013 Steering Committee and IOS Press. All rights reserved.
Subject Index
ABC framework 286 CPOE 151, 367
adverse drug event 162 critical thinking 68
agent-based model 383 cystic fibrosis 137
APEX 238 data collection 93
approaches 68 data integration 98
appropriateness 132 data management programs 251
autopoiesis 42 data mining 98, 111
baccalaureate nursing education 356 data quality assurance 37
benefits evaluation 15 data storage issues 227
bicycle accidents 93 data validation & verification 28
biomedical 189 database 238
biomedical informatics 151 decision making 107, 189
business case 28 decision quality 332
cancer registry 251 decision support system 132
cancer therapies 98 decision support tools 132
cardiology 63 dental 314
care coordination 21 dental informatics 314
CDSS 162 diabetes 57
challenges 9 diabetes mellitus 74
child 43 diagnostic imaging 132
chronic illness 308 digital stethoscope 63
clinical adoption 15 disadvantage 276
clinical decision support 98, discrete-event simulation model 383
116, 126 drug information system 168
clinical improvement 251 education 28, 49, 345
clinical informatics 251 efficiency 174
clinical practice guidelines 220 e-health 74, 220, 251, 271, 281
clinical research informatics 98 electronic health record 3, 37, 42,
clinical simulation 220 49, 232, 276
clinicianled development 3 electronic healthcare record
coding 28 (EHR) 28, 145, 151
collaboration 251 electronic medical record (EMR) 15,
collaborative agency 388 28, 49, 145, 214, 375
common data model 28 electronic prescribing 327, 383
common ground 388 electronic whiteboards 174
communities of practice 350 EPM model 332
competencies 345 foster 43
complex adaptive systems 209, 383 general practice 314
complex interventions 263, 375 general practice computerization 28
consumer health 302 GOMS-KLM 174
consumer informatics 244, 296, 302 Hajj 281
coordination 388 handoffs 21
394
Author Index
Aarts, J. 327 Cirelli, J. 126
Abrahamson, K. 350 Clayton, M. 314
Ahluwalia, R. 3 Cornett, J.A. 388
Ahmad, A. 367 Courtney, K.L. v, 79, 189
Alkadi, K. 57 Cummings, E. 137, 227
Al-Mutairi, M. 296 de Semir, M. 271
Alshaikh, A. 367 Del Fiol, G. 116
Alsughayar, A. 296 Dogar, N. 3
Alsulaiman, A. 195 Duncan, K. 98
Alsuweed, F. 367 Ebdon, N. 68
Amini, A. 87 Edwards, R. 116
Anderson, J.G. 350 Eisenstein, E.L. 116
Anstrom, K.J. 116 Elias, D. 302
Antillon, F.G. 251 Ellis, L. 137, 291
Arreola, M. 251 Fels, S. 286
Aylott, M. 63 Foster, J. 345
Bainbridge, M. 28 Fox, R. 350
Bains, A. 238 Frank, G. 63
Barnett, J. 232 French, J. 98, 232
Bassi, J. 49, 257 Frisch, N. 345, 356
Beach, S.R. 189 Fyfe, M. 93
Beggs, S.A. 137 Fyfe, M.L. 63
Bellwood, P. 93 Gavin, D. 98
Beuscart, R. 162 Gene-Badia, J. 271
Beuscart-Zephir, M.-C. 162, 337 Ghany, A. 209, 383
Birnbaum, D.W. 87 Gidora, H. 68
Black, B. 87 Glover, K. 68
Bonney, W. 195 Grajales III, F.J. 271
Borrego, L. 201 Grau, I. 271
Borromeo, C. 314 Graydon, B. 126
Borycki, E.M. 49, 68, 93, 111, 132, Griffith, J. 132
157, 227, 332, 345, 356 Hagens, S. 257
Bowen, M. 15, 168 Harmer, S. 68
Bras Da Costa, S. 337 Hauser, J. 137
Bricker, A. 302 Hebert, M.A. 74
Cameron-Tucker, H. 137 Henderson, J. 63
Chelsom, J. 3 Hess, S. 314
Cheney, J. 137 Ho, C. 98
Cheung, H. 98 Hochheiser, H. 314
Chia, S. 98 Hood, S. 232
Chio, D.T.S. 332 Horwood, T. 63
Christie, S. 195 Househ, M.S. 9, 244, 281, 296, 367
398