Computer-based patient simulations (CBPS) are common, effective, instructional methods for medical students, but have limitations.
The goal of this project was to describe the development of a CBPS designed to overcome some of these limitations and to perform an online evaluation.
In 1996, patients and families experiencing a common pediatric problem were interviewed, photographed and a chart review completed. A digital storytelling template was developed: 1. patient's story, evaluation and clinical course, 2. problem-based approach to the evaluation, and 3. discussion of disease process. The media was digitized and placed onto the Internet. The digital stories and a 10-question online survey were pilot tested. Online survey responses were collected from 1999–2003. Overall use of the digital stories was measured by computer server logs and by the number of hyperlinks to the CBPS.
Eight stories were created using this system. Over 4.5 years, 814,148 digital story pages were read by 362,351 users. Hyperlink citations from other websites to the CBPS were 108. Online survey respondents (N = 393) described the overall quality as excellent or very good (88.4%). The stores were clearly written (92%) at an appropriate level (91.4%). Respondents felt they could begin to evaluate a similar case presentation (95.4%), and would remember the case in the future (91%).
A new type of CBPS, the digital storytelling system, has been developed and evaluated which and appears to be successful in overcoming some of the limitations of earlier CBPS by featuring patient's stories in their own words, by focusing on problems rather than diseases, and by having stories that are quick for students to work through.
Keywords:medical students; medical education; computer based patient simulations; problem based learning; digital stories; PDL; CAI
There is a growing consensus suggesting physicians acquire clinical expertise by listening to, analyzing, and organizing patients' stories or "illness scripts" and medical student education has supported this learning method by exposing students to as many patients, or "cases," as possible during their training [1,2]. The art of storytelling in medicine has been shown to be a method for teaching values and attitudes, for professional development, and enhancement of interpersonal communication [3-8].
Problem based learning is a curriculum development and delivery system that relies on real world, multi-answer problems or cases to drive the curriculum. Students solve the problems themselves while being guided teachers or facilitators. Problem based learning problems are typically in the form of cases that are narratives of complex, real-world challenges common to the discipline being studied [9,10]. This is sometimes referred to as the case method . Cases can be in various forms including those presented in print, verbally by teachers or by computers. "Computer-based patient simulations are situational experiences that present problem conditions close to real life. They are used to reinforce principles and practices that are essential to a trade or profession ." In the past, computer-based patient simulations (CBPS) have been offered as a method of providing a common patient base for medical student education. CBPS challenge and expand thinking, allowing students to develop problem solving skills, engage the students in synthesizing skills, and develop basic and advanced empathetic skills. Strong visceral connections to their learning are made, and long term recall is enhanced [13,14].
Thus, CBPS are a way to use stories to teach clinical skills but have only met with limited success. Some CBPS depict idealized, standardized patients synthesized from many patients, giving them few of the memorable qualities of real patients nor the richness and complexity of real patients' problems. CBPS may emphasize the knowledge of the "disease" at the expense of the "problem" with which the patient and the family are dealing (e.g. mid-gut malrotation, a disease versus vomiting, a problem). Furthermore, CBPS may attempt to be all encompassing and require long periods of time for students to work through and immense amounts of time for faculty to create and evaluate [15,16].
Computers also allow evaluations to be collected online and these evaluations have advantages and disadvantages. The advantages include user convenience, ease in collecting and collating data, and produces results that are often available sooner and more easily distributed. The disadvantages include user authentication, response rates that may be higher or lower than other methods depending on a specific situation, and potential changes in the users' actual responses. For example, a voluntary, anonymous survey of website's users has the limitations of not being able to authenticate its users and a potential difficulty in accurately determining the denominator of the response rate.
Therefore, the goals of this project were to: 1. describe the development of a CBPS designed to overcome some of these limitations, and 2. perform an online proof-of-concept evaluation by measuring the value of the CBPS to users.
Digital storytelling system development
During the summer of 1996, digital stories were developed based on the following educational objectives:
1. Tell individual, memorable stories from real patients in their own words.
2. Emphasize the clinical problem faced by the patient and family and de-emphasize the disease the patient has.
3. Promote primary care principles (e.g. safety and health screening)
4. Written primarily at the level of the introductory pediatric clerkship for third year medical students.
5. Used for independent study by medical students locally and nationally (i.e. not as part of a specific course)
6. Take a limited amount of time for students to work through (approximately 20–30 minutes).
7. Take a limited amount of time to author.
Due to its emphasis on memorable, first person patient stories, which emphasize common pediatric problems, we called our CBPS a digital storytelling system.
To be consistent with national guidelines for medical student education, a list of common pediatric problems was developed during the summer of 1996 based on national pediatric curricula, national health statistics, and inpatient and outpatient census data from 2 hospitals [[17-20] and Unpublished data, collected by DMD during general pediatric fellowship at Children's Hospital, Boston 1993–1995.]. Patient stories related to these problems were then sought from families in outpatient clinics and inpatient wards at the University of Iowa. After informed consent, the patient and/or family members were interviewed and photographed. A second year medical student completed a chart review and imaging and pathology slides were obtained as appropriate.
A 4-section digital story template was developed
1. The patient's story, evaluation process and clinical course told through the patient's voice and a clinical narrator.
2. A problem-based approach to the evaluation of the patient's problem, including differential diagnoses, history, physical examination, and laboratory and imaging evaluation.
3. A discussion of the patient's disease process based upon journal articles.
4. A brief conclusion and follow-up told through the patient's voice.
The patient stories were initially written by a second year medical student and reviewed and edited by a practicing pediatrician and pediatric radiologist for accuracy. A literature review was conducted and a problem-based approach to the evaluation of the patient's problem and a discussion of the patient's actual disease process was written by a practicing pediatrician. A science educator and professional storyteller reviewed the initial stories for their storytelling abilities.
All media was digitized and entered into the digital story template by a medical student. The digital media was then assembled into Hypertext Markup Language (HTML) files and placed on the University of Iowa's Virtual Hospital® digital library at: http://www.vh.org/pediatric/provider/pediatrics/VirtualPedsPatients/PedsVPHome.html webcite. The Virtual Hospital allows free, anonymous use of its entire collection to educate patients, and healthcare providers for the purpose of improving patients' care, outcome, and lives .
Two independent measures of digital storytelling value were used: usage of the digital stories and measurement of hyperlink citations between Internet websites. From January 1, 1999–July 1, 2003 the usage of to the digital storytelling system was extracted from computer server log files and analyzed using Analog 1.2.3 (University of Cambridge Statistical Laboratory, Cambridge, England). This determined how many times the digital storytelling system was used and by how many users. On July 27, 2003, hyperlink citations were recorded using the Google search engine searching for the term "link: http://www.vh.org/Providers/Simulations/VirtualPedsPatients/PedsVPHome.html webcite" .
Additionally, a 10-item online survey of respondent attitudes with Likert scale, best answer and written comments was developed in 1997 and pilot tested using a group of medical educators and medical students http://www.vh.org/pediatric/provider/pediatrics/VirtualPedsPatients/OnlineSurvey.html webcite. The anonymous, voluntary survey was placed on the homepage of the digital storytelling system in May 1999. Automated responses were mailed electronically to one author (DMD). The responses from May 1999–July 2003 were coded using a pre-determined coding scheme by a research assistant. Coding was checked for accuracy by one author (DMD) and showed accurate coding.
Digital storytelling system
Eight digital stories were created for this digital storytelling system. The problems included children with an abdominal mass, fever, leg pain, sore throat, and vomiting and diarrhea. They also included an adolescent with leg pain and a newborn with vomiting. Another digital story begins with the child's father saying, "This is my son Bobby, he is 5 years old. He's always been a pretty normal kid." His father then describes the problem by saying, "Bobby has had a long history of bowel problems. About 2-1/2 years ago, we noticed that he was constipated a lot. " The problem-based evaluation includes a clinical differential diagnoses (i.e. constipation/encopresis, hypothyroidism, functional abdominal pain, plumbism or lead toxicity, sacral agenesis, neurologic abnormality) and laboratory differential diagnoses (i.e. plumbism with secondary constipation and iron deficiency anemia.) The prognosis and follow-up are told again by the father who says, "He's got another medicine to get the lead out of his blood and his bones. We can't go back to our home until we get the results of the Health Department's tests too. The doctors told us when he finished his treatment and the lead is gone, he'll be okay." The digital story ends with a general overview of ingestions and specifically lead intoxication. http://www.vh.org/pediatric/provider/pediatrics/VirtualPedsPatients/Case04/Case04.html webcite
The digital storytelling system contains a total of 63 pages (i.e. 8 patient story pages, 47 media pages and 8 support pages). Creation of the digital storytelling system took approximately 300 hours of development time. This included the time to contact and meet with consultants, digital story template development, procedure manual development, individual digital story development, etc. We estimate that a single digital story could now be developed with approximately 20 hours or less time inclusive.
A total of 814,148 pages of the digital stories were read by 263,351 users between January 1,1999 and July 31, 2003 with usage steadily increasing over time (Data not shown). The number of hyperlinks citations from other websites to the digital storytelling system was 108.
In 1997, a formative evaluation of the digital stories was completed by 8 medical students, who found the digital stories to be memorable, presented in a clear manner, written at the appropriate level, and took approximately 20–30 minutes to work through. Suggestions included more interactivity in the patient stories by using videoclips, questions and answers, and comparisons of medical student answers with expert clinicians' opinions.
From May 1, 1999–July 1, 2003 the online survey was filled out by a total of 393 respondents [Table 1]. Each respondent did not answer all survey questions. The percentages are based on the number of respondents to an individual survey question. Respondents described the overall quality of the digital storytelling system as excellent (46.3%) or very good (42.1%). Ninety-two percent felt the digital stories were written clearly and 91.4% felt they were written at the appropriate level for third year medical students. Respondents (76.6%) felt that the digital stories took the right amount of time to work through. Overall 95.4% felt they could begin to evaluate at least some aspect of a similar case presentation and 91.4% felt they would remember at least some part of the case in the future. Respondents (74.1%) were likely to recommend the digital stories to another person. Written suggestions are generally uniformly laudatory http://www.vh.org/pediatric/provider/pediatrics/VirtualPedsPatients/TOC/VirtualPedPtAwards.html webcite but suggestions include increasing the digital storytelling system interactivity or having longer-term follow-up of the patients included.
Table 1. Percent distribution of survey results
Medical students in this survey (a subgroup of all respondents) described the overall quality as excellent (50%) or very good (43.2%): higher than all respondents. The medical students thought the digital stories were clearly written (89.7%) and at an appropriate level (96.1%). Medical students thought the digital stories took the right amount of time to work through (70.5%); slightly lower than all respondents did. Medical students (98.8%) felt they could begin to evaluate at least some aspect of a similar case presentation and 91% felt they would remember at least some part of the case in the future. Medical students (78.2%) were more likely to recommend the digital stories to another person than all respondents.
Many different computer-based patient simulations (CBPS) have been developed, each with different educational objectives and integration into the educational setting for which they are designed. There are pediatric examples of CBPS in both open and closed systems (i.e. CBPS available outside of or only within the institution) which can be found elsewhere and which have experienced mixed success [23-26]. We believe we have fulfilled most of the educational goals for which the digital storytelling system was developed. Notably, the patient's own words are used to describe the problem, the presenting clinical problem is emphasized, the disease process is presented but de-emphasized (i.e. the disease discussion is short and presented at the end of the story), the stories were quick to work through and were written at the appropriate level.
The digital storytelling system is valued by its users, as usage of the digital storytelling system has been high with over a quarter million users in the past 4.5 years. The literature suggests that to be well connected to the Internet as a whole, a website needs to be hyperlinked to 11–20 other websites . The number of hyperlink citations from other websites to the digital storytelling system was 108, surpassing this number and suggesting that other website authors view the digital storytelling system as valuable enough to hyperlink their own website to the system.
Although originally designed for medical students, the digital storytelling system was used by a broad spectrum of health sciences students and practitioners whose responses on the survey indicate the digital stories were educationally useful. It appears that medical students in this survey learned from the digital stories because 98.8% of the students said they could begin to evaluate a similar patient problem and would remember (91%) at least some aspect of the digital stories in the future. Medical students felt the overall quality of the digital stories was very good or excellent 93.2% of the time and almost 78.2% were likely to recommend the digital stories to someone else (another measure of value to the respondent).
Survey respondents sometimes commented that they would like more interactivity in the digital stories. From a pedagogic standpoint this makes sense since it offers the ability to practice one's knowledge as one is learning. More interactivity can be easily applied to similar digital stories, but interactivity adds complexity to their development, increases development time and also adds time to work through the digital stories; especially in branching paradigms, where answers to questions leads the students to different pathways in the stories. Thus we chose to minimize interactivity to shorten development time and the time required to work through the stories.
The digital storytelling system development must be put into a historical context. It was developed in 1996 when the Internet was just beginning to become mainstream and long before many of the Web technological innovations we take for granted today had been standardized. Also, it was developed with limited funds over a limited time period (i.e. institutional medical student summer research stipend and project). Nonetheless, the success of the digital storytelling system measured by long-term overall usage and user satisfaction suggests that it can serve as a template for future CBPS authors. At the center of any new CBPS project should be explicit educational goals and objectives from which the inevitable developmental tradeoffs can be decided. An evaluation plan, designed before the project begins, is critical to allow the project to be improved over time as well as in judging the final success of the project. The use of online evaluations have also increased since 1996 as institutional cultures have changed particularly with the increased need for timely reporting. One example is the United States Medical Licensing Examination that is now given at computer testing centers nationally .
The limitations of this study should be emphasized. Only a small percentage of the overall users of the digital storytelling system responded to the online survey and therefore there may be a potential bias towards respondents who were more positive. Authentication log-ins and compulsory surveys decrease use of websites , which detracts from the purpose of allowing free use of the digital storytelling system. Therefore these methods were not used. The survey does not directly ask what the respondent learned from the digital stories. Respondents are asked to hypothesize if they will remember the stories in the future and could not be re-surveyed to see if they actually did remember. However, it appears, en face, that these survey respondents believe there is an educational value to the digital stories by their responses.
A new type of CBPS, the digital storytelling system, has been developed and evaluated which and appears to be successful in overcoming some of the limitations of earlier CBPS.
DMD developed the digital storytelling system, performed the data analysis and wrote the manuscript. TEL developed the digital storytelling system, helped perform the initial data analysis, and revised the manuscript. MPD developed the digital storytelling system, assisted with data analysis and revised the manuscript. All authors read and approved the final manuscript.
The authors wish to thank Ms. Susan Kinzer and Ms. Lindsay Huth for their help with data entry and analysis. This paper was presented in part at the Pediatric Academic Societies' 40th Annual Meeting, Boston, MA, May 12–16, 2000.
Acad Med 1990, 65:611-621. PubMed Abstract
Family Practice 1992, 9:127-29. PubMed Abstract
Acad Med 1992, 67:500-504. PubMed Abstract
Canadian Family Physician 1994, 40:2098-2102. PubMed Abstract
Journal of Continuing Education in Nursing 1995, 26:280-282. PubMed Abstract
Contemporary Nurse 1996, 5:28-35. PubMed Abstract
Office of Consultation and Research in Medical Education at the University of Iowa: Computers in the Curriculum. [http://www.medicine.uiowa.edu/ocrme/curriculum/cur_computers_cs.html] webcite
Acad Med 1994, 69:455-457. PubMed Abstract
University of Iowa: Virtual Hospital Mission and Goals. [http://www.vh.org/welcome/aboutus/missiongoals.html] webcite
Yamamoto LI, Inaba AS, DiMauro RM: Radiology Digital Stories in Pediatric Emergency Medicine. [http://www2.hawaii.edu/medicine/pediatrics/pemxray/pemxray.html] webcite
Kamin CS, Deterding RD, Wilson B, Armacost M, Breedon T: The development of a collaborative distance learning program to facilitate pediatric problem-based learning. [http://www.Med-Ed-Online.org] webcite
The pre-publication history for this paper can be accessed here: