We developed the Guide-O-Game^© following the format of TV game shows in which two teams of residents compete in answering questions that are based on recommendations of CPGs. We initially designed the game for teams instead of single users to compete in order to use it in group educational activities (e.g. noon conference) and allow the maximum number of residents to actively participate. The development of the Guide-O-Game^© consisted of: (1) the creation of a multimedia interactive tool, (2) the development of recommendation-based questions, and (3) the definition of the rules of the game.

A team of instructional technology specialists from the Educational Technology Center of the University at Buffalo http://www.etc.buffalo.edu programmed the multimedia interactive tool. We followed a systematic approach for developing the recommendation-based questions of the Guide-O-Game^© (Additional file 1). The approach included 3 steps: (1) developing a comprehensive list of guidelines for potential inclusion; (2) assessing these guidelines for inclusion, including a methodological quality assessment using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument 10; (3) developing questions based on the recommendations of the included guidelines. We tailored the Guide-O-Game^© in order to the specific context of residents and clinical practice guidelines and healthcare issues. Indeed, the pilot testing (see below) confirmed the need for such tailoring.

We pilot tested the Guide-O-Game^© in order to evaluate its feasibility in terms of integration in the curriculum, the functionality of its tool, the structure of its questions, and the usability of its rules. We conducted four weekly sessions of the game at a hospital training site for the Internal Medicine residency program at the University at Buffalo. A senior resident (RM) doing an EBM elective run the sessions 11 and participants consisted of a convenience sample of 30 residents rotating at the training site at that time. Each session lasted 45 minutes and was followed by 15 minutes qualitative feedback sessions. The feedback consisted of answering in writing open ended questions followed by an open verbal group discussion. We improved each of the three components of the game (tool, questions, and rules) through an iterative process of pilot testing, feedback and revision.

The 15 minutes qualitative feedback sessions included questions about the acceptability of the game in terms of interest in the educational strategy and engagement in the learning process. We used the results of the evaluation after each session to improve the game prior the following session. The Health Sciences Institutional Review Board at the University at Buffalo approved the study.

Figure 1 shows the tool workflow diagram. The tool uses a Macromedia Flash web application and consists of a manager interface and a user interface compiled by Revolution for Mac and PC. The manager interface allows the creation and editing of questions using a question editor (Figure 2). It also collects data and produces reports about which questions participants selected and what answers they chose. The user interface allows choosing the game settings and running the game. The instructions for the manager interface are available as a users' guide PDF file.

The user interface settings include the choice of two game styles with different sets of rules (see below), the game length and the question length (Figure 3). Additional features include a soundtrack, the ability to choose the competing teams' names, a countdown clock for the game time, a countdown ticking clock and a timeout sound alert for each question, sounds to indicate whether an answer was correct or incorrect, and automatically calculated and displayed scores. The instructions for the user interface are imbedded the interface itself in order to make it user friendly.

A moderator runs the game on a computer and projects it on a wide screen for participants viewing. The "main screen" displays 5 columns corresponding to 5 different guidelines (Figure 4). Each column has 6 rows corresponding to 6 different questions. The "question screen" presents the question with a number of answer options (Figure 5). After the correct answer is provided, a "rationale screen" provides an explanation for the recommendation (e.g., supporting evidence). The "review screen" is available at the end of the game and shows whether a question was answered correctly or incorrectly and by which team (Figure 6). It thus gives the moderator the opportunity to review questions and to provide the correct answers and the rationale.

We have created 16 sets of "Guide-O-Game^©" questions relating to nine CPGs (Additional file 2). The user interface allows choosing from two game styles "Classic" and "Rally" that differ by their rules (Additional file 3). The two styles differ mainly by the timing of the provision of the correct answer to an incorrectly answered question. In the classic strategy, designed for learning purposes, the answer is provided before moving to the next question. In the rally strategy, designed for competition purposes, the answer is provided at the end of the game.

Educators can integrate the Guide-O-Game^© into the curriculum in different ways. One way would be to include one Guide-O-Game^© session per week, with each session lasting 50 minutes followed by a 10 minutes review period. The chief medical resident could serve as the session facilitator and all residents completing their ward rotations and all medical students completing their medicine clerkship would participate. Prizes could be established for the winning team. The residents would be made aware of what guidelines are being "played" the following week. Copies of the guidelines would be made available to interested residents, e.g. through the chief medical resident, rather than distributed to all residents. This is following the first principle of the adult learning theory stating that adults are autonomous and self-directed 12. We are hypothesizing that residents, as a result of the educational game, would get engaged and take charge of their own learning by getting copies of the guidelines, reading them and becoming familiar with them for the next game session.

Integrating the four weekly sessions within the curriculum was feasible, although it required ahead-of-time planning. We encountered no major problems with the Guide-O-Game^© tool. One glitch with the tool interrupted the first session, but we were able to fix it and the subsequent sessions run smoothly. None of the participants considered that the structure of the questions or their answer options to be confusing. In fact, all discussions around the questions related to their content. We only tested the "Classic Guide-O-Game rules". Because of the slower pace of these rules, we were not able during the assigned 45 minutes to ask all the session questions. Having the full 60 minutes should resolve this issue.

Participants found the game to be an acceptable educational strategy. First, the game format raised the residents' interest in guidelines recommendations which they described as "a dry material". The interest further increased when we integrated a rationale for each recommendation in the game. Second, they felt playing the game is fun, especially when racing against time to select an answer and when the scores of the two teams were close. This helped them "relax a bit" during a time of the day when work pressure is high (sessions were run during noon conference time). Third, as a result of the fun and the increased interest, the residents reported being more engaged in the learning process in comparison with usual didactic lecture during which they become "easily distracted". Fourth, while residents became actively engaged during the session there were some indicators that they might have also become actively engaged after the session. Indeed, one resident asked which CPGs were to be played the following week so she could "review them in advance".

While participants found most of the questions relevant to their practice, they judged some of them as non-relevant. One example relates to the number of procedures a laboratory need to perform per year to be considered an appropriate percutaneous coronary intervention center. They also criticized not providing a rationale for each recommendation in the initial version of the tool. We did subsequently add such rationale which proved helpful in increasing residents' interest. The residents finally recommended having a review period at the end of each session which we used for discussing some of the answers and rationales.