Patient safety culture is how leader and staff interaction, attitudes, routines and practices protect patients from adverse events in healthcare. The Safety Attitudes Questionnaire is the most widely used instrument to measure safety attitudes among health care providers. The instrument may identify possible weaknesses in clinical settings, and motivate and guide quality improvement interventions and reductions in medical errors. The Safety Attitudes Questionnaire – Ambulatory Version was developed for measuring safety culture in the primary care setting. The original version includes six major patient safety factors: Teamwork climate, Safety climate, Job satisfaction, Perceptions of management, Working conditions and Stress recognition. We describe the results of a validation study using the Norwegian translation of the questionnaire in the primary care setting, and present the psychometric properties of this version.
The study was done in seven Out-of-hours casualty clinics and 17 regular GP practices employing a total of 510 primary health care providers (194 nurses and 316 medical doctors). In October and November 2012, the translated Safety Attitudes Questionnaire – Ambulatory Version was distributed by e-mail. Data were collected electronically using the program QuestBack, whereby the participants responded anonymously. SPSS was used to estimate the Cronbach’s alphas, item-to-own-factor correlations, intercorrelations of factors and item-descriptive statistics. The confirmatory factor analysis was done by AMOS.
Of the 510 invited health care providers, 266 (52%) answered the questionnaire - 72% of the registered nurses (n = 139) and 39% of the medical doctors (n = 124). In the confirmatory factor analysis, the following five factor model was shown to have acceptable goodness-of-fit values in the Norwegian primary care setting: Teamwork climate, Safety climate, Job satisfaction, Working conditions and Perceptions of management.
The results of our study indicate that the Norwegian translated version of the Safety Attitudes Questionnaire – Ambulatory Version, with the five confirmed factors, might be a useful tool for measuring several aspects of patient safety culture in the primary care setting. Further research should investigate whether there is an association between patient safety culture in primary care, as measured by the Safety Attitudes Questionnaire – Ambulatory Version, and occurrence of medical errors and negative patient outcome.
Keywords:Adverse events; General practice; Medical errors; Out-of-hours; Patient safety culture; Primary care; Quality improvement; Safety attitudes questionnaire
Over the last years, there has been an increasing focus on medical errors and patient safety. Until recently, patient safety issues have mainly been addressed in the hospital care setting [1-7]. However, adverse events are common also in primary care, where the largest volume of health care is delivered. For these reasons, there has been an increasing interest in factors related to patient safety also outside the hospital setting.
Patient safety culture is how leader and staff interaction, attitudes, routines and practices protect patients from adverse events in healthcare . The concept is developed within the framework and research of organizational psychology, and is regarded as a group phenomenon rather than that of an individual.
A number of instruments have been developed to measure safety attitudes of health care providers [1,9-13]. The Safety Attitudes Questionnaire (SAQ) is most widely used, and the original version includes six major patient safety factors: Teamwork climate, Safety climate, Job satisfaction, Perceptions of management, Working conditions and Stress recognition .
SAQ scores have been shown to correlate with patient outcome in care giving units [12,15-17]. The instrument may identify possible weaknesses in a clinical setting and motivate quality improvement interventions and reductions in medical errors [18-20]. The SAQ has been translated and validated in a number of different countries, including Norway .
The majority of the safety attitudes instruments are adapted to different clinical settings within hospitals. It has been shown that patient safety culture may vary substantially across hospitals, departments and - in particular - wards, the level closest to the patients . Interventions to improve patient safety should therefore also include ward level.
In 2007, the first questionnaire for measuring safety culture in the outpatient setting was described . This Safety Attitudes Questionnaire – Ambulatory Version (SAQ-AV) was developed from the original SAQ, and adjusted to the primary care setting. It has shown to be a reliable tool for comparing attitudes across different professional groups of health care providers outside hospitals . Further, SAQ-AV gives data that makes it possible to focus on patient safety improvement activities, in addition to measuring change in attitudes over time .
In this paper, we describe the results of a validation study using the Norwegian translation of the SAQ-AV in the primary care setting, both in out-of-hours (OOH) casualty clinics and regular General Practitioner (GP) practices. We wanted to study whether the factor structure in the translated version was the same as in the original questionnaire. We present the psychometric properties of the Norwegian translated version of the SAQ-AV. The study is a part of the Norwegian patient safety campaign “In Safe Hands”, which was launched in 2011 by the Ministry of Health.
The present study was done both in OOH casualty clinics and in regular GP practices. Seven OOH clinics in Norway function as especially designated “Watchtower Clinics”, established by the National Centre for Emergency Primary Health Care to deliver research data [23,24]. The seven Watchtowers were selected so as to be representative for OOH casualty clinics in Norway, and serve 4,6% (226,000 inhabitants) of the nations population in a total of 18 municipalities. In addition, all regular GP practices in Sogn & Fjordane County were invited to participate in the study. This is one of 19 counties in Norway, with a population of approximately 110,000 in 26 municipalities.
In order to protect the confidentiality of the respondents, we only included clinics and practices employing at least five health professionals having clinical patient contact. For this reason we replaced one of the seven Watchtowers with the OOH clinic in the neighbouring municipality. The seven OOH clinics participating in our study employed a total of 337 health professionals, of whom 231 medical doctors and 106 nurses. They serve a total population of 251,000.
Seven of the total of 30 regular GP practices in Sogn & Fjordane County were not included, as they had less than five employees working clinically. Of the remaining 23 regular GP practices, 17 agreed to participate in the study. These 17 regular GP practices employed a total of 173 health professionals, 85 medical doctors and 88 support medical staff. The professional background of the support medical staff varied, and included registered nurses, medical secretaries and bioengineers. In this paper, we use the term “nurses” for this group of support medical staff. The participating GP clinics serve a population of 70,000.
The original SAQ-AV questionnaire was translated following modified principles adapted from Beaton et al. . Initially, the original English version was translated into Norwegian using a professional translation bureau. Next, an expert committee with clinicians and researchers from the Research Group for General Practice (University of Bergen), the National Centre for Emergency Primary Health Care (Uni Research, Bergen) and the Norwegian Knowledge Centre for the Health Services (Oslo) adapted the initial translated version to the primary care setting in Norway. This adapted version of the questionnaire was translated back into English by a second independent translation bureau being blinded to the original version.
Based on this back-translated version, the expert committee made some adjustments in order to clarify misunderstandings. The prefinal Norwegian version was tested in a group of primary health care providers. Based on their feedback, the final version of the translated questionnaire was developed. Two Norwegian versions of the questionnaire were made, one for OOH casualty clinics and one for regular GP practices, with only minor modifications in a few of the questions – according to the setting. Pre-tests showed that it took approximately 15 min to complete the SAQ-AV questionnaire.
The SAQ-AV is a 62 item questionnaire where the respondents rate their agreement using a 5-point Likert scale: 1 = disagree strongly, 2 = disagree slightly, 3 = neutral, 4 = agree slightly, 5 = agree strongly. For all questions, “Not applicable” was included as a response category, and combined with missing values in the data analyses. Scores of negatively worded items were reversed, so that higher scores in the data set always indicate a more positive evaluation of the unit’s patient safety culture.
The seven OOH clinics and 17 regular GP practices provided the e-mail addresses of all employees having direct patient contact in their clinical work. In October and November 2012, the SAQ-AV was distributed by e-mail to all 510 primary health care providers in these 24 clinics/practices (316 medical doctors and 194 nurses). Data were collected electronically using the program QuestBack, whereby the participants responded anonymously. This program automatically sent a reminder to those who had not responded after two weeks. After four weeks, an additional reminder was sent to the administrative key persons in the OOH clinics and regular GP practices – asking them to motivate the clinical staff to participate in the study.
After the study, each of the participating OOH clinics and GP practices received a summary of the SAQ-AV results for their own unit. In this way, the health care providers were encouraged to focus on specific factors related to patient safety, and to discuss possible strategies for improvement within their clinical setting.
The QuestBack file with anonymous SAQ-AV data was converted into a SPSS file for further analysis. SPSS v.18 was used to estimate the Cronbach’s alphas, item-to-own-factor correlations, intercorrelations of factors and item-descriptive statistics. The confirmatory factor analysis was done by AMOS.
Cronbach’s alpha is a measure of factor score consistency. The test was done to demonstrate to which extent the responses of items within a factor correlated pairwise. Cronbach’s alpha scores above 0.7 were considered adequate. Item-to-own-factor correlations were checked to see if the items correlated more with the factor they were hypothesised to belong to, than to the other factors.
Hypothesised factor structure
The original SAQ, developed at the University of Texas at Austin , described six factors: Teamwork climate, Safety climate, Working conditions, Job satisfaction, Perceptions of management and Stress recognition (Table 1). Only 30 of the 62 items in the SAQ-AV are covered by the original six factors. The rest of the items were considered useful for local improvement processes and discussion and included to provide additional information regarding the work environment in the primary care setting. Since some of these items belonged thematically to the Teamwork climate factor (Q37, Q39, Q45) we included the items in the hypothesized factor structure.
Table 1. The six factors and corresponding items in the original Safety Attitudes Questionnaire (SAQ) version
One item (Q18) had been moved from the factor Perceptions of management to the factor Working conditions, in the validation of the Norwegian SAQ Short form 2006 . We replicated this since we also in the present study found that the item (Q18) fitted the Working condition factor better, indicating that respondents in primary care perceive that adequate staffing has less to do with leadership, although it is strongly related to working conditions. Since several studies find that the factor Stress recognition does not vary significantly between organizational units [21,26], and also has problems regarding construct validity , it cannot be considered a valid factor for measuring patient safety . In the present study, we based our analyses on the five remaining factors, which were included in the hypothesised factor model.
Confirmatory factor analysis (CFA)
Among the 160 respondents answering each of the items (no missing/not applicable), we tested the hypothesized factors from the five factor model of the SAQ using AMOS. The factors reflect the correlation structure in the item responses. Valid factors should thus reflect a thematic logic that is coherent with the purpose of the questionnaire. CFA provides goodness-of-fit indices, which show how the survey responses comply with the pre-hypothesised factor model.
The following goodness-of-fit indices were calculated: P, Pclose, Adjusted Goodness-of-Fit Index (AGFI), Comparative Fit Index (CFI), Normed Fit Index (NFI), Root Mean Square Error of Approximation (RMSEA) and Hoelter 0.05. Acceptable goodness-of-fit values indicate that the SAQ-AV measures patient safety culture in the hypothesised factors. The P and Pclose values should exceed 0.05, the AGFI should be close to 1, and the RMSEA should not exceed 0.10 . The CFI should be close to 1 and the NFI > 0.90 . The Hoelter 0.05 should exceed 200, an estimate of the largest sample for which a data set with these intercorrelations among the variables would confirm the model .
This study was based on data regarding patient safety culture among health care providers. It was conducted in compliance with the ethical guidelines of the Helsinki Declaration. All participants received written information about the purpose of the study, and that the data would be collected anonymously and treated in confidence. As the study was not affected by the Norwegian Health Research Act, approval from an ethics committee was not needed. The study was approved by the Norwegian Social Science Data Services – the governmental agency for protecting survey research respondent privacy according to the Norwegian Personal Data Act (Ref. No. 2012/30774).
Of the 510 invited health care providers, 266 (52%) answered the questionnaire - 72% of the nurses (n = 139) and 39% of the medical doctors (n = 124). Professional status is not known for three of the respondents. The response rate was higher among doctors in GP practices (55%) than doctors in OOH clinics (33%), while the corresponding rates for nurses were 73% and 71%, respectively.
Table 2 presents mean scores with standard deviations for each of the 62 items, expressing the degree of agreement with the statements in the questionnaire (1 = disagree strongly, 2 = disagree slightly, 3 = neutral, 4 = agree slightly, 5 = agree strongly). Missing values at item levels are also shown in the table, and were on average 2.4%, ranging from 0 to 8.3%. The strongest disagreement was found in the statements “I feel burned out from my work” and “Abnormal test results are frequently lost or overlooked” (both mean scores 1.7). The highest mean scores reflecting agreement were reported for the statements “I like my job” and “Attending physicians/primary care providers in this office are doing a good job” (both mean scores 4.7).
Table 2. Mean score for the 62 items in the Safety Attitudes Questionnaire – Ambulatory Version (SAQ-AV)
The confirmatory factor analysis showed that each of the five hypothesised factors (Teamwork climate, Safety climate, Job satisfaction, Working conditions and Perceptions of management) fitted the data well. The goodness-of-fit indices for the five factors are presented in Table 3. As the P was satisfactory for all factors (p > 0.05), Pclose is not presented in the table.
Table 3. Goodness of fit indices for patient safety culture factors among 160* primary health care providers
The goodness of fit of the entire five-factor model is presented in Figure 1. It is not as clearcut as that of the five single factor models: The CFI of the entire model was 0.86, its NFI was 0.73 and its P-value (and its Pclose-value) was < 0.001. However, its RMSEA was satisfactory (0.07), and the model’s chisquare-to-degrees-of-freedom ratio was excellent: 1.82.
Figure 1. Factor structure model.
The Cronbach alphas ranged from 0.67 to 0.83 for the factor scores Teamwork climate, Safety climate, Working conditions, Job satisfaction and Perceptions of management (Table 4). All items in the table correlated higher with the factor it has been related to than with any other factor.
Table 4. Item variation and internal consistency of five factors based on 160arespondents
We have described the results of a validation study using the Norwegian translation of the SAQ-AV in the primary care setting, both in OOH clinics and regular GP practices. As far as we know, this is the first systematic study of patient safety culture in a representative sample of OOH clinics in Norway. In addition, the SAQ has not earlier been validated in a Norwegian GP setting.
Among those responding to the questionnaire, there was a low degree of missing values. The overall response rate was, however, not optimal. It was substantially higher, almost twice as high, among nurses compared to medical doctors. This difference is in accordance with findings from a SAQ-study in a Norwegian hospital setting . A possible reason for this could be that nurses may feel closer linked to the clinical units than the doctors, since they spend a larger amount of their working time there. Many primary care doctors do clinical work different places (in GP practices, OOH clinics, nursing homes, well baby clinics), while most nurses are employed only in one clinical setting. This may imply an increased interest in creating a positive and safe working environment.
The response rate was higher among doctors in GP practices (55%) than doctors in OOH clinics (33%). As GPs commonly spend more working hours in GP practices than most OOH-doctors do in casualty clinics, the higher response rate increases the validity of the patient safety assessment in general practice. OOH-doctors commonly work in casualty clinics for only a limited part of their total working hours. This may lead to a poorer linkage to this particular work place, meaning that the rather low response rate should not reduce the validity of the patient safety assessment in OOH clinics very much. Nurses are more often employed only in one clinic, and spend most of their working time there. The high response rates among nurses both in GP practices (73%) and OOH clinics (71%), strengthen the validity of the patient safety assessments.
Nurses might have had the opportunity to fill in the questionnaire during their work time – whereas it was a free time, and thereby an unpaid, activity for the majority of the doctors. This may explain the higher interest among nurses for participating in a study of how their working environment supports patient safety. There also may be a gender difference in how to respond to the reminders that were sent. A majority of the responses tended to be skewed towards the favourable side of the scale, reflecting a positive attitude to patient safety.
The Cronbach alphas were above the recommended limit of 0.70 for four of the factors, and not much below for the fifth factor Perceptions of management (0.67). These values demonstrate internal consistency of the factors.
The five factor model (Teamwork climate, Safety climate, Job satisfaction, Working conditions and Perceptions of management) fitted the dataset from this Norwegian primary health care setting, with acceptable goodness-of-fit values. A model containing a sixth factor, Stress recognition – one of the original six SAQ factors – could not be confirmed. As mentioned above, several studies have found it to be invalid as an organizational climate scale, and that may explain our result.
The results of our study indicate that the Norwegian translated version of the Safety Attitudes Questionnaire – Ambulatory Version, with the five confirmed factors, might be a useful tool for measuring several aspects of patient safety culture in the primary care setting. Discussing the results at unit level may facilitate strategies to reduce risk of medical errors, by focusing on improvement of quality and health care provider attitudes. In future studies, possible patient safety culture differences between OOH clinics and GP practices should be investigated. Likewise, it needs to be clarified whether different professional background may influence attitudes to patient safety in the primary care setting. Further research should also validate the questionnaire by correlating the scores on the SAQ-AV domains to patient-associated outcomes, not only medical errors and negative health outcomes, but also patient satisfaction and employee-related outcomes, such as staff satisfaction, retention and burnout.
The authors declare that they have no competing interests.
GTB was responsible for designing the study, developing the Norwegian translated version of the SAQ-AV questionnaire, data collection, analysis and interpretation of data, and writing the manuscript. DH participated in developing the Norwegian SAQ-AV questionnaire, and was responsible for the statistical analysis and interpretation of results, in addition to revising the manuscript critically. EHH participated in designing the study, developing the Norwegian SAQ-AV questionnaire, data collection and revising the manuscript critically. ECTD participated in designing the study, developing the Norwegian SAQ-AV questionnaire, data analyses and interpretation, and writing the manuscript. All authors read and approved the final manuscript.
GTB, MD and PhD, is Professor in General Practice at the University of Bergen and Principal Researcher at the National Centre for Emergency Primary Health Care, Bergen, Norway. He is specialist in Family Medicine, and works clinically as General Practitioner in Bømlo, Norway.
DH is Professor at the Institute of Health and Society, Faculty of Medicine, University of Oslo. He is a social scientist.
EHH, MPH and PhD, is Associate Professor at Haraldsplass Deaconess University College of Bergen, Norway, and Research Adviser at The Norwegian Nurses Organization in Norway. She is a registered nurse.
ECTD, MD and PhD, is a Senior Researcher at the Health Services Research Unit at Akershus University Hospital and a Senior Advisor for the Norwegian Knowledge Centre, secretariat of the Norwegian patient safety campaign. She is a Consultant in internal medicine in an ambulatory clinic for rehabilitation of stroke patients, at Akershus University Hospital.
We thank the health care providers in the 7 OOH casualty clinics and 17 GP practices for participating in the study.
Jt Comm J Qual Saf 2004, 30:125-132. PubMed Abstract
Sorra JS, Nieva VF: Hospital Survey on Patient Safety Culture. Rockville, MD: Agency for Healthcare Research and Quality; 2004. [(Prepared by Westat, Under Contract No. 290-96-0004). AHRQ Publication No. 04–0041]
Sexton JB, Thomas EJ, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR: Frontline assessments of healthcare culture: Safety Attitudes Questionnaire norms and psychometric properties. Austin, TX: The University of Texas Center of Excellence for Patient Safety Research and Practice; 2004.
Technical Report No. 04–01
J Patient Saf 2005, 1:33-40. Publisher Full Text
Jt Comm J Qual Patient Saf 2007, 33:699-703. PubMed Abstract
Zohar D: Safety Climate: conceptual and Measurement Issues. In Handbook of Occupational Health Psychology. Edited by Quick JC, Tetrick L. Washington, D.C: American Psychological Association; 2003:123-142.
Sociol Methods Res 1983, 11:325-344. Publisher Full Text
The pre-publication history for this paper can be accessed here: