When eating healthy is not healthy: orthorexia nervosa and its measurement with the ORTO-15 in Hungary

Abstract

Background

For a better differential diagnosis of eating disorders, it is necessary to investigate their subtypes and develop specific assessment tools to measure their specific symptoms. Orthorexia nervosa is an alleged eating disorder in which the person is excessively preoccupied with healthy food. The ORTO-15, designed by Donini and colleagues, is the first and only at least partially validated instrument to measure this construct. The aims of the present study were to examine the psychometric properties of its Hungarian adaptation (ORTO-11-Hu), and to investigate its relationship to food consumption and lifestyle habits in order to contribute to a better description of the phenomenon.

Methods

The ORTO-11-Hu, a lifestyle habits questionnaire, a food choice list indicating foods the participants choose to consume, and ten additional orthorexia-related questions were administered to a group of 810 Hungarian participants (89.4% female) aged between 20 and 70 (M = 32.39 ± 10.37 years).

Results

Confirmatory factor analysis suggested a single factor structure for the 11-item shortened version of the instrument. Internal consistency of the measure was adequate (Cronbach’s alpha = 0.82). No significant differences were found between males and females on the ORTO-11-Hu. Age and body mass index were significantly associated with a tendency towards orthorexia nervosa. Additional orthorexia-related features were significantly correlated with ORTO-11-Hu scores: orthorexia nervosa tendency was associated not only with healthier food choices (eating more whole wheat cereals, less white wheat cereals, more fruit and vegetables) but with shopping in health food stores, as well as with some healthy lifestyle habits (more sports activity, specific dietary behaviors, and less alcohol intake). Individuals with higher orthorexia nervosa tendency also reported a greater tendency to advocate their healthy diet to their friends and family members.

Conclusions

These results provide evidence for the reliability of ORTO-11-Hu and some support for the construct validity of the instrument. The present study also contributes to the establishment of (diagnostic) criteria for this new subtype of eating disorders.

Background

To achieve a better differential diagnosis of eating disorders, investigating the various subtypes and developing specific assessment tools to measure specific symptoms is necessary. Orthorexia nervosa (ON) is an alleged eating disorder in which the person is excessively preoccupied with healthy food. First described by Bratman in 1997, ON entails a fixation on healthy food or a health food dependence [1]. The term orthorexia nervosa arises from the words orthos (=accurate) and orexis (=hunger) meaning obsession with healthy food and proper nutrition. Fears and worries about health, eating, and the quality of food are significant.

In extreme cases, the obsessive and compulsive characteristics of ON become pathological and dominate a person’s life. The preoccupation with quality of food and eating healthy comprise the principal elements of this disorder. The pathological obsession with biologically pure food and shops which sell it leads to a special lifestyle. Stringent dietary restrictions and eating plans, combined with a personality and attitude of superiority and obsessive-phobic behavioral characteristics define the core of ON. Transgressing the dietary rules leads to intense anxiety, feelings of guilt and shame and is followed by even more stringent dietary restrictions [1, 2].

At present, ON is not a formal disorder, but more and more clinical reports related to it have appeared as it becomes more and more familiar to health care professionals [3]. There is no consensus on the categorization of ON among mental disorders or even if it is a mental disorder. ON was debated on whether or not it should have a place in the recently published DSM-5 [4]. In sum, ON is a new and controversial concept.

Two instruments have been developed to assess ON. The Bratman test is based mostly on clinical experience, and its validity has not been investigated by the author himself or by others. In 2005, Donini and colleagues developed the ORTO-15 based on Bratman’s test [5]. However, this instrument also includes items not specifically characteristic of ON making the face validity of the test doubtful. The ORTO-11 scale, the only foreign language (Turkish) adaptation of the ORTO-15 to date, was designed after omitting four non-ON-specific items based on statistical considerations [6]. In addition, the psychometric properties of the ORTO-15 have only been partially examined by the original test authors. Due to not having a well-defined group of individuals with ON for this purpose, Donini and colleagues proposed a cutoff point to distinguish between individuals with or without ON based on the interrelationship of ORTO-15 scores with obsessive-phobic personality traits as measured by the Minnesota Multiphasic Personality Inventory. A further limitation of the documentation is the inadequate description of the scoring method.

Some clinical studies have also been carried out to better understand the characteristics of ON [7, 8]. These suggest that the difference between an ON sufferer and a health-conscious person include the extreme preoccupation with and the judging attitude towards others who do not follow a healthy diet. These studies also provide recommendations for distinguishing ON from anorexia nervosa, psychosis, and poisoning fears.

The aim of the present study was to describe the adaptation process of the ORTO-15 into Hungarian and to investigate the psychometric properties of this version in a large Hungarian sample. We investigated the relationship between the ORTO-15 and ON features such as food choices and nutritional supplement use, alcohol, drug, and cigarette use, and some demographic variables, and added supplemental ON specific questions to examine external criteria for ON. We hypothesized that individuals who have a tendency for ON, in addition to making healthier food choices, refuse to use drugs, alcohol, cigarettes, and take more nutritional supplements, engage in more sports activity and have more dietary restrictions than those without a tendency for ON. Beyond the psychometric aspects, we also intended to improve the knowledge in the field concerning cultural differences, the investigating of which is largely missing from the literature on ON.

Methods

Sample and procedure

The study protocol was approved by the Ethics Committee of Semmelweis University (registration number 4498–0/2011-EKU; 410/PI/11). Participants were recruited from different universities in Hungary: students from Semmelweis University, Eötvös Loránd University, the University of Pécs, and the University of Debrecen. The data were collected via internet using the online test battery developed by the authors. Participants received the link to the research called ‘Nutrition and health’ in an email from the secretary at each university. In total, 810 individuals participated in the study (724 women, 86 men; 89.4% vs. 10.6%). The mean age of the respondents was 32.39 ± 10.37 years (31.48 ± 11.68 for males and 30.38 ± 10.20 for females). Body-mass index of the participants ranged between 14.88 and 56.06, with a mean of 23.20 ± 4.89.

The majority of participants comprised students from the following areas: dietetics (N = 69, 8.5%), medicine (N = 107, 13.2%), other health-related professions (nursing, physiotherapy, midwifery, optometry, physical education, and sport sciences; N = 240, 29.6%), and from other fields (sociology, social work, teaching; N = 33, 4.1%). Other participants included graduated professionals: dieticians (N = 26, 3.2%), medical doctors (N = 18, 2.2%), other healthcare professionals (N = 55; 6.8%), and non-healthcare professionals (N = 262, 32.3%).

All instruments were administered online in order to reach as many participants as possible. Participants completed an informed consent form, entered demographic information, filled in the Hungarian translation of the ORTO-15 and a checklist of food choices, and several ON-specific questions developed for the purpose of this study – based on the clinical experiences and observations from other authors [1, 2, 7]. Finally, participants were asked about their lifestyle (sports activity, dieting) and other foodstuff consumption habits (nutritional supplements, medication, alcohol, drugs, and smoking).

Instruments

Orthorexia nervosa

ORTO-15 is a measure instrument for ON comprising 15 multiple-choice items [5]. This instrument is based on Bratman’s orthorexia model combined with MMPI obsessive-phobic personality trait questions [1, 9]. ORTO-15 is a self-report questionnaire with a 4-point Likert-scale (always, often, sometimes, never). The items address the selection, preparation, consumption, and effect of, and attitude towards presumed to be healthy food. The original test authors assumed three underlying factors: the cognitive-rational (items 1,5,6,11,12,14), the clinical (items 3,7,8,9,15) and the emotional (items 2,4,10,13) components of ON [5]. The authors also tested the efficacy, sensitivity, specificity, positive and negative predictive value of three different threshold values for ORTO-15: <35, <40, and <45. Compared to the other two cutoff points, 40 (lower scores refer to more ON features) was considered to be the most appropriate for distinguishing between individuals with and without ON. In one sample, this cutoff had 73.8% efficacy, 55.6% sensitivity, 75.8% specificity, 20.5% positive predictive value, 93.8% negative predictive value; while in another sample it showed 75% efficacy, 100% sensitivity, 73.6% specificity, 17.6% positive predictive value and 100% negative predictive value [5].

For developing the Hungarian version of the ORTO-15, the first and the third author of the present study translated ORTO-15 into Hungarian. No new expressions were added nor were changes made to the original structure. The final version was created by using the expert opinions of two clinical psychologists. A back translation was conducted by an independent bilingual translator, and was compared to the original test by yet another professional; no significant changes were necessary. The final version of the scale was administered to 20 people to evaluate the clarity of the items.

Lifestyle habits

Participants were asked about their sports activity, dietary restrictions, caffeine, alcohol, drug, cigarette, medication, and nutritional supplement use. Frequency of sports activity was assigned to the following categories: do not engage in any sport, casually, regularly as a hobby, compete regularly. Smoking status was coded as current, former, and nonsmoker. Alcohol, caffeine, medication, and nutritional supplement use were addressed with yes/no type questions.

Food choice list

Individuals were asked about the type of foods they usually eat. Food categories were developed by a dietician based on the nutrition pyramid [10]. Participants rated the frequency of their consumption of given food types. The food choice list contained the following food categories: whole wheat bread, white bread (rated on a 3-point scale: daily, weekly, monthly or rarely), and vegetables plus fruits (rated on a 6-point Likert-scale: several times a day, once daily, several times per week, once a week, once a month, never, or very rarely). Individuals were also asked whether and how often they shop at stores which sell primarily healthy food (regularly, occasionally, never).

Supplemental orthorexia nervosa-related questions

Ten further questions were added to the ORTO-15 based on the descriptions of reviews and clinical reports about ON [1, 2, 7]. These were related to food consumption habits (so-called health food, timing, food color choices), prejudices and attitudes related to obesity and problems with controlling desires. Individuals had to rate whether they agree or not with the items.

Supplemental orthorexia nervosa-related questions used in this study:

• 1. I consume only healthy foods.

• 2. I always eat according to my eating schedule.

• 3. Sexuality plays an important role in my life.

• 4. Being overweight is a sign of weakness.

• 5. I avoid food with specific colors.

• 6. I disapprove of people who cannot overcome their desires.

• 7. I think most people can be blamed for their own diseases.

• 8. I always eat the same meals.

• 9. I am critical of people who don’t follow the rules of a healthy lifestyle.

• 10. I spend a large amount of time preparing my meals.

Statistical analyses

In order to evaluate the factor structure of the Hungarian version of the ORTO-15, four confirmatory factor analytic models were tested. Model 1 was a three-factor (‘cognitive-rational attributes’, ‘clinical attributes’, and ‘emotional attributes’) model based on the assumptions of the original authors of the instrument [5]. Model 2 was a single-factor model with one factor responsible for all 15 item responses. Since neither of the two models adequately fit the data, a shortened version of the instrument was developed by omitting four items with the lowest item-total correlations and factor loadings. The factor structure of this 11-item version was evaluated by a third confirmatory analytic model using a single-factor solution. Finally, a fourth model was also evaluated containing covariances suggested by modification indices from Model 3.

Internal reliability of the original and the shortened versions was evaluated by calculating Cronbach’s alpha coefficients. Normality of the continuous variables was tested by the Shapiro-Wilk W statistics. Because of the non-normal distribution of our continuous variables, relationships among them were analyzed using nonparametric methods. The Mann–Whitney and the Kruskal-Wallis tests were applied to evaluate the relationships between categorical and continuous variables, while Spearman correlation coefficients were calculated to estimate the strength of the associations between two continuous variables. Confirmatory factor analyses were conducted using AMOS version 20.0, while all other statistical procedures were carried out using SPSS 20.0 software.

Results

Factor structure, item analysis, and reliability

First, we tested whether our data fit well with the factor structure hypothesized by the original authors of the instrument (Figure 1). The confirmatory factor analytic investigation revealed that the three-factor solution (Model 1), suggested by Donini and colleagues, had to be rejected (χ² = 931.2; p < 0.001; CMIN/DF = 10.7; CFI = 0.72; TLI = 0.66; RMSEA = 0.11; PCLOSE < 0.001). The single-factor solution (Model 2) showed slightly lower goodness of fit indices (χ² = 1080.8; p < 0.001; CMIN/DF = 12.0; CFI = 0.67; TLI = 0.62; RMSEA = 0.12; PCLOSE < 0.001); therefore, this model was rejected as well.

Figure 1

The factor structure of ORTO-15 as hypothesized by Donini and colleagues (displayed values are unstandardized regression weights from the present sample). Legend: Big circles represent factors, squares represent items, while small circles represent error terms.

Because of the poor fit indices of the first two models, an item analysis was conducted to evaluate the appropriateness of each item (Table 1). Based on the particularly low item-total correlations and factor loadings from the second confirmatory factor analytic model, we decided to omit item numbers 5, 6, 8, and 14. A further confirmatory factor analysis was conducted to examine the goodness of fit of the shortened version’s single-factor structure. Although this solution showed better fit indices (χ² = 530.8; p < 0.001; CMIN/DF = 12.1; CFI = 0.81; TLI = 0.76; RMSEA = 0.12; PCLOSE < 0.001) compared to the previous two models, the indicators still remained unsatisfactory. Therefore the variances of the error terms were analyzed through modification indices. Following the cutoff criteria of modification indices equal to or higher than 50, three covariances between error terms were incorporated into a fourth model generating a new single-factor structure (Figure 2). The error terms that were correlated were items (modification indices between parentheses) 10/11 (116.5), 10/12 (107.0), and 11/12 (96.4). The fit indices of this fourth model proved to be acceptable (χ² = 230.8; p < 0.001; CMIN/DF = 5.63; CFI = 0.92; TLI = 0.90; RMSEA = 0.076; PCLOSE < 0.001). Internal consistency of both the original 15-item and the shortened 11-item version proved to be adequate (Cronbach’s alpha = 0.78 and 0.82, respectively). The full text of the original ORTO-15 and its Hungarian version, ORTO-11-Hu can be found in the Table 2.

Table 1 Results of the item analysis of ORTO-15 in our sample

Figure 2

The best fitting, final model of the shortened, 11-item Hungarian version of ORTO-15 (displayed values are standardized regression weights). Legend: Big circles represent factors, squares represent items, while small circles represent error terms.

Table 2 Full text and scoring grid of the original ORTO-15 and the Hungarian version, ORTO-11-Hu

Descriptive statistics and relationships with sociodemographic variables

The mean score on the ORTO-11-Hu was 28.83 with a standard deviation of 3.15, while that of the original 15-item version was 37.87 ± 3.93. Using the original cut-off point of 40 on the original 15-item version, 74.2% (N = 601) of this study’s sample had a tendency for ON. Further, no significant sex differences were found on the ORTO-11 (females vs. males: 28.81 ± 3.14 vs. N. 29.02 ± 3.25). The relationship of ORTO-11-Hu scores with age and body mass index was also examined. There was a statistically significant but weak correlation present between orthorexia scores and age (r = 0.13, p < 0.01) but no significant association was found concerning body mass index.

Relationships to other ON-related constructs as support for construct validity

Health behaviors

Participants engaging in more sports activity had lower ORTO-11-Hu scores (meaning a higher tendency for ON) compared to people who did not engage in any sport. There were no significant differences in ORTO-11-Hu scores with respect to smoking status, caffeine use, and medicine and nutrition supplement intake. However, participants who never drank alcohol had a higher tendency for ON (lower scores) compared to others consuming alcohol. Individuals on special diets also showed significantly lower scores on ORTO-11-Hu (stronger ON tendency) than non-dieting individuals. Finally, individuals with a greater tendency to encourage relatives and friends to follow their supposedly healthy diet reported a higher tendency for ON compared to individuals who did not try to convince others (see Table 3 for details).

Table 3 Relationship of ORTO-11-Hu scores with health behaviors

Food choices

Individuals eating whole wheat cereals were more frequently characterized by higher ON tendency compared to others eating whole wheat cereals less often. In addition, participants who more frequently ate white wheat cereals scored higher on ORTO-11-Hu than others who ate white wheat cereals occasionally. Concerning vegetable and fruit consumption, only tendencies were observed in the expected directions: ORTO-11-Hu scores were lower (higher ON tendency) for individuals eating fruits and vegetables. Finally, ORTO-11-Hu scores were significantly lower in the groups of individuals who more often visited shops selling healthy food compared to those who did not shop at such stores (see Table 4 for details).

Table 4 Relationship of ORTO-11-Hu scores with food choices

Supplemental orthorexia nervosa-related questions

The data confirmed the link between a tendency towards ON and scores on the supplemental questions based on the clinical literature on this phenomenon. ORTO-11-Hu scores were significantly associated with the consumption of healthy food and eating according to a fixed schedule, indicating that respondents consuming mainly healthy food and following a strict eating schedule scored lower on ORTO-11-Hu (meaning a higher tendency for ON). Individuals who scored lower on ORTO-11-Hu considered being overweight more often as a sign of weakness, and were more prone to avoid foods with specific colors and to blame people for their diseases. In addition, individuals with a higher tendency for ON more often judged people negatively who could not overcome their desires and who did not follow the rules of healthy nutrition. Finally, respondents with more ON features had a higher tendency to eat the same food everyday and to spend a large amount of time with preparation of meals (see Table 5 for details).

Table 5 Relationship of ORTO-11-Hu scores with the supplemental orthorexia nervosa-related questions

Discussion

Orthorexia nervosa is a recently proposed eating disorder where the quality of food becomes an obsession. Until now, only some basic steps have been made to operationalize this construct: the psychometric status of the first and only measurement instrument (ORTO-15) for ON is still unclear. The aim of this study was to contribute to the scientific literature on the assessment of this possible new eating disorder and to describe its clinical features.

The confirmatory factor analytic findings of this study did not provide support for the three-factor structure recommended by the original test authors [5]. Our final analysis suggested a one-factor solution with 11 items (item numbers 1, 2, 3, 4, 7, 9, 10, 11, 12, 13, and 15) omitting item numbers 5, 6, 8, and 14 from the original ORTO-15 in this Hungarian sample (thus creating ORTO-11-Hu). Internal consistency of this shortened 11-item version proved to be adequate.

To provide our results with international context, it is worth mentioning that the Turkish adaptation of ORTO-15 also led to an 11 item-questionnaire with a one-factor solution. However, the item-composition of the Turkish version is somewhat different: it contains items 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14 from ORTO-15. The internal consistency of the Turkish ORTO-11 (0.62) [6] was lower than that of the Hungarian version in the present sample. Concerning item composition, the items that were dropped from the Turkish and the Hungarian questionnaire are different. The four items excluded from the Hungarian one were related to quality preference (item 5) and the financial aspect of food choices (item 6), the transgression of dietary rules (item 8) and beliefs about unhealthy food at the market (item 14). The Turkish version omitted the following items: the calorie aspect of food (item 1), finding food shopping confusing (item 2), mood specificity (item 9), and a preference for eating alone (item 15).

Cultural differences may explain at least some of these differences, including the fact that Turkish society is characterized by collectivism, contrary to Hungary which has a strongly individualist culture. Hungarians tend to view themselves mainly as individuals emphasizing the needs of a single person. In Turkey, however, people tend to view themselves mainly as members of groups (families, work units, tribes, nations), and usually consider the needs of the group to be more important than the needs of individuals [11]. Another important difference may be related to the Muslim religion that has different values compared to Christianity (restrictions in the consumption of certain food types, the meaning of fasting, attitudes towards hedonism, habits of eating in the family, etc.). These differences might also be associated with general cross-cultural differences in the expression of eating disorders, and with the availability and quality of different foods. Symptoms of ON may thus vary across different countries and cultures, which underlines the importance of further cross-cultural research on this phenomenon.

Our final confirmatory factor analytic model contained three covariances between error terms suggested by modification indices from previous analyses. The necessity of the inclusion of these associations might be explained by the fact that item numbers 10, 11, and 12 contain more words than the other items in the Hungarian version of ORTO-15. These items are less understandable because of their multiple complex structures (a characteristic of the Hungarian language). The beginning of these three sentences is also the same, and the end of the sentences contains the main message of the item. Further, all three sentences contain the expression ‘eating healthy food’. A further relevant factor can be the difficulty of distinguishing between self-esteem, lifestyle, and appearance in the context of eating healthy food.

Concerning the cutoff score, highly unexpected results were found in the present sample. Setting the cutoff score at 40 for defining the occurrence of ON, as suggested by the original test authors [5], yielded a 74.2% point prevalence in our sample (lower scores refer to higher tendency for ON). This high prevalence rate is similar to the prevalence rates found in other studies employing the same cutoff point [12–15]. The reason for such a high percentage can be either that the cutoff point is not correct or the validity of the scale is poor. The finding that scores on the ORTO-11-Hu were significantly associated with ON-related behaviors in our sample supports the previous opinion of the inadequacy of the cutoff point in addition to the construct validity of the instrument. Further research should contribute to the establishment of a reliable cutoff point to distinguish between individuals with and without ON.

The present findings also showed that the association between ORTO-11-Hu scores, age, and body mass index was statistically significant but negligible. These results are in line with the diverse results of other studies which reported no differences in age [6] or body mass index [6, 13, 16, 17]. In other studies, lower age was found to be associated with higher scores on ORTO-15 [12, 14]. However, these studies were conducted among medical students, so the level of education may be a moderator in this relationship: several studies report that an earlier stage of university studies is associated with orthorectic features but this relationship weakens towards the end of studies [18]. Only the study of Donini and colleagues [5] reported data inconsistent with these results. Consistent with our present findings, most of the former studies also did not find a significant association between ON features and body mass index [6, 13, 16, 17]. As some authors suggest, body mass index can predict orthorectic behaviors in combination with other variables such as medical reasons, dieting, and healthy nutrition [6]. However, a study reported higher body mass index scores associated with lower ORTO-15 scores among medical students [14], meaning that higher body mass index is related to orthorectic behaviors in this special population.

In the present Hungarian sample, there were no significant differences between male and female individuals, in contrast to the other studies where females had a higher tendency for ON [5, 6, 14, 19]. Because males were underrepresented in our sample, these results provide less reliable data to assess gender-related differences. Further studies should clarify whether our results can be explained by the low rate of male respondents or they reflect some true cultural specialty.

In line with our expectations, individuals in our sample with stronger ON tendency reported eating more whole wheat cereals and less white wheat cereals and more frequent shopping in healthy food stores. The orthorectic features were also related to lifestyle habits such as regular sports activity, more dietary restrictions and less alcohol intake and an inclination to persuade others about the importance of a healthy diet. These results are similar to the descriptions of Bratman [1] regarding ON which includes not only eating habits but other associated lifestyle characteristics.

At present, no comprehensive and standardized criteria for ON exists. Our knowledge in this regard is limited to the description of Bratman, some clinical observations by others, and a few explorative studies conducted with convenience samples from a small number of countries (for a detailed review, see [20]). Since there are no specific external criteria or other validated assessment tools for ON, evaluating the validity of the present instrument is impossible in a strict sense. However, it seemed to be reasonable to use some supplemental questions based on the clinical literature to perform a preliminary validation of the investigated measurement instrument. In line with our expectations, these ON features were significantly associated with ORTO-11-Hu scores supporting some evidence for the validity of the scale. Individuals with higher frequency of eating health food, having a schedule for eating, eating food with a particular color, eating mostly the same type of food and spending lots of time preparing meals showed a stronger tendency for ON. Further, individuals who found sexuality not to be important, considered being overweight as a sign of weakness, and disapproved of people who could not overcome their desires also had a higher tendency for ON. This was also the case for those who believed that people should be blamed for their own diseases, and those who criticized people who did not follow the rules of a healthy lifestyle. These results provide some further support for the construct validity of the ORTO-11-Hu.

Our study has several strengths and limitations. Internet-based sampling may have inherent selection biases towards a younger and more educated sample. Further, the connotation of the name of the study ‘Nutrition and health’ may have attracted individuals for whom nutrition and healthy eating are more important in their everyday life than for individuals in the general population. In addition, all results were based on self-reports not enabling confirmation of the accuracy of responses, which could be especially distorted concerning health behaviors or food choices. However, to the best of our knowledge, this is the first study in the literature investigating the factor structure of an ON scale using confirmatory factor analytic techniques. A further strength of the study is the relatively large sample consisting of individuals with diverse characteristics across occupation and age. In addition, this is the first study to investigate correlates of ON in a diverse and large sample from Eastern Europe.

Conclusions

We can conclude that the Hungarian version of the ORTO-15 questionnaire has adequate psychometric characteristics, thus providing preliminary support for the use of this instrument. Further research from other countries is needed to improve our knowledge on the psychometric characteristics of the instrument and to better understand the nature and correlates of ON. Future research in this area should identify the place for ON among mental disorders, presumably on the spectrum of eating and obsessive compulsive disorders.