Research article | Open | Open Peer Review | Published:
Physicians’ attitudes and knowledge concerning antibiotic prescription and resistance: questionnaire development and reliability
BMC Infectious Diseasesvolume 16, Article number: 7 (2016)
Understanding physicians’ antibiotic-prescribing behaviour is fundamental when it comes to improving antibiotic use and tackling the growing rates of antimicrobial resistance. The aim of the study was to develop and validate -in terms of face validity, content validity and reliability- an instrument designed to assess the attitudes and knowledge underlying physician antibiotic prescribing.
The questionnaire development and validation process comprised two different steps, namely: (1) content and face validation, which included a literature review and validation both by physicians and by Portuguese language and clinical psychology experts; and (2) reliability analysis, using the test-retest method, to assess the questionnaire’s internal consistency (Cronbach’s alpha) and reproducibility (intraclass correlation coefficient - ICC). The questionnaire includes 17 items assessing attitudes and knowledge about antibiotic prescribing and resistances and 9 items evaluating the importance of different sources of knowledge. The study was conducted in the catchment area covered by Portugal’s Northern Regional Health Administration and used a convenience sample of 61 primary-care and 50 hospital-care physicians.
Response rate was 64 % (49 % to retest) for primary-care physicians and 66 % (60 % to retest) for hospital-care physicians. Content validity resulted in 9 changes to professional concepts. Face validity assessment resulted in 19 changes to linguistic and interpretative terms. In the case of the reliability analysis, the ICC values indicated a minimum of fair to good reproducibility (ICC > 0.4), and the Cronbach alpha values were satisfactory (α > 0.70).
The questionnaire developed is valid -in terms of face validity, content validity and reliability- for assessing physicians’ attitudes to and knowledge of antibiotic prescribing and resistance, in both hospital and primary-care settings, and could be a very useful tool for characterising physicians’ antibiotic-prescribing behaviour.
Rates of antimicrobial resistance are growing worldwide, threatening public health and increasing morbidity, mortality and healthcare costs [1–3]. Research suggests that what underlies such high rates of antimicrobial resistance is the misuse of antibiotics in both human and veterinary practices [4–7].
Bearing in mind that antibiotic consumption in southern European countries is higher than that in the north of Europe , without obvious benefit to public health , it becomes fundamental to understand which factors underlie antibiotic misuse. Considering the pivotal role that physicians play in this process, an in-depth understanding of their attitudes to and knowledge of antibiotic prescribing and antimicrobial resistance is essential when it comes to developing effective interventions and improving antibiotic use [10, 11].
Two recent systematic reviews have highlighted physicians’ attitudes and knowledge as factors affecting physician antibiotic prescribing behaviour [12, 13]; those reviews were based on several qualitative and quantitative studies in different settings; however, any tool have been developed, validated and published which, based on these studies, allow researchers, health professionals and health authorities to measure these specific determinants of antibiotic prescribing.
Questionnaires are well-established tools for collecting data in health sciences and could be a very useful instruments for assessing physicians’ clinical-practice characteristics  and understanding their antibiotic-prescribing behaviour.
Several scales have been published for assessing antibiotic misuse and overuse ; these scales evaluate the relationship between patients’/parents’  or physicians’  characteristics and antibiotic misuse in the community. According to Alumran et al., however, such scales display weaknesses in development and assessment of validation, which hinder their use in different settings or countries . Measurement of factors underlying physician misprescription of antibiotics should be made using reliable scales because these factors are extremely difficult to identify and assess.
Indeed, several parameters should be assessed to ensure the usefulness of the data collected, so that each question measures what it is intended to measure, all words and sentences are clearly understood, and all participants interpret the questions in the same way .
This study sought to develop and validate -in terms of face validity, content validity and reliability- an instrument designed to assess the attitudes and knowledge underlying physician antibiotic prescribing, in order to understand and improve antibiotic use in both hospital- and primary-care settings.
A questionnaire was developed with a dual purpose, namely: to assess physicians’ attitudes and knowledge vis-à-vis antibiotic prescribing, antibiotic use and antibiotic resistance, and the usefulness of different sources of knowledge used in clinical practice; and to collect socio-demographic and professional data on the physicians surveyed, including age, gender, medical specialisation, workplace and workflow. The final questionnaire took the form of a two-page long, self-administered document that was easy to complete, not time-consuming, and made up of the following five sections:
Section 1: instructions to complete the questionnaire;
Section 2 (“Antibiotics and Resistance”): 17 statements regarding attitudes and knowledge about antibiotic prescribing, antibiotic use and antimicrobial resistance;
Section 3 (“In the treatment of respiratory infections, how would you rate the usefulness of each of these sources of knowledge?”): 9 statements regarding the importance of various sources of knowledge, which can helps to understand the sources of knowledge underlying antibiotic misprescription;
Section 4: Socio-demographic and professional data (age, gender, medical specialisation, workplace and workflow);
Section 5: free to express ideas and views on antibiotics and resistance.
In sections 2 and 3, agreement with statements was measured using a horizontal, continuous, visual analogue scale (VAS), 8 cm long and unnumbered, scored from full disagreement to full agreement.
Figure 1 depicts the different steps of the study. The study was previously approved by the Northern Regional Health Administration (Permit No. 010484/ 2011), primary-care facility directors and Hospital Administration & Ethics Committee (Permit Date 10/2011). Authorisation from the Portuguese Data Protection Authority (Comissão Nacional de Proteção de Dados/ CNPD) was also obtained (Permit No. 2886/ 2013).
Step 1. Content and face validity of the questionnaire
Content validity was evaluated in two different stages, in accordance with published guidelines :
The development stage, which included the build-up of the questionnaire and determination of the domain and concepts of the construct of interest. This stage consisted of a literature review [12, 13] and previous studies undertaken by a collaborative group in Spain . It is important to refer that section 2 (17 statements about antibiotics and resistances) resulted from the literature review and concern the most important attitudes identified as influencing antibiotic prescribing. On the other hand, section 3 present the sources of knowledge identified in the literature review as being used by physicians nowadays.
The judgment stage, in which the professional opinion of experts was evaluated. A convenience sample of two groups of physicians, one consisting of five primary-care physicians and the other of five specialists (2 internists, 2 paediatricians and 1 clinical pathologist) pre-tested the questionnaire and assessed the accuracy, clinical terminology, completeness and meaning of all the statements. An additional panel of two pharmacologists and two pharmacoepidemiologists also evaluated the content validity of the questionnaire.
Face validity, which includes an assessment of the grammar, syntax, organisation, appropriateness and logical sequence of the statements , was evaluated by two university professors, one an expert in clinical psychology and the other an expert in the Portuguese language.
Step 2. Reliability analysis
For an instrument to be valid, it must first be reliable . Reliability refers to the consistency of a test or measurement . To assess questionnaire reliability, two aspects were addressed, i.e., reproducibility and internal consistency.
In September 2013 we used the test-retest method to conduct a pilot test in a so-called NUTS II area (Nomenclatura das Unidades Territoriais para Fins Estatísticos/Nomenclature of Territorial Units for Statistics) of Portugal defined by the Northern Regional Health Administration (Administração Regional de Saúde do Norte, I.P./ARS-N), using a convenience sample of 61 primary-care and 50 hospital-care physicians. The questionnaire, accompanied by an explanatory letter outlining the study, was personally delivered to the head of each unit who then proceeded to distribute it to all participants who agreed to take part. In line with previous studies [23, 24], an interval of 2 to 4 weeks was allowed to elapse between the first and second administration of the questionnaire.
Agreement between answers was assessed by calculating the intraclass correlation coefficient (ICC), a measure of reproducibility of repeated measures on the same subject . A one-way ICC was chosen because the effect of trials is not cross-checked against subjects (replication study; hence one-way) and the analysis is used to generalise from a trial that used a sample (thus random) .
Internal consistency was evaluated using Cronbach’s alpha , which is a very useful parameter to describe the extent to which all the items in a test measure the same concept or construct, and it is thus connected to the inter-relatedness of the items within the test . When multiple traits underlie the items on a scale, the “tau equivalent model” assumption is not respected and Cronbach’s alpha underestimates internal consistency . Since Section-2 statements evaluate different concepts (physicians’ attitudes and knowledge), assessing Cronbach’s alpha may be inappropriate according to the assumptions of this parameter. On the other hand, in view of the fact that Section 3 of the questionnaire exclusively evaluates one concept (the utility of different sources of knowledge), Cronbach’s alpha would seem to be a very useful parameter for assessing the internal consistency of this section.
The response rates for the two groups of physicians studied were as follows: 64 % (n = 39) for the first questionnaire administered and 49 % (n = 30) for the retest among the 61 primary-care physicians, and 66 % (n = 33) and 60 % (n = 30) respectively among the 50 hospital-care physicians.
Based on data drawn from Section 4 of the questionnaire, Table 1 gives a breakdown of the socio-demographic and professional characteristics of the physicians included in the study. Notwithstanding the results reported above on the participants’ socio-demographic and professional characteristics, this study did not seek to evaluate the influence of any of these variables.
Step 1. Content and face validity of the questionnaire
The content-validity judgment stage consisted of the pre-test and professional appraisal of the questionnaire. It was assessed by two groups of physicians who made 9 changes (e.g., in Statement 6, “The use of antibiotics on animals is an important cause of the appearance of new resistance” was amended to read, “The use of antibiotics on animals is an important cause of the appearance of new resistance to pathogenic agents in humans”).
Face-validity assessment resulted in 19 changes to linguistic and interpretative terms in the questionnaire, e.g., Statement 2 was changed from “In a primary-care context, one should wait for the microbiology results to treat an infectious disease” to “In a primary-care context, one should wait for the microbiology results before treating an infectious disease.”
Step 2. Reliability analysis
Table 2 shows the ICC and Cronbach’s alpha results if any given item is deleted for both groups of physicians included in the study.
The Cronbach alpha values yielded for Section 3 of the questionnaire were satisfactory (α > 0.70)  for both groups of physicians studied (αHospital-care physicians: 0.783; αPrimary-care physicians: 0.770).
Applying the definition proposed by Roster B. , ICC values smaller than 0.4 can be taken to indicate poor reproducibility, those ranging from 0.4 to 0.75 to indicate fair to good reproducibility, and those above 0.75 to indicate excellent reproducibility. On this basis, all statements displayed a minimum of fair to good reproducibility in both groups of physicians studied and in both sections of the questionnaire. Eight (Section 2) and four (Section 3) statements by primary-care physicians and ten (Section 2) and two (Section 3) statements by hospital-care physicians study yielded excellent ICC values.
When comparing the ICCs obtained for the two groups of physicians, considerable differences were found with respect to six statements in Section 2 (S4, S6, S9, S10, S14 and S16) and four statements in Section 3 (S1’, S2’, S3’ and S7’).
Assessing knowledge and attitudes that guide antibiotic prescribing is an essential step when it comes to counteracting the problem of antimicrobial resistance. Several scales have been developed to measure the factors associated with antibiotic misuse world-wide [29–31] but most of them have not been fully validated . This study reports on the development process and reliability evaluation of a questionnaire (Additional file 1) designed to assess physicians’ knowledge and attitudes concerning antibiotic use. The results yielded show that the questionnaire: (i) has content validity, face validity and is reliable in terms of internal consistency and reproducibility over time; and, (ii) could be applied to physicians working in both primary-care and hospital-care settings.
In terms of public health issues and related policies, this questionnaire for assessing physicians’ knowledge and attitudes offers several advantages over other published instruments, in that: (i) it allows for assessment, identification and description of attitudes and knowledge regarding antibiotic prescribing and resistance among hospital- and primary-care physicians simultaneously, and this in turn allows for comparison between the two groups of professionals; and, (ii) it uses a VAS which is a highly interesting scale for assessing small differences between physicians and between groups of physicians.
Indeed, considering the growing rates of antimicrobial resistance and decreasing effectiveness of antibiotics, antibiotic misuse is a major public health issue around the world, to the point where it is threatening a return to the pre-antibiotic era. Bibliographic reviews [12, 13] of physician antibiotic-prescribing behaviour have highlighted intrinsic factors, such as their socio-demographic characteristics, attitudes and diagnostic uncertainty, as being at the root of antibiotic misprescription. The literature also points the collegial advice and the logistics of microbiology test results as factors affecting antibiotic prescribing . This questionnaire now enables factors -such as attitudes- related with antibiotic misuse to be identified.
Several interventions have been implemented around the world, targeting at improving antibiotic prescribing among primary-care  and hospital-care  professionals. Considering that these interventions should reflect the characteristics and barriers present in the setting where they are to be implemented, this questionnaire is an adequate instrument for assessing the factors affecting physician-prescribing behaviour, an essential pre-requisite for developing effective educational interventions and improving antibiotic use. When we aim to assess and improve attitudes influence on antibiotic prescribing, we can also use this questionnaire to evaluate the intervention effect (pre- vs post-assessment) on these determinants. Finally, use this questionnaire in different countries/ settings will also allows to compare the influence of this factors in different realities, which could help to understand different rates of antibiotic prescribing quality indicators in different countries.
Visual analogue scales are well-established instruments for collecting data in several different areas [35–37] and numerous studies have supported their usefulness, validity and reliability, when compared to Likert scales [38, 39]. While both scales capture similar information, a VAS assesses and enables the quantification of subjective phenomena, and might be able to discern subtle differences because of its greater range of possible scores [40, 41].
Face validity, as a form of validity that refers to a subjective assessment , and content validity, as a measure of the comprehensiveness and representativeness of a scale’s content , were assessed and guaranteed by the panels of experts, the literature review  and a qualitative study conducted by a collaborative group .
With regard to the methodology used in the pilot study (test-retest), the main problem lies in the potential for learning or recall effects that can affect the test . The 2-week interval was fundamental to ensure that the time was : (i) long enough to avoid carry-over effects due to memory; and, (ii) short enough to avoid changes in physicians’ knowledge or attitudes concerning antibiotic prescribing and antimicrobial resistance.
The high Cronbach’s alpha values obtained in Section 3 of the questionnaire (Primary-care physicians: 0.770; Hospital-care physicians: 0.783) revealed the internal consistency of the statements in this section, and we feel that this is linked to the objectivity of these statements, which exclusively evaluate the utility of different sources of knowledge.
A comparison of the results obtained for the two groups of physicians could be very useful for assessing the appropriateness of the questionnaire and its applicability in each setting. The usefulness of the questionnaire was not the same for both groups of physicians. We feel, however, that these differences were related to the specific characteristics of each group, e.g., the differences found in Statement 9 of section 2 (“In situations of doubt as to whether a disease might be of bacterial aetiology, it is preferable to prescribe an antibiotic.”) could well be related to the availability in a hospital setting of microbiological tests that are not to be found in primary-care facilities. As regards the differences seen in the Section 3 of the questionnaire, the fair ICCs obtained for statements about the information and courses provided by the pharmaceutical industry (S2 and S3), could be related to primary-care physicians’ greater exposure to sales representatives, something that is in line with the reported influence of pharmaceutical companies in primary-care physician training .
This study’s main limitation lay in the size and selection of the sample. Even so, 30 participants in each group of physicians would seem to be a reasonable number for a pilot study where the purpose is preliminary survey or scale development . However, this sample size doesn’t allow to confirm the construct validity using exploratory factor analysis, which should be performed in future research with larger samples. While sample selection by a convenience method is the most widely-used process in validation studies, it nevertheless constituted our study’s main limitation . TTo evaluate the construct validity of the questionnaire, a larger sample size is needed; however, to confirm the structure presented, future research should also evaluate this using exploratory factor analysis.
Bearing in mind that antibiotic resistance is one of the main public health problems world-wide, this questionnaire would seem to be a very useful tool for collecting data on physicians’ attitudes to and knowledge of antibiotic prescribing and resistance, which is one of the most important factors underlying this global concern. Hence, this questionnaire should be used to assess factors underlying physician-prescribing behaviour in hospital and primary care, along with other possible variables, such as socio-demographic or professional-practice variables (medical specialisation, workplace and workflow). We believe that the use of this questionnaire could prove of the utmost importance when it comes to identifying factors underlying antibiotic prescription and antimicrobial resistance in both hospital and primary care.
Questionnaire development would seem to be essential for improving the likelihood of success in achieving the main study objective, which, in this case, is to identify factors underlying antibiotic prescription and antimicrobial resistance. This study presents a very comprehensive questionnaire which could be used on both primary- and hospital-care physicians.
World Health Organization. Antibiotic resistance. Geneva: WHO; 2012 [Fact sheet No. 194].
Howard DH, Scott RD. The economic burden of drug resistance. Clin Infect Dis. 2005;41 Suppl 4:S283–6. doi:10.1086/430792.
Kollef MH. The importance of appropriate initial antibiotic therapy for hospital-acquired infections. Am J Med. 2003;115(7):582–4.
Albrich WC, Monnet DL, Harbarth S. Antibiotic selection pressure and resistance in Streptococcus pneumoniae and Streptococcus pyogenes. Emerg Infect Dis. 2004;10(3):514–7. doi:10.3201/eid1003.030252.
Wise R, Hart T, Cars O, Streulens M, Helmuth R, Huovinen P, et al. Antimicrobial resistance. Is a major threat to public health. BMJ. 1998;317(7159):609–10.
Goossens H, Ferech M, Vander Stichele R, Elseviers M. Outpatient antibiotic use in Europe and association with resistance: a cross-national database study. Lancet. 2005;365(9459):579–87. doi:10.1016/S0140-6736(05)17907-0.
Tseng R. An audit of antibiotic prescribing in general practice using sore throats as a tracer for quality control. Public Health. 1985;99(3):174–7.
European Centre for Disease Prevention and Control. Antimicrobial resistance surveillance in Europe 2011. Annual Report of the European Antimicrobial Resistance Surveillance Network (EARS-Net). Stockolm: ECDC; 2012.
Butler CC, Hood K, Verheij T, Little P, Melbye H, Nuttall J, et al. Variation in antibiotic prescribing and its impact on recovery in patients with acute cough in primary care: prospective study in 13 countries. BMJ. 2009;338:b2242. doi:10.1136/bmj.b2242.
Tonkin-Crine S, Yardley L, Little P. Antibiotic prescribing for acute respiratory tract infections in primary care: a systematic review and meta-ethnography. J Antimicrob Chemother. 2011;66(10):2215–23. doi:10.1093/jac/dkr279.
Chlabicz S, Pytel-Krolczuk B. Antibiotic treatment for respiratory tract infections in Polish primary care facilities: is it time to change national guidelines or doctor prescribing behaviour? J Eval Clin Pract. 2008;14(3):470–2. doi:10.1111/j.1365-2753.2007.00865.x.
Teixeira Rodrigues A, Roque F, Falcao A, Figueiras A, Herdeiro MT. Understanding physician antibiotic prescribing behaviour: a systematic review of qualitative studies. Int J Antimicrob Agents. 2013;41(3):203–12. doi:10.1016/j.ijantimicag.2012.09.003.
Lopez-Vazquez P, Vazquez-Lago JM, Figueiras A. Misprescription of antibiotics in primary care: a critical systematic review of its determinants. J Eval Clin Pract. 2012;18(2):473–84. doi:10.1111/j.1365-2753.2010.01610.x.
Den Heijer M, van Asperen CJ, Harris H, Nippert I, Schmidtke J, Bouhnik AD, et al. International variation in physicians’ attitudes towards prophylactic mastectomy - comparison between France, Germany, the Netherlands and the United Kingdom. Eur J Cancer. 2013;49(13):2798–805. doi:10.1016/j.ejca.2013.04.025.
Alumran A, Hou XY, Hurst C. Validity and reliability of instruments designed to measure factors influencing the overuse of antibiotics. J Infect Public Health. 2012;5(3):221–32. doi:10.1016/j.jiph.2012.03.003.
Belongia EA, Naimi TS, Gale CM, Besser RE. Antibiotic use and upper respiratory infections: a survey of knowledge, attitudes, and experience in Wisconsin and Minnesota. Prev Med. 2002;34(3):346–52. doi:10.1006/pmed.2001.0992.
Coenen S, Michiels B, Van Royen P, Van der Auwera JC, Denekens J. Antibiotics for coughing in general practice: a questionnaire study to quantify and condense the reasons for prescribing. BMC Fam Pract. 2002;3:16.
Taylor-Powell E. Pilot test your questionnaire. University of Wisconsin-Extension, Cooperative Extension. 2008. Available: https://www.team-psa.com/brfss/2012/pres/K_Trepanier_questionnaire.pdf.
Yaghmaie F. Content validity and its estimation. J Med Educ. 2003;3(1):25–7.
Vazquez-Lago JM, Lopez-Vazquez P, Lopez-Duran A, Taracido-Trunk M, Figueiras A. Attitudes of primary care physicians to the prescribing of antibiotics and antimicrobial resistance: a qualitative study from Spain. Fam Pract. 2012;29(3):352–60. doi:10.1093/fampra/cmr084.
Szklo M, Nieto J. Epidemiologia intermedia: conceptos y aplicaciones. Madrid: Ediciones Díaz de Santos; 2003.
Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005;19(1):231–40. doi:10.1519/15184.1.
dos Santos Pernas SI, Herdeiro MT, Lopez-Gonzalez E. da Cruz e Silva OA, Figueiras A. Attitudes of Portuguese health professionals toward adverse drug reaction reporting. Int J Clin Pharm. 2012;34(5):693–8. doi:10.1007/s11096-012-9675-6.
Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for test-retest reliability of health status instruments. J Clin Epidemiol. 2003;56(8):730–5.
Rosner B. Fundamentals of Biostatistics. 7th ed. Belmont: Thomson-Brooks/Cole; 2006.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.
Tavakol M, Dennick R. Making sense of Cronbach’ alpha. Int J Med Educ. 2011;2:53–5. doi:10.5116/ijme.4dfb.8dfd.
Martin Bland J, Altman DG. Statistics notes: Cronbach’s alpha. BMJ. 1997;314:572.
Liabsuetrakul T, Chongsuvivatwong V, Lumbiganon P, Lindmark G. Obstetricians’ attitudes, subjective norms, perceived controls, and intentions on antibiotic prophylaxis in caesarean section. Soc Sci Med. 2003;57(9):1665–74.
Gould IM, Mackenzie FM, Shepherd L. Attitudes to antibiotic prescribing, resistance and bacteriology investigations amongst practitioners and patients in the Grampian region of Scotland. Eur J Gen Pract. 2007;13(1):35–6. doi:10.1080/13814780601050798.
Weiss MC, Deave T, Peters TJ, Salisbury C. Perceptions of patient expectation for an antibiotic: a comparison of walk-in centre nurses and GPs. Fam Pract. 2004;21(5):492–9. doi:10.1093/fampra/cmh504.
Skodvin B, Aase K, Charani E, Holmes A, Smith I. An antimicrobial stewardship program initiative: a qualitative study on prescribing practices among hospital doctors. Antimicrob Resistance Infect Control. 2015;4:24. doi:10.1186/s13756-015-0065-4.
Arnold SR, Straus SE. Interventions to improve antibiotic prescribing practices in ambulatory care. Cochrane Database Syst Rev. 2005;4:CD003539. doi:10.1002/14651858.CD003539.pub2.
Davey P, Brown E, Fenelon L, Finch R, Gould I, Hartman G, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst Rev. 2005;4:CD003543. doi:10.1002/14651858.CD003543.pub2.
Miller MD, Ferris DG. Measurement of subjective phenomena in primary care research: the Visual Analogue Scale. Fam Pract Res J. 1993;13(1):15–24.
Herdeiro MT, Figueiras A, Polonia J, Gestal-Otero JJ. Physicians’ attitudes and adverse drug reaction reporting : a case–control study in Portugal. Drug Saf. 2005;28(9):825–33.
Figueiras A, Tato F, Fontainas J, Gestal-Otero JJ. Influence of physicians’ attitudes on reporting adverse drug events: a case–control study. Med Care. 1999;37(8):809–14.
Pfennings L, Cohen L, van der Ploeg H. Preconditions for sensitivity in measuring change: visual analogue scales compared to rating scales in a Likert format. Psychol Rep. 1995;77(2):475–80.
Grant S, Aitchison T, Henderson E, Christie J, Zare S, McMurray J, et al. A comparison of the reproducibility and the sensitivity to change of visual analogue scales, Borg scales, and Likert scales in normal subjects during submaximal exercise. Chest. 1999;116(5):1208–17.
Celenza A, Rogers IR. Comparison of visual analogue and Likert scales in evaluation of an emergency department bedside teaching programme. Emerg Med Australas. 2011;23(1):68–75. doi:10.1111/j.1742-6723.2010.01352.x.
Foley DK. Development of a visual analogue scale to measure curriculum outcomes. J Nurs Educ. 2008;47(5):209–13.
DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39(2):155–64. doi:10.1111/j.1547-5069.2007.00161.x.
Allen MJ, Yen WM. Introduction to measure theory. Monterey: Brooks/Cole; 1979.
Caamano F, Figueiras A, Gestal-Otero JJ. Influence of commercial information on prescription quantity in primary care. Eur J Public Health. 2002;12(3):187–91.
Johanson GA, Brooks GP. Initial Scale Development: Sample Size for Pilot Studies. Educ Psychol Meas. 2009;70(3):394–400.
Bonita R, Beaglehole R, Kjellstrom T. Basic epidemiology. 2nd ed. Geneva: World Health Organization; 2006.
The authors would like to express their sincere thanks to the following: the collaborative group at the University of Santiago de Compostela, namely, Juan Vázquez Lago, Paula López Vázquez and Ana López Durán; the experts who evaluated the questionnaire, namely, Professor Doutor João Torrão of the Languages and Cultures Department, University of Aveiro, and Professor Doutor João Caldas of the Psychology Department of the Advanced Institute of Health Sciences – North; and all the physicians who voluntarily participated in this study.
This work was supported by the Foundation for Science & Technology (Fundação para a Ciência e Tecnologia - FCT), Portuguese Ministry of Education & Science [PTDC/SAU-ESA/105530/2008] and co-financed by Fundo Europeu de Desenvolvimento Regional (FEDER) through the Programa Operacional Factores de Competitividade (COMPETE) Program.
The authors declare that they have no competing interests.
MTH and AF conceived the study and supervises the study development. ATR and MF conducted the study. FR and ER participated in the questionnaire content and face validity and, with AF also were involved in the analysis and results interpretations. All authors participated in the development and red the manuscript final version. All authors read and approved the final manuscript.
Description of data: Questionnaire evaluated in the study. (PDF 312 kb)