- Journal List
- BMJ
- v.324(7330); 2002 Jan 19
- PMC64520
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
BMJ. 2002 Jan 19; 324(7330): 156–159.
PMCID: PMC64520
PMID: 11799035
Janet Grant, professor of education in medicine
Author information Article notes Copyright and License information PMC Disclaimer
See editorial "Learning in practice: a new section in the BMJ " onpage125.
Learning needs assessment has a fundamental role in education and training, but care is needed to prevent it becoming a straitjacket
It might seem self evident that the need to learn should underpin any educational system. Indeed, the literature suggests that, at least in relation to continuing professional development, learning is more likely to lead to change in practice when needs assessment has been conducted, the education is linked to practice, personal incentive drives the educational effort, and there is some reinforcement of the learning.1 Learning needs assessment is thus crucial in the educational process, but perhaps more of this already occurs in medical education than we suspect. The key lesson might be for those who design new systems of education and training: for example, the postgraduate education allowance system in general practice was felt to fail the profession because it did not include needs assessment and so led to ad hoc education to fulfil the time requirements of the system rather than the needs of individual doctors or the profession as a whole. On the other hand, basing learning in a profession entirely on the assessment of needs is a dangerous and limiting tactic. So a balance must be struck.
Summary points
Learning needs assessment is a crucial stage in the educational process that leads to changes in practice, and has become part of government policy for continuing professional development
Learning needs assessment can be undertaken for many reasons, so its purpose should be defined and should determine the method used and the use made of findings
Exclusive reliance on formal needs assessment could render education an instrumental and narrow process rather than a creative, professional one
Different learning methods tend to suit different doctors and different identified learning needs
Doctors already use a wide range of formal and informal ways of identifying their own learning needs as part of their ordinary practice
These should be the starting point in designing formalised educational systems for professional improvement
Learning needs assessment in medicine
In 1998 both individual and organisational needs assessment became part of government policy in relation to the continuing professional development and personal development plans of all healthcare professionals.2 Thus, it has a role in the clinical governance of the service3 and is therefore much more than an educational undertaking. This integration of needs assessment, education, and quality assurance of the service was first made explicit in 1989 in relation to clinical audit, which would identify practices in need of improvement and ensure that educational and organisational interventions were made to address these needs.4 Accordingly, audit was described as “essentially educational” and the educational process surrounding it described.5
Long before these recent developments, needs assessment outside medicine was presented as an important part of managed education and learning contracts, which are the predecessors of the personal development plans to be developed for all NHS healthcare professionals.6 In his descriptions of adult learning Knowles assumed (he did not claim to have research evidence) that learners needed to feel a necessity to learn and that identifying one's own learning needs was an essential part of self directed learning.7 In medicine a doctor's motivation to learn would therefore derive from needs identified during his or her experience of clinical practice. So the pedigree and practice of learning needs assessment, if not the evidence, are well established.
The definition of need
As in most areas of education, for many years there has been intense debate about the definition, purpose, validity, and methods of learning needs assessment.8 It might be to help curriculum planning, diagnose individual problems, assess student learning, demonstrate accountability, improve practice and safety, or offer individual feedback and educational intervention. Published classifications include felt needs (what people say they need), expressed needs (expressed in action) normative needs (defined by experts), and comparative needs (group comparison).9 Other distinctions include individual versus organisational or group needs, clinical versus administrative needs, and subjective versus objectively measured needs.10 The defined purpose of the needs assessment should determine the method used and the use made of findings.
Furthermore, even though the concept of educational needs assessment is enshrined in practice, policy, and the educational canon, several factors indicate the need for careful planning and research in this subject (see boxes boxesB1B1 and andB2).B2). Exclusive reliance on formal needs assessment in educational planning could render education an instrumental and narrow process rather than a creative, professional one. This is especially so in a profession where there is inherent unpredictability and uncertainty. Members of any profession require wide knowledge and depth of experience—the relevance of some of which might not have been obvious at the time of learning. Certainly, learning needs can and should be identified on the basis of what has been experienced and of what more experienced members of the profession know to be relevant, but this must not deter other, more general or even speculative, learning that, at the time, seems to answer no specific need. Possibly no specific learning needs assessment would ever send a person to a large international conference on a generic subject (such as endocrinology, medical education, or management). It is, nevertheless, important that doctors attend such meetings and return with the unexpected and expected benefits that they accrue.
Box 1
: Need for careful planning in needs assessment
There is little evidence that needs assessment alone enhances educational effectiveness and outcomes, so it must be placed within the wider process of planned learning, relevance to practice, and reinforcement of learning in the appropriate context
Formal needs assessment can identify only a narrow range of needs and might miss needs not looked for, so breadth and flexibility of needs assessment methods should be embraced
In professional education it is not necessarily defensible to focus all learning on identified needs—wider professional learning not related to a specific need is also of fundamental value where practice is not predictable
Individual and group learning needs are different—group learning needs may produce an average picture that fails to address important needs and interests of individual members of the group—so a balance is required. Each approach has its uses and effects, but each must be used for the right purpose
Identifying individual learning needs, often not shared by others, may lead to an unimpressive cost-benefit analysis in terms of individually targeted use of educational resources if used inappropriately. Individual learning needs assessment is best used in the context of learning that occurs on an individual basis—such as in the relationship between general practitioner registrar and trainer
Box 2
: Need for research into needs assessment in medical education
What are the effects of and responses to needs assessment alone for students, trainees, and senior doctors at different stages of medical education?
What is the relative validity, reliability, or utility of different formal and informal methods of learning needs assessment in medical education at any level?
To what extent do needs assessment methods identify all important learning needs?
What are the relative effects and efficacy of identifying group and individual learning needs?
What methods of planning effective learning experiences are most effective on the basis of needs identified?
Methods of needs assessment
Although the literature generally reports only on the more formal methods of needs assessment, doctors use a wide range of informal ways of identifying learning needs as part of their ordinary practice. These should not be undervalued simply because they do not resemble research. Questionnaires and structured interviews seem to be the most commonly reported methods of needs assessment, but such methods are also used for evaluation, assessment, management, education, and now appraisal and revalidation.11 Together, these formal and informal methods might make an effective battery where there is clarity of purpose. The Good CPD Guide details 46 formal and informal methods of self assessment (see box boxB3B3).12
Box 3
: Good CPD Guide's classification of sources of needs assessment12
The methods listed are both formal and informal, planned and opportunistic, showing that day to day work and encounters have the potential to generate needs as much as do formal methods. Formal needs assessment methods include critical incident techniques, gap analysis, objective knowledge and skills tests, observation, revalidation, self assessment, video assessment, and peer review. Such methods are often used to identify group needs.14,15 Formal identification of needs can also arise from audit, morbidity patterns, adverse events, patient satisfaction surveys, and risk assessment. Most of these tools use quantitative methods that can generate computerised data and cover wider population ranges, but these are often unable to probe into the personal agendas and opinions of individuals.
Types of needs assessment
Methods of needs assessment can be classified into seven main types, each of which can take many different forms in practice.
Gap or discrepancy analysis
This formal method involves comparing performance with stated intended competencies—by self assessment, peer assessment, or objective testing—and planning education accordingly.9,16,17
Reflection on action and reflection in action
Reflection on action is an aspect of experiential learning and involves thinking back to some performance, with or without triggers (such as videotape or audiotape), and identifying what was done well and what could have been done better.18,19 The latter category indicates learning needs.
Reflection in action involves thinking about actual performance at the time that it occurs and requires some means of recording identified strengths and weaknesses at the time. The Canadian MOCOMP programme uses formalised reflection as its basic process.20 Similarly, PUNs and DENs (see box boxB3)B3) are well known in British general practice.
Self assessment by diaries, journals, log books, weekly reviews
This is an extension of reflection that involves keeping a diary or other account of experiences.21 However, practice might show that such documents tend to be written nearer the time of their review than the time of the activity being recorded.
Peer review
This is rapidly becoming a favourite method. It involves doctors assessing each other's practice and giving feedback and perhaps advice about possible education, training, or organisational strategies to improve performance. The Good CPD Guide describes five types of peer review—internal, external, informal, multidisciplinary, and physician assessment.11 The last of these is the most formal, involving rating forms completed by nominated colleagues, and shows encouraging levels of validity, reliability, and acceptability.22,23
Observation
In more formal settings doctors can be observed performing specific tasks that can be rated by an observer, either according to known criteria or more informally. The results are discussed, and learning needs are identified. The observer can be a peer, a senior, or a disinterested person if the ratings are sufficiently objective or overlap with the observer's area of expertise (such as communication skills or management).
Critical incident review and significant event auditing
Although this technique is usually used to identify the competencies of a profession or for quality assurance, it can also be used on an individual basis to identify learning needs.24 The method involves individuals identifying and recording, say, one incident each week in which they feel they should have performed better, analysing the incident by its setting, exactly what occurred, and the outcome and why it was ineffective.
Practice review
A routine review of notes, charts, prescribing, letters, requests, etc, can identify learning needs, especially if the format of looking at what is satisfactory and what leaves room for improvement is followed.
The difference between needs assessment and assessment
Needs assessment is not the same as assessment in the sense of examination of learning. Assessment systems that lead to academic or professional awards should show certain minimum characteristics, including measurement of performance against external criteria and standards, a decision on adequacy by an assessor, and standardised data gathering.25 Needs assessment might sometimes have these characteristics, but it also might be based on practice, reflection, professional judgment, discussion, and informal data. Needs assessment methods that are limited by the standards of assessment will fall into the trap of assessing only a narrow range of needs.
Learning for needs
The main purpose of needs assessment must be to help educational planning, but this must not lead to too narrow a vision of learning. Learning in a profession is unlike any other kind of learning. Doctors live in a rich learning environment, constantly involved in and surrounded by professional interaction and conversation, educational events, information, and feedback. The search for the one best or “right” way of learning is a hopeless task,1 especially if this is combined with attempting to “measure” observable learning. Research papers show, at best, the complexity of the process.
Multiple interventions targeted at specific behaviour result in positive change in that behaviour.26 Exactly what those interventions are is less important than their multiplicity and targeted nature. On the other hand, different doctors use different learning methods to meet their individual needs. For example, in a study of 366 primary care doctors who identified recent clinical problems for which they needed more knowledge or skill to solve, 55 different learning methods were selected.27 The type of problem turned out to be the major determinant of the learning method chosen, so there may not be one educational solution to identified needs.
Much of doctors' learning is integrated with their practice and arises from it. The style of integrated practice and learning (“situated learning”) develops during the successive stages of medical education.28 The components of apprenticeship learning in postgraduate training are made up of many activities that may be regarded as part of practice (see box boxB4B4).29 Senior doctors might also recognise much of their learning in some of these elements and could certainly add more—such as conversations with colleagues.
Box 4
: Components of apprenticeship learning in postgraduate training29
Learning by doing
Experience of seeing patients
Building up personal knowledge and experience
Discussing patients
Managing patients
Having errors corrected
Making teaching points during service
Listening to experts' explanations
“Picking things up”
Charismatic influences
Learning clinical methods from practice
Being questioned about thought and actions about patients
Teaching by doing
Using knowledge and skill
Bite-size learning from “bits and pieces”
Retrieving and applying knowledge stored in memory
Learning from supervision
Receiving feedback
Presentation and summarising
Observing experts working
Learning from role models
Learning from team interactions
Hearing consultants thinking aloud
Thinking about practice and patients
Thus, educational planning on the basis of identified needs faces real challenges in making learning appropriate to and integrated with professional style and practice. The first step in all of this is to recognise the needs assessment and learning that are a part of daily professional life in medicine and to formalise, highlight, and use these as the basis of future recorded needs assessment and subsequent planning and action, as well as integrating them with more formal methods of needs assessment to form a routine part of training, learning, and improving practice.
Notes
Editorial by Goldbeck-Wood and Peile
Footnotes
Competing interests: None declared.
References
1. Grant J, Stanton F. The effectiveness of continuing professional development. Edinburgh: Association for the Study of Medical Education; 2000. . (ASME medical education booklet.) [Google Scholar]
2. Department of Health. A first class service: quality in the new NHS. London: HMSO; 1998. [Google Scholar]
3. Department of Health. The new NHS: modern and dependable. London: HMSO; 1997. . (Cm 3807.) [Google Scholar]
4. Department of Health. Working for patients. London: HMSO; 1989. . (Cm 555.) [Google Scholar]
5. Batstone G, Edwards M. Management of learning and professional development through clinical audit. In: White T, editor. Textbook of management for doctors. London: Churchill Livingstone; 1996. pp. 341–358. [Google Scholar]
6. Knowles MS. Using learning contracts: approaches to individualising and structuring learning. San Francisco: Jossey-Bass; 1986. [Google Scholar]
7. Knowles MS. Self-directed learning: a guide for learners and teachers. New York: Association Press; 1975. [Google Scholar]
8. Stufflebeam DL, McCormick CH, Brinkerhoff RO, Nelson CO. Conducting educational needs assessment. Boston: Kluwer Nijhoff; 1985. [Google Scholar]
9. Gillam SJ, Murray SA. Needs assessment in general practice. London: Royal College of General Practitioners; 1996. . (Occasional paper 73.) [PMC free article] [PubMed] [Google Scholar]
10. Lockyer J. Needs assessment: lessons learned. J Contin Educ Health Prof. 1998;18:190–192. [Google Scholar]
11. Myers P. The objective assessment of general practitioners' educational needs: an under-researched area? Br J Gen Pract. 1999;49:303–307. [PMC free article] [PubMed] [Google Scholar]
12. Grant J, Chambers G, Jackson G, editors. The good CPD guide. Sutton: Reed Healthcare; 1999. [Google Scholar]
13. Eve R. Learnng with PUNs and DENs: a method for determining educational needs and the evaluation of its use in primary care. Educ Gen Pract. 2000;11:73–79. [Google Scholar]
14. Glazebrook R, Chater B, Graham P. Rural and remote Australian general practitioners' educational needs in radiology. J Contin Educ Health Prof. 2001;21:140–149. [PubMed] [Google Scholar]
15. Westkaemper R, Doherty M, Woolf AD. Multiple choice question quiz: valid test for needs assessment in CME in rheumatology and for self-assessment. Ann Rheum Dis. 2001;60:740–743. [PMC free article] [PubMed] [Google Scholar]
16. Knowles N. The adult learner: a neglected species. 4th ed. Houston: Gulf; 1990. [Google Scholar]
17. Moore DE. Needs assessment in the new health-care environment: combining discrepancy analysis and outcomes to create more effective CME. J Contin Educ Health Prof. 1998;18:133–141. [Google Scholar]
18. Schon DA. San Francisco: Jossey-Bass; 1987. Educating the reflective practitioner: towards a new design for teaching and learning in the professions. [Google Scholar]
19. Kolb DA, Rubin IM, McIntyre JM. Organizational psychology: an experiential approach. Englewood Cliffs, NJ: Prentice Hall; 1979. [Google Scholar]
20. Parboosingh IJT, Thivierge RL. The maintenance of competence (MOCOMP) programme. Ann R Coll Physicians Surg Can. 1993;26:512–517. [Google Scholar]
21. Campbell C, Parboosingh J, Gondocz T, Babitskaya G. A study of the factors that influence physicians' commitments to change their practice using learning diaries. Acad Med. 1999;74:S34–S36. [PubMed] [Google Scholar]
22. Ramsey PG, Carlene JD, Inui TS, Larson EB, LoGerfo JP, Wenrich MD. Predictive validity of certification by the American Board of Internal Medicine. Ann Int Med. 1989;110:719–726. [PubMed] [Google Scholar]
23. Paget NS, Newble DI, Saunders NS, Du J. Physician assessment pilot study for the Royal Australasian College of Physicians. J Contin Educ Health Prof. 1996;16:103–111. [Google Scholar]
24. Pringle M. Significant event auditing. In: van Zwanenberg T, Harrison J, editors. Clinical governance in primary care. Abingdon: Radcliffe Medical Press; 2000. pp. 105–115. [Google Scholar]
25. Jolly B, Grant J. The good assessment guide: a practical guide to assessment and appraisal for higher specialist training. Milton Keynes: Open University Centre for Education in Medicine; 1997. [Google Scholar]
26. Pyatt RS, Moore DE, Caldwell SC. Improving outcomes through innovative continuing medical education partnership. J Contin Educ Prof. 1997;17:239–244. [Google Scholar]
27. McClaren J, Franco E, Snell L. Type of clinical problem is a determinant of physicians' self-selected learning methods in their practice settings. J Contin Educ Health Prof. 1998;18:107–118. [Google Scholar]
28. Lave J, Wenger E. Situated learning: legitimate peripheral participation. Cambridge: Cambridge University Press; 1991. [Google Scholar]
29. Macdonald MM. Milton Keynes: Open University; 1998. Craft knowledge in medicine: an interpretation of teaching and learning in apprenticeship [thesis] [Google Scholar]
Articles from The BMJ are provided here courtesy of BMJ Publishing Group
