Thread by @dymonite69

#CBME (a long thread)

1)Competency in clinical practice is a complex construct

2)Competency is considered to be an integrated function of knowledge, skills and attitudes and can’t be reliably evaluated by assessing each of these separate elements in isolation

3)Observing common or important activities in an authentic clinical context is a meaningful way of evaluating this integration

4)If specific knowledge and skills are evaluated in isolation from how it is applied clinically, then that information needs to be analysed at a granular level and cross-referenced (triangulated) of how they would contribute successful application to an integrated activity

(e.g. triangulating respiratory history-taking with knowledge of respiratory pathology and therapeutics, and interpretation of chest radiology)

5)Practice experts who are adequately trained in assessment are best suited to evaluate competency

6)Observations should take in consideration that the difficulty of performing the same activity can be contextual based on patient and environmental factors

(e.g. taking a history in a patient with cognitive impairment, performing a lumbar puncture in a patient with limited mobility and indistinct land marks)

7)The validity of an assessment increases based on a combination of:

the number of assessments; the range of assessors; the quality, detail and contextual factors used in the assessments; the range of knowledge and skills being tested, and how those relationships are transparently, defensibly and meaningfully articulated.

8)It is appreciated that there will be idiosyncratic variation in supervisor standards but supervisor expectations should be made transparent to the learner as part of the process.

The impacts of variability is mitigated by random exposure to different supervisors and increasing the number of assessments

9)The definition of a competent, independent practitioner ready for graduation cannot be defined simply by objective criteria or numerical grades but reflects the net opinion of a group of expert practitioners gathering multiple assessments over a period of time.

Supervisors will make increasing entrustment decisions for different clinical tasks as the learner progresses. Entrustment lies on a continuum and is a probabilistic predictive statement made by experts that is not easily defined by pre-set criteria

10)Feedback is an important part of the learning cycle

11)Effective feedback, particularly of complex integrated performance is a collaborative and sensitive process requiring mutual buy-in.

It should take in the consideration of the perspective of the learner and how they are uniquely mentally processing information and translating that into behaviours and actions

12)Feedback aims to demonstrate the potential relationship between new points of learning and improved future performance

13)The feedback cycle must involve the learner creating an acceptable, evaluable and meaningful learning goal that will be reassessed in timely manner

14)Repeated failure to improve in response to clearly explained and documented feedback is grounds for elimination from the program

15)A feedback culture has to be addressed at both individual level and as a system. It has to be safe for both learner and supervisor that is both realistically actionable, meaningful, and defensible

16)Levels of competence for different activities may develop unevenly for individual learners due to different learning sequences and clinical exposure. This is true for undergraduate education and continues through post-graduate training and continuing professional development

17)Training need to be flexible and agile. It cannot simply rely on prolonged formal educational group sessions for an entire cohort.

Greater emphasis should be made on brief teaching and learning encounters that are more meaningful, and immediately relevant to the specific individual (e.g Teaching on the Run).

18)Direct observation and real time feedback of a range of different authentic activities (however brief) is critical for developing mastery

19)Learners must acknowledge the critical importance of brief teaching encounters as both summative and formative and the main means and motivation for progression

20)Training must include a component of equipping learners with evidence-based learning strategies and reflective learning techniques so that they can more effectively and strategically take ownership of their learning.

21)Learners have an equal responsibility to regularly evaluate, analyse and integrate all their various assessments and develop strategic learning plans. Trained learning coaches should be offered to those who need assistance with this task.

22)Logistical challenges of work-based assessment include multiple assessors, difficulties of longitudinal follow-up, fleeting observations and brief feedback encounters.

This MUST be addressed with a seamless, efficient information management system that can log individual assessments quickly and easily.

Automated learning analytics system are required to integrate data points in a meaningful manner that allows for interpretation later and to verify that progression is occurring.

23)There needs to be an ongoing program of developing supervisor assessment, teaching and feedback skills to ensure educational efforts are being matched with effective learner progression and outcomes