Thread by @tmhohl71

1/ One of the privileges and duties of becoming a full professor is to write letters of evaluation for junior faculty in the promotions process at outside institutions. I give these a lot of thought and have completed 6-8 per year in the past three years.

2/ Usually, I receive an e-mail inquiry from an outside departmental administrator. When I agree to the evaluation, I am sent a promotion package prepared by the faculty member as well as a letter/document with promotions criteria by the host institution.

3/ I am increasingly troubled by requests for comparative evaluations:

Here's an example from one request letter (redactions to maintain anonymity):

• Estimate their (Dr. X's) standing in the field and compare them with other faculty of roughly the same cohort.

4/ Here's a 2nd (worse) example:

• Comparative evaluations play a central role in our effort to appoint outstanding scholars and teachers. Thus, it will aid our deliberations if you would compare Dr. X to the following scholars (Dr. A, Dr. B, and Dr. C – at peer institutions),

5/ both in absolute and relative terms, taking into account any differences in career stage when framing your reflections. … Even if you are not familiar with the work of everyone on the list, we would still be interested in your views on those whose work you do know,

6/ or others you believe to be of equal or greater merit.

These letters come from the highly prestigious and respected institutions in the US that would be considered world leaders in biomedical research.

Why do I highlight this problem?

7/ In the first letter, the request is entirely vague, includes no criteria for "standing in the field", and asks for a comparison with faculty (unnamed) of roughly the same cohort (how would you define this?)

The second letter is worse, because it asks for direct comparisons

8/ with three named scientists at peer institutions (unbeknownst to the named scientists or their institutions). As a reviewer, I only have access to the promotions package of the candidate faculty and no comparable information on the three "comparator" faculty.

9/ I have no idea how these three were chosen. I am troubled by these requests.

To promotions committees: It is not the job of outside evaluators to compare your faculty candidate with faculty at other institutions - it is our job to provide an evaluation of how faculty

10/ candidates meet the stated criteria for promotion at your institution. Please provide reviewers with objective, clearly defined criteria (level of support, publications, teaching, mentorship, etc...) and not with requests for comparisons with faculty at other institutions.

11/ These evaluations are not akin to evaluations for a scientific prize in which all candidates have the opportunity to fill out a standard application, ensuring that comparisons are meaningful (but may still contain biases).

12/ I responded in the following way to the first request:
"Please note that I do not include individual comparisons of scientists in any promotion evaluations. I believe that it is impossible to do so without having access to equivalent information about all individuals

13/ to be compared and that such requests are prone to eliciting unconscious bias in faculty evaluations."

To the second request:
"I object in the strongest terms to comparing named candidates at different institutions and to assigning absolute and relative rankings.

14/ I do not have the same information package available for all named faculty and I do not feel that it should fall on a reviewer to compile this information. Moreover, the named scientists are not all in the same field of investigation. This comparison cannot be done in an

15/ unbiased manner, and I am concerned that this format can perpetuate structural inequalities. Thus, I am not comfortable in partaking in this exercise, even if it led to the conclusion that Dr. Y is the strongest candidate."

16/ Comment: I want to support junior faculty and the promotion process. I do not have that including person-to-person comparisons achieves this goal. believe it to be harmful, and that it introduces biases, particularly against members of groups that are historically

17/ underrepresented in biomedical research. IMHO, it's time to take a stand.
My pledge: I will write letters to support faculty who are going through the promotions process, but will not compare scientists head-to-head.

18/fin
Another way to help is to serve on promotions committee who make up the rules of promotion and evaluate candidates (full disclosure: I serve on a promotions advisory committee at @MSKCancerCenter).
#MedTwitter #IDTWitter @UshmaNeill @RuthGotian @MSKCancerCenter