Thread by @ewanbirney

Metrics on science (people, institutes) are crude tools and should be used with extreme care - my basic rules:

1. Never be lazy and use metrics when you can just read the science (even the goddamn abstracts is better!). We've all been there - don't do it. I have to book in time to read the papers / assessment, but this is potentially travel time

(its also fun reading science - that's why we love our science right - and broadens your own knowledge, so there are positives about being 'forced' to read papers)

Even if you are not an expert, presumably the committee *has* someone experts / written reviews on this topic (if not - what is going on? Complain loudly). I'd take 5 subjective unbiased opinions and average them over 1 hetregenous, hard to normalise, poorly discriminant metric

2. Metrics have some use in triage, but only for mid-career people onwards. Never use biblometrics for early stage careers - all they show is how well funded the persons "parent" lab was. Beware between field/sub-field differences

3. On your CV, quote metrics (fine to use the one with the best result, ie, Google scholar :)) - saves the review fiddling around - but mre importantly highlight your best 5/10 papers that bring out the science you are most proud of.

4. Beware the use of metrics in committee setting - I think we should ban them in the final ranking in particular at the "top" because I think they might have some bottom to top discrimination but hopeless in ranking the top 20% IMHO

Even having the metrics prominently available and in tables ("for easy comparison") makes the committee have in my experience an anchoring effect that the implied ranking is a good starting point. Very human thing.

What's better? - have everyone rank independently, sum ranks is my view (ranking rather than scoring implicitly normalises). Or score independently. Don't get freaked out by variance in scoring/ranking - it's healthy.

Frankly it should be a warning sign if the committee is too tightly correlated - I'd say this is some implicit anchoring effect, though on one-off examples hard to know.