Thread by @ProfDFrancis

Joel Giblett @joelgiblett brings up a very important question.

Where does scientific integrity come from? What makes me think I can trust papers written by him? I've never heard of him before this tweet, and I have no idea of his background.

Well, first of all, he seems genuinely curious about integrity. This is a good sign.

Second, he is scared of R&D and MHRA. This is a good sign that he tries to toe the line, pays his taxes, doesn't park in the disabled spots (unless entitled) and doesn't drop litter.

But it is not non-littering, or trembling when the R&D office sends an email, that CAUSES his research to be non-fake.

It is his own personal attitude, those of his colleagues, and the lack of tremendous incentives to fiddle.

Suppose I wanted to "discover" that (for example) boys are cleverer than girls, using clinical data of IQ tests they happened to have had.

Since it is routine clinical data, R&D office don't particularly get involved in the nitty-gritty.

And the identification of patients is done by my research fellow, whom (of course) I "pre-educate" (or select for gullibility) that "boys are much cleverer than girls, everyone knows that".

Then I just let him/her loose on the data.

I don't need to supervise much.

Just 3 hints: "Obviously use the most clinically appropriate and meaningful IQ measurement in each case, and don't count obviously ineligible cases. And don't assume the pre-entered sex is right, sometimes staff get it wrong when in a rush."

And I can be confident the fellow will apply reMEASURE, reMOVE and reCLASSIFY, and deliver a highly convincingly significant result.

In fact our Twitterverse has shown this in the last week here >

External Tweet loading...
If nothing shows, it may have been deleted
by @ProfDFrancis view original on Twitter

What stops me doing that?

Not MHRA. Not R&D office.

I think it is because

(a) I don't _need_ to accumulate rubbish papers. Nobody is forcing me. I hope nobody is forcing Joel.

(b) I don't _want_ to tilt the world towards believing or disbelieving X. I am more interested in getting a better view of whether X is true, for my own benefit, that of our patients and that of our health care system.

(c) I don't want to be _laughed at_ by the ORBITA HQ fellows, who are fans of fraudery and love picking up and mocking even the slightest hint of fiddling in each other's research.

But I can afford all these luxuries, and I hope Joel can too. We are not in imminent danger of being sacked for "not having enough papers" (I hope).

We have a comfortable life, not dependent on p<0.05, so that if one has disappointing results, we can simply say so.

For people without these comforts, life is much more difficult.

Suppose your endpoint was "getting your PhD". Everyone in your department has positive results. The gels did what they wanted them to do. Only yours didn't.

Their perception:
No positive results -> No papers -> No PhD -> failure

That is a horrible, horrible situation to face.

I've never faced it.
I wouldn't wish it on anyone.

Not the researcher's fault, but it is a lonely and personally crushing position.
It must be unspeakably horrific.

I try to give each of my fellows multiple hypotheses.
I encourage them to recognise BS when they see it, so they can declare an idea dead before they waste their life on it.
I try to lead by example by changing my point of view in light of data.

Looking beyond the PhD stage, suppose your aim is to get funding from a company for their drug, or stem cell, or device being good.

Clearly the funding will only keep coming as long as the results are positive for the company.

The fear is not for yourself: you probably have a relatively secure senior job.

It is for the various staff in your department who are funded through the company contract. The university cannot keep paying them without the company money.

So the results have to be positive.

The R&D office is not really there to make sure you do the research properly and unbiasedly.

It is there to show that the university made efforts to make you do it properly etc.

They don't see you doing the actual work. The recruitment, the measurement, the removal, the reclassification.

For prospective studies, they _DO_ make sure that the study existed in some form.

They make sure the consent forms exist.
They make sure there is evidence that the measurements were done.
But they don't supervise remeasure/remove/reclassify.
We are free to do that ourselves.

Which reminds me!

I am chasing a well-known journal that has been a bit slow on pulling the plug on a trial that I consider to be fictional.

The results make me happy, but they are not true, so it is my duty to get it de-published if I can.

Enjoy your Sunday, friends!