Thread by @vgr

I was briefly calling myself an independent researcher: somebody who self-funds spec R&D on their own ideas. In theory it’s something like indie-research : academic research :: blogging/self-publishing : traditional publishing.

But the idea doesn’t really work.

Unlike the market for general interest writing, the market for R&D is almost entirely institutional, and they don’t really “buy” indie R&D. It’s 99.9% crackpot inventors, 0.1% black swan stuff.

Most “research” indie consultants sell institutions is in the market-research class, not academic research

If I spent 6 months working on a new robot navigation algorithm or a study of government failure risk in Latin America or a deep study of some artist, nobody would buy it at a price that would pat for what it cost to do even in pure living costs terms.

You can at best target a research problem on a marketplace where someone has already declared a bounty, like say innocentive, nine sigma, etc (in which case they’re few and narrow). But that’s hardly independent. If you want to define and work on a problem you choose, it’s your $

Research =

a) long time between having an idea and having something to show for it that even the most sympathetic fellow crackpot would appreciate (not even pat for, just get)

b) a >10:1 ratio of background invisible thinking in notes, dead-ends, eliminating options etc

With a blogpost, it’s like a week of effort at most from idea to mvp, and at most a 3:1 ratio of invisible to visible. That’s sustainable as a hobby/side thing.

To do research-grade thinking you basically have to be independently wealthy and accept 90% deadweight losses

John Carmack announcing he was quitting oculus to do indie research on AGI finally made the stark equation clear. You need Carmack-level starting points (indie wealth, established R&D celebrity record) to go indie. That’s basically modern equivalent of old nobility.

That’s the real deal. Anyone below that threshold calling themselves an independent researcher is either talking about something like market research or trend research (which is really environmental intelligence work not R&D in the sense that interests me), deluded, or lying.

tldr: Get tenure, or get rich. Software has not eaten R&D, and won’t eat it anytime soon. Possible outliers like Satoshi (who may have been in an R&D institution or indie-wealthy in 2009 for all we know) notwithstanding.

Best-case outcome if neither applies: your idea has commercial potential and you can get a startup out of it. Which means the vast majority of ideas, both STEM and humanities, are out. Even I’m not optimist enough to think your R&D on 16rh century French poets can be a unicorn.

The reason software doesn’t eat R&D even though people think it should is that tools or open access to published lit/libraries, or free peer-reviewed publishing/presentation forums (if you want that) are only a tiny fraction of the cost for most ideas. The main host is time.

Even if the most expensive research tools like LHC or Hubble got as cheap as Arduinos, it wouldn’t help. That’s not the cost bottleneck. The bottleneck is researcher time. Even crappiest, cheapest conference paper I ever wrote took at least 6 months (~7k grad student salary then)

This isn’t cheap talk btw. I dropped 2k on a personal Matlab license after I went indie and kept it current for years, and went through dozens of false starts working on various ideas. Couldn’t find the time to develop any of them to even bad-conference-paper level.

Note: peer-reviewed publishing or patents just as depth cal9bration. I wouldn’t necessarily publish in such forms since I don’t think those institutional processes have ever added much to my work. Nor into them tbf, in the peer reviewing I’ve done. But I’d aim for that depth.

Note #2: I enjoy proper research, but wasn’t a great talent even at my peak and am now likely at 60%. So in a way this is an okay market outcome. If I were wealthy I might do self-indulgent mediocre research for the rest of my life, but no reason you should pay for it.

Note #3

Learning projects to prepare to go deep on a subject are not research even if it feels like it

Reading published literature and blogging a few derivative observations is not research

Critiquing/finding flaws in published papers is indie peer-review, not research

I find that autodidacts who haven’t experienced institutional R&D environments have a self-congratulatory low threshold for what they count as research. It’s a bit like vanity publishing or fan fiction. This mismatch doesn’t exist as much in indie art, consulting, game dev etc

I’m on a year-long fellowship right now, and this is partly what has made me admit the conceit in the “indie researcher” self-label. I’m doing this sort of thinking for the first time since ~2006 or so (the year I last submitted a paper to a peer-reviewed journal).

It’s also reminding me how much I enjoy doing the real thing. It’s like a year long vacation back to my own past.

Ironically, most modern tenured faculty don’t get to do much R&D either, at least in STEM. They’re too busy fund-raising. They’re more like angel investors and board advisors to grad students and postdocs. Humanities and social sciences may perhaps be better, I don’t know.

If you’re thinking kickstarter-like funding mechanisms, forget it. You’ll be asking for 10x as much money as typical artistic or startuppy projects, have no natural rewards to hand out, and a far lower likelihood of success.

A good public example: Breaking Smart S2 was a medium-depth research project I thought I could sustain on my own (a16z supported S1). I gave up looking for funding after a couple of half-hearted pitches to orgs I thought might be a fit for the topic.

The topic I wanted to research, and got about 1/3 the way through, was institution building in the Great Weirding. It’s the sort of thing that’s not a natural fit for any corporate funding source.

Current plan is to package and flush out what I have so far as a sort of “Christmas Special” about 1/5 the scope/ambition of S1.

A more basic problem with kickstarter type ideas is that anyone good at creating the buzz and hype for that is almost by definition going to be bad at R&D and vice versa. Content mismatch aside.

Writing grant proposals to satisfy a few bureaucrats is already enough of a mismatch to research personalities. Crowd-pleasing requires 10x that painful mismatch.

List of actually credible indie researchers

1. Stephen Wolfram (math/complexity/computation)
2. Jeff Hawkins (neuroscience)
3. John Carnack (AGI)

Notice something, besides the fact they’re smart and have the right subject-matter prep?

They got rich first 🙂

Don’t mean to be a downer. There’s possibly imaginative models that could work in indie mode that I simply haven’t thought of.

Many things get called “research” and a lot of ego-sensitivity gets attached to it. I think of it mainly in terms of (high risk of no valuable output)*(high ratio of invisible to visible output)*(high time demands). Let me try to pseudo-quantify this and take the ego element out:

1. Intelligence briefs
2. Gartneresque research
3. Investigative journalism
3. Market research
4. Broad societal trend research
5. Data-heavy trend research, pure math
6. Tech futures, humanities
7. S/W tech, social science
8. Generic STEM
9. Big science
10. Paradigm shifts

This scale isn’t commentary on the intelligence, creativity, or imagination of the people who do such work. Higher on the scale is simply riskier, more time-consuming, and requires more backend work, even holding the human factor constant

One thing I probably could do is raise funding for a small research institution/lab working on problems in the 6-8 range. Maybe 4-5 staff. The thing is I don’t want to run a research org, which is an entirely different interest/ambition than doing research. It takes a COO type.

Strange-looping in a secondary meta thread I did later.