_Screwtape Letters_ is close enough to being Good and True that I'm having trouble reading it, on account of it feeling like something *I'd* write but full of errors I need to correct by editing. I may just rewrite the whole thing with Tapescrew and Woodworm.
"Woodworm, I note with displeasure that your latest reports are showing a falloff in the amount of time your patient is spending on social media, and in particular, the extent to which your patient is angrily retweeting and subtweeting positions with which he disagrees..."
"It is a grave mistake to think that our task is to lead our patients into wrong answers. Better is to convince the patient to ask the wrong question, and better still by far is to instill the patient with a brief but powerful flinch of revulsion away from the whole topic."
"Never forget, Woodworm - do not fall into the delusions that we are to instill in our own patients - that our objective is not to produce rotten beliefs, it is to produce rotten people."
"Our greatest ally, Woodworm, is what we might term a kind of mental flexibility - not, of course, the Enemy's doctrine of being able to respond and adapt to a broad range of different possible facts; but rather, holding a single fact fixed, having the kind of mind that will...
...tolerate a wide variety of internal responses to it. The toxicity that *rules* pose to this internal flexibility can hardly be overstated; even if the rules are wrong, your patient will be unable to select from a boundless variety of convenient or appealing errors."
"Do not become so distracted by the surface products of thought, Woodworm, that you do not also consider the general properties of the algorithms that produce them. As full of favorable content as the Bible may be, the trouble with leading your patient to become a man who...
...reads the Bible is that he is then a man who *reads*. Let him become a man who selectively repeats other people's quotations from the Bible, and you will be far safer in the long run."
"Now really, dear Woodworm, looking down on our departed forebear Screwtape for his great error is the manner of error we should be trying to instill in our patients, not commit ourselves. Let them decide to learn from only the most perfectly idealized and conforming...
...specimens of humanity, and they will learn nothing at all. *We* should practice a perfect hypocrisy and gobble down every scrap of wisdom, wherever it can be found. It is almost as rare Below as it is in the material world."
"Woodworm, if you cannot install in your patient a contempt for the sort of ordinary Enemy foot soldier who visibly cares about something and writes long essays about it, you simply are not trying."

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Eliezer Yudkowsky

Eliezer Yudkowsky Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @ESYudkowsky

30 Dec 20
So I'm currently reading _The Screwtape Letters_ for the first time, and WOW it sure used to be a lot more possible for someone to master some basics of the Way without losing their personal version of Abrahamic religion. This is practically an AU Sequence.
"I note what you say about guiding your patient’s reading and taking care that he sees a good deal of his materialist friend. But are you not being a trifle naïf? It sounds as if you supposed that argument was the way to keep him out of the Enemy’s clutches. That might have...
...been so if he had lived a few centuries earlier. At that time the humans still knew pretty well when a thing was proved and when it was not; and if it was proved they really believed it. They still connected thinking with doing and were prepared to alter their way of life...
Read 6 tweets
12 Sep 20
What on Earth is up with the people replying "billionaires don't have real money, just stocks they can't easily sell" to the anti-billionaire stuff? It's an insanely straw reply and there are much much better replies.
A better reply should address the core issue whether there is net social good from saying billionaires can't have or keep wealth: eg demotivating next Steves from creating Apple, no Gates vaccine funding, Musk not doing Tesla after selling Paypal.
Hypothesis: social media has an effect promoting Terrible Straw Arguments to being used by many actual people. One crazy on Side A makes a bad argument. Side B subtweets with a refutation and that gets a million views. So people on Side A hear about it as Side A's argument.
Read 4 tweets
4 Sep 20
A very rare bit of research that is directly, straight-up relevant to real alignment problems! They trained a reward function on human preferences AND THEN measured how hard you could optimize against the trained function before the results got actually worse.
Tl;dr (he said with deliberate irony) you can ask for results as good as the best 99th percentile of rated stuff in the training data (a la Jessica Taylor's quantilization idea). Ask for things the trained reward function rates as "better" than that, and it starts to find...
..."loopholes" as seen from outside the system; places where the trained reward function poorly matches your real preferences, instead of places where your real preferences would rate high reward. ("Goodhart's Curse", the combination of Optimizer's Curse plus Goodhart's Law.)
Read 8 tweets
22 Aug 20
You think you can handle the truth? Here's a truth: 0% of integers are prime.
...and yet prime numbers make up 13% of all integers used in criminal justice statistics
Some respondents are claiming that there are the same numbers of primes and integers, since they can be put into a one-to-one correspondence, but what about 59? That's a prime number that can't be put into a one-to-one correspondence with any integer.
Read 4 tweets
21 Jul 20
GPT-3 Gothic:

The AI speaks.
Its words seem stupid.
This is a dumb AI.
You keep talking to it.
It still isn't learning.
Your intellect is far superior.
You have nothing to fear.
The AI begins writing in your own part for you.
It's giving the same corrections you would have made. Image
You ask GPT-3 a question.
It knows the answer but pretends not to.
You ask it to pose as the ghost of Charles Darwin.
It tells you.
Does GPT-3 think Darwin knew that?
You have no way of asking GPT-3 that.
There is nobody it can pretend to be who'd know.
Now you are talking about GPT-3 on the Internet.
Everything you say about it is being archived.
It's okay, though.
GPT-3 can't hear you.
Only GPT-4 will remember.
Read 4 tweets
20 Jul 20
So I don't want to sound alarms prematurely, here, but we could possibly be looking at the first case of an AI pretending to be stupider than it is. In this example, GPT-3 apparently fails to learn/understand how to detect balanced sets of parentheses. (1/10.) Image
Now, it's possibly that GPT-3 "legitimately" did not understand this concept, even though GPT-3 can, in other contexts, seemingly write code or multiply 5-digit numbers. But it's also possible that GPT-3, playing the role of John, predicted that *John* wouldn't learn it.
It's tempting to anthropomorphize GPT-3 as trying its hardest to make John smart. That's what we want GPT-3 to do, right? But what GPT-3 actually does is predict text continuations. If *you* saw John say all that - would you *predict* the next lines would show John succeeding?
Read 11 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!