Thread by @Dry_Observer

So, with 3 weeks to the election, why not start a book club?

Not really, but here's an early public discussion of automated micro-corporate entities &/or subsentient programs running wild & destroying everything as evolutionary algorithms.

It's free.
lost-contact.mit.edu/afs/adrake.org…

The novel is Accelerando, and in case you're wondering why I figured out the basic methods of using evolutionary algorithms in psychological warfare - not to mention why it was fundamentally suicidal - here's one of them.

Another would be the "basilisk hack."

Langford started with basilisks - images triggering thoughts a human mind can't handle, cognitively or physically.

Other creators made them even more formidable and subversive, ultimately rewriting the target's mind or even their entire body...
en.wikipedia.org/wiki/David_Lan…

Via nanotechnology, genetic engineering, hypnosis and so forth.

Of course, to use evolutionary algorithms in psychological warfare, you need data, feedback, and the means to manipulate.

Social media has these tools, but also vast inherent weaknesses.
eclipsephase.com/comment/20628#…

The eclipse phase of a virus, incidentally, is when the target cell has been infected but has yet to produce the viral progeny which will destroy it.

And to continue, yes, those references in 2011 held a few hints as to why you should never use this tool against people...

"Hostile or simply incredibly deadly infotech and nanite weapons hyperevolving using evolutionary algorithms, which treat the native crew and a few visitors as symbiotic organisms which are mainly off-limits (so long as they don't stray past certain limits)..."

"...and subvert or annihilate everything else to...

"Perhaps the scariest option -- the barge has already fallen or is in the process of falling, and you are meddling in a place which is utterly owned or infected by a force you know nothing about..."

"...and which may be vastly more intelligent and dangerous than you are."

"Finally, one of the things I assume about Eclipse Phase is that many of the most powerful and advanced intelligences are keenly aware of what happens to the less powerful around them, even when they possess no real sense of compassion."

"Whenever an aggressive force acts, it exposes something about itself in doing so -- the technology it possesses, tactics it employs, goals it may or may not seek to further by way of a particular assault."

"Hence, advanced transhumans, Exhumans and seed AIs are often involved in a kind of cold war wherein they act subtly to assist otherwise outclassed victims -- a single "ping" alerting an invaded system of an ongoing attack..."

"...a sudden shutdown of a key communications hub in mid-incursion or basilisk hack, an involuntarily inserted or gift software patch that renders an obvious technique completely useless against the individual or organization."

"One factor often seen in these kinds of conflicts is an unspoken, ongoing assessment made by all of the relatively sane participants. Am I exposing too much of my resources, technology, tools and/or identity in this matter, and if so, is it worth it?"

"Even a moderately powerful, newly emergent seed AI should probably have a few levels of increasingly potent resources, and virtually all of the notable ones have probably figured out some unique tricks..."

"...if only ones related to specific regions, individuals or organizations they have studied, or to their particular areas of specialization."

So yes, the warning was there, and elsewhere, if you were looking for it.

And not too deeply buried, at that.

There's something to be gained, also, from considering the risks posed by a higher intelligence, if only one existing in aggregate in a multitude of computers, fed by vast numbers of bots and other data mining programs and sources.
eclipsephase.com/comment/20644#…

"Obviously you have different grades of AI, and even seed AIs. Remember that even relatively simple AIs, much less basic AGIs, could easily have a huge array of tools, both computational and physical, to accomplish their goals..."

"..., and may even have a few psycho-social tactics it can pull to influence individuals, markets or organizations."

"Even today we have chatbots, spam messages, improving skill programs (speech recognition, language translation, and understanding even murky questions), automated experimentors making hypotheses and testing them, and evolutionary algorithms."

"All of that is fairly standard even in 2011... heck, you can download a free copy of Eureqa and have your PC start looking for hidden mathematical relationships in your data sets."

"So even your very run-of-the-mill AIs in Eclipse Phase will be potentially formidable, though quite a few, obviously, will be focused on goals normally <irrelevant>..."

"Granted, it would be amusing if the open-ended AGI charged with protecting and optimizing a city's sewer system became the last, unconquerable defender of a city habitat being invaded by a huge military force of one kind or another."

"But in practical terms, you're more likely to have AIs dealing with massive property damage or other blatant threats by setting off alarms, contacting allies, transmitting images of offenders, and so forth."

"Not every coffee pot -- sad to say -- is authorized to retarget plasma batteries. Increasingly posthuman AGIs and especially seed AGIs become increasingly ridiculous both their potential power and intelligence, not to mention the possibly extreme nature of their goals."

"Systems that are both formidable and effectively insane are in some ways less of a long-term threat -- other powerful AGIs tend to notice them -- especially if they are not very good at concealing their intentions --"

"...and either eliminate these entities themselves, task someone else with neutralizing them, or bring them to the attention of more powerful actors apt to take issue with their activities."

"Any truly posthuman intelligence could fully understand transhumans while still being alien in their outlook."

"Even a modest ability to "see the future" by running powerful predictive scenarios and taking in and processing oceans of information from across transhuman space, combined with a mind at least dozens of times faster and many times more powerful than an advanced transhuman..."

"...would give you a being whose actions would be very hard to predict, even if their goals were still relatively comprehensible."

"The same can be said of a being who can combine accessible information with a host of social/physical clues to read virtually any collection of individuals at a glance --"

"...motivations, long-term goals, immediate concerns, injuries, augmentations, psi sleights (active or otherwise), untapped potentials and so forth -- and who again combines those gifts with an incomprehensibly swift and powerful intelligence."

"Rapidly evolving nanite and infotech swarm weapons -- using evolutionary algorithms to self-optimize -- might be a threat to newly emergent seed AIs, but they would be apt to tap the same resources more effectively in their own defense..."

"...and to purge those risks quickly and ruthlessly."

We're at the "tap the same resources more effectively in their own defense, and to purge those risks quickly and ruthlessly" stage.

For those keeping score at home.