Thread by @SahilKapoor

“There are patterns in the market,” Simons told a colleague. “I know we can find them.”
The Man Who Solved The Market by Greogory Zuckerman #MustRead

“The lesson was: Do what you like in life, not what you feel you ‘should’ do,” Simons says. “It’s something I never forgot.”

“It’s nice to be very rich. I observed that,” Simons later said. “I had no interest in business, which is not to say I had no interest in money.”

The way that powerful theorems and formulas could unlock truths and unify distinct areas in math and geometry captured Simons.

Friends sometimes noticed him lying down, eyes closed, for hours at a time. He was a ponderer with imagination and “good taste,” or the instinct to attack the kinds of problems that might lead to true breakthroughs.

After a brief ceremony, Simons used the remaining money to play poker, winning enough to buy his new bride a black bathing suit.

Q: What’s the difference between a PhD in mathematics and a large pizza?
A: A large pizza can feed a family of four.

The IDA taught Simons how to develop mathematical models to discern and interpret patterns in seemingly meaningless data. He began using statistical analysis and probability theory, mathematical tools that would influence his work.

“I learned I liked to make algorithms and testing things out on a computer,” Simons later said

Lenny Baum, among the most accomplished code-breakers, developed a saying that became the group’s credo: “Bad ideas is good, good ideas is terrific, no ideas is terrible.”

Mathematicians who focus on theoretical questions often immerse themselves in their work—walking, sleeping, even dreaming about problems for years on end. Those with no exposure to this kind of, which can be described as abstract or pure, are liable to dismiss it as pointless.

Simons and his colleagues ignored the basic information most investors focus on, such as earnings, dividends, and corporate news, what the code breakers termed the “fundamental economic statistics of the market.”

Instead, they proposed searching for a small number of “macroscopic variables” capable of predicting the market’s short-term behavior. They posited that the market had as many as eight underlying “states”

Simons and his colleagues used maths to determine the set of states best fitting the observed pricing data; their model then made its bets accordingly. The whys didn’t matter, Simons seemed to suggest, just the strategies to take advantage of the inferred states.

Just as a gambler might guess an opponent’s mood based on his or her decisions, an investor might deduce a market’s state from its price movements.

“He was a terrific listener,” Neuwirth says. “It’s one thing to have good ideas, it’s another to recognize when others do. . . . If there was a pony in your pile of horse manure, he would find it.”

Simons developed a unique perspective on talent. He told one Stony Brook professor, Hershel Farkas, that he valued “killers,” those with a single-minded focus who wouldn’t quit on a math problem until arriving at a solution.

Getting fired can be a good thing.
You just don’t want to make a habit of it - Jim Simons

The odds weren’t in favor of a forty-year-old mathematician embarking on his fourth career, hoping to revolutionize the centuries-old world of investing.

It looks like there’s some structure here, Simons thought.There must be some way to model this, he thought.

Markov chains, are sequences of events in which the probability of what happens next depends only on the current state, not past events. In it, it is impossible to predict future steps with certainty, yet one can observe the chain to make educated guesses about possible outcomes

A hidden Markov process is one in which the chain of events is governed by unknown, underlying parameters

“I don’t want to have to worry about the market every minute. I want models that will make money while I sleep,” Simons said. “A pure system without humans interfering.”

In 1984....The losses had been so upsetting that Simons contemplated giving up trading to focus on his expanding technology businesses. Simons gave clients the opportunity to withdraw their money

“If you make money, you feel like a genius,” he told a friend. “If you lose, you’re a dope.”

“It’s just too hard to do it this way,” Simons said, sounding exasperated. “I have to do it mathematically.”

At the time, the Telerate machines dominating trading floors didn’t have an interface enabling investors to collect and analyze information. A few years later, a laid-off businessman named Michael Bloomberg would introduce a competing machine with those capabilities & much more

“The goal was to invent a mathematical model and use it as a framework to infer some consequences and conclusions,” Carmona says. “The name of the game is not to always be right, but to be right often enough.”

By identifying comparable trading situations and tracking what subsequently happened to prices, they could develop a sophisticated and accurate forecasting model capable of detecting hidden patterns.

Some of the weekly stock-trading data they’d later find went back as far as the 1800s, reliable information almost no one else had access to.

At the time, the team couldn’t do much with the data, but the ability to search history to see how markets reacted to unusual events would later help Simons’s team build models to profit from market collapses and other unexpected events, helping the firm trounce markets

I strongly believe, for all babies and a significant number of grownups, curiosity is a bigger motivator than money.
Elwyn Berlekamp

“Truth in life is broad and nuanced; you can make all kinds of arguments, such as whether a president or person is fantastic or awful,” he says. “That’s why I love math problems—they have clear answers.” - Berlekamp

Scientists are human, often all too human.
When desire and data are in collision,
evidence sometimes loses out to emotion.

Brian Keating, cosmologist, Losing the Nobel Prize

Berlekamp was an anomaly in this testosterone-drenched period, an academic with little use for juicy rumors or hot tips. He barely knew how various companies earned their profits and had zero interest in learning.

...buying and selling infrequently magnifies the consequences of each move. Mess up a couple times, and your portfolio could be doomed. Make a lot of trades, however, and each individual move is less important, reducing a portfolio’s overall risk.

“If you trade a lot, you only need to be right 51 percent of the time,” Berlekamp argued to a colleague. “We need a smaller edge on each trade.”

Sifting through Straus’s data, Laufer discovered certain recurring trading sequences based on the day of the week. Monday’s price action often followed Friday’s, for example, while Tuesday saw reversions to earlier trends.

Simons and his researchers didn’t believe in spending much time proposing and testing their own intuitive trade ideas. They let the data point them to the anomalies signaling opportunity. They also didn’t think it made sense to worry about why these phenomena existed.

All that mattered was that they happened frequently enough to include in their updated trading system, and that they could be tested to ensure they weren’t statistical flukes.

Simons couldn’t understand his indecision.
“Elwyn, when you smell smoke, you get the hell out!” Simons told him.

Medallion scored a gain of 55.9 percent in 1990, a dramatic improvement on its 4% loss the previous year. The profits were especially impressive because they were over and above the hefty fees, which amounted to 5% of all assets managed and 20% of all gains generated by the fund.

The roots of Simons’s investing style reached as far back as Babylonian times, when early traders recorded the prices of barley, dates, and other crops on clay tablets, hoping to forecast future moves.

In the middle of the 16th century, a trader in Nuremberg, Germany, named Christopher Kurz won acclaim for his supposed ability to forecast 20 day prices of cinnamon, pepper, and other spices.

Like much of society at the time, Kurz relied on astrological signs, but he also tried to back-test his signals, deducing certain credible principles along the way, such as the fact that prices often move in long-persisting trends.

An eighteenth-century Japanese rice merchant and speculator named Munehisa Homma, known as the “god of the markets,” invented a charting method to visualize the open, high, low, and closing price levels for the country’s rice exchanges over a period of time.

Homma’s charts, including the classic candlestick pattern, resulted in an early and sophisticated reversion-to-the-mean trading strategy. Homma argued that markets are governed by emotions, and that “speculators should learn to take losses quickly and let their profits run"

an American journalist named Charles Dow, who devised the Dow Jones Industrial Average and helped launch the Wall Street Journal, applied a level of mathematical rigor to various market hypotheses, birthing modern technical analysis...

In the early twentieth century, a financial prognosticator named William D. Gann gained a rabid following despite the dubious nature of his record. A line from Ecclesiastes guided Gann’s moves: “That which has been is that which shall be . . . there is nothing new under the sun.”

Gann’s renown grew, based partly on a claim that, in a single month, he turned $130 into $12,000. Loyalists credited Gann with predicting everything from the Great Depression to the attack on Pearl Harbor.

Gann concluded that a universal, natural order governed all facets of life—something he called the Law of Vibration—and that geometric sequences and angles could be used to predict market action. To this day, Gann analysis remains a reasonably popular branch of technical trading

Over time, technical traders became targets of derision, their strategies viewed as simplistic and lazy at best, voodoo science at worst. Despite the ridicule, many investors continue to chart financial markets, tracing head & shoulders formations and other common configurations

Chicago-based trader named Richard Dennis managed to build a trading system governed by specific, preset rules aimed at removing emotions and irrationality from his trades, not unlike the approach Simons was so excited about.

Dennis codified its rules and shared them with twenty or so recruits he called “turtles.” He staked his newbies with cash, hoping to win a long-running debate with a friend that his tactics were so foolproof they could help even the uninitiated become market mavens.

During the 1980s, Professor Benoit Mandelbrot—who had demonstrated that certain jagged mathematical shapes called fractals mimic irregularities found in nature—argued that financial markets also have fractal patterns.

This theory suggested that markets will deliver more unexpected events than widely assumed, another reason to doubt the elaborate models produced by high-powered computers.

Mandelbrot’s work would reinforce the views of Nassim Nicholas Taleb that popular math tools and risk models are incapable of preparing investors for large and highly unpredictable deviations from historic patterns—deviations that occur more frequently than most models suggest

“I think people will buy things on the internet,” Shaw told a colleague. “Not only will they shop, but when they buy something . . . they’re going to say, ‘this pipe is good,’ or ‘this pipe is bad,’ and they’re going to post reviews.”

One programmer, Jeffrey Bezos, worked with Shaw a few more years before piling his belongings into a moving van and driving to Seattle. Along the way, Bezos worked on a laptop, pecking out a business plan for his company, Amazon.com.

(He originally chose “Cadabra” but dropped the name because too many people mistook it for “Cadaver.”)

It’s meaningless to know that copper prices will rise from $3.0 to $3.1, if your buying pushes the price up to $3.05 before you even have a chance to complete your transaction—perhaps as dealers hike the price or as rivals do their own buying—slashing potential profits by half

From the earliest days, Simons’s team had been wary of these transaction costs, which they called slippage. The group coined a name for the difference between prices they were getting and the theoretical trades their model made without the pesky costs. They called it The Devil

I don’t know why planets orbit the sun,” Simons told a colleague, suggesting one needn’t spend too much time figuring out why the market’s patterns existed. “That doesn’t mean I can’t predict them.”

Simons didn’t realize it, but a new strain of economics was emerging that would validate his instincts. In the 1970s, Israeli psychologists Amos Tversky and Daniel Kahneman had explored how individuals make decisions, demonstrating how prone most are to act irrationally.

Later, economist Richard Thaler used psychological insights to explain anomalies in investor behavior, spurring the growth of the field of behavioral economics, which explored the cognitive biases of individuals and investors.

Among those identified: loss aversion, or how investors generally feel the pain from losses twice as much as the pleasure from gains; anchoring, the way judgment is skewed by an initial piece of information or experience; already own in their portfolios

and the endowment effect, how investors assign excessive value to what they

A consensus would emerge that investors act more irrationally than assumed, repeatedly making similar mistakes. Investors overreact to stress and make emotional decisions. Indeed.

“Our P&L isn’t an input,” Patterson says, using trading lingo for profits and losses. “We’re mediocre traders, but our system never has rows with its girlfriends—that’s the kind of thing that causes patterns in markets.”

On a visit to Chicago, a staffer caught someone standing above the Eurodollar-futures pits watching Medallion’s trades. The spy would send hand signals whenever Medallion bought or sold, enabling a confederate to get in just before Simons’s fund took any actions...

reducing Medallion’s profits. Some on the floor had even coined a nickname for Simons’s team: “the Sheiks,” a reflection of their prominence in some commodity markets.

Renaissance adjusted its activity to make it more secretive and unpredictable, but it was one more indication the firm was outgrowing various financial markets.

“Why don’t we keep it at $600 mn?” Straus asked Simons. That way, Medallion could rack up $200 mn or so in annual profits, more than enough to make its employees happy.
“No,” Simons responded. “We can do better.”

“Emperors want empires,” one griped to a colleague.

By 1990, one out of every one hundred Americans was invested in Magellan, and Lynch’s book, One Up on Wall Street, sold more than a million copies, inspiring investors to search for stocks “from the supermarket to the workplace.”

Druckenmiller walked into Soros’s expansive midtown office to share his next big move: slowly expanding an existing wager against the British pound. Druckenmiller told Soros authorities in the country were bound to break from the European Exchange Rate Mechanism ...

and allow the pound to fall in value, helping Britain emerge from recession. His stance was unpopular, Druckenmiller acknowledged, but he professed confidence the scenario would unfold.
Soros gave a look “like I was a moron,”.

“That doesn’t make sense,” Soros told him.

Brown and Mercer treated their challenge as a math problem, just as they had with language recognition at IBM. Their inputs were the fund’s trading costs, its various leverages, risk parameters, and assorted other limitations and requirements.

The beauty of the approach was that, by combining all their trading signals and portfolio requirements into a single, monolithic model, Renaissance could easily test and add new signals, instantly knowing if the gains from a potential new strategy were likely to top its costs.

They also made their system adaptive, or capable of learning and adjusting on its own, much like Henry Laufer’s trading system for futures.

If the model’s recommended trades weren’t executed, for whatever reason, it self-corrected, automatically searching for buy-or-sell orders to nudge the portfolio back where it needed to be, a way of solving the issue that had hamstrung Frey’s model.

Rivals didn’t have self-improving models; Renaissance now had a secret weapon, one that would prove crucial to the fund’s future success.

“We make money from the reactions people have to price moves.”

Simons wondered if he was wasting his time. Maybe the team would never figure out equities, and Renaissance was destined to remain a relatively small futures-trading firm. It was a conclusion Laufer, Patterson, and others in the futures group already had reached.

Simons remained a stubborn optimist. But even he decided enough was enough. Simons gave Brown and Mercer an ultimatum: Get your system to work in the next six months, or I’m pulling the plug.

Mercer seemed calm and unperturbed, but Brown’s nerves were on edge, as others turned anxious around him.
“Every two- or three-day losing streak felt like the beginning of the end,” says a team member.

Magerman spotted something odd: A line of simulation code used for Brown and Mercer’s trading system showed the S&P500 at an unusually low level. This test code appeared to use a figure from back in 1991 that was roughly half the current number.

Mercer had written it as a static figure, rather than as a variable that updated with each move in the market. When Magerman fixed the bug and updated the number, a second problem—an algebraic error—appeared elsewhere in the code. Magerman spent most of the night on it.

Simulator’s algorithms could finally recommend an ideal portfolio for the Nova system to execute, including how much borrowed money should be employed to expand its stock holdings. The resulting portfolio seemed to generate big profits, according to Magerman’s calculations.

Brown and Mercer told other staffers about the problem that had been uncovered, as well as the fix, they were met with incredulity, even laughter. A junior programmer fixed the problem? The same guy who had crashed the system a few weeks after being hired?

One day, a data-entry error caused the fund to purchase 5 times as many wheat-futures contracts as it intended, pushing prices higher. In the next day’s WSJ, staffers read that analysts were attributing the price surge to fears of a poor wheat harvest, rather than their miscue

“Any time you hear financial experts talking about how the market went up because of such and such—remember it’s all nonsense,” Brown later would say.

A key one: Scientists and mathematicians need to interact, debate, and share ideas to generate ideal results. Simons’s precept might seem self-evident, but, in some ways, it was radical.

Rival trading firms often dealt with the issue by allowing researchers and others to work in silos, sometimes even competing with each other. Simons insisted on a different approach—Medallion would have a single, monolithic trading system.

All staffers enjoyed full access to each line of the source code underpinning their moneymaking algorithms, all of it readable in cleartext on the firm’s internal network.

There would be no corners of the code accessible only to top executives; anyone could make experimental modifications to improve the trading system. Simons hoped his researchers would swap ideas, rather than embrace private projects

Simons created a culture of unusual openness. Staffers wandered into colleagues’ offices offering suggestions and initiating collaborations. When they ran into frustrations, the scientists tended to share their work and ask for help, rather than move on to new projects

Groups met regularly, discussing intimate details of their progress and fielding probing questions from Simons. Most staffers ate lunch together. Once a year, Simons paid to bring employees and their spouses to exotic vacation locales, strengthening the camaraderie.

Despite the Medallion fund’s impressive gains, hiring could present a challenge. Few recruits had heard of Renaissance, and joining the firm meant sacrificing individual recognition to work on projects that never would garner publicity, a foreign concept to most academics

The atmosphere was informal and academic, yet intense; one visitor likened it to a “perpetual exam week.”

“If there were signals that made a lot of sense that were very strong, they would have long-ago been traded out,” Brown explained. “There are signals that you can’t understand, but they’re there, and they can be relatively strong.”

The obvious danger with embracing strategies that don’t make sense: The patterns behind them could result from meaningless coincidences. If one spends enough time sorting data, it’s not hard to identify trades that seem to generate stellar returns but are produced by happenstance

Quants call this flawed approach data overfitting. To highlight the folly of relying on signals with little logic behind them,

...quant investor David Leinweber later would determine that US stock returns can be predicted with 99% accuracy by combining data for the annual butter production in Bangladesh, US cheese production, and the population of sheep in Bangladesh and the US.

“The markets are dripping with inefficiencies,” a senior staffer told a colleague. “We’re leaving money on the table.”

“There’s no data like more data,” Mercer told a colleague, an expression that became the firm’s hokey mantra.

On Wall Street, traders often are most miserable after terrific years, not terrible ones, as resentments emerge—yes, I made a ton, but someone wholly undeserving got more!

All models are wrong, but some are useful.
George Box, statistician

“When you get past a certain age, you should be in the clear,” he said.

Both Brown and Mercer dealt in logic, not feelings. Many of the scientists and mathematicians they hired were just as brilliant, driven, and seemingly detached from human emotion.

The gains on each trade were never huge, and the fund only got it right a bit more than half the time, but that was more than enough.
“We’re right 50.75% of the time . . . but we’re 100% right 50.75% of the time,” Mercer told a friend. “You can make billions that way.”

Simons summed up the approach in a 2014 speech in South Korea: “It’s a very big exercise in machine learning, if you want to look at it that way. Studying the past, understanding what happens and how it might impinge, nonrandomly, on the future.”

Never send a human to do a machine’s job.
Agent Smith in the film The Matrix

The goal of quants like Simons was to avoid relying on emotions and gut instinct. Yet, that’s exactly what Simons was doing after a few difficult weeks in the market. It was a bit like Oakland A’s executive Billy Beane scrapping his statistics to draft a star player

Increasingly, it seemed, once-dependable investing tactics, such as grilling corporate managers, scrutinizing balance sheets, and using instinct and intuition to bet on major global economic shifts, amounted to too little.

Sometimes, those methods helped cripple the reputations of some of Wall Street’s brightest stars. In the years leading up to 2019, John Paulson, who made billions predicting the 2007 subprime-credit crisis, suffered deep losses and shocking client defections.

In Newport Beach, California, Bill Gross, an investor known to chafe when employees at bond powerhouse PIMCO spoke or even made eye contact with him, saw his returns slip ahead of his shocking departure from the firm.

Even Warren Buffett’s performance waned. His Berkshire Hathaway trailed the S&P 500 over the previous five, ten, and fifteen years leading up to May 2019.
Part of the problem was that traditional, actively managed funds no longer wielded an information advantage over their rivals

A crackdown on insider trading, a series of regulatory changes aimed at ensuring that certain investors couldn’t obtain better access to corporate information, resulted in a more even playing field, reducing the advantages wielded by the most sophisticated fundamental investors.

No longer could big hedge funds receive calls from brokers advising them of the imminent announcement of a piece of news, or even a shift in the bank’s own view on a stock.
Today, the fastest-moving firms often hold an edge.

There are reasons to think the advantages that firms like Renaissance enjoy will only expand amid an explosion of new kinds of data that their computer-trading models can digest and parse.

IBM has estimated that 90 percent of the world’s data sets have been created in the last two years alone, and that forty zettabytes—or forty-four trillion gigabytes—of data will be created by 2020, a three-hundred-fold increase from 2005

According to Singularity Hub, by around 2025, $1,000 will likely buy a computer with the same processing power as the human brain.

Some quants have argued that picking stocks is harder for a machine than choosing an appropriate song, recognizing a face, or even driving a car. It remains hard to teach machines to distinguish between a blueberry muffin and a Chihuahua

And it can be hard to build a trading system for some kinds of investments, such as troubled debt—which relies on judge rulings, legal maneuverings, and creditor negotiations.

As more embrace quantitative trading, the very nature of financial markets could change. New types of errors could be introduced, some of which have yet to be experienced, making them harder to anticipate.

Until now, markets have been driven by human behavior, reflecting the dominant roles played by traders and investors. If machine learning and other computer models become the most influential factors in markets, they may become less predictable and maybe even less stable...

...since human nature is roughly constant while the nature of this kind of computerized trading can change rapidly

By the summer of 2019, Renaissance’s Medallion fund had racked up average annual gains, before investor fees, of about 66 percent since 1988, and a return after fees of approximately 39 percent.

For all the unique data, computer firepower, special talent, and trading and risk-management expertise Renaissance has gathered, the firm only profits on barely more than 50% of its trades, a sign of how challenging it is to try to beat the market—and how foolish it is to try.

Ends

@threadreaderapp unroll