Thread by @nimkef

20 Feb 2020

On a wall of fame for stars of the Chinese company were several former employees of Nortel, the Canadian telecommunications giant that suffered a spectacular collapse a decade ago.

nationalpost.com/news/exclusive…

“These are (now) Huawei employees associated with great tech accomplishments … & I recognized so many of them,” said Calof, a U. of Ottawa business prof visiting the site w/ MBA students. “At one level you’re proud to be a Canadian, at the same time you’re upset to be Canadian”

The ex-Nortel engineers’ place of honour in Shenzhen underscores how the two companies’ fortunes unfurled for years in striking parallel, and yet with starkly different outcomes.

They produced similar equipment, competed for the same contracts & tried to negotiate JVs. As one grew, the other collapsed. In Nortel’s waning days, Huawei backed a bid to keep it alive, only to ultimately walk away. Then snap up many of the bankrupt firm’s most-skilled staff.

For at least 10 years, it was revealed in 2012, the company was invaded by hackers based in China who stole hundreds of sensitive internal documents from under the noses of its top executives.

[Note:

"NorTel and Telecom people would probably pick something easy and universal. What password best fitted that description?

username: nortel

password: nortel

It worked.”

From Underground by Julian Assange, 1997]

Before that, the Canadian Security Intelligence Service (CSIS) warned Nortel of Beijing-led human spies in its midst. Later reports suggested that actual listening devices had been planted in Nortel’s Ottawa R&D complex, now Canada’s National Defence headquarters.

And never previously reported are allegations by former Nortel security personnel that a customer tied to Huawei returned a piece of equipment that had been pulled apart and “reverse engineered” to divine its secrets.

But on social media, and among some cyber-security experts and ex-Nortel staff, suspicions live on that Nortel died at least in part because its intellectual property was plundered, as an upstart Chinese rival soared past it in the telecom industry.

Such views will not be disabused by a new indictment that U.S. prosecutors filed last week. It accuses Huawei of using various methods to steal other companies’ intellectual property for decades.

“What people need to hear is that economic espionage caused Nortel’s failure,” insists Brian Shields, the security advisor who uncovered the massive hack. “So others better beware lest they succumb to the same fate.”

That’s by no means a universally held view.

Nortel had in fact been in trouble for years, suffering huge losses when the dotcom bubble burst, never recovering from that disaster and making widely criticized management decisions during its final years.

In a major study into the reasons behind the company’s demise, Calof and colleagues at his university’s Telfer School of Management did not even mention the hacking or other espionage.

https://twitter.com/nimkef/status/1298846167523119105?s=20

Despite its stellar reputation for developing cutting-edge technology, customers eventually just weren’t buying what it had to sell, he said.

“They lost sales not because of technology copying, not because of inferior technology, they lost because the customers lost faith in them,” says Calof. “They did not believe that Nortel would be alive in 10 years.”

As part of what is now a $15B annual R&D budget, Huawei has spent $650M on R&D over the last decade in Canada, where it has 1,200 employees, notes company spokesman Alykhan Velshi.
Huawei Canada’s vice-president of corporate affairs Alykhan Velshi.

Birthed by Bell Telephone, once known as Northern Electric and then Northern Telecom, it led the way in developing digital telephone networks worldwide in the 1970s and 1980s.

By the turn of the last century, having expanded to provide a wide array of Internet-based networking solutions, it boasted over 90,000 employees and a market capitalization that accounted for a third of the worth of companies on the Toronto Stock Exchange.

Its technological prowess is still legendary. Though never put into production, Nortel researchers in Ottawa even developed the Orbitor, a mobile phone with a user-interface screen, nine years before the iPhone revolutionized the market with such a device.

Then, in 2000, the speculative Internet bubble that had so elevated Nortel suddenly burst. The company’s stock plummeted overnight, decimating Canadian pension funds and triggering thousands of Nortel layoffs.

The 2000s were generally rocky years for the company, marked early on by an accounting scandal that saw CEO Frank Dunn and two other executives charged with fraud — though later acquitted — followed by a desperate struggle for survival.

If the telecom industry and Nortel itself can be traced 146 years back to Alexander Graham Bell’s family farm outside Brantford, Ont., Huawei is a relative baby.

Founded by Ren Zhengfei, a former People’s Liberation Army engineer, it began as a small producer of phone switches in 1987, branching out into building telecommunications networks and making mobile phones.

Helped by relatively cut-rate prices, it soon began gobbling up market share.

By 2012, it had surpassed Ericsson as the world’s biggest supplier of telecommunications equipment, and in 2018 became the second-largest smartphone producer.

In 2005, when Huawei won a small but significant part of British Telecom’s huge 21Century Network project, its first breakthrough in the West.

Nortel and British counterpart Marconi were frozen out; within a few years both would be gone.

Battling to turn around the company, Nortel struck a tentative deal with its Chinese rival in 2006. The JV to develop “ultra” broadband networks was later quietly shelved, and subsequent joint venture talks with Huawei also fizzled, then-CEO Mike Zafirovski later revealed.

Nortel suffered another Huawei blow, on home turf no less. In 2008, the Chinese firm bested it to win a major contract with Telus and Bell. In January, 2009, Nortel filed for bankruptcy protection.

Even then, Huawei figured in its story. A group of former Nortel exec, convinced Nortel could be turned around w/ better management, put together a buy-out bid. It would be backed by Huawei acting as a minority shareholder. The bid collapsed when Huawei’s board rejected the deal

Ren has offered a different version of events, telling the Globe and Mail last year he was interested in buying Nortel, but that the plan died after the nearly insolvent Nortel said it wanted a controlling share of Huawei.

What didn’t become public until years later was espionage traced back to China.

Michel Juneau-Katsuya was head of the CSIS Asia-Pacific desk in the late 1990s when the service became aware of “spying activities the Chinese were conducting” against Nortel.

“What we knew from my point of view was about the agents, the people, human actors in and around Nortel,” he said. “Definitely Nortel was targeted.”

When the intelligence agency warned the company, it all but ignored CSIS. This led Juneau-Katsuya to a startling conclusion:

“To this day, I believe there might have been one or more agents of influence controlled by the Chinese in [Nortel] which succeeded in neutralizing our warning.”

A little later, around 2000, U.S.-based security staff inside the company believe they got an early taste of corporate espionage from China with an alleged incident never before publicized. It involved Huawei itself, say three former employees.

With an office across the freeway from a Nortel facility in Texas, Huawei returned a fibre card — essentially a computer device — used in Nortel data switches and asked for a refund, recalls Lawrence Bill, a forensic analyst who worked on the subsequent investigation.

Former colleagues Tony Anastasio and Brian Shields confirmed the incident, though say they recall it was a front company that had bought the equipment and passed it on to Huawei.

Regardless, when Nortel engineers looked closely, they realized the “bleeding-edge” gear had been disassembled and reverse engineered, says Bill.

“It was returned in pieces, where things were taken apart,” recalled Anastasio, who was an assistant to Nortel’s VP of security then

Meanwhile, the company started noticing knock-off versions of some of its products in Asian markets, he says.

Nortel considered suing, but dropped the matter after the Huawei office across the road in Texas closed down, says Bill.

Velshi says he asked counterparts in Huawei around the world if they knew anything about such an episode 20 years ago and came up blank. The Canadian government has never accused it of anything like that, he says.

It first came to light in the spring of 2004 when a Nortel employee in the U.K. noticed some documents he’d stored in the company’s “LiveLink” database had been downloaded by a senior executive in Canada.

The Brit helpfully emailed the manager — optical-networks president Brian McFadden — to say he was available to answer any questions McFadden might have about the material. The executive’s response? I have no idea what you’re talking about.

Nortel’s security staff in Raleigh, N.C., were promptly alerted.

Larry Bill, based in Raleigh, noticed a troubling fact: Logs indicated that McFadden had signed into the Nortel system from multiple locations around the world, places he had never visited.

“That’s what threw all the alarms,” Bill recalls today. “We just knew that we had a major problem, that an executive’s credentials are being hacked and being used to exfiltrate documents.”

The hacking of McFadden’s account turned out to be just the tip of the iceberg.

Security advisor Brian Shields discovered that not one, but seven Nortel executives, including CEO Frank Dunn, had been hacked, and that the hackers were vacuuming an alarming volume of sensitive material out of its databases.

By the end of his investigation, Shields says he was able to track the theft of over 1,400 documents from the LiveLink server, and that was only during a six-month period when bosses allowed him to monitor the stealing.

He found evidence the break-in of Nortel’s internal computer network had started no later than 2000, and probably began in the 1990s. He says it lasted past 2009, when he was laid off.

He traced most of the hacks back to IP addresses and four Internet service providers (ISPs) in China. When material was actually downloaded from Nortel, it mostly ended up at an ISP in Shanghai.

The structured nature of the IP addresses used to siphon off those Nortel files, not to mention the sophistication that allowed the incursion to go undetected for years, point to the involvement of a skilled, government-directed outfit, Shields says.

He cites a 2013 report by cyber-security firm Mandiant, which revealed the existence of a major Internet-espionage organization in Shanghai, likely “Unit 61398” of the People’s Liberation Army. Mandiant tracked thefts of data from 141 companies in 20 major industries.

What was pilfered from Nortel during the short window when Shields tracked the losses?

The reams of material included a doc that laid out current Nortel technology and the direction various products were headed; a sales proposal that would include pricing and network design;

technical papers on aspects of optical circuits; and an analysis of how Nortel lost a contract with former Internet firm Genuity.

“It was sickening then and it is sickening now to see what was stolen,” says Shields, Nortel’s rep on the Network Security Information Exchange, a U.S. government initiative to help protect the national telecom infrastructure.

“This was a very capable adversary. You have to be pretty darn good to achieve such a level of stealth.”

He has no evidence of who ultimately received the docs, but notes that only a Nortel competitor would benefit from the info

According to Shield, one document taken by the hackers in 2004 was called “high speed data over UMTS Quad.”

Four years later, Huawei beat out Nortel on that Telus/Bell contract. It was its first major project in North America, says the company’s milestones page, involving a form of mobile data transmission called universal mobile telecommunications service — UMTS.

Shields cannot prove that Huawei benefited from the hacking, but is convinced that its rise to a world telecommunications superpower — as Nortel simultaneously withered away — is no coincidence.

NP has viewed the first two pages of the internal report Shields and Bill prepared in 2004, the remaining 12 taken up by a list of the stolen docs.

And yet he is certain the Nortel CEO never saw that report.

His investigation wound down after a few months, and it appears no one notified firms that later bought Nortel assets that its computers might be infected.

Early 2009 CSIS offered to help Nortel with the hacks, Shields says, but by then it was too late. Within a week, Nortel had filed for bankruptcy. Concerned that buyers of Nortel units were unaware of the Chinese intrusions, Shields finally revealed the story to the WSJ in 2012.

A few years later, as the National Defence Department prepared to take over Nortel’s former research campus in Ottawa, it discovered evidence of another type of spying — old-school listening bugs implanted in the building during Nortel days, a senior Defence officer said.

DoJ indictment last Thursday replaces one filed earlier against Huawei and its CFO, Meng Wanzhou, now under arrest in Vancouver in response to an American extradition request. It alleges a systematic campaign by the company to steal secrets from six U.S. technology companies.

Prosecutors accuse Huawei of using employees of rival firms, university researchers working for it covertly and other methods to pilfer intellectual property.

In one incident, a Huawei employee was caught opening up the networking equipment of other Co.'s at a tech conference in Chicago & photographing the boards. Nortel was among a list of 6 competitors the engineer had w/ him when nabbed by a security guard.

The 2014 study that Calof led, arguably the most extensive analysis of why Nortel failed, involved dozens of interviews with the company’s competitors, former executives and customers.

It made no mention of IP theft, but attributed Nortel’s fall to a medley of problems inc. “hubris” among its leaders that made it unresponsive to customer needs; ill-chosen purchases of other Co.'s in the 1990s and a “black cloud” that hung over it due to the financial troubles.

Nortel failed to secure the British Telecom contract because it did not offer what the operator’s tender requested, ironically providing a foothold for Huawei itself, Calof argues.

He concedes that some of the Canadian firm’s adversaries may have gained an unfair and accelerated leg-up in the market by stealing technology.

But the business professor stresses that others in the telecos & networking equipment industry were subject to IP theft by Asian actors as well, and to lower-cost competition like Huawei. Most of those firms, such as Cisco, Nokia, Ericsson and Alcatel-Lucent, survived

Meanwhile, telecom operators have often been quoted as lauding Huawei’s speedy work, fine products and responsiveness to customers.

Others, though, still suspect that Nortel’s once-sparkling future was not lost, it was brazenly stolen

It’s “entirely plausible” that tech robbed and copied by a rival was the final straw that took down an already struggling Co., says David Skillicorn, a Queen’s U computing prof

Bill, the former forensic analyst, says he believes Nortel was hit by a perfect storm: poor management in later years, discount pricing by Huawei and a hack that funneled away its “crown jewels.”

“It was shocking,” he says, “when we finally started pulling back the covers.”