Thread by @jgkoomey

This is a nice example of how speed of computation is a function of ALGORITHMS. More efficient algorithms make computation faster. It's not just about hardware (e.g. Moore's law) and in fact we will come to rely more on algorithms to speed things up as we approach physical limits

https://twitter.com/Rainmaker1973/status/1310919839955984384

Leiserson, C E., N C. Thompson, J S. Emer, B C. Kuszmaul, B W. Lampson, D Sanchez, and T B. Schardl. 2020. "There’s plenty of room at the Top: What will drive computer performance after Moore’s law?" Science. vol. 368, no. 6495. pp. eaam9744. [science.sciencemag.org/content/368/64…]

There's a field called "codesign", in which software and hardware are optimized together to make certain calculations faster.

Shalf, J., D. Quinlan, and C. Janssen. 2011. "Rethinking Hardware-Software Codesign for Exascale Systems." Computer. vol. 44, no. 11. pp. 22-30. [doi.ieeecomputersociety.org/10.1109/MC.201…]

Micheli, Giovanni De, Rolf Ernst, and Wayne Wolf, ed. 2001. Readings in Hardware/Software Co-Design (Systems on Silicon). Burlington, MA: Morgan Kaufmann.

Krueger, Jens, David Donofrio, John Shalf, Marghoob Mohiyuddin, Samuel Williams, Leonid Oliker, and Franz-Josef Pfreund. 2011. Hardware/software co-design for energy-efficient seismic modeling. Seattle, Washington: ACM.

Most people don't understand that long-term trends in computation are estimated based on general purpose computers, but by moving big homogeneous loads to SPECIAL PURPOSE computers optimized for particular computations, we can speed things up a lot.

Nordhaus, William D. 2007. "Two Centuries of Productivity Growth in Computing." The Journal of Economic History. vol. 67, no. 1. March. pp. 128-159. [econ.yale.edu/~nordhaus/home…]

Koomey, Jonathan G., Stephen Berard, Marla Sanchez, and Henry Wong. 2011. "Implications of Historical Trends in The Electrical Efficiency of Computing." IEEE Annals of the History of Computing. vol. 33, no. 3. July-September. pp. 46-54. [doi.ieeecomputersociety.org/10.1109/MAHC.2…]

Koomey, Jonathan, and Samuel Naffziger. 2016. "Energy efficiency of computing: What's next?" In Electronic Design. November 28. pp. [electronicdesign.com/microprocessor…]

For more on approaching physical limits, see the discussion of Feynman's schematic estimate of the efficiency of a three atom transistor, here:

Koomey, Jonathan G., H. Scott Matthews, and Eric Williams. 2013. "Smart Everything: Will Intelligent Systems Reduce Resource Use?" The Annual Review of Environment and Resources. vol. 38, October. pp. 311-343. [arjournals.annualreviews.org/eprint/wjniAGG…]

PS: Researchers at Purdue (the university not the chicken company) in collaboration w/ colleagues at the University of New South Wales and Korea institution of science and technology innovation, have demonstrated a one atom transistor that operates at liquid helium temperatures.

Fuechsle, Martin, Jill A. Miwa, Suddhasatta Mahapatra, Hoon Ryu, Sunhee Lee, Oliver Warschkow, Lloyd C. L. Hollenberg, Gerhard Klimeck, and Michelle Y. Simmons. 2012. "A single-atom transistor." Nat Nanotechnology. vol. 7, no. 4. 04//print. pp. 242-246. [dx.doi.org/10.1038/nnano.…]