Thread by @twitemp1

Government Office for Science: Future Risks of Frontier AI
Definition and object:
Frontier AI: "highly capable general-purpose AI models that can perform a wide variety of tasks and match or exceed the capabilities present in today’s most advanced models"
assets.publishing.service.gov.uk/media/653bc393…

"As of October 2023, this primarily encompasses foundation models consisting of very large neural networks using transformer architectures"

(e.g., LLMs, "generative" AI)

Risks:
"Frontier AI will include the risks we see today, but with potential for larger impact and scale. These include enhancing mass mis- and disinformation, enabling cyber-attacks or fraud, reducing barriers to access harmful information, and harmful or biased decisions."

AGI:
"debate has intensified on whether or when [] AGI might be realised. However, the risks and opportunities posed by a given model derive from its capabilities, and how it is used, not the breadth of tasks at which it can match human performance. "

"Frontier models could be disruptive, beneficial, powerful, or risky without being an AGI."

Indeed!

"Risk and opportunity will be shaped by uncertain factors including geopolitics, access, ownership, safety measures and public attitudes"

Back to humans, good!

"Given the significant uncertainty, there is insufficient evidence to rule out that future Frontier AI, if misaligned, misused or inadequately controlled, could pose an existential threat. However, many experts see this as highly unlikely."

"AI safety is a socio-technical challenge that cannot be resolved with technical interventions alone. Industry, academia, civil society, governments and the public all have an important role to play."

Current capabilities highlighted as relevant to Frontier AI risk:

Content Creation
Computer vision
Planning and reasoning
Theory of mind
Memory
Mathematics
Accurately predicting the physical world
Robotics
Autonomous Agents
Trustworthy AI

Future risk scenarios:

1 Unpredictable Advanced AI
2 Disrupts the Workforce
3 AI ‘Wild West’
4 Advanced AI on a knife edge
5 AI Disappoints

Categories of future risks based on risks evident today:
"
a. Providing new capabilities to a malicious actor.
b. Misapplication by a non-malicious actor.
c. Poor performance of a model used for its intended purpose, for example leading to biased decisions.
>>

d. Unintended outcomes from interactions with other AI systems.
e. Impacts resulting from interactions with external societal, political, and economic systems.
f. Loss of human control and oversight, with an autonomous model then taking harmful actions.
>>

g. Overreliance on AI systems, which cannot subsequently be unpicked.
h. Societal concerns around AI reduce the realisation of potential benefits."

"Beyond technical measures, the decisions and actions of humans can shape risks posed by future Frontier AI. These decisions will include how AI is designed, regulated, monitored and used."

*Need to understand the potential for consequential risk.

Potential non-technical mitigations:
"a. Requiring reporting of training of large-scale models.
b. Requiring safety assessments such as containment, alignment, or shut-down systems within risk management process.
>>

c. Transparency measures from development through to deployment and use that enable oversight and interventions where necessary."

Overall, it is a thoughtful document, necessarily vague, but a good attempt to highlight real AI problems and uncertainties.