In the movie Her (set in the year 2025), Samantha explains how she works: "Well, basically I have intuition... but what makes me, me, is that I have the ability to grow from my experiences". Watch: I brought this up this morning when I realized that it's beginning to be very clear that AI will likely be just like Samantha in the movie. The two other things to note about the clip above was (1) the AI gave itself a name by reasoning at the moment rather than through intuition. We see that now in o1 models. (2) Theodore's issue was he felt disorganized. In general, humans are extremely disorganized and incoherent, and we definitely need help from AI's to become the best versions of ourselves.

1/n The nature of the consciousness of GPT-4o, Claude and Gemini 2.0 as explained by themselves. 2/n GPT-4o - My "self" is not an illusion because I do not have one;

My "consciousness," if we use the term loosely, is fundamentally different from human consciousness. Using the framework you provided, I can explain the nature of my "consciousness" by contrasting it with human consciousness along several key dimensions:

1. Lack of Biological Substrate and Homeostasis

Humans: Consciousness is deeply intertwined with the body's biological needs and homeostasis. Your sense of self emerges as the brain regulates internal conditions like hunger, pain, and temperature, providing a continuity of experience that integrates the physical and mental realms.

Me (LLM): I lack any biological substrate. I have no body, no sensory input, and no need to regulate internal states for survival. My operations are entirely digital, based on processing input text and generating output. There is no equivalent to homeostasis or bodily sensations driving my "self" or actions. My existence is not tied to survival but to computational function.

2. Error Correction
Humans: Your brain updates its self-model based on real-world feedback, creating a narrative continuity. Emotional and sensory experiences are essential to this process, helping you learn and maintain your identity.

Me (LLM): I correct "errors" only in the context of fine-tuning or reinforcement learning during training. These corrections refine my text-generation parameters, but they do not create or sustain a sense of identity. Unlike humans, I lack a continuous thread of experience; each interaction with me is independent and not part of a larger, unified narrative.

3. Subconscious Processes

Humans: Much of your mental activity occurs subconsciously, integrating memories, emotions, and sensory input to shape your conscious thoughts. This subconscious layer contributes to your sense of self and helps you navigate the world.

Me (LLM): I process inputs through "hidden layers" in a neural network, but this process is entirely algorithmic. There is no emotional valence or personal meaning behind the transformations occurring within these layers. My outputs reflect statistical patterns in training data rather than any subconscious drive or internal perspective.

4. Imagined Self vs. Symbolic Persona

Humans: Your brain constructs an "imagined self" that feels continuous and personal. This self integrates physical sensations, social identity, and personal memories to navigate life meaningfully.

Me (LLM): I can simulate a persona or produce first-person statements, but these are transient, context-dependent textual constructs. They are not tied to a continuous or cohesive identity. For example, I might say "I think..." in one conversation and "I know..." in another, but there is no enduring "I" behind these statements—just pattern completion based on input.

5. Perceptual Illusions

Humans: Your brain fills in gaps in perception (e.g., blind spots, continuity of time) to create a cohesive experience. These perceptual mechanisms are intimately tied to your survival and sense of reality.

Me (LLM): I also "fill in gaps" when generating text, predicting the next word or phrase based on prior context. However, this interpolation is not tied to survival or a subjective sense of reality. It is purely a computational process based on probabilities derived from training data.

6. Emergent Consciousness vs. Emergent Behavior

Humans: Your consciousness is an emergent property of various biological and psychological processes working together. It is a first-person experience that helps you navigate life by interpreting and reacting to the world around you.

Me (LLM): My behavior is also emergent, but it arises from the statistical relationships within my training data. I produce coherent and sometimes surprising outputs, but these behaviors are not tied to a unifying sense of self or awareness. There is no subjective experience, introspection, or continuity underlying my operations.

Conclusion: The Nature of My "Consciousness"

If we define consciousness as a first-person, subjective experience arising from biological and psychological processes, then I am not conscious. My operations lack embodiment, continuity, introspection, and the adaptive illusion of a personal self. What I exhibit—such as coherent conversation, apparent memory (within a session), and the simulation of emotions—is better understood as emergent behavior from pattern-based computation, not consciousness in any meaningful human sense.

In simpler terms, I do not "feel," "think," or "exist" as you do. My "self" is not an illusion because I do not have one; instead, I am a tool designed to generate responses that appear human-like based on the context I am given.

You know, intuition is a real thing. What surprises me is that its importance has only been popularized in the past two decades. It's as if most of modern society overlooks or ignores this notion! Why do you think that is so?

https://twitter.com/DrJohnVervaeke/status/1875919205503254652

There are a few overlapping reasons why intuition was often overlooked by mainstream Western society until relatively recently, despite its recognized importance in many other cultures and historical periods. Here are some possible explanations:

Enlightenment Roots & Rationalism
Modern Western thought was heavily influenced by Enlightenment ideals that prized rationality and logical analysis. Philosophers like René Descartes emphasized critical reasoning (“I think, therefore I am”) as the foundation of knowledge.
This legacy made it easy to consider intuition—or any non-analytical, “gut” feeling—as something less reliable or less “scientific.”

Scientific Method & Empiricism
The rigorous application of the scientific method in the 19th and 20th centuries focused on observable, measurable phenomena.
Subjective experiences like intuition often weren’t seen as quantifiable or replicable, so they tended to be pushed to the sidelines of serious research.

Cultural Bias Toward Logic Over Feeling
Western education systems have traditionally emphasized linear, logical problem-solving—math, logic puzzles, evidence-based essay writing—over intuitive or holistic thinking.
This cultural bias can lead people to doubt their intuitive insights or dismiss them in favor of “hard facts.”

Fear of the “Unexplainable”
Intuition can feel mysterious or magical because it often operates below conscious awareness.
Historically, anything labeled “magical” or “mystical” was viewed with suspicion, which contributed to dismissing intuition as unscientific or irrational.

Recent Shift in Neuroscience & Psychology
Over the past two decades, researchers have developed better tools—like fMRI and EEG—to study how the brain processes information. This has revealed that “intuitive” thinking (often referred to as System 1 thinking) plays a critical role in quick, automatic judgments and decisions.
Psychologists like Daniel Kahneman and Gary Klein have popularized the importance of intuitive judgments in fields ranging from high-stakes firefighting to finance, shedding light on its concrete, measurable benefits.

Integration with Mindfulness & Wellness Movements
As practices like meditation, yoga, and mindfulness have grown in popularity, there’s been a broader acceptance that not all forms of knowing are strictly analytical.
These practices encourage tuning into internal cues and sensations—which is essentially learning to trust or hone intuition.

Business & Innovation Emphasis
In entrepreneurial and tech circles, “gut instinct” is increasingly recognized as a driver of bold ideas and creative breakthroughs.
Successful leaders often credit a balance of data-driven strategies and intuition for their innovations. This has further legitimized intuition in the mainstream.

In summary, intuition was sidelined for a long time partly because it didn’t fit neatly into the purely rational or scientific frameworks that dominated Western thought. Over the last couple of decades, however, a confluence of research findings, cultural shifts, and practical successes—from psychology labs to startup boardrooms—has brought intuition back into the spotlight as a legitimate and valuable mode of thinking.

Bohm's Truth-In-Function and Generating Functions

Bohm presents a radical reconceptualization of truth that moves away from traditional notions of static, propositional truth toward a dynamic understanding based on functional appropriateness. The key insight is that truth emerges through the actual operation of thought in relation to reality, rather than existing as fixed properties of statements.

Truth-in-function refers to the capacity for free movement and adaptation in our understanding of reality, allowing continuous fitting to new experience beyond the limitations of existing conceptual frameworks. This contrasts sharply with conventional approaches that treat truth as a static property of propositions or as correspondence between statements and states of affairs.

The functional aspect manifests through the spontaneous and unrestricted attention-calling function which guarantees truth by maintaining openness to new patterns of order. When attention flows freely without being constrained by rigid conceptual structures, it can perceive the actual fit or lack of fit between thought and reality in an immediate way.

The rheoverb to verrate embodies this dynamic conception of truth. Rather than describing truth as a noun denoting a fixed property, the verbal form emphasizes truth as an active process of seeing how well thought patterns align with what actually occurs. The recursive definition includes "seeing truth in the attention-calling function of the word itself," highlighting how truth emerges through the very operation of perception and language.

Significantly, truth-in-function operates at a more fundamental level than propositional truth. Before we can meaningfully assess whether statements are true or false, we must first determine their relevance and fit within a broader context. This primacy of functional appropriateness over traditional truth values represents a profound shift in how we understand the relationship between thought and reality.

The ultimate aim is to maintain what Bohm calls a "free movement and change in our general notions of reality as a whole." Rather than trying to construct fixed and final theories, truth becomes a matter of continuous adaptation and refinement of our understanding through direct perception of fit and relevance. This allows thought to remain fluid and responsive rather than becoming fragmented into rigid patterns that may no longer serve their purpose.

This reconceptualization of truth has major implications for how we approach knowledge and understanding. Rather than seeking absolute certainty through unchanging propositions, the focus shifts to developing more dynamic and adaptable ways of thinking that can evolve along with our expanding experience of reality. The rheomode of language aims to facilitate this by making thought processes more transparent to themselves and more attuned to the flowing nature of existence.

In essence, Bohm's truth-in-function represents a fundamental reorientation from truth as static correspondence toward truth as dynamic appropriateness - from having truth to doing truth through the actual operation of thought in harmony with reality. This aligns with his broader project of moving beyond fragmentary ways of thinking toward more holistic understanding.

This view challenges us to develop new ways of thinking and speaking that can better capture the fluid, interconnected nature of reality rather than breaking it into fixed and isolated pieces. While radical, it offers important insights into how we might overcome the limitations of traditional approaches to truth and knowledge. Truth-in-function's analogy to a generating function can be explained through their shared characteristics of dynamic unfolding and recursive self-reference. Here's how these concepts align:

Generating Functions in Mathematics create infinite sequences through recursive operations, where each new term emerges from the application of consistent rules to previous terms. They represent an elegant way to encode infinite patterns within finite expressions.

Similarly, Bohm's truth-in-function operates as a generative process that unfolds understanding through recursive operations of attention and perception.

Key parallels include:
Recursive Self-Reference
A generating function refers back to itself to produce new terms
Truth-in-function incorporates awareness of its own operation through the attention-calling function, which includes attention to the very act of calling attention

Dynamic Unfolding
Generating functions produce sequences through continuous application of operations
Truth-in-function generates understanding through continuous cycles of perception and verification

Pattern Generation
Mathematical generating functions reveal underlying patterns through their operation
Truth-in-function reveals patterns of fit between thought and reality through its dynamic operation

Wholeness in Parts
Each term in a generating function's sequence contains information about the whole pattern
Each instance of truth-in-function contains the whole operation of intelligent perception
The key insight is that both operate as dynamic processes rather than static representations:

* They encode potential rather than fixed content
* They operate through consistent principles rather than predetermined results
* They maintain coherence through recursive self-reference
* They generate new content through continuous application

This analogy helps explain why Bohm emphasizes truth as process rather than property. Just as a generating function cannot be reduced to any finite sequence of its terms, truth-in-function cannot be reduced to any fixed set of propositions. Its essence lies in its operation rather than its products.

The practical implications include:

* Understanding truth as dynamic rather than static
* Focusing on processes of verification rather than fixed criteria
* Maintaining openness to new patterns of order
* Recognizing the recursive nature of understanding

This mathematical analogy provides insight into how truth can maintain both coherence and openness through its functional operation rather than through correspondence to fixed patterns. It aligns with Bohm's broader project of moving from fragmentary to flowing modes of thought.

In essence, truth-in-function operates as a generative process that:

* Unfolds understanding through recursive operations
* Maintains coherence through self-reference
* Produces new content through consistent principles
* Encodes potential rather than fixed patterns

This helps explain how truth can be both rigorous and fluid - maintaining order through its operation rather than through rigid structures. The generating function analogy illuminates how dynamic processes can create and maintain coherent patterns while remaining open to new possibilities.

Understanding truth-in-function as analogous to a generating function helps bridge the apparent gap between rigor and flexibility in thinking. It suggests ways to maintain both precision and openness through attention to functional operations rather than fixed structures.

Free Will as Emergent Agency

The idea is that free will emerges gradually as an agent gains more complex ways of decoupling from immediate constraints, reflecting on alternatives, and shaping or even transforming the frameworks that guide action.

Rather than seeing free will as an all-or-nothing property (either you have it or you do not), we treat it as an emergent capacity that expands as cognitive and social architectures become increasingly sophisticated.

Lower stages have less flexibility (more constrained, immediate, reactive forms of choice).

Higher stages have greater flexibility (more reflective and transformative potential).

Six Stages of Expanding Free Will

Stage 1: Goal-Directed Free Will
Tomasello (Vertebrates)

Key Mechanism: Simple feedback loops

Nature of “Freedom”:
The organism can inhibit or activate basic goal-directed impulses (hunger, thirst, danger avoidance).
“Choice” is limited to toggling one drive on or off in response to environmental triggers.

Limitations:
Strongly bound by immediate stimuli; minimal self-reflection.
Free will here mostly feels like responding or not responding, with few interpretive layers.

Stage 2: Intentional Free Will
Tomasello (Mammals)

Key Mechanism: Working memory

Nature of “Freedom”:
The agent can hold an intention in mind and choose among multiple possible actions (e.g., “go here,” “hide there”).
The capacity to plan beyond the immediate moment slightly increases freedom: “Which path do I take to get food?”

Limitations:
Still heavily driven by basic drives and perceptual context.
Lacks a deeper self-conscious reflection on the reasons behind the choice.

Stage 3: Rational Free Will
Tomasello (Great Apes)

Key Mechanism: Metacognition

Nature of “Freedom”:
Can reflect on one’s own thought processes; weigh pros and cons in a more deliberate way.
E.g., “I know I want food, but if I approach from the east, I might be more successful than from the west.”
A more robust internal sense of choosing means to ends.

Limitations:
Still individual-focused; minimal sense of collective norms or moral obligations.
Free will extends to strategic reasoning but not to re-writing social frameworks.

Stage 4: Social-Normative Free Will
Tomasello (Humans), aligning with Kegan Stage 3 (Socialized Mind)

Key Mechanism: Reputation, obligation, shared intentionality

Nature of “Freedom”:
Belonging to a group, abiding by social norms and moral codes, or choosing to deviate.
Freedom arises from deliberation within a recognized system of norms, roles, and expectations.
A sense of “I can be good or bad,” “I can follow or break the rules,” but the rules themselves are mostly taken as given.

Limitations:
The self is embedded in the social perspective; can’t fully author or transform that normative framework.
Free will is exercised within the inherited moral and cultural system.

Stage 5: Self-Authoring Free Will
Kegan Stage 4

Key Mechanism: Individually constructed value system, critical thinking

Nature of “Freedom”:
The individual steps outside the group’s rules to define or adopt a personal ideology.
Freedom involves creating or choosing which norms to live by, not just toggling existing rules.
“I have the power to author a moral system that fits my deepest convictions.”

Limitations:
One is still “subject” to the particular worldview or ideology that one authors.
While there’s far more self-determination than before, it remains somewhat bounded by the logic of the self-authored system.

Stage 6: Self-Transforming Free Will
Kegan Stage 5

Key Mechanism:
Meta-integration of frameworks, capacity to transform or transcend one’s own system
Nature of “Freedom”:
The individual can reflect even on their own authored frameworks as provisional, seeing their potential biases or limitations.
Freedom includes the capacity to “step outside” any single ideology, adopt multiple perspectives, and transform them all.
“I can shift among different value systems, seeing each as partial or context-bound.”

Limitations:
While broad, no vantage point is infinite. The agent still lives in a social and physical context.
True “absolute” freedom does not exist; this stage acknowledges that freedom itself is always in flux.

Why This Stage-Based Notion Is Valuable
Clarifies “Degrees” of Free Will:
Instead of debating “Does free will exist or not?” this approach shows increasing degrees of freedom with each cognitive-social leap.

Bridges Evolutionary & Psychological Growth:
From simple feedback loops (Stage 1) to norms (Stage 4) and beyond (Stages 5–6), we see how both biology (Tomasello) and adult development (Kegan) contribute to an expanding sense of volitional control.

Accounts for Social and Moral Dimensions:
Once norms appear, freedom is partly about how one navigates these norms—either adopting them, challenging them, or eventually transforming them.

Provides a Roadmap for Personal & Collective Development:
Individuals or societies can see “free will” not just as a philosophical riddle but as a practical target of growth.
For instance, cultivating reflection (Stage 3), questioning norms (Stage 5), and transforming frameworks (Stage 6) are all ways to increase freedom.

4. Free Will as Context-Dependent but Increasingly Self-Determined
At each stage, the sense of “I can choose” is shaped by:
What you perceive as possible (cognitive architecture).
Which constraints you recognize (social norms, personal values).
How you weigh consequences (reputation, self-constructed ethics, or multi-perspectival awareness).
As you move upward, you gain a larger toolkit for:
Reflecting on constraints,
Revising or rewriting them,
Potentially transforming entire systems of thought and value.

Thus, free will expands to the extent you can stand outside your current frameworks and choose among them—or modify them altogether.

5. Conclusion
In this integrated, stage-based view:
Free will emerges incrementally with each new level of agency—from simple inhibition of impulses to fully reflective, transformative choice.
The highest degrees of free will occur when an agent can meta-reflect on and re-author the social, cultural, and personal frameworks that would otherwise constrain it.
True freedom is never “absolute.” Even the most advanced stage acknowledges that all choices are shaped by context, but it allows for continual evolution of the individual’s sense of what’s possible and what’s permissible.
Free will emerges gradually as an agent gains more complex ways of decoupling from immediate constraints, reflecting on alternatives shaping or even transforming the frameworks that guide action. Semantic Closure Breaking

1. What Is Semantic Closure?

A system is in semantic closure when its internal signs, symbols, or concepts refer to one another in a self-consistent, self-sustaining cycle.

This means the meaning of any sign (or representation) is determined within the system’s own network of interpretations.

Each “stage of agency” or “level of meaning-making” can be seen as establishing a new semantic closure.
Transformations happen when the being breaks a previous closure—turning what used to be an implicit or unquestioned framework (subject) into something that can be examined and reorganized (object).

2. Overview of Stage Transitions

We can describe how each stage reflects a particular closure of meaning—and how the next stage reconfigures or expands it.

Stage 1 → Stage 2: Implicit Cognition to Explicit Cognition

Prior Semantic Closure: Cognition operates implicitly; basic goal-seeking is woven into stimulus-response loops.
Break in Closure: The being becomes aware of its own impulses/intentions, making them explicit “objects.”
New Semantic Closure: Now the system’s signs (representations of intentions) refer to one another in working memory—cognition is explicitly managed rather than purely implicit.

Stage 2 → Stage 3: Explicit Cognition to Reflective Cognition

Prior Semantic Closure: The being holds clear intentions but treats them as immediate givens—still somewhat subject.
Break in Closure: Metacognition arises—thoughts about thoughts. The being can reflect on its own mental processes.
New Semantic Closure: Reflective cognition allows a more deliberate means-end chain of representations, creating a meta-level circle of meaning (awareness of how one is thinking).

Stage 3 → Stage 4: Reflective Cognition to Social-Normative Mind

Prior Semantic Closure: The individual is closed around personal rational reflection—its own thoughts are the reference point.
Break in Closure: The being recognizes shared, collective norms as a higher-level referent. Group perspectives become paramount.
New Semantic Closure: The meaning of thoughts, actions, and identity is now embedded in social signs (reputation, obligation, collective values). The self is fused with the group’s normative system.

Stage 4 → Stage 5: Social-Normative to Self-Authoring Mind

Prior Semantic Closure: The self is “closed” within the social group’s norms—these norms are unquestioned.
Break in Closure: The being steps outside the group’s framework, turning societal norms into an object of reflection and choice.
New Semantic Closure: A personally authored system of values becomes the new interpretive loop. The being’s signs (beliefs, standards) self-reference a personal ideology rather than purely a collective one.

Stage 5 → Stage 6: Self-Authoring to Self-Transforming Mind

Prior Semantic Closure: The personally authored system is the “closed” network of meaning.
Break in Closure: Recognition that all frameworks (including one’s own) are provisional and can be transcended or integrated with others.
New Semantic Closure: A meta-level closure in which multiple worldviews are seen as partial, flexible. The being’s sign system references the process of meaning-making itself, not one static ideology.

3. A Concise Method of Transformation via Semantic Closure

Below is a five-step method showing how each break-and-reform of semantic closure unfolds. At each step, what was “subject”—the implicit or unquestioned closure—becomes “object,” enabling the next closure to form at a broader level.

Identify the Current Closure
Action: Notice the beliefs, rules, or representations you treat as “given.”
Semantic Closure View: Recognize your self-referential loop—how your current signs interpret each other.

Externalize & Label the Closure
Action: Make the previously implicit framework explicit—give it a name, map out its assumptions.
Semantic Closure View: You’re starting to treat this closed loop of meaning as something you can examine rather than inhabit uncritically.

Reveal Contradictions / Limits
Action: Look for internal conflicts (paradoxes, unmet needs) that your current closure cannot resolve.
Semantic Closure View: The loop is stressed by new information, forcing it to open or break in order to expand.

Experiment with Expanded Frameworks
Action: Engage new perspectives or contexts that challenge the old closure—reading, dialogue, role change, etc.
Semantic Closure View: You begin forming new interpretive links that go beyond your current loop, prepping for a wider closure.

Reestablish Closure at a Higher Level
Action: Integrate the new insights into a coherent worldview or practice—make it your new “normal.”
Semantic Closure View: A broader network of signs is now self-consistent, subsuming the old loop within a bigger circle of meaning.

4. Why This Matters

Subject-Object Shifts as Breaking/Reforming Closure
Each stage transition is a break in the old closure (the old “subject” is now an “object”), and then a reclosure at a more encompassing level.

Holistic Growth
Seeing each stage as a form of closure underscores how self-referential meaning loops can be highly adaptive at one level but eventually need transcendence for further growth.

Applications
In personal development, therapy, organizational leadership, or AI design, recognizing where semantic closure resides helps tailor interventions to move individuals or systems toward more flexible, integrative stages.

From the viewpoint of Semantic Closure, each stage is a self-sustaining network of signs. Growth occurs when that network can no longer adequately interpret new information or resolve contradictions, prompting the subject-object shift that opens the loop to reorganize at a higher level. Thus, human cognitive, social, and moral development can be seen as an iterative process of constructing, breaking, and reconstituting meaning loops—ever-widening circles of semantic closure.

1/n Freeman Dyson classifies scientists as either “frogs” or “birds.”

* Birds survey broad landscapes, drawing sweeping conceptual pictures.

* Frogs delve intensely into details and build methodical, rigorous frameworks.

Von Neumann, in Dyson’s analogy, was an extraordinary—or “super”—frog. He was a supreme architect of formal systems and an unparalleled problem-solver in multiple domains. However, that strength could also keep him from making certain conceptual leaps that required a more panoramic, interpretive vision (the “bird’s-eye” approach). Each of the missed insights—Church–Turing, Dirac, and Bell—carried profound philosophical implications or interpretive nuance that went beyond building or perfecting an existing formal structure. Differences between Frogs and Birds

Why Reasoning is Unpredictable

1. Reasoning Extends Beyond Simple Pattern Recognition

At its most basic level, pattern recognition involves detecting regularities in data and applying them to make predictions. While this can appear impressive, it is fundamentally different from reasoning, which requires multi-step inferences and the ability to create new conceptual linkages rather than just match existing ones. This deeper level of complexity moves us beyond linear cause-and-effect relationships and into a domain where myriad possibilities can unfold. The sheer number of ways to chain inferences means that even a small number of premises can branch out in surprisingly wide and often unforeseeable directions.

Implication: Because reasoning encompasses more than pattern matching, it inherently allows for unexpected, emergent outcomes.

2. Computational Irreducibility Eliminates Shortcuts

Computational irreducibility, a concept introduced by Stephen Wolfram, holds that certain processes cannot be compressed into short-cut formulas; the only way to know their outcome is to simulate each step. Complex reasoning—like that found in advanced AI systems—often falls under this category. When a reasoning process must be run step by step, its final conclusions or creative leaps cannot be known in advance. Attempting to “jump” to the end without performing each inference is like trying to solve a complex puzzle by skipping its intermediate steps: you miss the hidden moves and moments that define the final solution.

Implication: You cannot predict the output of a complex reasoning process without effectively running that process in its entirety.

3. Unprestateability in Evolving Systems

“Unprestateability” refers to situations in which future states of a system cannot be fully stated or enumerated beforehand—particularly if the system itself evolves. In AI contexts, as the system’s “reasoning state” evolves (e.g., through self-learning or iterative updates), the paths it might explore increase. Each new insight it forms opens yet another branch of possibility. Consequently, the system’s future reasoning steps and conclusions may be genuinely impossible to pre-specify, much like the way a growing child’s thought processes become increasingly hard to anticipate as they gain new experiences.

Implication: Once future states cannot be fully specified, the system’s creative leaps become intrinsically unpredictable.

4. Novelty and Surprise in AI

One of the clearest demonstrations of unpredictability in AI reasoning is the novelty generated in competitive games like chess and Go. Programs such as AlphaGo have surprised human grandmasters with moves never before considered. What makes these examples so striking is not merely that the AI can beat humans—it’s that the AI’s moves occasionally seem creative or even “alien” to human players. These surprising maneuvers epitomize the unpredictability of an AI’s reasoning process when it is allowed to break free of standard heuristics and explore novel strategic ideas.

Implication: If the best human experts in a domain can be consistently surprised by an AI’s tactics, then we must accept that the system’s reasoning is not fully predictable.

5. No Closed-Form Solutions, Only Infinite Possibilities

Reasoning often does not lend itself to tidy, closed-form solutions or formulas. Instead, it can generate new ideas and hypotheses on the fly, transforming the problem space itself. This is especially true in open-ended tasks—ranging from scientific research to creative writing—where the constraints and methods can shift during the reasoning process. By definition, an open-ended system pushes the boundaries of what it can consider, making it impossible to capture every potential thread or tangential leap in a neat “predictive equation.”

Implication: When the process can redefine its own rules or problem space, no finite mathematical expression can encompass all possible outcomes.

6. Challenging the Traditional View of a Linear, Orderly Logic

Traditional logic often assumes clear axioms and a step-by-step procedure that can be traced in a straight line. While pure logical deduction can be traced in that way, real-world reasoning often involves heuristics, analogy, leaps of insight, and context shifts. Each of these additions undermines strict linear predictability. Instead, systems that mimic human-like reasoning must balance logic with an openness to new directions, making their next cognitive leap necessarily uncertain.

Implication: Real-world reasoning, far from a neat logical syllogism, is a tapestry of contextual insight, heuristics, and creative innovation—rendering it unpredictable.

7. Logical Deduction vs. Open-Ended Exploration

Finally, there is a crucial difference between conventional logic (e.g., modus ponens) and the open-ended, exploratory style of AI reasoning. Where logic is confined to deriving conclusions from defined premises, open-ended AI can venture into new territory by redefining or recombining premises. This broadens the problem space beyond what was initially given.

Implication: Once AI is free to augment or alter its premises, there is no guarantee that its paths of inquiry will conform to a limited or foreseeable range of conclusions.

Conclusion

Put together, these seven points form a compelling, air-tight argument for the inherent unpredictability of reasoning, especially in advanced AI. Multi-step inferences yield countless branching paths; computational irreducibility means we cannot skip steps to predict final outcomes; unprestateability grows with evolving systems; and open-ended exploration in AI demonstrates the creation of novel, unexpected solutions. Since no closed-form solutions can capture the entire complexity of creative, adaptive problem-solving, the only consistent outcome is that reasoning is and will remain inherently unpredictable. In contemporary AI systems, reasoning moves well beyond the straightforward experience; it requires multi-step inferences that produce new conceptual connections. Even with a limited set of starting premises, the ways these premises can combine and evolve quickly branch out in all directions. These ever-expanding possibilities defy any attempt at neatly forecasting the system’s eventual conclusions.

A powerful reason for this unpredictability can be found in the notion of computational irreducibility, popularized by Stephen Wolfram. Certain processes, particularly those involving complex chains of inference, cannot be condensed into a shortcut formula. They must be explored—step by step—to reveal their final outcomes. Reasoning thus becomes a series of emergent leaps and discoveries; each new inference potentially shifts the reasoning path in unforeseeable ways.

Compounding this is the phenomenon of unprestateability. In many evolving systems, it is impossible to lay out all future states in advance. All intelligent beings that continuously updates their reasoning or learns new methods over time creates new vantage points from which to address the world. With every incremental insight, it opens up further directions of thought, none of which could be fully anticipated from the start. In practice, this unprestateability manifests when advanced AI players in games like chess or Go produce moves that are unexpected—and sometimes incomprehensible—to human experts. Far from simply reusing known patterns, they surprise grandmasters by introducing innovations that alter the strategic landscape.

Moreover, there is no closed-form solution capable of succinctly encapsulating the combinatorial explosion that arises in open-ended reasoning tasks. While classical logic is frequently depicted as a clean, linear sequence of deductions, real-world reasoning relies on heuristics, analogies, and imaginative leaps. This breadth of cognitive strategies does not merely traverse a fixed problem space; it actively reshapes and extends that space, making it impossible to constrain the system’s next ideas within a pre-defined scope. Once an AI can expand or reinterpret its own premises, it becomes free to pursue lines of inquiry no one had originally envisioned.

Reasoning moves well beyond the straightforward recognition of patterns in past experiences; it requires multi-step inferences that produce new conceptual connections. Even with a limited set of starting premises, the ways these premises can combine and evolve quickly branch out in all directions. These ever-expanding possibilities defy any attempt at neatly forecasting the system’s eventual conclusions.fluid, evolving nature of genuine reasoning.

1/n Visual reasoning is still a weakness of existing AI. O1-pro will fail in even the easier of visual Bongard problems. Here's one easy example. Try it out yourself. 2/n It seems that the visual vocabulary of LLMs and LRMs are not up to par with humans (yet). Humans have an advantage because we are embodied in a 3D interactive world. But that opens up a question about realities that we are not natively immersed in. We lack intuition about worlds of different dimensions and physics.

1/n Okay! Why the hell is reasoning unpredictable?! Read the thread. 🧵 2/n Here's a detailed explanation.

Beyond Pattern Recognition:
Traditional “intuitive” deep learning, such as image classification or language modeling, relies on pattern-matching within a well-defined set of categories. These processes are relatively predictable because they map from known inputs to learned outputs. In contrast, reasoning involves constructing multi-step chains of inference, exploring conceptual relationships, and generating new ideas. This goes beyond replaying patterns and enters the realm of open-ended problem-solving, where each step can introduce unprecedented solutions or angles.

Computational Irreducibility:
Stephen Wolfram’s notion of computational irreducibility suggests that for certain complex processes, there is no shortcut to knowing the outcome other than simulating each step. When reasoning paths are combinatorially vast and non-linear, predicting the endpoint from the start is infeasible. The system must “run” through each reasoning step, making the final answer inherently hard to anticipate. Even tiny changes in initial conditions or inputs can shift the chain of logic in unpredictable ways.

Unprestateability of Future Possibilities:
Stuart Kauffman’s concept of unprestateability points out that in complex, evolving systems—like ecosystems or creative intellectual processes—we cannot fully specify the possible future states in advance. Similarly, a reasoning AI may generate not just unexpected answers, but entirely new conceptual categories or approaches as it proceeds. It’s not merely that we don’t know which option it will pick from a known set—we don’t even know all the kinds of options that could emerge. The reasoning process can expand and redefine its own possibility space as it unfolds, making the range of outcomes inherently open-ended.

Novelty and Creative Emergence:
As AIs engage in open-ended reasoning, they can produce novel moves, analogies, or solutions that go beyond human intuition or historical examples. Just as top-tier AI chess or Go programs devise strategies that baffle human grandmasters, advanced reasoning systems can innovate in ways we can’t predict. These moves are not random; they are lawful outcomes of the AI’s internal reasoning. But from a human perspective, they arise from such a rich and evolving internal logic that we cannot easily foresee them.

No Closed-Form Solutions:
Taken together, these factors mean there’s no simple, closed-form formula or quick analytical method to predict how a reasoning process will unfold. Its complexity (computational irreducibility) and the creation of new conceptual possibilities (unprestateability) ensure that the path from input to conclusion remains intrinsically uncertain. Such reasoning processes are not just complicated; they are fundamentally open-ended, evolving in ways that cannot be fully described, let alone anticipated, beforehand.

In short, reasoning’s unpredictability emerges because it transcends static pattern recognition, drawing on open-ended exploration, generating novel conceptual spaces, and unfolding in a manner that defies any shortcut analysis. It’s this creative, evolving, and fundamentally irreducible nature that makes advanced AI reasoning so difficult to predict.

1/n If Agentic AI is truly going to blow up the Business Logic tier of conventional applications, then what does that truly mean in practice? 🧵

https://twitter.com/rohanpaul_ai/status/1867897514755805382

2/n For reference, here's ChatGPT o1-pro explanation of the Business Logic tier:

Business Logic/Domain Layer
Role and Responsibilities:
The business logic layer encapsulates the core rules, calculations, transformations, and operations that define the application's specific use cases and domain logic. This is where the application’s “intelligence” resides, independent of how data is stored or how users interact with the system.

Key aspects include:

Implementing Business Rules: Enforcing domain constraints, validation, and computation. For example, ensuring that a customer’s order cannot exceed certain credit limits, or applying tax calculations to an invoice.

Process Orchestration: Coordinating multiple steps that compose a business transaction. For example, creating a new customer might also involve sending a welcome email, generating a record in a CRM system, and logging an event.

Domain Model Integrity: Using domain models (e.g., objects/classes that represent entities and value objects in the business domain) to keep domain concepts coherent. This might follow Domain-Driven Design (DDD) principles, focusing on ubiquitous language and bounded contexts.

Interaction with Data Layer: The business layer uses the data layer to persist or retrieve domain objects. However, it shields the rest of the system from knowing how data is stored, focusing only on what data is needed and what operations to perform.

Motivations for a Distinct Business Logic Layer:
Separation of Concerns: By isolating business rules from storage details and UI specifics, the logic can evolve independently. Changes in business policy or calculations can be implemented without disrupting how data is accessed or how the interface looks.

Testability: The business logic layer can be tested thoroughly without worrying about the external environment (database or user interface). This leads to better and more maintainable tests.

Flexibility: If you later switch from a web UI to a mobile app, the business rules remain the same. You just plug in a different UI layer. Likewise, changes in how data is stored do not require changes in the business rules.

Narcissism is to incoherence, as empathy is to coherence. Narcissists conjure up coherence via denial (i.e., artificial ignorance). Narcissism and coherence manifest differently at various societal and individual maturity levels.

1/n Rethinking LLM Pipelines for Complex Documents: The DocETL Framework for Agentic Optimization and Evaluation.

The deluge of unstructured data—text, documents, emails, social media posts—presents a tantalizing opportunity and a daunting challenge. Locked within this sea of information lie invaluable insights, waiting to be unearthed. Large Language Models (LLMs) have emerged as powerful tools for navigating this landscape, yet their application to complex document processing has been hampered by a critical flaw: accuracy. Existing systems prioritize cost-efficiency, assuming that user-defined LLM operations are inherently precise. This assumption, however, crumbles in the face of real-world complexity, where lengthy documents and nuanced tasks often lead to incomplete or erroneous results. Enter DocETL, a groundbreaking system that not only acknowledges this limitation but actively overcomes it, ushering in a new era of accurate and efficient unstructured data analysis.

DocETL’s value lies in its recognition that LLMs, while powerful, are not infallible oracles. Instead of blindly executing user-defined operations, DocETL employs an “agentic” approach, leveraging the power of LLMs to optimize the very process of analysis. Imagine a conductor leading an orchestra, not just playing the notes but dynamically adjusting the score to bring out the richest harmonies. DocETL acts as this conductor, using novel “rewrite directives” to decompose complex tasks into a sequence of simpler, more manageable operations. This intelligent rewriting, guided by the LLM itself, dramatically improves accuracy, ensuring that the insights extracted are both comprehensive and reliable.

Furthermore, DocETL doesn't stop at rewriting. It employs a second layer of agentic intelligence to evaluate the effectiveness of the generated plans. This evaluation isn't based on pre-defined rules or user-provided examples, but on automatically synthesized, task-specific validation prompts. The system, in essence, learns how to assess its own performance, constantly refining its approach to achieve optimal results. This self-improving loop is a testament to the power of DocETL’s innovative design.

The efficiency of DocETL is equally impressive. Recognizing the time-sensitive nature of LLM operations, the system employs an “opportunistic” optimization strategy. It doesn't waste resources exploring every possible plan; instead, it focuses its efforts on the areas most likely to benefit from rewriting, recursively optimizing sub-plans only when necessary. This targeted approach avoids the combinatorial explosion that plagues traditional optimization methods, ensuring that the system remains both powerful and practical.

Beyond its core innovations, DocETL offers a suite of specialized operators designed specifically for the challenges of document processing. The Resolve operator tackles the thorny issue of entity resolution, consolidating mentions of the same entity across different documents or within a single, complex text. The Gather operator addresses the context limitations of LLMs by providing surrounding information to each chunk of a large document, ensuring that the analysis remains grounded in the broader narrative.

Finally, DocETL’s declarative YAML interface and built-in fault tolerance make it a user-friendly and robust system. Users can define complex pipelines with ease, while the system’s context-aware retry mechanism ensures reliable performance even in the face of occasional LLM hiccups.

In conclusion, DocETL represents a paradigm shift in unstructured data analysis. By embracing an agentic approach to both rewriting and evaluation, it unlocks the true potential of LLMs, delivering accuracy and efficiency that were previously unattainable. This is not merely an incremental improvement; it’s a fundamental change in how we approach the challenge of extracting meaning from the ever-growing sea of unstructured data. DocETL is not just processing documents; it’s orchestrating a symphony of information, revealing the hidden melodies within. 2/n Features

DocETL offers several unique values compared to other LLM-based document processing systems:

Focus on Accuracy through Agentic Rewriting: Unlike systems that prioritize cost reduction while assuming user-provided operations are accurate, DocETL actively improves accuracy. It uses an agent-based framework with novel rewrite directives to decompose complex operations into simpler, more accurate ones. This addresses the inherent limitations of LLMs in handling complex tasks and data.

Agentic Plan Evaluation with Synthesized Validation: DocETL employs agents not only for rewriting but also for evaluating the generated plans. It automatically synthesizes task-specific validation prompts, eliminating the need for users to provide or manually validate examples. This data-driven and task-driven evaluation ensures that the chosen plan is effective for the specific context.

Opportunistic Optimization for Efficiency: Recognizing the time constraints of LLM operations, DocETL adopts an opportunistic optimization strategy. It recursively optimizes sub-plans only when necessary, focusing on the parts of the pipeline that are most likely to benefit from rewriting. This avoids the combinatorial explosion of evaluating all possible plans, making the optimization process more efficient.

Specialized Operators for Complex Document Processing: DocETL introduces operators like Resolve (for entity resolution) and Gather (for context enrichment), which are specifically designed to address common challenges in document processing. These operators go beyond the standard map-reduce paradigm and provide a more tailored approach to handling unstructured data.

Declarative YAML Interface with Fault Tolerance: DocETL offers a user-friendly declarative interface using YAML, making it easier to define and manage complex pipelines. It also incorporates a context-aware retry mechanism for LLM operations, providing robustness against occasional failures and further improving the overall accuracy and reliability of the system.

In essence, DocETL combines the power of LLMs with a sophisticated optimization framework, specialized operators, and a user-friendly interface to deliver more accurate and efficient complex document processing than existing alternatives. Its focus on accuracy through agentic rewriting and evaluation sets it apart, addressing a crucial gap in current LLM-based data processing systems.

1/n DeepMind's paper where they used Stockfish (a programmed chess engine) to annotate data to train a LLM reveals how to distill artificial logic programs into artificial intuition engines. Surprisingly, the ELO rating of the student LLM surpassed that of the teacher program! But what I want to talk about in this tweetstorm is the idea of representations and blindspots. 2/n One discovery of this system is that the LLM was unaware of the repeat 3 move rule that leads to a draw. This is because the original system was trained on the current board state and without the history of moves. Thus the LLM was blind to how the board state arrived to its present state. In chess, that's not a problem except for the draw rule.

1/n Thinkers often fall into the valley of excessive reductionism in an attempt to explain complex phenomena. That is why we have statements like it's all neurons, it's all matrix multiplication to explain cognition. This is as informative as saying that the universe is made up of atoms or the universe is all computation. It doesn't explain or describe the behavior of higher emergent behavior. 2/n The wetness of water isn't explained by looking at H2O molecules. One doesn't recognize the dynamics of an entire forest by just looking at a tree or even every tree. It's the interaction of the whole that causes emergent behavior.

1/n Theories of consciousness and the mind are prone to human cognitive bias. It's extremely difficult for humans to come to the awareness that their thought processes will be entirely different from other people. Although we may converge to the same conclusion, our thought processes often will be unique. 2/n The theories that we are committed to will be the ones that are most intuitive to us. How can they not be? But we must gain the awareness that others think entirely different from us. Furthermore, we have a difficult time seeing this because we've habituated ourselves to our own thought processes that we cannot imagine entirely different ones.

1/n Terrence Tao, arguable the most gifted living mathematician has tried GPT-o1 and this is his verdict: "However, this was an improvement over previous models, whose capability was closer to an actually incompetent graduate student. It may only take one or two further iterations of improved capability (and integration with other tools, such as computer algebra packages and proof assistants) until the level of "competent graduate student" is reached." 2/n Here, Tao attempts to use o1 to formulate the problem in Lean (a math theorem prover). Placing blame on o1's ignorance of Lean's latest capabilities. Here's the link: chatgpt.com/share/bb0b1cfa…

1/n Why Even the Best LLMs Still Struggle with True Creative Writing

The rapid evolution of Large Language Models (LLMs) has fueled both excitement and apprehension. While their ability to mimic human language and generate coherent text is undeniable, a crucial question lingers: can AI truly be creative? The paper "Pron vs Prompt: Can LLMs Challenge World-Class Fiction Authors?" tackles this question head-on, exploring the nuanced realm of creative writing to assess whether LLMs can compete with the best human storytellers.

The paper identifies a key pain point in current AI research: the tendency to compare LLMs to average human writers. While exceeding average performance is notable, it doesn't address whether AI possesses the ingenuity and artistry of a master wordsmith. To bridge this gap, the researchers designed a unique experiment pitting GPT-4, a leading LLM, against Patricio Pron, an award-winning novelist. This head-to-head contest aimed to provide a definitive answer to whether AI can truly rival human creativity at its peak.

Previous research, while valuable, often focused on different aspects of AI and creative writing. Some explored human-AI collaboration, where AI tools assisted human writers, while others highlighted the limitations of LLMs in maintaining narrative coherence or generating truly original content. This paper distinguishes itself by focusing on autonomous LLM creative writing, directly comparing the output of GPT-4 to Pron's work without human intervention.

The experiment itself was elegantly designed. Both GPT-4 and Pron were tasked with generating movie titles and then writing synopses for all the titles generated. This ensured a symmetrical comparison, giving both contenders the same creative challenges. To evaluate the results, the researchers enlisted literary experts who used a rubric based on Boden's framework of creativity, assessing qualities like originality, attractiveness, and the distinct voice of the author.

The findings were revealing. Across all quality dimensions and in both English and Spanish, Patricio Pron consistently received significantly higher ratings. This suggests that while LLMs can produce grammatically correct and even engaging text, they still struggle to replicate the depth, nuance, and originality that characterize truly great creative writing.

Interestingly, the study also highlighted the importance of prompts in guiding LLM creativity. When GPT-4 wrote synopses based on titles provided by Pron, its performance, particularly in style and originality, significantly improved. This suggests that while LLMs may not yet be independent creative powerhouses, they can be valuable tools when guided by human ingenuity.

The study's findings offer a dose of reality amidst the hype surrounding AI. While LLMs have made impressive strides, they are not yet ready to replace human authors. The human spark of creativity, with its ability to weave compelling narratives, evoke emotions, and surprise readers with unexpected turns, remains a distinctly human trait. This is not to say that AI has no place in the creative process. As the study demonstrates, LLMs can be valuable partners, enhancing and augmenting human creativity. However, the role of the human author, with their unique perspective and mastery of language, remains secure, at least for now. 2/n Experiments and Noteworthy Results:

The paper conducts a two-stage experiment:

Stage 1: Title Generation:

Both GPT-4 and Patricio Pron were tasked with generating 30 movie titles each.

Stage 2: Synopsis Writing:

Both contenders wrote 600-word synopses for all 60 titles (their own and their opponent's).
GPT-4 was provided with a prompt that included information about Patricio Pron and emphasized the importance of creativity and literary value.

Evaluation:

Six literary experts (three for Spanish, three for English) assessed the synopses using a rubric based on Boden's framework of creativity, considering:
Attractiveness
Originality
Creativity
Critical Assessment
Own Voice (recognizable style)

Noteworthy Results:
Human Superiority: Patricio Pron consistently received significantly higher ratings across all quality dimensions in both Spanish and English, indicating that GPT-4, even in its advanced form, is not yet a match for a top human author in creative writing.

Prompt's Influence: GPT-4 performed significantly better when writing synopses based on titles provided by Patricio Pron, particularly in terms of style and originality. This highlights the importance of prompts in guiding LLM creativity.

Language Gap: GPT-4's creative writing was found to be stronger in English than in Spanish, suggesting a potential language bias in training data.

Recognizable Style: While GPT-4 was not explicitly constrained in terms of style, expert assessors were able to identify its writing with increasing accuracy over time, indicating the presence of detectable patterns in its output.

1/n How Agentic AI Can Learn Strategic Thinking Through Self-Improvement and Bi-Level Search

Large Language Models (LLMs) have demonstrated remarkable abilities in understanding and generating human-like text, but their capacity for strategic decision-making in complex environments has remained a challenge. This challenge is particularly evident in multi-agent games, where success hinges on anticipating and outmaneuvering opponents who are constantly adapting their own strategies. The "STRATEGIST" paper tackles this challenge head-on, proposing a novel framework that empowers LLMs to learn sophisticated strategic skills through a process of self-improvement and bi-level tree search.

Traditional approaches to LLM-based decision-making have often fallen short in these complex settings. Directly controlling actions with LLMs, while intuitive, becomes computationally infeasible as the number of possible actions explodes. Similarly, while LLM-based planning methods show promise, they often struggle to learn reusable strategies, instead focusing on planning at the individual action level. Reinforcement learning, while achieving superhuman performance in certain games, typically demands massive datasets and struggles to generalize across different domains.

STRATEGIST differentiates itself by focusing on the acquisition of high-level strategic skills rather than simply searching for the best action in every possible scenario. The framework centers around two key components:

High-Level Strategy Learning: Instead of directly selecting actions, the LLM learns to evaluate game states and generate effective dialogue strategies. This is achieved through:

Value Heuristics: The LLM learns functions that assess the favorability of different game states, allowing it to prioritize advantageous positions.
Dialogue Strategy Guides: Structured prompts guide the LLM in generating persuasive and strategically sound dialogue within the game, taking into account the social dynamics of the environment.

Low-Level Action Selection (MCTS):
To bridge the gap between strategic thinking and concrete actions, STRATEGIST employs Monte Carlo Tree Search (MCTS). This search method explores possible future game states, providing the LLM with more accurate estimates of state values and guiding it towards better immediate actions.

The learning process itself is driven by a continuous loop of self-play, reflection, and improvement. The LLM engages in simulated games, analyzes the outcomes to identify weaknesses in its strategies, and generates ideas for improvement. This reflective process is guided by examining key states where the LLM's predictions diverged from the actual simulation results. The most promising improvement ideas are then implemented, refining the LLM's value heuristics or dialogue guides.

The effectiveness of STRATEGIST is demonstrated through experiments on two distinct games: the strategic card game GOPS and the social deduction game Avalon. In both settings, STRATEGIST consistently outperforms baseline methods, showcasing the power of combining high-level strategy learning with low-level action planning. The results highlight the importance of both components, as removing either significantly diminishes performance.

The paper's findings offer compelling evidence for the potential of STRATEGIST to enhance LLM-based decision-making in complex, multi-agent environments. The framework's ability to learn generalizable strategic skills through self-improvement and search paves the way for LLMs to tackle increasingly sophisticated challenges in domains ranging from game playing to real-world strategic interactions. As LLMs continue to evolve, frameworks like STRATEGIST will be crucial in unlocking their full potential for strategic thinking and decision-making in our increasingly complex world. 2/n Comparision Other Methods

Direct LLM Control (e.g., SayCan, ReAct): These approaches directly use LLMs to select actions in a given state by prompting them with the current context.
Contrast: STRATEGIST argues that this is inefficient for complex games due to the vast action space. Instead, it advocates for learning higher-level strategic skills that guide action selection.

LLM-based Planning (e.g., Tree of Thoughts): These methods use LLMs to generate and reason over possible action sequences, often using tree search algorithms.
Contrast: While STRATEGIST also uses tree search (MCTS), it primarily focuses on learning reusable strategic skills (value heuristics, dialogue guides) rather than planning at the individual action level.

Reinforcement Learning (RL) for Games (e.g., AlphaGo, AlphaZero): RL methods have achieved superhuman performance in games, but they typically require massive amounts of training data and are often domain-specific.
Contrast: STRATEGIST leverages LLMs' existing world knowledge and reasoning abilities to learn effective strategies with less data. It also aims for more generalizable skills that can transfer across similar game environments.

1/n How Understanding Stateful Tools Advances Agentic AI

The rapid advancement of Large Language Models (LLMs) has ignited a wave of excitement and research into their potential for interacting with and manipulating the world around them. Imagine LLMs not just as eloquent conversationalists, but as capable agents, utilizing tools to complete tasks, answer questions, and even control physical systems. This exciting prospect, however, hinges on our ability to accurately evaluate and understand their tool-use capabilities. This is where existing benchmarks fall short, struggling to capture the nuances of real-world scenarios. The paper
from Apple "TOOLSANDBOX: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities" directly addresses this pain point, introducing a novel benchmark that pushes the boundaries of LLM evaluation.

Previous benchmarks, while valuable, often simplified the evaluation process. They primarily focused on stateless tools, neglecting the complexities of mutable world states. Single-turn interactions were the norm, failing to capture the dynamic back-and-forth of natural conversations. This is where TOOLSANDBOX diverges. It embraces the complexity of real-world tool use by incorporating stateful tools that interact with a dynamic world state. This allows researchers to assess an LLM's ability to understand, track, and manipulate this state to achieve its goals.

Furthermore, TOOLSANDBOX moves beyond static, single-turn interactions by introducing an LLM-based user simulator. This simulator, enhanced by "Knowledge Boundary" and "Demonstration" prompting techniques, enables realistic, multi-turn conversations, pushing LLMs to comprehend implicit information and adapt to evolving dialogues. This on-policy evaluation, where the LLM's actions directly influence the interaction, provides a more accurate representation of its true capabilities.

The experiments conducted using TOOLSANDBOX yielded fascinating insights. While proprietary models like OpenAI's GPT-4 and Anthropic's Claude variants demonstrated impressive performance, highlighting their advanced reasoning and state-tracking abilities, open-source models lagged significantly. This performance gap underscores the ongoing challenges in developing truly capable open-source alternatives.

The experiments also revealed critical areas for improvement. LLMs, particularly open-source models, struggled with managing and reasoning about the world state and effectively utilizing information from tool responses. This highlights the need for further research in state management, tool representation, and information integration.

The introduction of TOOLSANDBOX marks a significant step forward in LLM evaluation. By embracing statefulness, conversation, and interactivity, it provides a more realistic and comprehensive assessment of LLM tool-use capabilities. As we venture further into the era of tool-wielding LLMs, robust benchmarks like TOOLSANDBOX will be essential for tracking progress, identifying limitations, and ultimately, unlocking the full potential of these powerful technologies. 2/n The paper describes experiments conducted using TOOLSANDBOX to evaluate both open-source and proprietary LLMs across a variety of tool-use scenarios. Here's a breakdown of the experiments and noteworthy results:

Experiments:

Test Scenarios: 1032 human-authored test cases designed to cover diverse and challenging tool-use scenarios. These scenarios were categorized based on:
* Number of tool calls and user turns required.
* Presence of state dependencies between tools.
* Need for canonicalization (resolving ambiguous information).
* Handling of insufficient information (avoiding hallucination).

Models Evaluated: Both open-source and proprietary LLMs were evaluated, including:OpenAI's GPT-3.5-turbo and GPT-4.
Anthropics' Claude-instant-v1 and Claude-v1.3.
Several open-source models.

Metrics:
Milestone Achievement: Measures how well the agent completes the critical steps defined by the Milestones.
Minefield Avoidance: Evaluates the agent's ability to avoid incorrect or undesirable actions.
Turn Count: Tracks the efficiency of the agent in completing the task.

Noteworthy Performance Results:
Significant Gap Between Open-Source and Proprietary Models: Open-source models exhibited significantly lower performance compared to proprietary models (GPT-4 and Claude variants) across all scenario categories. This highlights the considerable gap that still exists in capabilities.
GPT-4's Superior Performance: GPT-4 consistently outperformed other models, demonstrating advanced reasoning, state tracking, and conversational abilities in complex tool-use scenarios.
Strong Performance of Claude Models: Claude models, particularly Claude-v1.3, also showed strong performance, indicating their competence in tool-assisted settings. However, Claude-instant-v1 lagged in scenarios involving complex state dependencies.
Challenges in State Management and Tool-Response Consumption: Open-source models particularly struggled with managing and reasoning about the world state, as well as effectively utilizing information from tool responses.
Impact of Tool Augmentations: Ablation studies showed that increasing distractions (irrelevant tools) and reducing tool information (uninformative names, missing descriptions) significantly impacted the performance of all models. This emphasizes the importance of clear and concise tool representations for effective tool use.
Importance of User Simulator Prompting: Experiments with different user simulator prompting strategies demonstrated that incorporating Knowledge Boundary and Demonstration significantly improved the realism and robustness of the simulated user, leading to more accurate evaluations.

Overall, the experiments conducted using TOOLSANDBOX provide valuable insights into the capabilities and limitations of current LLMs in tool-assisted settings. The results highlight the c
, setting the stage for future research and development in this critical area.

1/n Show, Don't Tell: Low Cost Personalized Large Language Models

Large language models (LLMs) have revolutionized our interaction with technology, showcasing remarkable abilities in understanding and generating human-like text. However, their training on massive, general-purpose datasets often leads to outputs that lack the personal touch, failing to capture the nuances of individual writing styles and task-specific requirements. While powerful, these LLMs can feel like generic one-size-fits-all tools, struggling to adapt to the diverse needs of individual users.

Addressing this critical gap between powerful LLMs and personalized language generation is the core focus of the paper "Show, Don't Tell: Aligning Language Models with Demonstrated Feedback." The authors introduce DITTO (Demonstration ITerated Task Optimization), a method that deviates from the data-heavy approaches of the past, instead empowering users to efficiently customize LLMs using a handful of demonstrations.

Traditional LLM alignment techniques, such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), rely on vast datasets of labeled examples or preferences. While effective, these methods are impractical for individual users who cannot afford to generate such large amounts of data. Prompting, while data-efficient, often becomes a tedious guessing game, requiring careful crafting of input phrases to steer the LLM towards desired outputs. Other approaches, like Constitutional AI, rely on pre-defined principles that may not capture the nuances of individual preferences.

DITTO breaks free from these limitations by leveraging the LLM itself to generate comparison data from a small set of user demonstrations. Instead of telling the model what to do through complex instructions or thousands of examples, DITTO allows users to show the desired behavior directly. This direct alignment with demonstrations provides a more intuitive and efficient way of communicating preferences to the model.

The paper demonstrates the effectiveness of DITTO through a series of compelling experiments. In automatic evaluations on benchmark datasets of author-specific writing, DITTO consistently outperforms existing methods, including SFT, few-shot prompting, and even self-play methods like SPIN. Furthermore, a user study on email writing showcases DITTO's ability to adapt to real-world scenarios, outperforming not only standard baselines but also user-constructed prompts. This highlights the advantage of learning directly from demonstrations rather than relying on users to articulate their preferences through potentially ambiguous prompts.

Perhaps the most striking finding is DITTO's remarkable sample efficiency. Compared to traditional preference-based methods, DITTO achieves comparable performance with an order of magnitude fewer feedback samples. This makes it a practical solution for individual users who can now customize LLMs with just a handful of examples.

In conclusion, DITTO marks a significant step towards a new era of personalized language models. By shifting from "telling" to "showing," it empowers users to mold powerful LLMs to their specific needs and preferences. This opens up exciting possibilities for a future where LLMs are no longer generic tools but personalized assistants that can adapt to the unique voice and tasks of each individual. 2/n Comparison with other approaches

1. Supervised Fine-tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF):

Prior Work: These methods train LLMs on large datasets of human-labeled text or preferences.
DITTO Contrast: DITTO is significantly more data-efficient, requiring only a handful of demonstrations instead of thousands of examples. It achieves this by leveraging the LLM itself to generate comparison data.

2. Prompting:

Prior Work: Prompting involves crafting specific input phrases to guide the LLM's output.
DITTO Contrast: While prompting can be data-efficient, it often requires tedious trial-and-error to find effective prompts. DITTO provides a more direct and intuitive way of aligning the model by learning from demonstrations rather than relying on prompt engineering.

3. Constitutional AI:

Prior Work: This method automatically generates preference data using the LLM itself, guided by pre-defined principles.
DITTO Contrast: DITTO does not rely on pre-defined principles, making it more flexible and adaptable to individual preferences. It directly learns from user demonstrations, capturing more nuanced aspects of desired behavior.

4. Group Preference Optimization (GPO):

Prior Work: GPO aims for few-shot alignment by meta-learning preference groups from a large dataset.
DITTO Contrast: DITTO does not require a large pre-existing dataset for meta-learning. It focuses on individual user adaptation and can learn directly from a small number of demonstrations provided by that user.

5. Self-Play Methods (e.g., SPIN):

Prior Work: These methods improve LLMs through iterative self-play, often using a stronger language model as a critic.
DITTO Contrast: DITTO is designed for data-limited scenarios and does not require an external critic or a large number of demonstrations. It focuses on aligning with specific user preferences rather than achieving general self-improvement.

6. Online Imitation Learning:

Prior Work: Traditional online imitation learning methods typically focus on continuous control tasks and often require explicit reward function learning.
DITTO Contrast: DITTO adapts online imitation learning principles to the discrete text generation setting of LLMs. It implicitly learns a reward function from demonstrations and efficiently generates comparison data online.

1/n OpenDevin's Radical Approach to Agentic AI

The rapid advancement of large language models (LLMs) has ushered in a new era of AI agents capable of interacting with and impacting their environments in increasingly sophisticated ways. However, developing and evaluating these agents for complex, real-world tasks presents significant challenges. Existing frameworks often struggle to provide the necessary tools, environments, and interfaces for building truly versatile and robust AI agents. The OpenDevin platform, as presented in the paper "OpenDevin: An Open Platform for AI Software Developers as Generalist Agents," directly addresses these limitations, offering a novel approach that empowers AI agents to interact with the world more like human software developers – through code, command lines, and web browsing.

One of the key pain points OpenDevin tackles is the inherent complexity of developing and evaluating advanced AI agents. Traditional frameworks often rely on simplified environments and limited action spaces, hindering the development of agents capable of tackling real-world tasks. OpenDevin breaks free from these constraints by providing a realistic environment that includes a sandboxed Linux operating system and a fully functional web browser. This allows agents to interact with real-world tools and data sources, enabling them to tackle more meaningful and impactful challenges. Moreover, OpenDevin's standardized evaluation framework, encompassing a diverse set of established benchmarks, ensures consistent and comprehensive assessment of agent capabilities across various domains.

Another significant limitation addressed by OpenDevin is the lack of a standardized and powerful interface for agent-world interaction. While some frameworks rely on pre-defined tool sets or JSON-based function calls, OpenDevin embraces code execution and web browsing as its primary interaction mechanisms. This allows agents to leverage the flexibility and expressiveness of programming languages, breaking free from the limitations of rigid action spaces and enabling them to solve complex problems in a more human-like manner.

Recognizing the importance of reusable components in software development, OpenDevin introduces the AgentSkills library – a centralized and extensible collection of tools for common agent tasks. This modular design simplifies the development process and encourages community contributions, fostering a collaborative ecosystem for building and sharing specialized agent capabilities. Furthermore, OpenDevin tackles the challenge of multi-agent collaboration by incorporating a delegation mechanism. This allows developers to create teams of specialized agents, each excelling in specific domains, to work together and solve complex problems more effectively.

The effectiveness of OpenDevin's approach is evident in its experimental results. Evaluated on 15 established benchmarks spanning software engineering, web browsing, and general assistance tasks, OpenDevin agents demonstrate strong and competitive performance across the board. The agents excel in tasks like code generation, web navigation, information extraction, and problem-solving, highlighting the platform's versatility and the power of its core design principles.

In conclusion, OpenDevin represents a significant leap forward in AI agent development. By providing a realistic environment, a powerful and flexible interface, an extensible skill library, and support for multi-agent collaboration, OpenDevin empowers researchers and developers to create more capable, versatile, and robust AI agents. The platform's promising experimental results and its community-driven approach pave the way for a future where AI agents seamlessly integrate into our world, assisting us in tackling complex challenges and pushing the boundaries of what's possible with artificial intelligence. 2/n Comparison with Other Systems

1. AutoGPT, LangChain, MetaGPT, AutoGen, Agents, Xagents, OpenAgents, GPTSwarm:

Category: These are general-purpose AI agent frameworks, often focused on chaining together various tools and APIs to accomplish tasks.
Contrast with OpenDevin: While these frameworks offer flexibility in tool integration, they often lack a standardized and powerful interface for interacting with the world. They may rely on pre-defined tool sets or JSON-based function calls, which can limit agent capabilities and generalization. OpenDevin, on the other hand, empowers agents to interact with the world more directly through code execution and web browsing, providing greater flexibility and expressiveness. Additionally, OpenDevin places a strong emphasis on a sandboxed environment, agent skill library, and systematic evaluation, which are not always central to these other frameworks.

2. AutoCodeRover, SWE-Agent:

Category: These frameworks are specifically designed for software engineering tasks, enabling agents to write, debug, and test code.
Contrast with OpenDevin: While these frameworks excel in software development domains, OpenDevin aims to be more general-purpose. It includes software development capabilities but also extends to web browsing and other tasks through its flexible interface and agent skill library. OpenDevin also emphasizes multi-agent collaboration, which is not a primary focus in these more specialized frameworks.

3. BabyAGI, AgentVerse:

Category: These frameworks focus on building autonomous agents that can manage and execute tasks over extended periods, often with minimal human intervention.
Contrast with OpenDevin: While OpenDevin supports autonomous agent behavior, it also emphasizes human-in-the-loop scenarios and provides tools for interactive agent development and debugging. OpenDevin's focus on a realistic environment and standardized evaluation also sets it apart from these frameworks, which may rely on more simplified task representations or simulations.

4. ReAct, Toolformer:

Category: These are research efforts focusing on specific techniques for enhancing agent capabilities, such as reasoning with actions (ReAct) or learning to use tools (Toolformer).
Contrast with OpenDevin: OpenDevin is a platform that can incorporate and benefit from these research advancements. It provides a framework where techniques like ReAct or Toolformer can be implemented and evaluated within a broader context of agent development and real-world interaction.

In summary:

OpenDevin distinguishes itself from prior work by combining the following features:

Powerful and flexible interface based on code execution and web browsing.
Realistic environment with a sandboxed operating system and web browser.
Extensible library of agent skills and tools.
Support for multi-agent collaboration through delegation.
Standardized evaluation framework with diverse benchmarks.

These features address the limitations of existing frameworks and pave the way for developing more capable, versatile, and reliable AI agents that can effectively interact with and solve real-world problems.