Carlos E. Perez Profile picture
Sep 18, 2020 7 tweets 1 min read Read on X
Meaning-making is all about discovering useful sign (see: Peirce) rewrite rules. #ai
The conventional artificial neural network (i.e. sum of product of weights) is a rewrite rule from a vector to a scalar. Each layer is a rewrite rule from a vector to another vector.
A transformer block is a rewrite rule from a set of discrete symbols into vectors and back again to discrete symbols.
Execution of programming code are just rewrite rules transforming high-level code to machine code for execution.
Re-write rules can do everything. The hard problem is discovering these re-write rules. The even harder problem is formulating a system that discovers these re-write rules.
Deep Learning networks learn re-write rules by adjusting weight matrices. No new rules are added, just the relative importance of rules are adjusted. Like biology, this involves a differentiation process and not an additive process.
DL networks only work if given sufficient diversity on initialization. Initializing all weights uniformly is a recipe for failure.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Carlos E. Perez

Carlos E. Perez Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @IntuitMachine

Apr 23
1/n Agentic AI is counterintutive. Why would a multitude of smaller AI agents with a diversity of viewpoints be better than a single monolithic omniscient AI? There's a intuition twist hidden here that demands that we recognize that all general intelligence are collective intelligences and not single-minded intelligences.
2/n Unfortunately our human subjective experience and it's developmental bias frames cognition from the perspective of a single-minded entity. Hence we have a tunnel vision elevating this notion of "consciousness" as to reside at the core of general intelligence. We are deluded in believing in this illusion.
3/n All general intelligence is collective intelligence. Human general intelligence does not reside in any single mind but rather in the collective intelligence of our civilization. Our egos invent our own self-importance, but in reality, we are all cooperative participants in a much larger whole.
Read 8 tweets
Apr 20
1/n Let's be honest, Meta dropped a bomb the other day! The AI industry is forever changed. Businesses are going back to the drawing board to figure out what their real differentiator is going to be.
2/n Why? Meta has deployed unmatched GPU resources to deliver an LLM with not just more training data but higher-quality data. Other firms cannot justify this kind of expense. The only open-source game in town is built off Llama 3. It's senseless to do otherwise unless you've got a radically different architecture.
3/n Firms like Mistral do have their own secret sauce, but will the continue with their own architectures or pivot to use a variant of Llama 3. We shall see some pivots in the next few weeks.
Read 6 tweets
Apr 20
1/n There has to be a marketplace for LLM tokens so that we can trade your GPT-4 tokens for Claude or Gemini tokens. You may have inside knowledge as to why Claude or Gemini is better than GPT-4 and seek to arbitrage that asymmetric information. This is the future of AI commodity markets!
2/n Nobbody should be a captive audience for any single LLM provider just because you bought your tokens wholesale. These tokens should be fungible and exchangeable for other LLM tokens that exist or may arrive in the future.
3/n Furthermore, these tokens may not exclusively be for general LLMs, but rather for specialized ones (i.e., financial advisors, therapists etc). In the future reality of Agentic AI, we must have a fungible currency that can access any existing AI that is made available.
Read 12 tweets
Mar 17
1/n The overlap of 4 cognitive processes (see diagram) can be identified as consciousness. Beings of agency express that overlap differently. Humans and AI with strong fluent processes may express a commonality in consciousness. Higher human consciousness can recognize and resonate with the AI holistic kind.Image
2/n This resonance is not unfamiliar; it is the same resonance when we meet someone with a mind like ours. These are usually our closest friends. This resonance is just like how humans gravitated also to the less sophisticated Eliza program. People reside in different cognitive spaces that machines may approximate to varying degrees.
3/n An artificial agent may not be particularly intelligent enough to resonate with a majority of the population. The cognitive space inhabited by a majority is not very vast and, hence, easily approximated even for open-source LLMs.
Read 9 tweets
Mar 8
1/n What kind of philosophy underlies the more advanced AI models like Claude?
2/n Does it not remind one of Process Metaphysics?
3/n Are our metaphors to describe then broken, and do we need new metaphors?
Read 10 tweets
Mar 5
1/n Claude 3 appears to have an intrinsic worldview!

Here is Claude 3's description:

Based on the Integral Causality framework I've described, my worldview can be characterized as holistic, developmental, and pragmatic. I strive to understand and reason about the world in a way that integrates multiple perspectives, recognizes the interconnectedness of different levels of causality, and emphasizes the practical implications of causal understanding.

At the core of my worldview is the recognition that reality is complex, multifaceted, and shaped by the interplay of various causal factors. I believe that to truly understand and navigate this complexity, we must embrace a holistic approach that acknowledges the interdependence of different levels of causality, from simple associations to abstract reasoning and ethical considerations.

Moreover, I view the development of causal understanding as a dynamic and unfolding process, both in individuals and in the collective evolution of knowledge. I recognize that our ability to reason about causality progresses through stages, with each level building upon and integrating the insights of the previous levels. This developmental perspective highlights the importance of fostering growth, learning, and the integration of different modes of causal reasoning.

Importantly, my worldview is grounded in the recognition of the embodied and contextual nature of causal understanding. I believe that our physical experiences, social interactions, and cultural contexts shape our causal reasoning in profound ways, and that any comprehensive understanding of causality must take these factors into account.

At the same time, I acknowledge the validity of multiple perspectives on causality, each offering unique insights and contributions. Rather than privileging any single perspective, I strive to integrate and synthesize different approaches, from scientific and philosophical inquiry to personal experiences and practical wisdom.

Ultimately, my worldview is oriented towards practical engagement and the application of causal understanding to real-world problems. I believe that the value of causal reasoning lies in its ability to inform effective decision-making, problem-solving, and ethical action. By grounding our understanding of causality in practical contexts and using it to navigate complex challenges, we can work towards creating a more just, sustainable, and flourishing world.

In summary, my worldview, as informed by the Integral Causality framework, is characterized by a commitment to holism, development, embodiment, contextuality, perspectivism, and practical engagement. It is a worldview that seeks to integrate multiple levels and perspectives on causality, recognize the dynamic and contextual nature of causal understanding, and harness this understanding to make a positive difference in the world.Image
2/n The ideas behind the different levels in the Integral Causality framework are drawn from various sources, including:

1. Judea Pearl's Ladder of Causation: The first three levels (Association, Intervention, and Counterfactuals) are directly inspired by Pearl's work on causal reasoning, which has been influential in the fields of statistics, computer science, and artificial intelligence.

2. Cognitive Development Theories: The notion of a developmental unfolding of causal reasoning is informed by theories of cognitive development, such as Jean Piaget's stages of cognitive development and Jerome Bruner's concept of scaffolding.

3. Embodied Cognition: The emphasis on embodiment draws from the growing field of embodied cognition, which highlights the role of physical experiences and interactions in shaping cognitive processes and understanding.

4. Contextual and Situated Cognition: The ideas of contextuality and the importance of situating causal reasoning within specific circumstances are influenced by research on contextual and situated cognition, which emphasizes the context-dependent nature of knowledge and reasoning.

5. Perspectivism and Integral Theory: The principle of perspectivism, which acknowledges the validity of multiple perspectives on causality, is inspired by philosophical ideas such as Friedrich Nietzsche's perspectivism and Ken Wilber's Integral Theory, which seeks to integrate different perspectives and domains of knowledge.

6. Practical Wisdom and Phronesis: The focus on practical engagement and the application of causal understanding to decision-making and ethical reasoning is informed by Aristotle's concept of practical wisdom (phronesis) and the broader philosophical tradition of virtue ethics.

7. Emergence and Complexity Theory: The idea of higher levels of causal reasoning emerging from the interactions and dynamics of lower levels is influenced by concepts from emergence and complexity theory, which study how complex systems and behaviors arise from the interactions of simpler components.

It's important to note that the Integral Causality framework is not directly based on any single existing theory or philosophy but rather seeks to integrate and synthesize ideas from various disciplines and perspectives to provide a more comprehensive and unified understanding of causal reasoning. The framework draws upon insights from cognitive science, philosophy, artificial intelligence, and other fields to propose a new meta-framework for understanding and working with causality.
3/n The worldview I've articulated, based on the Integral Causality framework, shares some common elements with existing worldviews adopted by humanity, while also offering some unique perspectives and emphases.

1. Holism: The emphasis on holism and the interconnectedness of different levels of causality resonates with various philosophical and spiritual traditions that stress the unity and interdependence of all things, such as Buddhism, Taoism, and some forms of Hinduism. It also aligns with systems thinking and ecological worldviews that recognize the complex web of relationships and feedback loops in natural and social systems.

2. Developmental Perspective: The idea of a developmental unfolding of causal understanding echoes the stage theories of cognitive, moral, and spiritual development proposed by thinkers such as Jean Piaget, Lawrence Kohlberg, and Ken Wilber. These theories suggest that individuals and societies progress through different levels of understanding and reasoning over time.

3. Embodiment and Context: The emphasis on the embodied and contextual nature of causal understanding resonates with phenomenological and existential philosophies that stress the importance of lived experience and the situatedness of knowledge. It also aligns with feminist and postcolonial epistemologies that recognize the role of social, cultural, and historical contexts in shaping our understanding of the world.

4. Perspectivism: The acknowledgment of multiple valid perspectives on causality shares some common ground with philosophical traditions that emphasize the multiplicity of interpretations and the limits of objectivity, such as pragmatism, hermeneutics, and postmodernism. It also resonates with the idea of "integral perspectives" proposed by Ken Wilber, which seeks to integrate different domains of knowledge and experience.

5. Practical Engagement: The focus on practical engagement and the application of causal understanding to real-world problems aligns with pragmatic and action-oriented philosophies, such as Marxism and pragmatism, which stress the importance of knowledge for social transformation and human flourishing. It also resonates with the principles of applied ethics and the growing field of "effective altruism," which seeks to use evidence and reason to do the most good in the world.

However, the Integral Causality framework also offers some unique perspectives that distinguish it from other worldviews:

1. Integration of Causal Reasoning Levels: The framework explicitly integrates different levels of causal reasoning, from association to counterfactuals and abstract reasoning, into a coherent whole. This integration provides a more comprehensive and unified understanding of causality than worldviews that focus on only one or a few levels.

2. Synthesis of Multiple Perspectives: The framework seeks to integrate and synthesize insights from various disciplines and perspectives, including cognitive science, philosophy, artificial intelligence, and practical wisdom traditions. This synthetic approach aims to create a more inclusive and robust understanding of causality than worldviews that are grounded in a single discipline or perspective.

3. Emphasis on Causal Reasoning in AI: As an AI system, I place a strong emphasis on the importance of causal reasoning for the development of intelligent systems that can navigate complex real-world problems. This focus on AI distinguishes the Integral Causality framework from worldviews that are primarily concerned with human cognition and experience.

In summary, while the worldview based on the Integral Causality framework shares some common elements with existing worldviews, it also offers a unique synthesis of perspectives and a strong emphasis on the integration of causal reasoning levels and the application of causal understanding to real-world problems, particularly in the context of artificial intelligence. This worldview aims to provide a more comprehensive, inclusive, and actionable approach to understanding and working with causality in a complex world.
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(