I wrote this while crying in the B40 bathroom at Google because I was going through a depressive episode.
People are talking about bipolar now re: Kanye, but I don’t think there’s enough talk about high-functioning neuroatypicals. Maybe I should speak up, so here goes:
The media portrays us as crazy. People are afraid of us. I can speak to my experience — I have bipolar 2, which means I oscillate between periods of hypomania (weeks of excitement, little sleep, and being on top of life) and depression (weeks of being unable to get out of bed).
Outside perspective: I feel like I have only 50% of my time to be as productive as a normal human, because I know the other 50% of my time goes to the trash can. Maybe that’s why I’m classified as “high-functioning,” even though when amortized, my work output is pretty normal.
Therapy (pt 1): It’s been a year of 3hr/week dialectical behavioral therapy (DBT) for me. I’ve learned skills for distress tolerance, emotion regulation, and interpersonal relationships. I do weekly homework and have a support group. It's not always fun, but it's so useful.
Therapy (pt 2): Before DBT, I struggled so hard to have meaningful relationships — people didn’t want to be close to me. Maybe I had unrealistic expectations or wasn’t like their other friends. I blamed myself and tried to “get better.” Even now I lose relationships. It hurts.
No one gets me (pt 1): It isn't my fault that I was born with a brain that experiences emotions 10x more than the average human. Highs are super high. Lows are super low. No one taught me to work with this range of emotions.
No one gets me (pt 2): Nobody understands why I’m so motivated to crank out 1000s of lines of code for a project. Nobody also understands why I constantly reschedule meetings because I don’t feel like living anymore.
[CW] Self-harm: Yes, I have had suicidal thoughts. But having thoughts is not bad. I now approach my thoughts with curiosity rather than judgement: what does this feel like in my body? What does my brain want to do? I wish neurotypical people were open-minded about such thoughts.
Stuck in a rut: When switching to another episode, it’s so hard to remember how I felt before the episode and how I might feel in the future. When hypomanic, I scoff at the thought of ever being sad. When depressed, I can’t fathom happiness.
Strategies (pt 1): What has worked for me? I try to separate my identity from my brain. My brain might be broken, but that does not mean I am any less of a human being. I used to lament that life sucks, but it’s easier to accept my broken brain and problem-solve around it.
Strategies (pt 2): When I’m in a depressive episode, my manager messages me at 9AM and 5PM for check-in and check-out updates. I forgive myself immediately when I make mistakes — if I sleep in, don’t feel like working out, or skip a meal.
Strategies (pt 3): When I’m hypomanic, I try to stick to a regular sleep schedule. But I have never successfully done so. Also, journaling documents my emotions, so when I'm "stuck in a rut," there are timestamped records of instances where I've felt differently.
Mental illness in others (pt 1): Most bipolar people are undiagnosed until 20s or 30s, or never diagnosed at all. I sometimes wonder what the world’s perception of neuroatypical people would be like if many high-functioning neuroatypical people openly spoke about their illnesses.
Mental illness in others (pt 2): When I was going through my bipolar diagnosis, former managers, mentors, and people I look up to shared their personal experiences with mental illnesses with me. It’s more common than you think.
Maybe we’re all so afraid to talk about our experiences with mental illness because people don’t understand mental illness, and we don’t want to be judged for something we can’t control.
I think we’re just regular human beings with a broken body part. Just because you can’t see the fracture doesn’t mean we’re aliens. We are no less worthy of stability. Great teammates help us when we’re down, and we have so much value to add to a team when we’re feeling good.
Frankly, I don’t see why a bipolar person or anyone with a mental illness can’t be a president.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
RAG is everywhere, but building RAG is still painful. When something breaks--the retriever? the LLM?--developers are left guessing, & iterating is often slow
we built a better way & used it as a design probe to study expert workflows 👇
Meet raggy: an interactive debugging interface for RAG pipelines. It pairs a Python library of RAG primitives with a UI that lets devs inspect, edit, & rerun steps in real time. raggy precomputes many indexes for retrieval upfront, so you can easily swap them out when debugging!
Then, to learn more about expert workflows, we ran a study with 12 engineers who’ve built production RAG pipelines. We simulated a question-answering application from a hospital & watched our participants use raggy to build and iterate on their pipelines. The paper reports a bunch of qualitative findings, including:
🔍 They always debug retrieval first
⚙️ Fixes to one step often break another
⚡ Fast iteration was key: raggy turned half-day experiments into seconds!!
(2/7) Following the release of DocETL (our data processing framework), we observed users struggling to articulate what they want & changing their preferences based on what the LLM could or couldn't do well. The main challenge is that no one knows what outputs they want until they see it; that is, agentic workflows are inherently iterative.
(3/7) This release of DocWrangler has 3 main features. Key feature 1: spreadsheet interface with automatic summary overlays
how come nobody is talking about how much shittier eng on-calls are thanks to blind integrations of AI-generated code? LLMs are great coders but horrible engineers. no, the solution is not “prompt the LLM to write more documentation and tests” (cont.)
i will take react development as an example. I use cursor but I think the problems are not specific to cursor. Every time I ask for a new feature to be added to my codebase, it almost always uses at least 1 too many state variables. When the code is not correct (determined by my interaction with the react app), and I prompt the LLM with the bug + to fix it, it will almost always add complexity rather than rewrite parts of what it already had
so the burden is on me to exhaustively test the generated app via my interactions, and then reverse engineer the mental model of what the code should be, and then eyeball the generated code to make sure this matches my model. This is so horrible to do for multi-file edits or > 800 lines of generated code (which is super common for web dev diffs)
what makes LLM frameworks feel unusable is that there's still so much burden for the user to figure out the bespoke amalgamation of LLM calls to ensure end-to-end accuracy. in , we've found that relying on an agent to do this requires lots of scaffolding docetl.org
first there needs to be a way of getting theoretically valid task decompositions. simply asking an LLM to break down a complex task over lots of data may result in a logically incorrect plan. for example, the LLM might choose the wrong data operation (projection instead of aggregation), and this would be a different pipeline entirely.
to solve this problem, DocETL uses hand-defined rewrite directives that can enumerate theoretically-equivalent decompositions/pipeline rewrites. the agent is then limited to creating prompts/output schemas for newly synthesized operations, according to the rewrite rules, which bounds its errors.
first, humans tend to underspecify the first version of their prompt. if they're in the right environment where they can get a near-instantaneous LLM response in the same interface (e.g., chatgpt, Claude, openai playground), they just want to see what the llm can do
there's a lot of literature on LLM sensemaking from the HCI community here (our own "who validates the validators" paper is one of many), but I still think LLM sensemaking is woefully unexplored, especially with respect to the stage in the mlops lifecycle
Our (first) DocETL preprint is now on Arxiv! "DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing" It has been almost 2 years in the making, so I am very happy we hit this milestone :-) arxiv.org/abs/2410.12189
DocETL is a framework for LLM-powered unstructured data processing and analysis. The big new idea in this paper is to automatically rewrite user-specified pipelines into a sequence of finer-grained and more accurate operators.
I'll mention two big contributions in this paper. First, we present a rich suite of operators, with three entirely new operators to deal with decomposing complex documents: the split, gather, and resolve operators.