Post

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @omarsar0

elvis

@omarsar0

Aug 13

The Illusion of Progress

It's well known that there are caveats with benchmarks and metrics that measure LLM capabilities.

It's no different for hallucination detection.

"ROUGE fails to reliably capture true hallucination"

Here are my notes:

Overview

The paper argues that common QA hallucination detectors look better than they are because evaluations lean on ROUGE.

In human‑aligned tests, many detectors drop sharply. Simple response‑length heuristics rival complex methods, revealing a core evaluation flaw.

ROUGE misaligns with humans.

In a human study, LLM‑as‑Judge matches human labels much better than ROUGE.

Results show that LLM‑as‑Judge F1 0.832 vs ROUGE 0.565, with far higher agreement.

Read 8 tweets

elvis

@omarsar0

Aug 12

Unlocking Long-Horizon Agentic Search

AI agents still struggle with long-horizon tasks.

This paper sheds light on how to improve long-horizon agentic search with RL.

Here are my notes:

Overview

It introduces ASearcher, an open-source framework for training LLM-based search agents capable of long-horizon, expert-level search.

Addresses 2 major limitations in prior open-source approaches: short turn limits (≤10) and lack of large-scale, high-quality QA data.

Fully asynchronous RL for long-horizon search

Unlike batch generation RL, ASearcher decouples trajectory execution from model updates, avoiding bottlenecks from long trajectories.

This enables relaxed turn limits (up to 128), with training showing >40 tool calls and >150k tokens in a single trajectory.

Read 8 tweets

elvis

@omarsar0

Aug 11

Getting huge productivity boosts by combining Claude Code with Obsidian vaults.

Everything in Obsidian is .md, so this is like the most delicious context for LLMs.

Everything is in one place: notes, bookmarks, instructions, LLM context, AI outputs, and so on.

The part I like about Obsidian is that, finally, I feel like I own my notes.

I can access them everywhere.

Modify them when I want.

And leverage them with LLMs all the time.

To be clear, the free version of Obsidian is more than enough. I like the fact that I can sync notes/bookmarks from different devices.

I can edit the .MD files with any editor, too. This is what I really love about it. You own the notes.

You can also version your notes, too, for those who want to leverage Claude Code+GitHub actions on the road.

Read 4 tweets

elvis

@omarsar0

Aug 7

BREAKING: OpenAI introduces GPT-5

Here's everything you need to know:

Altman claims that with GPT-5, it is now like talking to an expert.

It can write entire programs from scratch. Software-on-demand is a defining characteristic.

PhD-level experts in your pockets.

GPT-5 brings a higher level of reasoning.

It thinks just the perfect amount to generate the perfect answer.

Good for math, physics, law, and many other domains.

They claim that GPT-5 is the best coding model today.

Read 30 tweets

elvis

@omarsar0

Aug 3

The Agentic Web is upon us!

If you want to learn about the Agentic Web, look no further.

This new report is a banger!

It presents a detailed framework to understand and build the agentic web.

Here is everything you need to know:

Agentic Web

This paper introduces the concept of the Agentic Web, a transformative vision of the internet where autonomous AI agents, powered by LLMs, act on behalf of users to plan, coordinate, and execute tasks.

It proposes a structured framework for understanding this shift, situating it as a successor to the PC and Mobile Web eras.

It's defined by a triplet of core dimensions (intelligence, interaction, and economics) and involves fundamental architectural and commercial transitions.

Read 15 tweets

elvis

@omarsar0

Aug 2

Hierarchical Reasoning Model

This is one of the most interesting ideas on reasoning I've read in the past couple of months.

It uses a recurrent architecture for impressive hierarchical reasoning.

Here are my notes:

The paper proposes a novel, brain-inspired architecture that replaces CoT prompting with a recurrent model designed for deep, latent computation.

It moves away from token-level reasoning by using two coupled modules: a slow, high-level planner and a fast, low-level executor.

The two recurrent networks operate at different timescales to collaboratively solve tasks

Leads to greater reasoning depth and efficiency with only 27M parameters and no pretraining!

Read 9 tweets

Share this page!

Enter URL or ID to Unroll

elvis

Try unrolling a thread yourself!

More from @omarsar0

elvis

elvis

elvis

elvis

elvis

elvis

Did Thread Reader help you today?

Don't want to be a Premium member but still want to support us?

Send Email!