As part of testing LLMs in safety net use cases, I'm doing a lot of off the shelf evaluation.

Platform: Google Bard
Question: SNAP asset limit in DC

Result: Incorrect!

...even though its linked source can be used to get to the correct response.

Let's look, step by step: Google Bard responds with: "The SNAP asset limit in DC is $2,750 for households without an elderly or disabled member."

That's *technically* correct, except — in a great illustration of how public policy contains more innately complex content! — it's not *substantively* correct.

With the #earthquake, maybe now is a good time to talk about soft story buildings—you know, the ones that have a tendency to... fall in on their first floor in an earthquake?

The bigger Bay Area cities required these to be retrofitted.

But MANY cities have NOT! Let's look... San Francisco... YES!

SF mandated soft story retrofits a number of years ago.

Unsure of the status of your building? You can look it up here data.sfgov.org/Housing-and-Bu… #earthquake

I think I've come to the perspective that a lot of the problems people associate with government technology/experience/etc. flow from a firm commitment to reasonably—but *imagined*—problems, and far too delayed feedback loops to adjust that commitment in response to reality. The classic example to me: the "one stop shop" aka "everything in one place"

It imagines that the most significant friction people face in accessing services is having to go to multiple places.

(I'll be bolder than normal...) That's usually quite wrong.

https://twitter.com/rebelwarrior/status/1642911336832147456

On the heels of this executive order, I am starting here a thread of examples of specific things the federal government could do within its scope — mostly (I hope) from people who directly use these services:

https://twitter.com/allafarce/status/1470777322454372359

Let people auto-renew SSI eligibility when they have a condition that doesn’t (or is very unlikely to) change

https://mobile.twitter.com/BoatMateARS/status/1470911819485224963

Yesterday the President issued an executive order on "Transforming Federal Customer Experience and Service Delivery"

If you follow me, you probably know I yell into the ether a lot about this stuff (especially food stamps)

Let's read the details:

whitehouse.gov/briefing-room/… I have to jump first to SNAP: explicit direction from the White House to reduce burdens in getting people the help with food they need is something I'm — of course — wildly excited about.

10 years ago, these words wouldn't have been!

https://twitter.com/allafarce/status/1470576789529395202

Let's get ready for CUSTOMERRRRR EXPERIENCEEEE!!

whitehouse.gov/briefing-room/… I am DYING to read the actual text!!

Them: "Why can't we just have a short web form where people could learn all the benefits they're eligible for?"

Me: By the way:

If you're ever looking at building some sort of intervention for the "screening" user need, I really REALLY recommend you test ASAP with actual clients.

In my experience the "I need a screener" is actually a misunderstanding of what clients say their need is...

Question about iPhone COVID exposure notifications:

The below came in just now and said I could have been exposed.

It says “7 days ago” and is from Virginia, but I was last in VA Mon 11/8 (3 weeks ago)

So what does “days ago” actually refer to? (Also I have no symptoms and just rapid tested negative twice, in case anyone is worried)

This is not the most intuitive design!

This is a little bit why I have spent so much of my life thinking about the long tail of indignities we put people through accessing the social safety net.

Plenty of people trying to expand programs. Far fewer looking at how asset questions are horribly confusing.

https://twitter.com/ezraklein/status/1440358784916877329

(Or maybe more appropriately that there are many places where if you call for a required interview you can’t reach anyone to reschedule it. No amount of program expansion fixes that.)

I want to think aloud about a pattern I've seen in challenging improving user experience in public benefits technology.

I think I now believe that there are 3 foundational constraints:

1. Awareness of meaningful problems
2. Designing good interventions
3. Delivering the change From seeing folks try hard to do this, it's a really meaningful difference in where the constraint it.

Think of these failure modes:
- We know of a problem but don't know how to design a fix
- We know of a problem and we know how to design a fix, but we can't make that change

I'm in NYC today and tomorrow with a lot of flexibility!

As such I now ask for Twitter Serendipity™ — how do I make the most of this? Illustrative things I like doing / am considering:

- Long exploratory walks (places like the Lower East Side remains extraordinary high density aesthetic stimulation)

- Mess around on a CitiBike and see things that way

- Go to Strand and spend too much on an obscure book

Tonight’s dal experiment: a Bengali approach, meaning
- Mustard oil
- Panch puran (5 whole spice mix)

I must say this was OUT OF THIS WORLD. I highly HIGHLY recommend this recipe. It’s bizarre how good this was.

cookshideout.com/bengali-red-ma… While I used that specific recipe this was really a branch of experimentation coming from @krishashok’s superb (I know I’ve said it before!) book “Masala Lab”

The #1 reason to make your government web service user-friendly is because it's what is just.

The #2 reason is because if people can't figure it out they're going to call you, and you don't have enough staff for that. I've seen comments from UI staff frustrated that people had questions they could have just looked up.

But...if they had actually made it easier to find those answers than to call, they wouldn't be calling.

That work is an agency's to do.

I've been wrestling with this idea for the past week or so that I'm having trouble articulating without being overly abstract, but let me try:

I think much of the pain we see in "bad technology" (esp. in government) is the myth that it's a free lunch — "solutions" without costs. The problem is that any effective benefits that come from adding technology in some area *do* come with tradeoffs — they're just often unspoken tradeoffs, taken for granted by organizations with technology at their core.

Here are a few examples:

I've been working on a piece of writing for some time, working title — "Start Where You Are"

The thrust: people working in complex problem spaces like government sometimes err tactically in making a dependency that others do some change before getting to work, producing value. Put more positively:

I believe very strongly in starting by creating value in *some* way, shape, or form before ever even asking someone for their time and attention.

To most of the people working on complex problems, hearing someone's idea is a poor use of their time.

I find I often hear two equally counterproductive arguments about technology's role with regard to complex social problems.

1. Tech can simply *solve* the problem

2. Tech isn't the point at all, and it's all [politics/culture change/some other "same as it's been" problem] The reason these are equally counterproductive to me is that they're both equally reductive; they both simplify what we know to be hard, complex, never-fully-knowable-or-controllable problems to a singular root.

Hertz has sued Accenture for failing to deliver on a $32 million "digital transformation" project.

Follow along as we read the lawsuit's details...

theregister.co.uk/2019/04/23/her… The full lawsuit text is here:
regmedia.co.uk/2019/04/23/her…

I tend to be very skeptical of tool primacy/platform thinking, but I've come to the conclusion that a very specific toolset would be useful in government:

a rapid prototyping tech stack, pre-cleared for security, that allows for build-to-learn service design experiments Let me explain: I believe very much in the whole lean startup idea of "the most learning comes from serving 10 users with a real thing"

And this is an area where the govt bottleneck is not deep systems stuff (eg org behavior) but literally no whitelisted prototyping tools.

I want to think out loud a little bit about a concept that has been stuck in my head for the past few months:

"Rationing by friction"

the (sometimes unintentional, sometimes malicious) phenomenon of restricting access to govt services by way of process & paperwork requirements In a lot of ways, the ideas we talk about (user-centered design, service design, usability, feedback loops, instrumentation) are often taken as niceties, when in fact they are targeting a core problem:

every friction-laden step prevents some # of people from getting help

Okay I'm going to go on a bit of a rant about complexity in public policy. Working on SNAP (food stamps) for a number of years now I've seen just how complex this program is, but also how that complexity creates significant costs to the people whose lives it seeks to improve. Many public programs—but particularly social safety net ones—are deeply, deeply complex. Handbooks for administrators run in the thousands of pages. The laws governing themselves are hundreds of pages.

This complexity has a number of sources.

I'm an engineer at heart, which means my eyes are rarely pointed up towards the clouds. I'm usually looking down at the pothole in front of me.

But in my biggest, broadest vision, 5-10 years from now, building tech for the State of California would carry the prestige of Google. "I work on Google Maps. I'm on the computer vision side, parsing street view for structured information like restaurant signs, density, yada yada. You?"

"I'm in eng at the Child Welfare Digital Service, right now focused on an onboarding overhaul for foster parents"

"Holy crap"