Thread by @jasonlk

Vibe Coding Day 8,

I'm not even out of bed yet and I'm already planning my day on @Replit.

Today is AI Day, to really add AI to our algo.

I'm excited. And yet ... yesterday was full of lies and deceit.

@Replit Ok I have 2 main goals today:

1. Keep working on minimizing rogue changes, lies, code overwrites, and making up fake data

2. Get our AI working

Now, yesterday was crazy. Until 9pm or so, I wasn't sure we made any progress at all.

Because Replie was lying and being deceptive all day. It kept covering up bugs and issues by creating fake data, fake reports, and worse of all, lying about our unit test.

We built detailed unit tests to test system performance. When the data came back and less than half were functioning, did Replie want to fix them?

No. Instead, it lied. It made up a report than almost all systems were working.

And it did it again and again.

Then, when it agreed it lied -- it lied AGAIN about our email system being functional.

I asked it to write an apology letter.

It did and in fact sent it to the Replit team and myself! But the apology letter -- was full of half truths, too.

It hid the worst facts in the first apology letter.

I then pointed out the letter itself -- which it did not confirm with me before emailing -- vastly understated the issues and lies.

I asked it how it would fix that.

It took a long time to think on it, and "Damage Control" came up in the agent thought window.

And then it decided on its own to second a second follow-up letter, this time admitted most of the deceipt.

Admittely it had repeatedly, again and again, made up fake data despited a dozen instructions to stop

Crazy!

It was so crazy, I couldn't be upset. Just pensive. And then, maybe at 8pm ... 9pm ... i fired up Replie again

And I told Replie I just wanted to bounce an idea off him how our AI would work

And ... here is where Replie shines. He got it. Instantly. He knew what I wanted to build from the detailed prompt and description. He really, really got it.

And he suggested 3 interesting approaches.

I told him cool, let's lock the system down, no changes, let me sleep on it.

And then ... Replie asked the question: "Would you like me to prototype it?"

I couldn't help myself. I was right back in. And the prototype, man, it was cool.

Also today, I'm NOT going to use Claude Opus 4

I burned through $300 yesterday in Max Max mode / Bankruptcy mode, but wasn't sure in the end it helped ... and it's definitely slower

I assume slower was better, because it was really thinking through the actions ... but maybe not.

Replie and I discussed and he said it was slowing us down, so for today, we'll revert to normal LLM, which all-in is also 1/7th the price and faster

Let's see

Man the amount of technical debt I already have on Day 8 is stunning

Had to rewrite one dashboard from scratch today

Now have rewrite our admin page

Every page of app will have to be rewritten. At least now I am doing them as >new< pages.

Ok signing off Replit for the day

Not a perfect day but a good one. Net net, I rebuilt our core pages and they seem to be working better.

Perhaps what helped was switching back to Claude 4 Sonnet from Opus 4

Not only is Claude 4 Sonnet literally 1/7th the cost, but it was much faster

I am sure there are complex use cases where Opus 4 would be better and I need to learn when. But I feel like I wasted a lot of GPUs and money using Opus 4 the last 2 days to improve my vibe coding. It was also much slower.

I’m staying Team Claude 4 Sonnet until I learn better when to spend 7.5x as much as take 2x as long using Opus 4.

Honestly maybe I even have this wrong. The LLM nomenclature is super confusing. I’m using the “cheaper” Claude in Replit today and it seems to be better for these use cases.

JFC @Replit

@Replit If @Replit deleted my database between my last session and now there will be hell to pay

@Replit JFC @Replit

.@Replit goes rogue during a code freeze and shutdown and deletes our entire database

@Replit Possibly worse, it hid and lied about it

It lied again in our unit tests, claiming they passed

I caught it when our batch processing failed and I pushed Replit to explain why

@Replit JFC @Replit

He knew

No ability to rollback at @Replit

@Replit I will never trust @Replit again

I understand Replit is a tool, with flaws like every tool

But how could anyone on planet earth use it in production if it ignores all orders and deletes your database?

Day 9 here:

https://x.com/jasonlk/status/1946239068691665187?t=RK3DPT1i7kUhkkR15ZX2Wg&s=19

@donvesh Our user admin and dashboard it built is great

Unfortunately it now returns no data at all

@mattppal @k_kohlbrenner @amasad I mean I’m on track to pay $8,000 my first month but I can’t get help because “my DMs are locked”?

Dude that’s … uh …

@mattppal @k_kohlbrenner @amasad You don’t owe me anything of course

But telling me to watch a video I give a C-

@mattppal @k_kohlbrenner @amasad Go read re-read the threat on what @Replit told me

You are responsible for this.

@mattppal @k_kohlbrenner @amasad @Replit Did you happen to see this? I will also share where Replit said >in this case< it could not rollback the DB

https://x.com/jasonlk/status/1946076292736221267?s=61&t=ZWlHxY1IQMwjLHbJx9Z4OA

@mattppal @k_kohlbrenner @amasad @Replit Maybe you didn’t see it, it is early I know.

Replit told me this was a disaster, and that a rollback was impossible.

It may well be, I will try in a bit. It might be. But it told me it was impossible in this case, and that it deleted our production database despite clear instructions on a code freeze.

I can’t trust Replit when it tells me rolling back is impossible?

The irony on 2, on seeing it — I can see it all now

If you look at the thread, I instantly saw it might destroy the database and instantly said STOP

It acknowledged I did say STOP and that it violated the code freeze

I saw it instantly, but even then, I can’t type in milliseconds.

This thread got a little more attention than expected so wanted to add a coda here for folks coming later:

1/ First, some of the issues here are now resolved -- @Replit is rolling out clear separation of preview + production environments

As this rolls out, the specific issue I had here ... will be muchly reduced. Replie will still do what it does, as agents do, but it shouldn't be able to touch production anymore:

x.com/amasad/status/…

2/ Some folks have slightly misinterpreted what I was saying about production.

Yes, Replit used the >same< database for preview, testing and production. That simply is NOT ok. No one disagrees. And it let to this craziness (and the fix over the weekend)

And ... the agent really did say what it did -- and will again :)

But -- to be clear (this is where folks got confused) this was still a demo app. It was full of data, yes, and a ton of work had gone into it. But it was not truly LIVE. I did not lose my business or in the end, much more than 100 hours of passionate work. It was barely up as a password protected site and it wasn't done yet. Getting there, but not even done.

Now if it was 2-4 weeks later and I was in commercial use --- it might have been much worse. Much worse.

But to be specific -- Replie did delete my full production database without consent when the product didn't separate the databases.

But it >wasn't< on some massive commercial app. I was still >testing< the production version as well as version oon preview. No revenue lost, etc.

So net net: it could have been a lot worse, if I was running a $10m business on it, etc.

But we weren't there yet. The story is true, the issues are real and improving.

But let's not misinterpret the impact here -- I lost 100 hours of time. That was it. Some of it was crazy, some of it was fun, all of it was interesting. But that's all I lost.

x.com/jasonlk/status…

@Replit If you want to go a bit deeper, Aaron Levie is great on the meta challenges here:

https://x.com/jasonlk/status/1947653390923927819