,
7 tweets,
2 min read
Read on Twitter
Yesterday I had a situation that just totally epitomizes the futility of software development.
1. My website was not updating after I pushed to master.
2. Hm, I wonder why. Check Travis CI - build is green.
3. Must be the CDN. Check the CDN, purge cache.
4. Still no update.
⬇️
1. My website was not updating after I pushed to master.
2. Hm, I wonder why. Check Travis CI - build is green.
3. Must be the CDN. Check the CDN, purge cache.
4. Still no update.
⬇️
5. Rebuild Travis a few times, no dice.
6. Finally read the log carefully. "Update failed, out of disk space".
7. Hunt for my YubiKey. SSH into my server. 28GB of 30GB used. 😱
8. No worries, I will just size up the instance until I figure it out.
9. Log in to DigitalOcean admin.
6. Finally read the log carefully. "Update failed, out of disk space".
7. Hunt for my YubiKey. SSH into my server. 28GB of 30GB used. 😱
8. No worries, I will just size up the instance until I figure it out.
9. Log in to DigitalOcean admin.
10. Oh, 30GB disk space is the maximum for all instances.
11. Crap. What do I do now? *checks* WEBSITE DOWN. Crap.
12. Googles "What is the s3 of Digital Ocean?"
12. Spaces. I need a Space. 250GB of space, I will just put my static images there (my ASSUMED problem).
11. Crap. What do I do now? *checks* WEBSITE DOWN. Crap.
12. Googles "What is the s3 of Digital Ocean?"
12. Spaces. I need a Space. 250GB of space, I will just put my static images there (my ASSUMED problem).
13. But how am I going to back this up?
14. I will create a git LFS repo and sync it to the space and then proxy with Nginx! Problem solved! Symbolic link for local dev!
15. Takes one hour to upload images to my new repo.
16. Didn't properly set images to LFS. One more hour.
14. I will create a git LFS repo and sync it to the space and then proxy with Nginx! Problem solved! Symbolic link for local dev!
15. Takes one hour to upload images to my new repo.
16. Didn't properly set images to LFS. One more hour.
17. How am I going to sync this automatically?
18. Hm, I just need it to run one command (s3cmd a Python util).
19. I will try to do this with GitHub actions.
20. Waste HOURS trying to make the GitHub action work. Battle Docker. Battle Python. Battle s3cmd.
21. Give up go to bed.
18. Hm, I just need it to run one command (s3cmd a Python util).
19. I will try to do this with GitHub actions.
20. Waste HOURS trying to make the GitHub action work. Battle Docker. Battle Python. Battle s3cmd.
21. Give up go to bed.
22. It's morning. Today. Think to myself, "How can these images possibly be 20+ GB? My MacBook Air would never have space for that."
23. Log in to the server. `du -sh` around.
24. Realize that my problem is the `.git` directory is TWENTY GIGABYTES.
25. Run `git gc`.
23. Log in to the server. `du -sh` around.
24. Realize that my problem is the `.git` directory is TWENTY GIGABYTES.
25. Run `git gc`.
26. Size of .git goes from 21GB to >1GB.
TLDR: I spent probably 8+ hours troubleshooting and "solving" a problem that was fixed with one command.
🙃
TLDR: I spent probably 8+ hours troubleshooting and "solving" a problem that was fixed with one command.
🙃
