In 2002, I was working for a nation-wide retailer. About 6 months after building a new store, my car broke down. I lived so far away from the store that I was only able to get two rides home from different colleagues. I ultimately took to walking 23 miles to work (each way) 🧵 The moment I got home from walking, I had to immediately sleep for 4.5 hours, wake up, take a quick shower, and then start walking back to work

Colleague: hey, your dashboard says this one user is pounding the NFS server we are trying to sunset, can you help us find the rogue Linux process that is writing to this NFS server?
Me: Why, sure!
(60 seconds later)
Colleague: Holy Shit!
<let us explore what just happened> 🧵 Me, armed with the user's name, logs into the NFS server on the command-line. Executes "nfsdtop -cwU <user>" and now I have a realtime display of the machines the user is using to access the NFS server. I then use "ssh" to log into the node where the activity is coming from

When you wield God-like abilities, people come to you with humble requests like this and within seconds you just blow their mind -- alternate introduction: "what can you do with bpftrace on Linux?" Let's go … We're sunsetting an NFS server, replacing it with Ceph. Problem: There are hundreds of systems using the NFS server. Here is the request that has come-down from on-high:

Red Team, Blue Team, why not both? Why not go purple and use Red Team's tactics against them? Let's go 🧵 Achtung: if you have problems with anxiety, turn back now

While Gmail is clamping down on MX relay by requiring SPF (at a minimum) and DKIM, now seems to be a good time to drop a bomb 🧵 💣 why there has been no better time to run your own mail server E-Mail providers for decades have been built on the premise that account holders want to be able to send mail to people anywhere. As if e-mail at-large must be a proxy for physical mail, but I am about to turn that on its head for fun and profit

ssh has secrets. Too many to share in one tweet. One of which is how it acts as a serial-line processor for secret keyboard functionality you probably never knew about. For example, why, when you press ENTER and then ~ immediately after, does the ~ not appear right away? Thread… ssh secret No 1 is that you can tell how many ssh-layers deep you are (how many hosts you've hopped-through to get where you are) simply by pressing ENTER and then counting how many times you have to press ~ before it shows up on your prompt (important for tip No 2)

As happens quite often, I was pulled-in to help solve a mystery at work. In migrating to Enterprise Linux 9, some RPMs were failing to install due to build-id errors. A few hours later and I uncover a dirty change over 20 years in the making Some individuals at RedHat who shall remain nameless decided to turn on signing by default for all packages, traditionally only opt-in for debuginfo packages

@TonyBenoy They did bring me into the review process. And ignored 90% of what I said. Then had their manager approve it anyway. Then had my boss make me deploy it.

Today, I just got authorization to turn off their code en-masse and enable my new code which I wrote to replace their Python @TonyBenoy "TLDR : A junior dev rewrote your code and the code had a bug?"
Sending 2M e-mails in the course of 4 weeks and causing HR to reach out to systems because several managers cannot receive e-mail.
Wasting ~600k USD in R&D
Wasting ~350k USD in electricity
Don't down-play the effects

A little lesson about soft-skills. [thread]

Boss was on vacation last week.
Shit hit the fan while he was gone.

First msg to boss:
"Welcome back! I hope vacation was super fun. I expect to hear tales of fun in today's meeting."
Boss: Aunt got COVID! Wife showing symptoms 😲
Uh, yeah, glad I decided to do a soft-opening on this week before the meeting. I'll be holding back from retelling of the shit-storm week we had while he was gone.

He's got a lot on his plate.

Ok, remember that custom OOM killer I wrote in Perl which got rewritten to Python by a junior software engineer? And management put it into production?

Oh ho ho, grab the popcorn because the train derailed and took out an entire Potemkin village (a juicy thread) Management put the Python into production. It ran. It sent 100k e-mails a day to my inbox, that of my boss, and many others, including the boss of the person that wrote said Python.

And it ran. I started sounding bells and whistles, but nobody in management (except MY boss) ...

Remember when I said that a certain co-worker was chartered to rewrite my work from Perl to Python because he wanted to make a change but didn’t know Perl? Have I got a new extended edition of that story for you. Strap-in, cause this ride is about to get turbo-charged! There were *several* concepts in my Perl code that were completely opaque to him and his team, not because it was Perl, but because they simply could not imagine why I “over-engineered” the thing.

Remember that term. This is going to be a lesson in humility for lots of folks

Courtesy of Gunny and friends at @LumberJocks, I have a workshop apron now for my birthday. From my favorite leather company, @hideanddrink whom also makes my leather chisel roll So beautiful and hand crafted

I am going to make a tiny acrylic push block to push small pieces into my trim router Step 1 is complete, cutting some contact “paper” to the size of the block. Next step is to glue it to the block and then clamp it before we put the handle on

The first thing you'll always do when starting a new #bpftrace script is selecting a probe. In this thread we will discuss how to select the probes that will give you the information you desire. 1/ 2/ This is arguably the most difficult step. First, we need to know the landmarks. Use the very basic command of "bpftrace -l" to list all the probes. On my system, that produces 40032 lines

I am releasing details today. All information has been gathered, timelines analyzed, old code vetted, major flaw fixed, and results scrutinized. Strap-in boys and girls, this rollercoaster goes to 11! Sun Microsystems #GridEngine has been hiding an ace 🂡 up its sleeve for 16 yrs

https://twitter.com/freebsdfrau/status/1274627170573225985

Show you the code? OK ... github.com/FrauBSD/sge/

Still interested in how a decades-old job scheduler might have contained the necessary bits to dethrone even future contenders?

Stick around and we'll see how Sun #GridEngine may just surprise you

Hey, want to see a crazy trick?
+ Compile package from port on #FreeBSD 12.1
+ Convert to old-style FreeBSD package
+ Install on FreeBSD 9.2
Only works with packages containing no binaries, of course; security/acme.sh for example So let's get into it. Because that probably sounds like dark magic (well, it is, and I'm going to show you how to wield it).

This probably sounds like Debian/Ubuntu's "alien" utility but for FreeBSD. Well, it kind of is.

Thread.

I've publicly released nearly everything you need to make these dashboards. The only thing you need is a BeeGFS cluster, the rest is easy.
github.com/FrauBSD/beegfs…
github.com/FrauBSD/beegfs… Step 1: Download/Install beegfstop

That's easy, it's a shell script. Just download it and put it in /usr/bin (if you're too lazy to say "make" in the redhat dir to make an RPM or you're using a non-RH based Linux distro)

raw.githubusercontent.com/FrauBSD/beegfs…

Don't make these mistakes in #Grafana when using #InfluxDB Let's go over each mistake [thread]

Details still pending, but what if I told you I discovered a way to improve cluster scheduling performance under load by up to 60000% or more? Sun Microsystems hid a really amazing optional feature in #GrodEngine and it has largely been unknown/unused/unfinished for 19 years! The problem arises when you have over 100k pending jobs. Things start to slow down, job wait times sky-rocket, and the end-result is low-throughout. All-bad if you spent $8MM for your equipment and it is under-performing, and not returning your investment. Well...

Watching @JurassicWorld I witnessed "dmidecode --isolate" and throughout the remainder of the film I could not stop thinking about this. movies.stackexchange.com/questions/9006… I am positive I saw "dmidecode --isolate" and now I'm thinking how I could make that a real thing There's something called the DPDK for Linux. dpdk.org Stands for Data Plane Development Kit. It has an option for isolating CPUs. If you're going to add an option to dmidecode to "isolate" something/everything -- and in the premise of the @JurassicWorld movie -- 1/

I wonder how many of my followers know about ssh's builtin session management prompts? To explore this, I first need to explain why initial tilde(s) (~) seem to be "eaten" by your shell. Strap in for a fun one, you either already knew about this or this might just shock you 1/? 2/? While ssh'd to another host, after you press ENTER (this only works after pressing ENTER) press tilde (~). It won't appear on screen. Press tilde again. If you are ssh'd more than once (e.g., ssh -> ssh -> shell), it still won't appear. For each ssh, +1 tilde is eaten