Liz Fong-Jones (方禮真) Profile picture
Principal 🐝 @honeycombio; governance @OpenTelemetry; views her own. labor advocate, board @coworkerfund. 🇺🇸 in 🇨🇦, ♿, 🏳️‍⚧️ & queer, #pluralgang ❄️🐆
Joanna (ze/zir/zey) Profile picture Nat Welch Profile picture 🇦🇺 techvedi™ 🇺🇸 Profile picture Apoorv Singh Profile picture jordan k Profile picture 11 added to My Authors
10 Jun
Tomorrow morning, I am keynoting at #hnycon with a "live" demo of how we @honeycombio bees release to production and debug when things inevitably break in the turbulent world that is customers & production traffic! Hope to see you at hnycon.io!
@honeycombio There is zero fakery involved in this demo, everything I did, I _actually_ did to our production systems, the only "cheat" is skipping forward by an hour mid-demo to skip past the "CI does its job, CD does phased deploy across environments with tests green" step.
Funny behind the scenes: share.getcloudapp.com/DOuqQyRd from @metaforgotten
Read 6 tweets
4 Jun
Always be deploying to production less than 3 hours after you write the code to maximize situational awareness.

That means code reviews need to happen within less than an hour, builds need to take less than 10 minutes, and you need to push to production at least hourly.
ps want to be part of a team like this? @honeycombio is hiring for a telemetry engineer, frontend engineer, and engineering manager! honeycomb.io/careers
pps before you passionately disagree saying "fast is not safe // releases are painful, so they need to happen as infrequently as possible", please do some background reading on DevOps and small batches, for instance from my friend @yesthattom :)

queue.acm.org/detail.cfm?id=…
Read 5 tweets
3 Jun
One more reason not to pick New Relic or Observe Inc/Snowflake: the companies have leadership that is pro-discrimination and anti-diversity (except for "ideological diversity" of course)

(bloombergquint.com/business/obser…)
Also you want to talk about "merit"?

"Sutter Hill is committed to supplying as much as $100 million in debt financing to Observe and doesn’t want the startup to have to raise money from other investors yet."

That was with them only having one (1) paying customer.
So ironic that people who have benefitted so much from inside connections want to make it about "merit not diversity" when they go back to the giant piles of cash from their connections every time.
Read 7 tweets
1 Jun
Storytime: this problem of preventing deadnaming was a project that Demma Rosa had been working on for years. Google even trumpeted her work. accelerate.withgoogle.com/stories/meet-g…

She no longer works at Google as of Aug 2020. And Google put up so many impediments to her finishing that work.
This isn't a brand new issue that Google can plead ignorance of or say it's scrambling to fix.

Trans Googlers as part of Gayglers, then as part of Trans@Google, raised these complaints for over a decade, including while I was head of the trans ERG at Google.
We were phenomenally lucky to have Demma Rosa as an ally on our side. But even with her escalating up through Ben Fried (CIO)'s org, we were unable to make traction on fixing the issue; it was almost as if Google just didn't care about being responsive to minority needs.
Read 4 tweets
4 May
how to fail at even pretending to be anti-racist:

thank and coddle a senior white supremacist employee who claims the idea of white supremacy is "racist against white people", claim that it has to be like the Holocaust to be racist, then refuse to denounce white supremacy.
also, like, that is the kind of viewpoint that's being coddled as "we have to avoid making these people feel uncomfortable" -- it's not about what the marginal tax rate should be, they're literally treating overt white supremacist language as a form of difference you celebrate
and that's what's trendy among the exec set: saying "everyone has to be welcome no matter how they're different, especially my white supremacist friend" at the expense of the Black, brown, minority gender, and queer folks at their companies that they're driving out.
Read 4 tweets
3 May
you have about 18 more hours to get a submission into @o11ycon! we're interested in hearing about your #observability journey, whether you're a Honeycomb customer or not!
@o11ycon Have a few minutes to share a topic you'd like to hear more about or have an anecdote about your #observability journey you'd like to share? Our Call for Presentations for @o11ycon is open until tonight! go.hny.co/3e5XcOY

Don't dawdle, put something in before signing off!
Hi US East Coast, it's about to be 7pm, remember, we're not extending this deadline any further. Even if it's just some partial thoughts, put in for @o11ycon the topic of your talk and a few points you might want to cover! #OpenTelemetry #Observability go.hny.co/3e5XcOY
Read 4 tweets
18 Dec 20
That was a giant freaking scare. A loose USB port connection on my #honeycomb_lx2k carrier board took it out of commission half of today.

Given I was already down one machine, this wasn't looking promising for stream tonight; but I'm burning it in with a temporary fix in place.
sometimes the answer to "why is my system unstable" is actually "because the USB ports are bad".
the burn-in is looking good so far, so we're on for the stream tonight!

Meanwhile, in other computer troubles, ASRock thinks the 5600X is a dud and that the B550M motherboard is fine, so on to asking AMD to RMA a grey market scalped (but authentic/sealed) processor...
Read 5 tweets
17 Dec 20
It's on! #AdventOfCode from my #Honeycomb_LX2K ARM64 machine starting in 5 min! twitch.tv/lizthegrey obs running showing a wayland desktop
Screensharing glitched first 5 min then was rock stable 60 FPS! #AdventOfCode

My application of De Morgan's Laws wasn't rock stable. I wasted 30 minutes on !(a || b) that should have been !a && !b instead of !a || !b

Video:
Code: github.com/lizthegrey/adv…
Timestamps say I was stuck for 35 minutes on that two character mistake. and reviewing it, I see _exactly_ what happened. It wasn't just !(a || b) expansion, I was simultaneously refactoring !(f(x) == 2 || f(x) == 3) to only eval f(x) once, so I forgot the intention #AdventOfCode
Read 4 tweets
16 Dec 20
You know who else was taken aside before becoming a manager and told that she couldn't be as vocal? This trans woman of color.

"Why do you have ‘concerns’ about me being a manager? The conversation was “She seems to be unhappy, so let’s not make her a manager.” " --@timnitGebru
I was told that part of the bargain of becoming a manager was being invited to sit at the decision-making table, but in exchange to commit and disagree on any decisions made there.

But I wasn't ever invited to the decision-making table about the ethical issues that mattered.
I was invited to participate in decisions about technical work of my team. And in performance evaluation for my team. And sure, I disagreed and committed on those things.

But to muzzle someone from criticizing any area of Google once they become a manager is a terrible bargain.
Read 8 tweets
16 Dec 20
In 2.5 hours: #AdventOfCode live with me in #GoLang. twitch.tv/lizthegrey

Just 10 more puzzles left!
Starting in on #AdventOfCode now!

Honestly hoping for a quick one as I have some job stuff to clean up after, having done errands/paperwork earlier in day, but we'll see what @ericwastl has in store for us! twitch.tv/lizthegrey
Oof. Got tripped up on parsing and then nested data structure mangling. But we got there with the help of the #AdventOfCode community!

Video: (processing may take a while, it's almost an hour...)
Code: github.com/lizthegrey/adv…
Read 4 tweets
16 Dec 20
Bought a 5600X from one of the more legit scalpers in .ca (AeroMalls, has a Newegg storefront, lists 20% above Canadian MSRP but not 50% or 100%+ above).

As the goblins say, "time is money, friend!" Swapping it in will wait for weekend, don't want to risk AoC stream or work day. Ryzen 5 5600X processor new...
Sequence of operations: unplug both AMD64 machines, leaving the ARM64 workstation and the Chromebook as backup.

Swap B550 mobo into the larger case and the X470 mobo into the smaller case. Remove 3400G from X470, insert 4750G from B550. Plug 5600X into B550. Move RAM/GFX. Pray.
I wish I'd put the 4750G in the X470 to begin with (since both are PCIE 3.0 only), but they only recently released BIOS updates enabling 5000 and 4000 series on the X470 so that wasn't possible in September :/
Read 9 tweets
5 Dec 20
and for research.google/pubs/pub45764/ and research.google/pubs/pub46939/ and the countless conference talks I gave while a Googler, when the approver approved after vetting the content was generally on the right track and didn't leak secrets, their judgment was trusted.
even if the talk was added to pubapprove 2 days before the talk, if the approver got to it before the talk, the talk was approved. fullstop.

(if the approver _didn't_ get to it in the 2 days, then you'd have to retract/withdraw, of course. but not the pattern of facts here)
In fact, I am pretty damn sure that the vast majority of talks added to pubapprove _aren't_ submitted for review a full 2 weeks before deadline. I would eat my hat if every single Googler were diligent enough to work 2 weeks ahead of schedule.
Read 4 tweets
4 Dec 20
miss me with your sex-negativity and whorephobia. Yes, both YouTube and Pornhub and Vimeo and any site where videos are shared will have people who post non-consensual videos against ToS.

But this is part of a systematic effort to stigmatize sex work, not to stop "trafficking".
"oh my god, there are videos of women being asphyxiated with plastic bags" -- have you considered for a moment that maybe some of those women _enjoy_ it and are getting paid to, and it's a fun day at work for them?
Seriously, if I didn't have a tech career, being a sex worker would also actually be pretty high up on my list, because I'd get paid to do something I like?

I seriously considered applying to work w/ kink\.com when I was in my first year of my tech career. It seemed fun.
Read 7 tweets
3 Dec 20
What Google is doing to @timnitGebru is unconscionable and is disparate treatment. When I sent this letter to my manager and HR, I was not immediately terminated.

The company did not honor my Feb 25 date (and pushed me out by Jan 26 instead), but that's still miles different.
*Jan 29, correction, I got my 6 and 9 mixed up while typing fast :/
This. They are making an example of @timnitGebru because she is Black.
Read 11 tweets
13 Oct 20
at the moment, shooting rogue drone structures gives derived standings to edencom and to trigs because edencom and trigs both hate rogue drones. #TweetFleet #EVEOnline standings screen showing -5.0 and -2.5 trig and edencom to r
Rogue drone structures can be found in Drone Squad and Drone Gathering anoms in k-space, among other locations (combat sigs too); it isn't required to go into Pochven to find them. wiki.eveuniversity.org/index.php?sear… #EVEOnline #TweetFleet
A listing of all drone destructible sentries is at everef.net/groups/383 -- and probably can be cross-referenced to anom/combat site listings to figure out what will give standings. #EVEOnline #TweetFleet
Read 10 tweets
7 Oct 20
To be a little bit less conciliatory, what the fucking fuck?

$35M in funding (incl Series A, debt in lieu of Series B), stealth mode throughout, and only 15 customers?!!! If only @mipsytipsy & @cyen had the privilege of white men. crn.com/slide-shows/cl…
The bar around revenue, customer base is held super high for founding teams made up of marginalized folks, but the instant you have white dudes with pedigrees leading your company, VCs throw in tens of millions of dollars without you having traction & product/market fit.
p.s. my q about unit economics has now been answered by the article: they charge you more the more you query, and pass on the costs of having an expensive indexing engine to you. Making customers think twice before querying is a hell of a business model.
Read 7 tweets
7 Oct 20
Do I read correctly that tiered storage is GA in @confluentinc platform 6.0?

I absolutely must try it if so. docs.confluent.io/current/kafka/…
Confluent Platform 6.0 upgrade is baking in the oven...
so far, it's been amazingly unremarkable, we hardly even noticed from the consumers/producers that anything had changed (aside from rolling broker restarts).

A few alarming blips to the "requests total time" for fetch.consumer.mean, but other than that smooth sailing.
Read 15 tweets
2 Oct 20
til a "walk-in clinic" is different from a "primary clinic". why is health provision so difficult?
backstory is I know exactly what specialist center I need to go to to get my methotrexate refilled and get my bloodwork done, but it's gated on seeing a primary care doctor first to get a referral :/
that was _lucky_. This primary care doctor had treated autoimmune patients taking methotrexate before and was willing to write a prescription and order the correct labs (and also knew folks also take folic acid w/ MTX).

I will still need referral for eye exam, but whew.
Read 4 tweets
28 Sep 20
#planeselfie because this is the only one I'm going to get for the next year at least...

Vancouver, here we come! Looking forward to being reunited with Elly after a month apart. Lizzes wearing a mask seated at the rear of the business cla
.@theflurbs looked very dejected when I abandoned her with CATSA at oversize baggage drop-off. Her second time on an airplane in her life, we think. Flurry, a biscuit colored Samoyed-Golden mix, looks towards
microcovid.org says at current rates in Ontario, with 3 other business class passengers near me within 6 feet for 6 hours, it's ~20 microcovid (10-70 confidence range), so it's between a few days and a month of error budget if I want 99.99% odds of surviving thru 2021.
Read 7 tweets
25 Sep 20
whoa, I've healed enough that I momentarily blanked on my rapist's name.

It's Richard DeVaul (@rdevaul). After resigning from Google & deleting his Twitter after that story, he's back on tech twitter, and is the CTO of a company called Prometheus Fuels, according to his LinkedIn
@rdevaul Two years later, there's been no accountability. He's CTO of a Y Combinator backed startup, after having sexually assaulted me (and many others), & having sexually harassed @starsandrobots.

@daiwaka and @ktbenner's reporting has been met by a shrug by the "innovation ecosystem"
They're perfectly fine to hire a serial rapist and sexual harasser, who has literally had articles about his bad conduct printed in the NYT on page A1. Because their mission is about "innovation" and "solving climate change".

Some of y'all who follow me, follow him. 🤮🤮🤮
Read 10 tweets
23 Sep 20
Hot take: if you're a SaaS provider of observability tooling, and didn't build your own data store & query engine specialized in the observability use case, your unit economics will suck.

(there may be other redeeming features of your product though, so it's not a fatal flaw)
the o11y use case is just... so different from every case on the market. It's not data warehousing, as you need results near-instantly rather than minutes or hours later. It's a write once, read specific rows almost never but when you do it matters, drilldown/slice ALL the time.
There's a reason that we didn't build Monarch or Dapper on top of Spanner or Bigtable in the end but instead had to go all the way down to RecordIO/ColumnIO at Google. Likewise if you look at every major player in our biz, they wrote their own telemetry store.
Read 4 tweets