Pierre Vincent Profile picture
Apr 5, 2022 10 tweets 8 min read Read on X
First talk of the day on the #QConLondon production track, by @glenathan and a challenge: can we build observable services without logs? Image
"We needed to build a new service in Go, without our usual existing scaffolding in Clojure... That led to some bikeshedding but also gave a chance for experimentation!" @glenathan #QConLondon
"Before this, we spent a lot of money to know what our applications were doing in production" Image
Giving up logs, for an event-first approach. "Events are just good logs" @glenathan #QConLondon ImageImageImageImage
Totally agree with @glenathan: it's ok to "clutter" your code with telemetry code, because it's right there in front of you, and encourages to do it consistently. Other approach like aspect oriented can hide it away, and gets forgotten.
Telemetry first! #QConLondon
It's not only about high-cardinality (eg user id...) but expressing more about units of work with good span names, gives a solid high level view of traces. @glenathan #QConLondon ImageImage
... But still do the high cardinality stuff! Throw all information, context and ids available at that stage (it's doesn't cost anything extra in a volume-based pricing!) @glenathan #QConLondon Image
Instrumenting with trace first doesn't mean we can't have logs, especially when developing locally or running tests. With a bit of magic on pretty printing all of the json events, and outputting events during failed tests. @glenathan #QConLondon ImageImage
So that's logs and traces from events. Next is metrics...which can also be derived from the events, with the same source of truth. @glenathan #QConLondon ImageImageImage
So the 3 pillars are actually all the same thing, just from a different angle. @glenathan #QConLondon ImageImageImage

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Pierre Vincent

Pierre Vincent Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @PierreVincent

Apr 5, 2022
Here we go, last talk of the day on the production track at #QConLondon, with @rdelvira and "an entertaining outage story" (his own words) when slack rolled out DNSSEC Image
"Who here tried to rollout DNSSEC?, Ok one person... Now how failed when trying to rollout DNSSEC? Welcome to the club!" 😂 @rdelvira #QConLondon
"We planned DNSSEC carefully, with the necessary changes and replicated most of our DNS use cases... And you'll see later why I said 'most'..." @rdelvira #QConLondon Image
Read 8 tweets
Apr 5, 2022
Continuing the #QConLondon production track with @yurynino, and using visual metaphors to understand our production data in a different way. Image
"In our field, observability is about humans and about how humans interact with technology" @yurynino #QConLondon Image
Collecting metrics and signals are only one part of the solution - observability has to come with good visualisation, and engineering a solution for humans. @yurynino #QConLondon ImageImage
Read 8 tweets
Apr 5, 2022
Next up on the production track, @mhausenblas on Continuous Profiling #QConLondon Image
"Observability is the capability to continuously generate and discover actionable insights based on signals from the system under observation with the goal to influence that system" and that's for both people (eg debugging) and automation (eg autoscaling) @mhausenblas #QConLondon Image
Observability can go beyond usual metrics, logs and traces: @mhausenblas introducing profiles and eBPF #QConLondon ImageImageImage
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(