Finally found time to look at this a little more closely.
It's interesting that in the recent 2-3 years, the newer crop of logging systems have shied away from ELK style full text indexing but instead treat log aggregation and querying as a stream processing problem of sorts.
It's interesting that in the recent 2-3 years, the newer crop of logging systems have shied away from ELK style full text indexing but instead treat log aggregation and querying as a stream processing problem of sorts.
Where queries are akin to grep against time partitioned files. One of the important aspects of observability is refining the search space (I believe @el_bhs was the one who mentioned this in one of his recent talks) - which isn't to be confused with visualizing the search space.
"no full text indexing" doesn't mean there's no indexing whatsoever. Most (if not all) of these systems index based on timestamp. Loki indexes Prometheus metadata labels which I presume (haven't read the source as yet) it extracts from raw logs using mtail (h/t @jaqx0r)
How and what to index is often an exercise in tradeoffs with different pros and cons. With Kafka + kSQL, there's no indexing involved (correct me if I'm wrong @gwenshap). SaaS offerings like Honeycomb and Humio doesn't use any indexing either (bit.ly/2zTQCqE).
