We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Log Management: From grep to Full-Text Search and Back

Formal Metadata

Title
Log Management: From grep to Full-Text Search and Back
Title of Series
Number of Parts
69
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Logs are everywhere. But they have gone through an interesting development over the years: - grep: This works well as long as you have a single instance to search on. Once you need to SSH into many machines and try to piece together the results of multiple grep commands, things tend not to work that well anymore. - Splunk: Centralizing those logs and letting users search through them with a piped language in Splunk is the logical step to fix that issue. However, the more data you centralize, the slower this will get. - ELK: The solution to that idleness is using full-text search. Elasticsearch, in combination with Logstash and Kibana (plus Beats), gave logs a major performance boost. But at what cost? - Loki: Reducing the scope and going back to a smart data structure combined with grep gives Loki the possibility to reduce costs while still providing good performance. - Closing the gap: So what are the tradeoffs between the different systems, and are they potentially closing some gaps between performance and cost?