Log Management: From grep to Full-Text Search and Back
Formal Metadata
| Title | Log Management: From grep to Full-Text Search and Back |
|
| Title of Series | |
| Number of Parts | 69 |
| Author | |
| License | You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. |
| Identifiers | |
| Publisher | |
| Release Date | |
| Language | |
Content Metadata
| Subject Area | |
| Genre | |
| Abstract | Logs are everywhere. But they have gone through an interesting development over the years:
- grep: This works well as long as you have a single instance to search on. Once you need to SSH into many machines and try to piece together the results of multiple grep commands, things tend not to work that well anymore.
- Splunk: Centralizing those logs and letting users search through them with a piped language in Splunk is the logical step to fix that issue. However, the more data you centralize, the slower this will get.
- ELK: The solution to that idleness is using full-text search. Elasticsearch, in combination with Logstash and Kibana (plus Beats), gave logs a major performance boost. But at what cost?
- Loki: Reducing the scope and going back to a smart data structure combined with grep gives Loki the possibility to reduce costs while still providing good performance.
- Closing the gap: So what are the tradeoffs between the different systems, and are they potentially closing some gaps between performance and cost? |
|