Why do you need a Centralized Logging System like Graylog?

Suppose that your application is designed on top of Microservices architecture and a particular request goes through services A, B, and C. Each of these services run a load balancer that spreads requests among 10 different servers running application servers and reverse proxies (Nginx, for example). Also, these services use PostgreSQL and Redis instances running on Amazon RDS and Amazon Elasticache respectively. 

Now answer me, how can you keep track of what's going on when something goes wrong in this particular scenario? Are you going to ssh 10 production servers from service A, 10 production servers from service B, 10 production servers from service C, and for each of them use commands like tail -f logfile | grep "ERROR" for the Nginx access.log and error.log as well as the application server logs? Don't forget that on top of that you would have to monitor the AWS logs for PostgreSQL and Redis as well. 

Well, I think you got the idea. The bottom line is that the traditional way of visualizing logs (analyzing specific log files stored on the disk) doesn't scale at all. In fact, there are many other problems related to this traditional approach. Let's list out some of them:

Security issues: you have to ssh production servers every time you want to see the logs, and as you know, some angry system administrators that fight on UFC for fun don't like that for security reasons, especially if you are not an infrastructure expert. As a result, you can get very hurt;

Logs are streams, not files: you cannot assume that logs are being directly stored into files on the disk for cloud-native applications that follow the 12-Factor logging practices. Instead, the only guarantee you have in the cloud is that the disk is ephemeral;

Filtered content: some log entries might contain sensitive information and must be hidden while others don't. How to control that and avoid that someone that is directly logged into the server could see the sensitive log entries?

Inefficiency: when something goes wrong, the time spent on logging into many machines and greping log files one by one is probably the time that would take to query Graylog, figure out what's wrong and start working on the fix.

Now, let's list out some benefits you instantaneously get when you use Graylog as your centralized logging system:

Increased Security: people don't need to have access to the production servers as the log visualization is displayed on the beautiful Graylog Web Interface through their preferred modern browser. Keep in mind that if you try to use something like Internet Explorer 6, your problem is not logs, my friend!;

Authentication and Authorization: it's possible to manage users and roles very easily. Actually, it's even possible to integrate it with your existing LDAP solution (if you have one);

Powerful Querying Capabilities: you can use a beautiful and powerful search syntax very close to the Apache Lucene syntax rather than relying on insanely ugly regular expressions;

Built-in Alerting: Graylog has a built-in alerting mechanism, so you don't need to be all the time worried about the behavior of your application and suffering from nightmares at night. Instead, you can define certain patterns that would trigger an alert and then get alerted on your mobile with a graceful message like this one: "Sorry to bother you, Mr. Jorge Acetozi, but it looks like someone attempted to ssh your production server (IP: XXX.XXX.XXX.XXX) for 5 consecutive times in less than a minute, so I was wondering here if you could take a look at it. By the way, your coffee is ready and waiting for you on your desk. Thank you very much for your time!";

Input Everything: you don't have to restrict the centralized logging system to your application logs only. You can input every type of logs in it, such as the sshd daemon in the alerting example before, operating system logs, firewall logs... Well, pretty much everything!;

Custom Dashboards: create custom dashboards and display them on televisions spread around your company's office;

Geolocation: add geolocation metadata to log specific log entries (such as Nginx access.log) so that you can visualize where the requests are coming from;

Reports: create reports that could even be useful for Marketing decisions, such as the more frequently accessed pages.

Well, and the list goes on! 

However, everything comes at a price. Maintaining a production infrastructure of a centralized logging system like Graylog is neither easy nor cheap. As far as Graylog is concerned, it relies on different tools such as Elasticsearch and MongoDB to get the work done. In production, such tools should be deployed as clusters to allow high-availability, performance, and scalability, which of course leads to a increased complexity of the overall solution.

Hopefully, my book will provide a good understanding of each technology involved in Graylog's architecture in a hands-on (and not boring) fashion so that you can get started with Graylog as soon as possible!

You can get the in progress version of the book for only $9,99 on Leanpub and download a new chapter every week. If you get the bundle with my other book Continuous Delivery for Java Apps using Kubernetes and Jenkins, the Graylog book will cost you only $5 bucks!

Please reach me out at https://www.jorgeacetozi.com/about if any issues come up throughout your Graylog / Continuous Delivery journey!