Monitoring Mapify analytics with Grafana

Grafana is an open-source analytics and visualization platform that allows you to create charts, graphs, register logs, and set up alerts. Its dashboards are fully customizable to the needs of the project.

It’s an extremely popular tool for monitoring stacks, often used with time series databases like InfluxDB, Prometheus, and Graphite.

Grafana comes with really cool features, which can save you time and some headaches.

1. The ability to create dashboard templates that you can share with your team.
2. You can use scripts to automate the creation of dashboards.
3. There are plugins to extend Grafana’s capabilities and integrate it with other software.
4. It’s possible to set up alerts in Slack and other communication channels for specific events.
5. It allows you to turn a SQL database into metric data and then a graph.
   

Why we decided to start using Grafana

Our team needed to improve the operational performance of Mapify. To do that, it was crucial to track Mapily analytics with the help of alerts and colorful dashboards which now allow us to make more informed decisions and act faster when problems arise.

Before Grafana, our analytics consisted of Google Cloud analytics. They were quite limited to what we wished to monitor since it only gave us analytics about our infrastructure. Some examples are CPU utilization percentage, memory usage, network bytes in/out, keys in database.

Now Grafana is the one providing us with application analytics, such as queue size, messages in progress in regards to our workflows, and rows per table and table size rate regarding our datasets. About the former, we can detect if a single dataset is growing abnormally.

How we applied Grafana to our database

To start, there are a couple of things we had to go through before we could use Grafana.

1. Install Prometheus;
2. Connect Prometheus to our database;
3. Connect Prometheus to Kubernetes;
4. Connect Prometheus to the Mapify API;
5. Connect Prometheus to Redis;
6. Install Grafana;
7. Connect Grafana to Prometheus;
8. Ready to play with Grafana!
   

After configuring every dashboard on Grafana, this is what it looks like now:

We have alerts for all of these applicational metrics. In case the workflows and datasets metrics go over a certain value, we are automatically notified on Slack. Here’s an example:

What are the next steps?

Grafana is really a long-term investment for your team, so we’re looking forward to seeing how this will impact Mapify’s operations. At the moment, we still have a few more things to tweak on Grafana before we can call it a day.

We’ll be sure to update you on our Grafana journey in a few months!