Observing Python Applications Using Prometheus

We’ll start with a Python application which runs a web server exposing an endpoint. On successful requests this endpoint displays a rendered HTML page but sometimes returns a server error (5XX). We want to start observing the behavior of this application at runtime by tracking and exporting metric data to understand the effect this is having on the user experience of our application.

Metrics allow us to have time based data which provides insight into how our applications are performing. Together with Logs and Tracing they make up the three pillars of Observability. Metrics help us proactively identify unwanted behavior by incorporating them in alerts and dashboards.

We will instrument our application using the time-series database system Prometheus, which uses a "pull" method to extract data from running applications. This means that applications need to "expose" their metric data so that Prometheus is able to "scrape" it. This is typically done via an HTTP endpoint (/metrics, by convention).

We will use the Prometheus Python client library to track metrics in our code as well as explore the importance of monitoring, useful metrics and the collaboration between SREs and Application Engineers to ensure operational excellence.

You can expect a combination of discussing theory, demonstrations and practical challenges which will enable you to implement the concepts on your own. In the end, you will be equipped with the knowledge to later apply these learnings to your own applications.

Get ready to start adding metrics to your own services!

Agenda

• Theory of Observability and Metrics
• Exploration and implementation with Prometheus Python Client
• Tying it all together, devops as a culture

Prerequisites

This workshop is aimed at those who already have some experience with engineering and building applications in containers using Docker. To participate in the workshop you will be required to have Python 3, Python Poetry and Docker running on your machine

About the speaker:

Jessica Greene (she/her)

Having explored careers as a camera assistant in film & television, and as a specialty coffee roaster for a renowned European micro Roastery. Jessica made the decision to retrain in 2017 and add programming to her skill set. After completing Rails Girls Summer of Code scholarship in 2018 she joined the team at Ecosia.org to use her new skills to help plant trees. As a self taught/ community taught developer she is thankful to the local tech communities who supported her in making the move into IT. She organises events to support other women & gender minorities in making the move into tech.

Github: https://github.com/sleepypioneer

Vanessa Aguilar

Vanessa Aguilar (@veernacular) is a Mexican-American Site Reliability Engineer living and working in Berlin. When she isn’t in front of her computer, she is cuddling with her Pug “Connie”, cooking Oaxacan food, listening to music, or doing all three at once. Through engineering, she aims to create ways to empower her community and share her love for technology.

DataTalks.Club is the place to talk about data. Join our slack community!