Prometheus.jl

Introduction

This package is a Julia client for Prometheus. If you are not familiar with Prometheus it is recommended to browse the upstream documentation. The documentation here focuses on the Julia client.

Two of the basic concepts of a Prometheus client are Registries and Collectors. Registries are collections of collectors, and the collectors are the units responsible to record and capture metrics. Client libraries implement a default registry which all collectors implicity register with, so for basic usage there is no need to interact with a registry (see Default registry).

The third important concept is Exposition of the collected metrics. Typically metrics are exposed over a HTTP server, as in the Quickstart-example just below. See the section about Exposition for more details and examples on how metrics can be exposed.

Quickstart

  1. Install Prometheus.jl and HTTP.jl using the package manager:

    pkg> add Prometheus HTTP
  2. Paste the following code into a Julia REPL.

    # Load the packages
    using Prometheus, HTTP
    
    # Create a Counter metric
    const request_counter = Prometheus.Counter("request_count", "Number of handled requests")
    
    # Start a HTTP server on localhost port 8000 to server the metrics
    server = HTTP.listen!(8000) do http
        Prometheus.inc(request_counter) # Increment the request counter
        return Prometheus.expose(http)  # Expose the metrics
    end
  3. Visit http://localhost:8000 in your browser. You will see something like the following

    # HELP gc_alloc_bytes_total Total number of allocated bytes
    # TYPE gc_alloc_bytes_total counter
    gc_alloc_bytes_total 365578814
    
    [...]
    
    # HELP request_count Number of handled requests
    # TYPE request_count counter
    request_count 1

    The output contains some default metrics related to the running process, as well as the request counter that we added ourselves. Every time you refresh, the counter will increment its value. close(server) will shutdown the server.

Collectors

This section documents the collectors that are currently supported. This include the "basic" collectors (Counter, Gauge, Histogram, Summary) as well as some custom collectors (GCCollector, ProcessCollector). There is also a section on how to implement your own collector, see Custom collectors.

Upstream documentation:

Counter

Quoting the upstream documentation:

A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart. For example, you can use a counter to represent the number of requests served, tasks completed, or errors.

Do not use a counter to expose a value that can decrease. For example, do not use a counter for the number of currently running processes; instead use a gauge.

Counter API reference

Prometheus.CounterMethod
Prometheus.Counter(name, help; registry=DEFAULT_REGISTRY)

Construct a Counter collector.

Arguments

  • name :: String: the name of the counter metric.
  • help :: String: the documentation for the counter metric.

Keyword arguments

  • registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Pass registry = nothing to skip registration.

Methods

source
Prometheus.incMethod
Prometheus.inc(counter::Counter, v::Real = 1)

Increment the value of the counter with v. The value defaults to v = 1.

Throw a Prometheus.ArgumentError if v < 0 (a counter must not decrease).

source

Gauge

Quoting the upstream documentation:

A gauge is a metric that represents a single numerical value that can arbitrarily go up and down.

Gauges are typically used for measured values like temperatures or current memory usage, but also "counts" that can go up and down, like the number of concurrent requests.

Gauge API reference

Prometheus.GaugeMethod
Prometheus.Gauge(name, help; registry=DEFAULT_REGISTRY)

Construct a Gauge collector.

Arguments

  • name :: String: the name of the gauge metric.
  • help :: String: the documentation for the gauge metric.

Keyword arguments

  • registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Pass registry = nothing to skip registration.

Methods

source
Prometheus.incMethod
Prometheus.inc(gauge::Gauge, v::Real = 1)

Increment the value of the gauge with v. v defaults to v = 1.

source
Prometheus.decMethod
Prometheus.dec(gauge::Gauge, v::Real = 1)

Decrement the value of the gauge with v. v defaults to v = 1.

source
Prometheus.setMethod
Prometheus.set(gauge::Gauge, v::Real)

Set the value of the gauge to v.

source
Prometheus.@timeMacro
Prometheus.@time collector expr

Time the evaluation of expr and send the elapsed time in seconds to collector. The specific action depends on the type of collector:

  • collector :: Gauge: set the value of the gauge to the elapsed time (Prometheus.set)
  • collector :: Histogram and collector :: Summary: add the elapsed time as an observation (Prometheus.observe)

The expression to time, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.

Prometheus.@time collector <expr>

Prometheus.@time collector begin
    <expr>
end

It is also possible to apply the macro to a function definition, i.e.

Prometheus.@time collector function time_this(args...)
    # function body
end

to time every call to this function (covering all call sites).

source
Prometheus.@inprogressMacro
Prometheus.@inprogress collector expr

Track the number of concurrent in-progress evaluations of expr. From the builtin collectors this is only applicable to the Gauge – the value of the gauge is incremented with 1 when entering the section, and decremented with 1 when exiting the section.

The expression, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.

Prometheus.@inprogress collector <expr>

Prometheus.@inprogress collector begin
    <expr>
end

It is also possible to apply the macro to a function definition, i.e.

Prometheus.@inprogress collector function track_this(args...)
    # function body
end

to track number of concurrent in-progress calls (covering all call sites).

source

Histogram

Quoting the upstream documentation:

A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values.

Histogram API reference

Prometheus.HistogramMethod
Prometheus.Histogram(name, help; buckets=DEFAULT_BUCKETS, registry=DEFAULT_REGISTRY)

Construct a Histogram collector.

Arguments

  • name :: String: the name of the histogram metric.
  • help :: String: the documentation for the histogram metric.

Keyword arguments

  • buckets :: Vector{Float64}: the upper bounds for the histogram buckets. The buckets must be sorted. Inf will be added as a last bucket if not already included. The default buckets are DEFAULT_BUCKETS = [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0, Inf].
  • registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Pass registry = nothing to skip registration.

Methods

source
Prometheus.observeMethod
Prometheus.observe(histogram::Histogram, v::Real)

Add the observed value v to the histogram. This increases the sum and count of the histogram with v and 1, respectively, and increments the counter for all buckets containing v.

source
Prometheus.@timeMacro
Prometheus.@time collector expr

Time the evaluation of expr and send the elapsed time in seconds to collector. The specific action depends on the type of collector:

  • collector :: Gauge: set the value of the gauge to the elapsed time (Prometheus.set)
  • collector :: Histogram and collector :: Summary: add the elapsed time as an observation (Prometheus.observe)

The expression to time, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.

Prometheus.@time collector <expr>

Prometheus.@time collector begin
    <expr>
end

It is also possible to apply the macro to a function definition, i.e.

Prometheus.@time collector function time_this(args...)
    # function body
end

to time every call to this function (covering all call sites).

source

Summary

Quoting the upstream documentation:

Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.

Summary API reference

Prometheus.SummaryMethod
Prometheus.Summary(name, help; registry=DEFAULT_REGISTRY)

Construct a Summary collector.

Arguments

  • name :: String: the name of the summary metric.
  • help :: String: the documentation for the summary metric.

Keyword arguments

  • registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Pass registry = nothing to skip registration.

Methods

source
Prometheus.observeMethod
Prometheus.observe(summary::Summary, v::Real)

Add the observed value v to the summary. This increases the sum and count of the summary with v and 1, respectively.

source
Prometheus.@timeMacro
Prometheus.@time collector expr

Time the evaluation of expr and send the elapsed time in seconds to collector. The specific action depends on the type of collector:

  • collector :: Gauge: set the value of the gauge to the elapsed time (Prometheus.set)
  • collector :: Histogram and collector :: Summary: add the elapsed time as an observation (Prometheus.observe)

The expression to time, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.

Prometheus.@time collector <expr>

Prometheus.@time collector begin
    <expr>
end

It is also possible to apply the macro to a function definition, i.e.

Prometheus.@time collector function time_this(args...)
    # function body
end

to time every call to this function (covering all call sites).

source

GCCollector

A collector that exports metrics about allocations and garbage collection (for example number of allocations, number of bytes allocated, time spent in garbage collection, etc). These metrics have the julia_gc_ prefix in their name.

A GCCollector is registered automatically with the default registry, see Default registry for more details.

GCCollector API reference

Prometheus.GCCollectorMethod
Prometheus.GCCollector(; registry=DEFAULT_REGISTRY)

Create a collector that exports metrics about allocations and garbage collection.

Keyword arguments

  • registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. The default registry is used by default. Pass registry = nothing to skip registration.
Note

A GCCollector is registered automatically with the default registry. If necessary it can be removed by calling

Prometheus.unregister(Prometheus.DEFAULT_REGISTRY, Prometheus.GC_COLLECTOR)
source

ProcessCollector

A collector that exports metrics about a running process, for example CPU seconds and metrics about I/O operations. Metrics from this collector have the process_ prefix in their name. This collector is only available on Linux since it requires the /proc file system.

A ProcessCollector for the current process is registered automatically with the default registry, see Default registry for more details.

ProcessCollector API reference

Prometheus.ProcessCollectorMethod
Prometheus.ProcessCollector(pid; registry=DEFAULT_REGISTRY)

Create a process collector for the process id given by the pid function. The collector exposes metrics about the process' CPU time, start time, memory usage, file usage, and I/O operations.

Arguments

  • pid :: Function: a function returning a process id as a string or integer for which to collect metrics. By default the "self" pid is used, i.e. the current process.

Keyword arguments

  • registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. The default registry is used by default. Pass registry = nothing to skip registration.
Note

A ProcessCollector for the current process is registered automatically with the default registry. If necessary it can be removed by calling

Prometheus.unregister(Prometheus.DEFAULT_REGISTRY, Prometheus.PROCESS_COLLECTOR)
Note

The process collector is currently only available on Linux since it requires the /proc file system. On Windows and macOS this collector will not expose any metrics.

source

Custom collectors

RandomCollector

Labels

Prometheus allows attaching labels to metrics, see the upstream documentation:

In this package labeling of collectors is done with Prometheus.Family. A collector family consist of a number of regular collectors, the children, with unique labels.

A concrete example is a HTTP request Counter, where we might also want to keep track of the target resource and the status code of the request. Such instrumentation can be implemented as follows

# Custom label struct
struct RequestLabels
    target::String
    status_code::Int
end

# Create the counter family
request_counter = Prometheus.Family{Prometheus.Counter}(
    "http_requests", "Total number of HTTP requests", RequestLabels
)

# Extract a Counter for a specific set of labels
counter = Prometheus.labels(request_counter, RequestLabels("/api", 200))

# Increment the counter
Prometheus.inc(counter)

Note that using a custom label struct is optional, refer to the constructor Prometheus.Family and Prometheus.labels for alternative methods.

Family API reference

Prometheus.FamilyMethod
Prometheus.Family{C}(name, help, args..., label_names; registry=DEFAULT_REGISTRY, kwargs...)

Create a labeled collector family with labels given by label_names. For every new set of label values encountered a new collector of type C <: Collector will be created, see Prometheus.labels.

Arguments

  • name :: String: the name of the family metric.
  • help :: String: the documentation for the family metric.
  • args...: any extra positional arguments required for Cs constructor, see Prometheus.labels.
  • label_names: the label names for the family. Label names can be given as either of the following (typically matching the methods label values will be given later, see Prometheus.labels):
    • a tuple of symbols or strings, e.g. (:target, :status_code) or ("target", "status_code")
    • a named tuple type, e.g. @NamedTuple{target::String, status_code::Int} where the names are used as the label names
    • a custom struct type, e.g. RequestLabels defined as
      struct RequestLabels
          target::String
          status_code::Int
      end
      where the field names are used for the label names.

Keyword arguments

  • registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Pass registry = nothing to skip registration.
  • kwargs...: any extra keyword arguments required for Cs constructor, see Prometheus.labels.

Methods

Examples

# Construct a family of Counters
counter_family = Prometheus.Family{Counter}(
    "http_requests", "Number of HTTP requests", (:target, :status_code),
)

# Increment the counter for the labels `target="/api"` and `status_code=200`
Prometheus.inc(Prometheus.labels(counter_family, (target="/api", status_code=200)))
source
Prometheus.labelsMethod
Prometheus.labels(family::Family{C}, label_values) where C

Get or create the collector of type C from the family corresponding to the labels given by label_values. If no collector exist for the input labels a new one is created by invoking the C constructor as C(name, help, args...; kwargs..., registry=nothing), where name, help, args..., and kwargs... are the arguments from the family constructor, see Family.

Similarly to when creating the Family, label_values can be given as either of the following:

  • a tuple, e.g. ("/api", 200)
  • a named tuple with names matching the label names, e.g.(target="/api", status_code=200)
  • a struct instance with field names matching the label names , e.g. RequestLabels("/api", 200)

All non-string values (e.g. 200 in the examples above) are stringified using string.

Tip

Base.getindex is overloaded to have the meaning of Prometheus.labels for the family collector: family[label_values] is equivalent to Prometheus.labels(family, label_values).

Note

This method does an acquire/release of a lock, and a dictionary lookup, to find the collector matching the label names. For typical applications this overhead does not matter (below 100ns for some basic benchmarks) but it is safe to cache the returned collector if required.

source
Prometheus.removeMethod
Prometheus.remove(family::Family, label_values)

Remove the collector corresponding to label_values. Effectively this resets the collector since Prometheus.labels will recreate the collector when called with the same label names.

Refer to Prometheus.labels for how to specify label_values.

Note

This method invalidates cached collectors for the label names.

source
Prometheus.clearMethod
Prometheus.clear(family::Family)

Remove all collectors in the family. Effectively this resets the collectors since Prometheus.labels will recreate them when needed.

Note

This method invalidates all cached collectors.

source

Registries

Default registry

Exposition

Prometheus support

Prometheus.exposeFunction
Prometheus.expose(file::String, reg::CollectorRegistry = DEFAULT_REGISTRY)

Export all metrics in reg by writing them to the file file.

source
expose(io::IO, reg::CollectorRegistry = DEFAULT_REGISTRY)

Export all metrics in reg by writing them to the I/O stream io.

source
expose(http::HTTP.Stream, reg::CollectorRegistry = DEFAULT_REGISTRY; kwargs...)

Export all metrics in reg by writing them to the HTTP stream http.

The caller is responsible for checking e.g. the HTTP method and URI target. For HEAD requests this method do not write a body, however.

source