This guide explains how your Quarkus application can utilize OpenTelemetry to provide distributed tracing for interactive web applications.

Prerequisites

To complete this guide, you need:

  • less than 15 minutes

  • an IDE

  • JDK 11+ installed with JAVA_HOME configured appropriately

  • Apache Maven 3.8.1

  • Docker

Architecture

In this guide, we create a straightforward REST application to demonstrate distributed tracing.

Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can skip right to the completed example.

Clone the Git repository: git clone https://github.com/quarkusio/quarkus-quickstarts.git, or download an archive.

The solution is located in the opentelemetry-quickstart directory.

Creating the Maven project

First, we need a new project. Create a new project with the following command:

mvn io.quarkus:quarkus-maven-plugin:2.0.2.Final:create \
    -DprojectGroupId=org.acme \
    -DprojectArtifactId=opentelemetry-quickstart \
    -DclassName="org.acme.opentelemetry.TracedResource" \
    -Dpath="/hello" \
    -Dextensions="resteasy,quarkus-opentelemetry-exporter-otlp"
cd opentelemetry-quickstart

This command generates the Maven project with a REST endpoint and imports the quarkus-opentelemetry-exporter-otlp extension, which includes the OpenTelemetry support, and a gRPC span exporter for OTLP.

If you already have your Quarkus project configured, you can add the quarkus-opentelemetry-exporter-otlp extension to your project by running the following command in your project base directory:

./mvnw quarkus:add-extension -Dextensions="opentelemetry-otlp-exporter"

This will add the following to your pom.xml:

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-opentelemetry-exporter-otlp</artifactId>
</dependency>

Examine the JAX-RS resource

Open the src/main/java/org/acme/opentelemetry/TracedResource.java file and see the following content:

package org.acme.opentelemetry;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import org.jboss.logging.Logger;

@Path("/hello")
public class TracedResource {

    private static final Logger LOG = Logger.getLogger(TracedResource.class);

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        LOG.info("hello");
        return "hello";
    }
}

Notice that there is no tracing specific code included in the application. By default, requests sent to this endpoint will be traced without any required code changes.

Create the configuration

There are two ways to configure the OTLP gRPC Exporter within the application.

The first approach is by providing the properties within the src/main/resources/application.properties file:

quarkus.application.name=myservice (1)
quarkus.opentelemetry.enabled=true (2)
quarkus.opentelemetry.tracer.exporter.otlp.endpoint=http://localhost:55680 (3)
1 All spans created from the application will include an OpenTelemetry Resource indicating the span was created by the myservice application. If not set, it will default to the artifact id.
2 Whether OpenTelemetry is enabled or not. The default is true, but shown here to indicate how it can be disabled
3 gRPC endpoint for sending spans.

Run the application

The first step is to configure and start the OpenTelemetry Collector to receive, process and export telemetry data to Jaeger that will display the captured traces.

Configure the OpenTelemetry Collector by creating an otel-collector-config.yaml file:

receivers:
  otlp:
    protocols:
      grpc:

exporters:
  logging:

  jaeger:
    endpoint: jaeger-all-in-one:14250
    insecure: true

processors:
  batch:

extensions:
  health_check:

service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger]

Start the OpenTelemetry Collector and Jaeger system via the following docker-compose file that you can launch via docker-compose run -d:

version: "2"
services:

  # Jaeger
  jaeger-all-in-one:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"
      - "14268"
      - "14250"
  # Collector
  otel-collector:
    image: otel/opentelemetry-collector:latest
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "13133:13133" # Health_check extension
      - "4317"        # OTLP gRPC receiver
      - "55680:55680" # OTLP gRPC receiver alternative port
    depends_on:
      - jaeger-all-in-one

Now we are ready to run our application. If using application.properties to configure the tracer:

./mvnw compile quarkus:dev

or if configuring the OTLP gRPC endpoint via JVM arguments:

./mvnw compile quarkus:dev -Djvm.args="-Dquarkus.opentelemetry.tracer.exporter.otlp.endpoint=http://localhost:55680"

With the OpenTelemetry Collector, Jaeger system and application running, you can make a request to the provided endpoint:

$ curl http://localhost:8080/hello
hello

Then visit the Jaeger UI to see the tracing information.

Hit CTRL+C to stop the application.

Additional instrumentation

Some Quarkus extensions will require additional code to ensure traces are propagated to subsequent execution. These sections will outline what is necessary to propagate traces across process boundaries.

The instrumentation documented in this section has been tested with Quarkus and works in both standard and native mode.

SmallRye Reactive Messaging - Kafka

When using the SmallRye Reactive Messaging extension for Kafka, we are able to propagate the span into the Kafka Record with:

Metadata.of(TracingMetadata.withPrevious(Context.current()));

The above creates a Metadata object we can add to the Message being produced, which retrieves the OpenTelemetry Context to extract the current span for propagation.

OpenTelemetry Configuration Reference

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

OpenTelemetry support. OpenTelemetry support is enabled by default.

boolean

true

Comma separated list of OpenTelemetry propagators which must be supported. Valid values are b3, b3multi, baggage, jaeger, ottrace, tracecontext, xray. Default value is traceContext,baggage

list of string

tracecontext,baggage

Support for tracing with OpenTelemetry. Support for tracing will be enabled if OpenTelemetry support is enabled and either this value is true, or this value is unset.

boolean

Comma separated list of resources that represents the entity that is producing telemetry. Valid values are beanstalk, ec2, ecs, eks, host, lambda, os, process, processruntime.

list of string

A comma separated list of name=value resource attributes that represents the entity producing telemetry (eg. service.name=authservice).

list of string

The sampler to use for tracing Valid values are off, on, ratio. Defaults to on.

string

on

double

If the sampler to use for tracing is parent based Valid values are true, false. Defaults to true.

boolean

true

Suppress non-application uris from trace collection. This will suppress tracing of /q endpoints. Providing a custom io.opentelemetry.sdk.trace.samplers.Sampler CDI Bean will ignore this setting. Suppressing non-application uris is enabled by default.

boolean

true

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

OTLP SpanExporter support. OTLP SpanExporter support is enabled by default.

boolean

true

The OTLP endpoint to connect to. The endpoint must start with either http:// or https://.

string

The maximum amount of time to wait for the collector to process exported spans before an exception is thrown. A value of 0 will disable the timeout: the exporter will continue waiting until either exported spans are processed, or the connection fails, or is closed for some other reason.

Duration

10S

About the Duration format

The format for durations uses the standard java.time.Duration format. You can learn more about it in the Duration#parse() javadoc.

You can also provide duration values starting with a number. In this case, if the value consists only of a number, the converter treats the value as seconds. Otherwise, PT is implicitly prepended to the value to obtain a standard java.time.Duration format.