Adding Custom Spans and Metrics ¶
Auto-instrumentation covers HTTP, database, and messaging calls out of the box. To trace your own business logic or track application-specific events, add custom spans and metrics using the OpenTelemetry SDK.
Prerequisites ¶
- Application running on Nais with auto-instrumentation enabled
- Access to your application's source code
1. Add custom spans ¶
Custom spans let you trace operations not covered by auto-instrumentation — business logic, batch jobs, or complex workflows.
Kotlin with @WithSpan ¶
The @WithSpan annotation is the simplest way to trace a method. The OpenTelemetry Java agent picks it up automatically.
Add the dependency:
dependencies {
implementation("io.opentelemetry.instrumentation:opentelemetry-instrumentation-annotations:2.27.0")
}<dependency>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-instrumentation-annotations</artifactId>
<version>2.27.0</version>
</dependency>Then annotate methods you want to trace:
import io.opentelemetry.instrumentation.annotations.WithSpan
import io.opentelemetry.instrumentation.annotations.SpanAttribute
class PaymentService {
@WithSpan("process-payment")
fun processPayment(
@SpanAttribute("payment.amount") amount: Long,
@SpanAttribute("payment.currency") currency: String
): PaymentResult {
// This method call becomes a span in your traces.
// @SpanAttribute adds the parameter values as span attributes.
validate(amount, currency)
return execute(amount, currency)
}
@WithSpan
private fun validate(amount: Long, currency: String) {
// Nested span — appears as a child of "process-payment"
}
}Starting an independent trace ¶
Sometimes you want a span that starts a new, independent trace instead of being nested under the current one — for example, a background job triggered by a request that should have its own trace.
Use inheritContext = false on the annotation (requires annotations library ≥ 2.14.0):
@WithSpan("background-sync", inheritContext = false)
fun syncToExternalSystem() {
// This span starts a new trace, not nested under the caller's trace.
}Kotlin suspend functions
@WithSpan(inheritContext = false) may not work correctly on Kotlin suspend functions.
Suspend functions are compiled into a state machine, and the OpenTelemetry Java agent may not
handle context propagation correctly for them.
If you need an independent trace from a suspend function, use the Tracer API with setNoParent() instead (see below).
Java / Kotlin with the Tracer API ¶
For more control (dynamic span names, adding events, setting status), use the Tracer API directly:
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.api.GlobalOpenTelemetry;
public class MyService {
private static final Tracer tracer =
GlobalOpenTelemetry.getTracer("my-app");
public void doWork() {
Span span = tracer.spanBuilder("custom-operation").startSpan();
try (var scope = span.makeCurrent()) {
// Your business logic here
span.addEvent("checkpoint-reached");
} catch (Exception e) {
span.recordException(e);
span.setStatus(io.opentelemetry.api.trace.StatusCode.ERROR);
throw e;
} finally {
span.end();
}
}
}To start an independent trace using the Tracer API, use setNoParent():
import io.opentelemetry.api.GlobalOpenTelemetry
val tracer = GlobalOpenTelemetry.getTracer("my-app")
fun syncToExternalSystem() {
val span = tracer.spanBuilder("background-sync")
.setNoParent()
.startSpan()
try {
span.makeCurrent().use {
// Your logic here — this span is a root span in a new trace.
}
} catch (e: Exception) {
span.recordException(e)
span.setStatus(io.opentelemetry.api.trace.StatusCode.ERROR)
throw e
} finally {
span.end()
}
}Node.js ¶
import { trace } from "@opentelemetry/api";
const tracer = trace.getTracer("my-app");
async function processOrder(orderId) {
return tracer.startActiveSpan("process-order", async (span) => {
try {
span.setAttribute("order.id", orderId);
const result = await executeOrder(orderId);
return result;
} catch (error) {
span.recordException(error);
span.setStatus({ code: 2 }); // ERROR
throw error;
} finally {
span.end();
}
});
}2. Add custom metrics ¶
Custom metrics track application-specific counters, gauges, or histograms. These are exported to Mimir and available in Grafana.
Kotlin / Java ¶
import io.opentelemetry.api.GlobalOpenTelemetry
import io.opentelemetry.api.metrics.LongCounter
object AppMetrics {
private val meter = GlobalOpenTelemetry.getMeter("my-app")
val ordersProcessed: LongCounter = meter
.counterBuilder("orders_processed_total")
.setDescription("Number of orders processed")
.build()
}
// Usage:
AppMetrics.ordersProcessed.add(1, Attributes.of(
AttributeKey.stringKey("status"), "completed"
))OTel Metrics API
Use GlobalOpenTelemetry.getMeter() (not GlobalMeterProvider.getMeter(), which was removed in OpenTelemetry Java 2.x).
Node.js ¶
import { metrics } from "@opentelemetry/api";
const meter = metrics.getMeter("my-app");
const ordersProcessed = meter.createCounter("orders_processed_total", {
description: "Number of orders processed",
});
// Usage:
ordersProcessed.add(1, { status: "completed" });3. Verify in APM ¶
- Deploy your application
- Open the Nais APM and find your service
- Check the Operations tab — your custom spans appear as operations
- In Grafana Explore, query your custom metrics with PromQL