--- title: "Observability and tracing" description: "Export OpenTelemetry traces and listen to correlated Farm runtime events in development and production." canonical_url: "https://mcp.farmjs.dev/docs/observability" markdown_url: "https://mcp.farmjs.dev/docs/observability.md" last_updated: "2026-10-20" x_farming_labs_generated_preamble: true --- # Observability and tracing URL: /docs/observability LLM index: /llms.txt Description: Export OpenTelemetry traces and listen to correlated Farm runtime events in development and production. # Observability and tracing Farm has three complementary observability layers: - OpenTelemetry traces show the request timeline and export to any OTLP-compatible backend. - OpenTelemetry metrics cover Farm's own cache, PPR, and streaming outcomes. - Farm events expose detailed framework lifecycle data for logs, alerts, and custom integrations. The same lifecycle events are attached to the active OpenTelemetry span, and delivered events include `traceId`, `spanId`, and `traceSampled` for correlation. ## OpenTelemetry quick start Install Farm's optional Node SDK setup package: ```bash pnpm add @farm.js/otel ``` Create one instrumentation file at `src/instrumentation.ts` or `instrumentation.ts`: ```ts import type { FarmInstrumentationContext } from "@farm.js/core/instrumentation"; export async function register(context: FarmInstrumentationContext) { if (context.runtime !== "nodejs") return; const { registerOTel } = await import("@farm.js/otel"); return registerOTel({ serviceName: "storefront", serviceVersion: process.env.APP_VERSION, }); } ``` Enable Farm spans in `farm.config.ts`: ```ts import { defineConfig } from "@farm.js/core"; export default defineConfig({ observability: { tracing: { attributes: { "deployment.environment": process.env.NODE_ENV ?? "development", }, }, }, }); ``` `@farm.js/otel` uses the OTLP HTTP trace exporter by default. Configure the destination with standard OpenTelemetry environment variables: ```bash OTEL_EXPORTER_OTLP_ENDPOINT=https://otel-collector.example.com OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer%20TOKEN" ``` Use `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` when traces need a different endpoint. You can also pass a custom `traceExporter`, `spanProcessors`, sampler, resource, instrumentation list, or `metricReaders` to `registerOTel`. Both pieces are intentional: `instrumentation.ts` starts an SDK/exporter, while `observability.tracing` tells Farm to create framework spans. Without an SDK, the OpenTelemetry API remains a no-op. ## What Farm traces | Span | Meaning | | ----------------------------------------- | ----------------------------------------------------------------------------- | | `GET /products/:id` | Server request span with method, route, path, status, and propagated context. | | `farm.render /products/:id` | Completed page render. | | `farm.middleware auth` | Completed middleware execution. | | `farm.api POST /api/products` | Completed API route execution. | | `farm.integration stripe.checkout.create` | Completed integration operation when that event is emitted. | | `farm.storage query` | Completed integration database operation when that event is emitted. | | `farm.ppr.refresh /dashboard` | Completed PPR refresh. | | `farm.plugin plugin.hook` | Completed plugin hook when that event is emitted. | Incoming W3C `traceparent` headers are extracted automatically. A matched Farm route renames the request span from the raw pathname to the route pattern and sets `http.route`. Status codes of 500 or higher and thrown errors mark spans as errors. To wrap application work in a typed Farm-aware child span: ```ts import { runWithFarmSpan } from "@farm.js/core/observability"; const result = await runWithFarmSpan( "catalog.recommendations", () => loadRecommendations(productId), { kind: "integration", attributes: { "product.id": productId }, }, ); ``` Use `getFarmTraceContext()` when a log or provider call needs the current trace and span IDs. ## What Farm measures `@farm.js/otel` also subscribes to Farm's event bus and records the framework's own cache, PPR, streaming, and middleware outcomes. Auto-instrumentation only sees generic HTTP, filesystem, and database work, so this is the layer that makes cache hit rate, tag invalidation, and shell TTFB visible. | Instrument | Kind | Attributes | | ----------------------------------- | --------------- | --------------------------------------------------------------------------------------------------- | | `farm.cache.lookups` | Counter | `farm.cache.result` (`hit`, `miss`, `stale`, `dedupe`, `bypass`), `farm.cache.stale` on a stale hit | | `farm.cache.invalidations` | Counter | `farm.cache.operation` (`revalidateTag`, `updateTag`, `revalidatePath`, `invalidated`) | | `farm.cache.invalidated_entries` | Counter | `farm.cache.operation`, incremented by the number of entries the operation removed | | `farm.ppr.shell.lookups` | Counter | `farm.ppr.result` (`hit`, `miss`, `cached`, `bypass`, `invalidated`) | | `farm.ppr.refresh.duration` | Histogram, `ms` | none | | `farm.render.stream.shell.duration` | Histogram, `ms` | none, this is shell TTFB | | `farm.middleware.short_circuits` | Counter | `farm.route`, the configured matcher rather than the request path, and `farm.http.status_class` | Metric attributes are deliberately bounded. A cache `key` embeds the serialized arguments of the cached call, a tag is authored at runtime, and the `route` on cache, PPR, and streaming events is the request pathname rather than a route pattern, so none of them becomes a metric attribute. That detail is recorded as an event on the active span instead, where per-request cardinality is free and a stale serve stays attributable to the request that caused it. When `observability.tracing` is enabled Farm already writes those span events itself, so `@farm.js/otel` does not add a second copy. Metrics need a metric reader, and Farm registers one only when you ask for it. Name an exporter with `OTEL_METRICS_EXPORTER`, or pass `metricReaders` to `registerOTel`: ```bash OTEL_METRICS_EXPORTER=otlp # otlp, console, prometheus, or none ``` Without it there is no meter provider, the instruments are no-ops, and the span events still work. That opt-in is deliberate: left to its own defaults the Node SDK exports metrics to `localhost:4318` whether or not an app configured metrics, and with nothing listening there every recorded metric turns graceful shutdown into the exporter's retry window, around eight seconds. Traces are unaffected and keep using the OTLP trace endpoint. Pass `farmEvents` to narrow or disable the mapping: ```ts return registerOTel({ serviceName: "storefront", // Leave Farm's event bus unsubscribed. farmEvents: false, // Or keep one half of it. // farmEvents: { metrics: true, spanEvents: false }, }); ``` An app that starts its own SDK can map the bus without `registerOTel`. Call it after the SDK has started, because an instrument created before a meter provider is registered stays a no-op, and dispose it on shutdown so a development restart does not stack subscribers. ```ts import { recordFarmEvents } from "@farm.js/otel"; const disposeFarmEvents = await recordFarmEvents(); ``` ## Instrumentation lifecycle Farm calls `register(context)` once before the development server or production request runtime starts. The context contains: - `mode`: `development`, `production`, or `test` - `runtime`: `nodejs`, `edge`, or `bun` - `root`: the runtime working directory `register` may return a cleanup function or an object with `shutdown()`. An exported `shutdown()` function is also supported. Farm runs cleanup during development server close and production graceful shutdown, after active requests drain, so batched spans are flushed before the process exits. Keep Node-only SDK imports inside the `runtime === "nodejs"` branch. Edge and Bun deployments can initialize a runtime-compatible OpenTelemetry SDK in their own branch without changing Farm's tracing API. ## Subscribe to events **farm.config.ts** ```ts import { onFarmEvent } from "@farm.js/core/observability"; onFarmEvent((event) => { if (event.level === "error") { console.error("[farm]", event.type, event); } }); ``` ## Event families | Family | Examples | | -------------------- | -------------------------------------------------------------------------------- | | Request | request.start, request.complete, request.error | | Server | server.start, server.ready, server.shutdown | | Routing | route.discovered, route.matched, route.notFound, route.redirect | | Rendering | render.start, render.complete, render.stream.shellReady, render.error | | Cache | cache.hit, cache.miss, cache.set, cache.revalidateTag | | PPR | ppr.shell.hit, ppr.shell.cached, ppr.shell.invalidated | | Middleware | middleware.start, middleware.complete, middleware.shortCircuit, middleware.error | | Integrations | integration.ready, integration.api.call.start, integration.webhook.verified | | MCP | mcp.tool.complete (registered callback outcome, not an agent run) | | Agents | agent.read (Markdown mirror, llms.txt, or API catalog read) | | Integration database | storage.query.start, storage.schema.ready | | Build | build.start, routes.generated, types.generated, manifest.generated | The `storage.query.*` and `storage.schema.*` event names are compatibility identifiers for integration database activity. They do not describe `getStorage()` KV operations. ## Configure in farm.config.ts Use config-level observability when an app should log or forward events from startup. ```ts import { defineConfig } from "@farm.js/core"; export default defineConfig({ observability: { logs: true, tracing: true, events: ["cache.hit", "cache.miss", "ppr.shell.cached"], onEvent(event) { if (event.type === "cache.miss") { console.log("Cache miss", event.key, event.reason); } }, }, }); ``` `events` is optional. Leave it out to receive every emitted event. Filtering event delivery does not disable span creation, the events recorded on active spans, or the `@farm.js/otel` metrics; use `farmEvents: false` for those. ## Runtime subscription Use `onFarmEvent` for tests, local debugging, or integration packages that want to register listeners without owning app config. ```ts import { onFarmEvent } from "@farm.js/core/observability"; const unsubscribe = onFarmEvent((event) => { if (event.type === "integration.webhook.failed") { console.error(event.integration, event.reason); } }); unsubscribe(); ``` ## Useful event types | Area | Events | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Cache | `cache.hit`, `cache.miss`, `cache.set`, `cache.stale`, `cache.bypass`, `cache.invalidated`, `cache.revalidatePath`, `cache.revalidateTag`, `cache.updateTag`, `cache.error` | | PPR | `ppr.shell.hit`, `ppr.shell.miss`, `ppr.shell.cached`, `ppr.shell.bypass`, `ppr.shell.invalidated`, `ppr.suspense.holeDetected`, `ppr.refresh.start`, `ppr.refresh.complete`, `ppr.refresh.error` | | API | `api.request.start`, `api.request.complete`, `api.validation.failed`, `api.error` | | MCP | `mcp.tool.complete` — configured server, transport route, tool name, outcome and duration; no tool payloads | | Agents | `agent.read` — surface (`markdown`, `llms`, `llms-full`, `api-catalog`), path, Markdown view (`outline`, `section`, `parts`, `actions`), status, and the agent the User-Agent declares | | Integrations | `integration.registered`, `integration.config.validated`, `integration.ready`, `integration.disposed`, `integration.api.call.start`, `integration.api.call.complete`, `integration.webhook.verified`, `integration.webhook.failed` | | Middleware | `middleware.start`, `middleware.complete`, `middleware.shortCircuit`, `middleware.error` | | Integration database | `storage.query.start`, `storage.query.complete`, `storage.query.error`, `storage.schema.ready`, `storage.schema.error` | Middleware events include the matched middleware route, request pathname, middleware file/config name, duration for completes, status for short circuits, and the thrown error for failures. MCP completion events distinguish tool errors from successful HTTP transport responses. They cover registered callbacks only, excluding authorization and SDK rejections before callback execution. See [MCP tool observability](/docs/plugins/mcp#observe-tool-executions) for payload and coverage details. ## Agent reads `agent.read` fires each time an agent surface answers: a Markdown mirror or `page.md` (including `?outline`, `?section`, `?max_tokens` parts, and `?actions`), `/llms.txt` or `/llms-full.txt`, and `/.well-known/api-catalog`. It carries the path served, never the query string, the status, and the agent the request's User-Agent declares: ```ts onFarmEvent((event) => { if (event.type !== "agent.read") return; // { surface: "markdown", route: "/pricing.md", view: "parts", status: 200, // agent: { family: "openai", category: "agent", identity: "self-reported" } } metrics.increment("agent_reads", { surface: event.surface, family: event.agent.family }); }); ``` `agent.category` is `agent` for user-triggered fetchers such as ChatGPT-User, Claude-User, and Perplexity-User, and `crawler` for indexing and training crawlers such as GPTBot, ClaudeBot, and PerplexityBot. The identity is `self-reported`: any client can send these tokens, so use it to understand traffic, never to grant access. Farm's own reads, such as llms-full.txt reading each page, are not reported. `classifyFarmAgent(userAgent)` from `@farm.js/core` returns the same identity for any request, for example in middleware. ## Middleware event payloads Middleware events are emitted for both `farm.config.ts` middleware entries and `src/app/**/middleware.ts` files in development and production. | Event | Payload | | ------------------------- | ----------------------------------------- | | `middleware.start` | `route`, `pathname`, `name` | | `middleware.complete` | `route`, `pathname`, `name`, `durationMs` | | `middleware.shortCircuit` | `route`, `pathname`, `name`, `status` | | `middleware.error` | `route`, `pathname`, `name`, `error` | ```ts import { defineConfig } from "@farm.js/core"; export default defineConfig({ observability: { onEvent(event) { if (event.type === "middleware.shortCircuit") { console.log(event.pathname, event.status); } }, }, }); ``` ## Production notes - Keep `@farm.js/otel` in `dependencies`, not `devDependencies`, so it is available in the deployed server bundle. - Set `OTEL_SERVICE_NAME` or pass `serviceName` explicitly; use deployment/version resource attributes for release comparisons. - Farm's production lifecycle waits for active requests and then shuts down instrumentation, allowing the batch processor to flush. - `forceFlushOTel()` flushes spans and, when a metrics pipeline is configured, recorded metrics. Call it from `after()` on hosts that freeze the process between requests, where the periodic metric reader never gets to export. - Filter high-volume events before shipping them to a log drain. - Use the emitted trace and span IDs to correlate Farm events with application logs. - Treat event payloads as operational metadata; do not put secrets in event fields. - Watch `api.validation.failed`, `integration.webhook.failed`, `cache.error`, and `ppr.refresh.error` in production. ## Sitemap See the full [sitemap](/sitemap.md) for all pages. Docs-scoped sitemap: [/docs/sitemap.md](/docs/sitemap.md). Well-known sitemap: [/.well-known/sitemap.md](/.well-known/sitemap.md).