Telemetry & Observability

Every agent action produces OpenTelemetry spans — LLM calls, tool invocations, subagent delegations, chain/graph steps, retrieval. Spans land in an in-memory exporter for the live view, a local SQLite database for historical query, and optionally an external OTLP endpoint.

Flow

sequenceDiagram participant Agent participant SDK as OTel SDK (OpenLLMetry) participant Mem as InMemorySpanExporter participant DB as TelemetryPersistenceService participant Ext as OTLP HTTP exporter Note over Agent, Ext: Init SDK->>SDK: setupInstrumentation()<br/>appName: "codebuddy-agent" SDK->>Mem: Attach + SimpleSpanProcessor SDK->>DB: Open SQLite, create tables, prune old Note over Agent, Ext: Runtime Agent->>SDK: LLM / tool / subagent SDK->>SDK: Auto-instrument LangChain + LangGraph SDK->>Mem: Completed span (live view) SDK->>DB: Persist (span_id, trace_id, attrs, events) opt OTLP enabled SDK->>Ext: HTTP batch export<br/>(DNS-pinned to resolved IP) end

What gets traced

OpenLLMetry auto-instruments LangChain + LangGraph:

LLM calls — model, provider, token counts (input / output / cache_read / cache_creation), latency, streaming status
Tool invocations — tool name, input, output, duration
Subagent delegations — subagent type, description, result
Chain / graph steps — LangGraph node transitions, state changes
Retrieval — vector search queries, result counts

Each span carries: span_id, trace_id, parent_id, name, kind, start/end (ns), status, attributes (JSON), events (JSON).

Persistence

SQLite at ~/.codebuddy/telemetry/traces.db via sql.js (WASM).

CREATE TABLE spans (
    span_id       TEXT PRIMARY KEY,
    trace_id      TEXT NOT NULL,
    parent_id     TEXT,
    name          TEXT NOT NULL,
    kind          INTEGER,
    start_time_s  INTEGER,
    start_time_ns INTEGER,
    end_time_s    INTEGER,
    end_time_ns   INTEGER,
    status_code   INTEGER,
    status_message TEXT,
    attributes    TEXT,   -- JSON
    events        TEXT,   -- JSON
    links         TEXT,   -- JSON
    session_id    TEXT,
    created_at    TEXT
);

Indexed on trace_id, created_at, session_id, name.

Batched writes: buffered → flushed every 5 s or at 50 spans, whichever first. Transactional INSERT OR IGNORE.

Auto-pruning: spans older than retentionDays (default 7) removed on startup.

Query API — `ObservabilityService` facade

Method	Source	Returns
`getTraces()`	In-memory exporter	Live spans from current session
`getPersistedTraces(days, limit)`	SQLite	Historical spans with computed duration
`getSessions()`	SQLite	Distinct session IDs + span counts + date ranges
`getMetrics()`	PerformanceProfiler	Runtime performance data
`getRecentLogs()`	Logger	Last 1000 log entries from circular buffer

OTLP export

Set an endpoint:

{ "codebuddy.telemetry.otlpEndpoint": "https://your-langfuse.example/api/public/otel" }

Or the dedicated Langfuse settings for the vendor-specific auth flow — see the code base for the exact key names (codebuddy.telemetry.langfuse.*).

DNS-rebinding pinning: OTLP exports use validateAndPinOutboundUrl (landed 2026-07-10) — the resolved IP is pinned to the socket connect via a custom dns.LookupFunction. Closes the check-then-connect window where a malicious resolver could flip a public answer to a private IP.

Compatible platforms:

Platform	Endpoint
Langfuse	`https://<host>/api/public/otel`
LangSmith	`https://api.smith.langchain.com`
Jaeger (OTLP)	`http://localhost:4318`
Grafana Tempo	Your Tempo OTLP HTTP endpoint

Any OTLP HTTP endpoint works.

Scores & user feedback (Langfuse)

Beyond spans, CodeBuddy writes scores to Langfuse over a separate REST path (/api/public/ingestion) — scores don’t flow through OTLP. Batched, fail-open, and DNS-pinned via validateAndPinOutboundUrl. Recorded scores include:

User feedback — thumbs up/down on an answer from the chat UI.
Diff acceptance — whether proposed changes were applied or rejected.
Test outcomes and other post-session derived metrics.
LLM-as-judge — an optional model-scored quality pass.

Enable with the Langfuse settings below; the secret key is stored in the OS keychain via CodeBuddy: Set Langfuse Secret Key, not in settings.json.

Anonymous user id (opt-in)

codebuddy.telemetry.anonymousUserId (default false) attaches a non-reversible, per-install id (sha256 of a local random nonce) to traces so you can group a session’s activity without identifying the user. Opt-in only.

Structured logging

Every log event carries the current OTel traceId so logs + traces correlate.

interface ILogEvent {
  timestamp: string;
  level: string;
  module: string;
  message: string;
  data: unknown;
  sessionId: string;
  traceId: string;  // matches the active OTel trace
}

Destinations:

VS Code Output channel (“CodeBuddy”)
Log files — .codebuddy/logs/codebuddy-*.log (structured JSON, one event per line)
Circular buffer — last 1000 entries, accessible via getRecentLogs()
Telemetry — optionally forwarded to the persistence service

Rotation & retention. The file sink is a buffered async write stream (not a per-line appendFileSync), rolls at 5 MB, and prunes on three axes: age (7 days) → count (10 files) → total size (50 MB). An environment gate keeps shipped builds from ever creating .codebuddy/logs (file logging is source/dev only, or when codebuddy.debug.fileLogging is on); the sink fails open on stream error and drains on deactivate().

Settings

Setting	Default	Purpose
`codebuddy.telemetry.persistTraces`	`true`	Persist traces to SQLite across sessions
`codebuddy.telemetry.retentionDays`	`7`	Auto-prune after N days (1–90)
`codebuddy.telemetry.otlpEndpoint`	`""`	Generic OTLP HTTP endpoint (empty = disabled)
`codebuddy.telemetry.langfuse.enabled`	`false`	Enable Langfuse observability + scores
`codebuddy.telemetry.langfuse.publicKey`	`""`	Langfuse public key (`pk-lf-…`)
`codebuddy.telemetry.langfuse.baseUrl`	`"https://cloud.langfuse.com"`	Langfuse host (cloud or self-hosted)
`codebuddy.telemetry.langfuse.allowInsecure`	`false`	Allow non-HTTPS Langfuse (self-hosted only)
`codebuddy.telemetry.anonymousUserId`	`false`	Attach an anonymous per-install id (opt-in)

The Langfuse secret key is set via CodeBuddy: Set Langfuse Secret Key (kept in the OS keychain), not as a setting.

Dependencies

Package	Version	Role
`@opentelemetry/api`	^1.9.0	OTel API
`@opentelemetry/sdk-trace-base`	^2.5.1	Trace SDK
`@opentelemetry/exporter-metrics-otlp-http`	^0.212.0	OTLP HTTP export
`@traceloop/node-server-sdk`	^0.22.7	Auto-instrument LangChain/LangGraph

Security — SSRF + DNS-rebinding gates that OTLP export inherits
Cost tracking — how token counts on spans get priced