Observability

Nagare instruments itself with System.Diagnostics.Activity and System.Diagnostics.Metrics — the same primitives ASP.NET Core and EF Core use. Any OpenTelemetry collector picks them up.

There is no Nagare.OpenTelemetry package, and there isn't going to be one. The ActivitySource and Meter are part of the core library so a single .AddSource("Nagare") and .AddMeter("Nagare") is enough — same pattern as Marten, Wolverine, and MassTransit.

Tracing setup

Register Nagare's activity source and meter with your OpenTelemetry configuration:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddSource("Nagare")
        .AddOtlpExporter())
    .WithMetrics(metrics => metrics
        .AddMeter("Nagare")
        .AddOtlpExporter());

That captures every span and counter the framework emits. The source name is also exposed as NagareActivity.SourceName if you prefer not to hard-code the string.

Traced operations

Nagare creates spans for six operations:

Span name	Kind	When
nagare.aggregate.ask	internal	A command is sent to an aggregate
nagare.eventstore.append	producer	Events are written to the store
nagare.eventstore.read	internal	Events are read from the store
nagare.subscription.handle	consumer	A subscription processes an event
nagare.subscription.checkpoint	internal	A subscription saves its position
nagare.outbox.dispatch	consumer	The outbox runner delivers a side-effect

A typical command flow produces three nested spans: aggregate.ask wraps eventstore.read (loading the aggregate) and eventstore.append (writing new events). Subscription processing produces subscription.handle with periodic subscription.checkpoint spans. Outbox delivery produces outbox.dispatch per row, linked back to the original eventstore.append via the stored traceparent.

Tags

Each span carries attributes that let you filter and group traces:

Tag	Value
nagare.aggregate.id	The aggregate instance ID
nagare.aggregate.type	The aggregate's type name
nagare.event.type	The event stream's type name
nagare.event.count	Number of events in this operation
nagare.command.type	The command's type name
nagare.subscription.id	The subscription's identifier
nagare.position	Global event store position
nagare.version	Aggregate version number
nagare.outbox.dispatch_id	Stable id of the outbox row being delivered
nagare.outbox.target	Logical name of the destination (handler/sink)
nagare.outbox.attempt	Attempt counter — 1 on first try
nagare.outbox.outcome	dispatched, transient, or dead
nagare.outbox.source_process	Process aggregate that produced the dispatch, if any
nagare.replay	true when the handler was driven by a catch-up replay

You can find every command handled by a given aggregate, trace from the HTTP request through the command to the events it produced, and follow those events through subscriptions and outbox deliveries.

Snapshot write failures

Snapshot writes happen after the events are committed, so a failed snapshot cannot fail the command — rolling back would make the caller retry and double-apply events that are already durable. Instead, Nagare records a nagare.snapshot.write_failed event on the current span and leaves the last-snapshot marker unchanged, so the next command re-attempts the snapshot. Snapshots are only a read-time optimisation; the events are the source of truth.

The event carries these tags:

Tag	Value
aggregate.id	The aggregate instance ID
snapshot.version	The version the snapshot was being written at
exception.type	Full name of the exception thrown by the snapshot store
exception.message	The exception message

This is the only observability surface for snapshot failures — Nagare deliberately takes no ILogger dependency, so there is no log entry. The event appears on the span that was current when the command was persisted (usually nagare.aggregate.ask), whenever an ActivitySource("Nagare") listener is active. Alert on it like any other span event.

One exception to the swallow-and-continue rule: cancellation. An OperationCanceledException from the snapshot write propagates to the caller — a cancelled command must not report success.

Metrics

Counters and a histogram are exposed under the same Nagare meter:

Instrument	Type	Description
nagare.outbox.dispatched	counter	Rows successfully dispatched
nagare.outbox.dead	counter	Rows that exhausted retries and were marked Dead
nagare.outbox.attempts	counter	Dispatch attempts, tagged by outcome
nagare.outbox.dispatch.duration_ms	histogram	Per-row dispatch latency
nagare.outbox.retention.deleted	counter	Rows removed by the retention sweep

Tag the dashboards by nagare.outbox.target to break the totals down per sink.

Linking subscription spans to writes

Events written today might be processed by a subscription seconds later — or replayed by a projection rebuild years later. Naive parent-child propagation breaks the second case: the original write trace is long gone from the backend, leaving handler spans pointing at nothing.

Because of that, handler spans are deliberately not parented to the writing span. This matches the OpenTelemetry messaging convention, stays deterministic across replays, and degrades gracefully when the writer trace has been sampled out or aged out of the backend.

The write side is captured at the append site: the framework reads Activity.Current when Append is called, formats a W3C traceparent, and stores it in event metadata. (The writing process must have a listener active for the Nagare source at that moment; if no Activity is current, no traceparent is stored.) That stored traceparent is what powers outbox dispatch linking — each dispatch span attaches to the producing span across the async hop.

Pipeline handler spans are emitted as stream.handle on the Nagare.Streams source, tagged with nagare.stream_id, nagare.position, and nagare.event_type. They start as their own roots; to correlate a handler span back to the original write, follow the traceparent stored in the event's metadata.

For most users, the link default is the right call. Turn parent-child on when you specifically want the demo-friendly view and your write traces are reliably retained.

Event metadata

Beyond tracing spans, you can attach metadata to individual events. This metadata is persisted in the event store and available everywhere the event is read.

var metadata = new EventMetadata(
    CorrelationId: requestId,
    CausationId: $"http:{Request.Path}",
    ActorId: currentUser.Id,
    Headers: new Dictionary<string, string> { ["tenant"] = currentUser.TenantId },
    Timestamp: DateTimeOffset.UtcNow);

await aggregate.Ask(new BorrowBook("user-42"), metadata);

In projections, the metadata is available on the envelope:

public async Task Handle(EventEnvelope<BookEvent> envelope)
{
    var actorId = envelope.Metadata?.ActorId;
    var correlationId = envelope.Metadata?.CorrelationId;
    var tenant = envelope.Metadata?.Headers?.GetValueOrDefault("tenant");
    // ...
}

What to put in metadata

Field	Purpose	Example
CorrelationId	Trace a chain of events back to the original request	HTTP request ID
CausationId	Identify what caused this event	The command or event that triggered it
ActorId	Stable id of whoever asked. Pairs with CommandSource.	user-42 for HTTP, a job name for the scheduler, a migration tag for system tasks. Prefer internal ids over PII — events are immutable.
CommandType / CommandSource	Auto-attribution	Filled by Aggregate.Ask and ProcessGrain; you usually don't set these.
Headers	Free-form IReadOnlyDictionary<string, string> for ride-along context	Tenant id, branch id, feature-flag bucket. Surfaces in dashboards and projections; isn't interpreted by the framework.
Timestamp	Custom timestamp	Override the store's default timestamp
TraceParent	W3C trace context captured at append time	Auto-populated; do not set by hand
TraceState	Vendor-specific trace propagation companion to TraceParent	Auto-populated; do not set by hand

TraceParent and TraceState are written by the framework at append time when an Activity is in scope — you don't set them. They drive the linking decision described above. Custom IEventMetadata types are passed through untouched, so spans-linking only works when you use the built-in EventMetadata record.

A middleware is a good place to attach metadata automatically:

public class CorrelationMiddleware(IHttpContextAccessor http) : ICommandMiddleware
{
    public async Task<IReply> InvokeAsync(AskContext context, AskDelegate next)
    {
        var requestId = http.HttpContext?.TraceIdentifier;
        var actorId = http.HttpContext?.User.FindFirst("sub")?.Value;

        var enriched = context with
        {
            Metadata = new EventMetadata(
                CorrelationId: requestId,
                ActorId: actorId)
        };

        return await next(enriched);
    }
}

Register it once and every command carries correlation data.

Health checks

Nagare registers three health checks that report whether the system is ready to serve traffic.

Event store readiness

EventStoreReadyHealthCheck reports healthy once the event store's database table has been created and verified. It reports unhealthy during startup while the initialization service runs CREATE TABLE IF NOT EXISTS.

Subscription readiness

SubscriptionsReadyHealthCheck tracks each subscription individually. It reports healthy only when every registered subscription has completed its initial catch-up (replayed historical events up to the current position). During startup, it lists which subscriptions are still initializing.

This is useful for Kubernetes readiness probes. A service shouldn't receive traffic until its projections have caught up. Otherwise, queries against read models return stale or empty results.

Repository storage readiness

RepositoryStorageReadyHealthCheck reports healthy once all document store tables have been created. Like the event store check, it transitions from unhealthy to healthy during startup.

Using health checks

The health checks are registered automatically when you add event stores, subscriptions, or repository stores. Wire them into ASP.NET Core's health check endpoint:

app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = _ => true
});

In Kubernetes, point your readiness probe at this endpoint:

readinessProbe:
  httpGet:
    path: /health/ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10

The service stays out of the load balancer until the event store is initialized, all subscriptions have caught up, and all document store tables exist.

Connecting the pieces

A production observability setup ties these together:

1. Tracing shows you what happened: which command was issued, what events it produced, how long each step took
2. Metadata shows you why: who issued the command, what request triggered it, what earlier event caused it
3. Health checks show you readiness: is the system caught up and safe to serve traffic

The tracing spans and metadata flow into your existing observability stack (Datadog, Jaeger, Grafana Tempo, Azure Monitor). The health checks integrate with your existing orchestrator. Nagare doesn't impose its own monitoring layer. It fits into whatever you already run.