“Free self-hosting is a game-changer.” That comment, upvoted into the stratosphere on r/LangChain hours after LangSmith ended its free tier, captures the mood of a maturing market. As of mid-2026, over 66 tools claim to solve AI observability — yet few were built for the messy, multi-step reality of production AI agents. This guide strips away the marketing and pits three heavyweights against each other: Langfuse (open-source darling, now part of ClickHouse), Weights & Biases Weave (the ML ops giant stretching into LLMs), and Datadog (the infrastructure behemoth adding an LLM lens). If you’re deploying agents to customers and can’t afford silent failures, the platform you choose is a foundation-level decision.
Traditional monitoring was born for request-response APIs. A 200 status code means everything worked. An agent doesn’t work that way. It might chain a dozen LLM calls, call a vector store, execute a tool, and then — after 47 steps — deliver a result that looks correct but hallucinated an entire contract clause. Logs won’t catch it. Error dashboards won’t surface the moment a retrieval step quietly returned bilingual policy docs that tripled token usage downstream. Cresta, an enterprise conversational AI platform, ran into exactly this. Their agents aren’t single model invocations; they’re intent detection, knowledge retrieval, tool execution, and safety validation stitched together. With Langfuse, Cresta organized every run into a trace tree where each node captured structured data — prompt/completion pairs, document counts with relevance scores, guardrail pass/fail outcomes. Debugging flipped from “something is slow” to “the response synthesis step used 3x the expected tokens because the retrieval returned bilingual documents.” That granularity separates tools that understand agent workflows from those that treat LLM calls as independent log lines. Research backs this up. Multi-agent observability work published in early 2026 showed that monitoring systems correlating semantic intent with system-level events catch failure patterns that log-level monitoring misses entirely. A single tool failure at step 2 can silently corrupt every subsequent step, and LLM-first tools that just log independent events leave teams manually reconstructing causality from raw JSON.
The Three Contenders at a Glance
| Langfuse | Weights & Biases (Weave) | Datadog LLM Observability | |
|---|---|---|---|
| License | MIT open source (core) | Proprietary (Weave OSS Apache 2.0, platform closed) | Proprietary |
| Deployment | Self-host or cloud; air-gapped ready | Managed (enterprise self-host) | SaaS only |
| Starting price | Free (self-host), $29/mo cloud | $50–$60/user/mo + usage | $31/host/mo + LLM add-on |
| Architecture | Agent-native, causal trace model, ClickHouse-backed | Agent session lifecycle, ML heritage | Infrastructure-first, APM at its core |
| Best for | Data sovereignty, cost control, open-source teams | Existing W&B ML shops | Enterprises already on Datadog stack |
| Now let’s see how these platforms hold up under the scrutiny of real production teams. | |||
| ## Langfuse: The Open-Source Workhorse That Outgrew Its Hobbyist Image | |||
| Langfuse launched in 2023 with an MIT license, and by mid-2026 it had crossed 28,900 GitHub stars — the largest community among open-source LLM observability tools. A pivotal moment came in January 2026 when ClickHouse acquired the project. Now Langfuse stores traces directly in ClickHouse, with a redesigned data model that writes every observation once to a single wide, immutable table. Initial table loads for large data volumes dropped from seconds to tens of milliseconds. | |||
| What that means for your team: No join storms, no read-time deduplication. You get sub-second query performance even as your agent generates tens of thousands of steps. The engineering benchmarks are public, a rare transparency move that build trust with infrastructure buyers. | |||
| Langfuse covers tracing, prompt management, LLM-as-judge evaluations, human annotation queues, and cost/latency monitoring. The self-hosted option is first-class, not an afterthought: you can run it air-gapped in your VPC with no vendor contact. | |||
| Pricing that makes CFOs smile | |||
| Langfuse’s metered cloud pricing (verified June 2026) starts at $29/month for the Core plan with 100k units and unlimited users. Overage costs $8 per 100k units, dropping to $6 once you exceed 50 million units per month. A team processing 1 million events per month would pay roughly $101/month — for unlimited users. Self-hosting is free. | |||
| For comparison, LangSmith at the same 1M events volume would cost $2,514/month for a single seat with 14-day data retention. That cost cliff is the most cited reason teams migrate, and it’s fueling Langfuse’s enterprise momentum. | |||
| Where Langfuse shines | |||
| SumUp, a fintech serving 4 million merchants across 35 markets, uses Langfuse for tracing and prompt management. The payoff: a 50% conversation deflection rate and 300,000 monthly requests blocked from reaching human agents, slashing external outsourcing costs by 30%. Intuit presented a real-world Langfuse case at ClickHouse Open House 2026, using it to track complex multi-service LLM pipelines. | |||
| SOC2 and ISO27001 compliance kick in with the Pro tier. HIPAA BAAs are available. Enterprise SSO via Okta adds $300/month. For regulated industries, Langfuse Pro or Enterprise hits the compliance checkbox without the eye-watering price tags of proprietary SaaS. | |||
| Rough edges to know | |||
The async streaming story still has gaps. In Python, the @observe decorator can lose context propagation in async generators, causing spans with zero duration and broken parent-child relationships. The Langfuse team acknowledges the limitation and is working on improvements, with some relief coming from OpenTelemetry span re-parenting in related projects. Until that’s fully baked, teams with heavy streaming workloads need to manually manage context or use the start_as_current_span() context manager. |
|||
| Also, Langfuse doesn’t yet offer a built-in gateway or simulation layer in the same product. For some advanced agent use cases, you’ll complement it with a separate gateway. The core plan lacks SOC2/ISO27001, so regulated industries must step up to Pro. Still, when a Reddit top comment reads “We self-host Langfuse and are pretty happy so far,” it’s a sign that the pain points are manageable for most teams. | |||
| ## Weights & Biases Weave: ML’s Heavyweight Enters the LLM Ring | |||
| W&B earned its reputation helping OpenAI train GPT-4. Weave is their LLM observability and evaluation platform that traces full agent conversation lifecycles — sessions, tool calls, and multi-turn context. If your organization already tracks experiments with W&B, Weave feels like a natural extension: same UI, same workflows, unified lineage from training data to production traces. | |||
| The platform auto-instruments supported agent harnesses like Claude Code and the OpenAI Agent SDK with zero code changes. It integrates with LiteLLM to wrap calls in versioned operations and track function-calling behavior. And in April 2026, Weave updated its OpenTelemetry integrations to follow the latest semantic conventions, with cost calculations now including cache token usage. | |||
| The good, the expensive, and the ML baggage | |||
| Pricing starts at $50–$60 per user per month, plus usage-based ingestion fees. At 10 users, you’re at $500/month before any trace volume. Scale to 30 seats, and the math gets painful — a reality reflected in community migration discussions where teams cite cost as a primary exit driver. One DevOps lead on r/LLMDevs noted, “W&B can become expensive at large scale; the per-user model scales poorly above 30 seats.” | |||
| The deeper issue for LLM-only teams: you’re paying for ML experiment tracking, hyperparameter tuning, model registry, and reports — features you may never touch. The dashboard is filled with ML primitives, and every time you navigate to your LLM traces, you pass through a world designed for researchers iterating on batch training jobs. Weave was bolted on in 2023, and despite improvements, it still feels like an add-on to many LLM developers. | |||
| Migrating off W&B when the cost crunch hits is painful. “Re-instrumenting Python services for OpenTelemetry was the part that always bit us,” wrote one lead in a migration retrospective. The SDK lock-in is real, which means the decision to adopt Weave should be treated as long-term and deeply intentional. | |||
| Where Weave wins | |||
| If your team does both classical ML training and LLM agent development, W&B’s unified platform is genuinely powerful. CoreWeave announced production agent evaluations in Weave at NVIDIA GTC 2026, connecting research iteration to live user feedback. The built-in playground for experimenting with prompts, models, and configurations is solid. And for small teams (fewer than 30 seats) already vested in W&B, Weave eliminates the cognitive overhead of learning a new tool. | |||
| ## Datadog LLM Observability: The Infrastructure Giant with an AI Lens | |||
| Datadog sat in the Leaders quadrant of Gartner’s 2026 Magic Quadrant for Observability for the sixth straight year, positioned highest in Ability to Execute. Its platform unifies IT operations, development, security, and business metrics — and now, with the LLM Observability add-on, traces every prompt and response, runs online evaluations over telemetry data, and uses its Bits AI suite to autonomously investigate alerts and surface root causes. | |||
| The MCP server gives agents structured access to observability data, while the Pup CLI brings much of the API surface into the terminal. Agent Skills package common AI engineering tasks into single commands. For enterprises already running Datadog for APM, logs, RUM, and security, adding LLM observability means everything lives in one pane — and that correlation is Datadog’s superpower. | |||
| The cost reality check | |||
| That superpower gets expensive fast. Datadog’s APM starts at $31 per host per month, plus $1.27 per million ingested spans and $2.55 per million indexed spans. The LLM Observability SKU meters at $8 per 10,000 monitored LLM requests. A single 16-microservice application can easily cost $174/day. One public cost breakdown from April 2026 showed a team paying over $10,000/month, with APM and span overages being the largest line item. | |||
| “Datadog is excellent infra observability that happens to support LLMs,” reads one r/learnmachinelearning comment, “but it’s not really LLM-native.” Another observation: Datadog treats AI outputs like any other microservice — a stream of signals where every metric carries equal weight. That’s fine for tracking latency and error rates, but evaluating response quality, hallucination scores, or whether a tool call fetched the right document inside an agent workflow? That’s not baked into the core data model; it’s layered on as an add-on. | |||
| There are also compatibility quirks. GitHub issues show that ddtrace can lose up to 80% of LLM calls when using Celery with gevent pools, and multiple users reported that recent dd-trace versions broke LLM API calls entirely. For a team debugging production agents at 2 a.m., that’s a crisis. | |||
| Where Datadog justifies its price tag | |||
| If your organization already runs Datadog and has budget, the Bits AI suite is genuinely compelling. Bits Investigation has performed hundreds of thousands of autonomous investigations, and Bits Code acts as an always-on developer companion that fixes issues without pulling engineers from feature work. In June 2026, Datadog acquired Adaptive ML, an RL ops startup, signaling serious investment in agentic LLM observability and post-training infrastructure. For large enterprises, having one vendor for infrastructure, security, and AI observability can simplify procurement and tool sprawl. | |||
| ## Head-to-Head: What Actually Matters in Production | |||
| Agent-native architecture | |||
| Langfuse and Weave are built to understand causal chains: a tool call at step 2 that fails can corrupt every subsequent step, and the platform surfaces that chain. Datadog treats each span as an independent event; you correlate them yourself. When 38 incidents all stem from one root pattern, automatic clustering turns a weekend firefight into a single click. Langfuse’s trace trees and Weave’s session lifecycle model deliver this; Datadog is still catching up on agent-native modeling. | |||
| Cost at real deployment scale | |||
| Scenario | Langfuse | W&B Weave | Datadog |
| --- | --- | --- | --- |
| Free tier | 50k units/mo | Limited seats/storage | Trial only |
| Entry paid | $29/mo (flat) | $50–$60/user/mo | $31/host/mo + LLM add-on |
| 1M events/month | ~$101/mo (unlimited users) | ~$2,500+/mo (10+ users) | $1,000+/mo (variable) |
| These numbers make the case bluntly. Langfuse’s usage-based model scales linearly; Datadog and W&B scale with seats or hosts in ways that surprise finance teams. The community phrase “cost cliff” exists for a reason. | |||
| Open source and data sovereignty | |||
| Langfuse is MIT-licensed, full self-hosting is free, and air-gapped deployments are a documented path. Arize Phoenix (Elastic License 2.0) is often mentioned as a lighter-weight alternative, but Langfuse’s ClickHouse-backed architecture supports heavier workloads. Datadog and W&B’s core platform are proprietary, with self-hosting only at enterprise tiers — and even then, not fully equivalent to their SaaS offerings. For European firms under GDPR, or any business in finance and healthcare, the ability to run observability inside your own VPC without vendor egress is table stakes. | |||
| Evaluation and feedback loops | |||
| All three platforms support LLM-as-judge evaluations, but the depth varies. Langfuse bakes prompt experiments, dataset tracking, and human annotation queues into the flow; some of these require the Pro tier when self-hosting. Weave links evaluations to versioned prompts and models, great for teams that treat prompts as artifacts. Datadog runs online evaluations over telemetry but lacks prompt management and systematic dataset experimentation. If you need a tight feedback loop from user feedback to prompt iteration, Langfuse or Weave fit more naturally. | |||
| Community and ecosystem | |||
| Langfuse’s open-source community dwarfs Weave’s LLM-specific following. When LangSmith ended its free tier, Reddit’s top reply was “We self-host Langfuse and are pretty happy so far.” That sentiment is backed by migration momentum. W&B’s LLM community is smaller, and much of its support comes from the broader ML ecosystem. Datadog’s strength is its enterprise support org, not organic developer community. | |||
| ## The Decision Framework | |||
| Pick Langfuse if: | |||
| - Open source and data sovereignty are non-negotiable | |||
| - You need predictable, per-unit pricing with unlimited users | |||
| - Your stack is framework-agnostic and you want the broadest integration coverage | |||
| - Self-hosting in air-gapped environments is required | |||
| Pick Weights & Biases Weave if: | |||
| - Your org already runs W&B for ML experiment tracking | |||
| - You want unified classical ML and LLM observability in one platform | |||
| - Auto-instrumentation for Claude Code or OpenAI Agent SDK saves you weeks | |||
| - Your team is small enough that per-user pricing doesn’t hurt | |||
| Pick Datadog if: | |||
| - Datadog is already your standard for infrastructure monitoring | |||
| - Correlating AI behavior with security, infrastructure, and RUM is critical | |||
| - Autonomous incident investigation through Bits AI fits your ops model | |||
| - Your observability budget comfortably absorbs five-figure monthly contracts | |||
| A hybrid reality | |||
| Many enterprises are landing on a combination: Langfuse for deep agent tracing and prompt management, paired with Datadog for infrastructure correlation. OpenTelemetry makes this swapping possible without rip-and-replace. An Elastic survey from early 2026 found 85% of organizations plan to enable LLM observability, yet only 8% have completed implementation. The market is open, and the tooling is still settling. | |||
| ## The Lock-In Elephant in the Room | |||
| Every observability platform wants to own your traces. Langfuse’s MIT license and first-class self-hosting make it the hardest to lock you in. Datadog and W&B tie you to their SDKs and data models; migrating off either means re-instrumenting codebases, which, as community post-mortems show, is always the painful part. OpenTelemetry continues to gain ground as a vendor-neutral layer, and platforms that embrace it natively — Langfuse and Datadog both now do, with Weave catching up — give you an exit path if costs balloon or requirements shift. | |||
| The 2026 LLM observability market is projected to hit $26.9 billion annually, growing at over 36% year-over-year. As agentic workflows become the default, the platform you choose today determines not just what you can see about your agents, but whether you can afford to keep seeing it at scale. | |||
| “Once an agent hits production you need to see what it did and score whether it was any good.” That line, from a DEV community LLM observability index, sums up the baseline. The question isn’t which platform checks the most feature boxes — it’s which one gives your team the clarity to answer that question before your customers do. |
Editorial Disclosure: This commercial analysis is compiled from global informational platforms and developer community discussions. Due to rapid technical cycles, readers are advised to independently verify volatile metrics. COMPUTE VIEWS HUB maintains structural objectivity and independent neutrality. more
This publication is intended solely for commercial, educational, and informational purposes.
Articles may include news reporting, editorial opinions, technical analysis, software tutorials,
deployment guidance, benchmark testing, hardware evaluations, workflow optimization strategies,
pricing references, market intelligence, developer resources, and enterprise technology commentary.
Product specifications, APIs, licensing models, cloud pricing, benchmark results, software capabilities,
commercial terms, and hardware availability are subject to change without notice. Any performance figures
or comparisons are based on publicly available information, vendor documentation, independent testing,
or specific test environments and should not be interpreted as universally representative. Readers are
encouraged to verify all technical and commercial information directly with official vendors before
making engineering, purchasing, investment, or operational decisions. Unless explicitly labeled as
sponsored content, advertising, affiliate content, or paid partnerships, editorial decisions remain independent.
COMPUTE VIEWS HUB does not warrant the completeness, accuracy, or future availability of third-party products,
services, software, or information referenced within this publication.