LLM observability in 2026: revisit Langfuse, Arize Phoenix and OpenLIT before you ship another AI feature without traces

Somewhere in your company today, a team is shipping an AI feature that calls a model, maybe a tool, and returns an answer. When a customer reports a wrong summary or a leaked field, the usual reply is still a guess: look at application logs, ask the developer who wrote the prompt last week, and hope the provider dashboard still has the request. Logging is not enough once prompts, retrieval steps and tool calls form a chain.

LLM observability is the habit of recording those chains as traces you can search, score and improve. In this post we revisit three active open-source, self-hostable projects in this space, Langfuse, Arize Phoenix and OpenLIT, and look at history, licences, architecture, OpenTelemetry, prompts, evaluations and security defaults.

This is not a ranking. No benchmark or hands-on drill was run for this post; the facts come from each project’s own GitHub repository, LICENSE files, documentation and release notes as of early November 2026. We also do not repeat neighbouring layers. If you are deciding how to route model traffic, read our earlier post on AI gateways with LiteLLM, Bifrost and Agent Router. If you run models yourself, see LLM inference serving with vLLM, SGLang and llama.cpp. For the chat screen people use day to day, see self-hosted AI chat front ends with Open WebUI, LibreChat and AnythingLLM. General-purpose distributed tracing is a different tool; our earlier post on Jaeger, Grafana Tempo and SigNoz covers that layer.

Why revisit LLM observability now

All three projects ship often, and each recent release changes something an operator should know about:

• Langfuse v4.76.0 was released on 5 November 2026 (IST), after a dense run of v4.7x releases in late October and early November. The notes cover session views, Topics summaries in the public API, audit-log reads for traces and sessions, and continued work on the optional AI Gateway. The core product remains MIT-licensed; enterprise folders stay behind a separate licence.
• Arize Phoenix v21.0.0 (package arize-phoenix) was released on 30 October 2026 (IST). It is a breaking release: when PHOENIX_ENABLE_AUTH is unset, authentication now follows the bind address, so non-loopback hosts (including the Docker image default of ::) turn auth on. Client packages such as arize-phoenix-client v3.7.0 followed on 5 November 2026 (IST).
• OpenLIT 2.1.1 was released on 26 October 2026 (IST). The notes highlight a first-party OTLP receiver on ports 4317 and 4318 (replacing a bundled collector), a trace-level telemetry view, more connectors, and clearer Community versus Enterprise licence boot behaviour. The repository was still receiving commits into early November 2026, so we keep OpenLIT as the third active option rather than switching to Helicone, Lunary or Opik for this post.

Once AI features leave a demo notebook, traces become the system of record for prompts and completions. That is a different risk profile from ordinary APM spans, so old assumptions need to be revisited.

Shared concepts before you compare

Unlearn the idea that “logging is enough”. An LLM feature needs a few ideas that plain logs rarely give you:

• Traces and spans. A trace is one user request end to end. Spans are the steps inside it: retrieval, embedding, model call, tool call, post-processing. Parent and child links show the order and timing.
• Generations (model observations). The model call itself needs prompt, completion, model name, token counts, latency and cost metadata. Without that, you cannot debug quality or spend.
• Scores and evaluations. Human labels, code checks and LLM-as-a-judge scores attach quality signals to traces so you can compare versions, not only read them.
• Prompts and datasets. Versioned prompts and fixed test sets turn “we changed the prompt” into something you can reproduce and score again.
• PII in prompts. Customer names, KYC fields, account numbers and internal tickets often sit inside the prompt text. Storing them in a trace store is a privacy decision, not only a debugging convenience.
• Storage and retention. Trace backends grow quickly. Decide where data lives (Postgres, ClickHouse, object storage, SQLite), who can read it, and when it is deleted. Backups keep deleted text longer than the UI does.

Compare these jobs, not the colour of the dashboards.

Langfuse

Langfuse started in May 2023 (repository created 18 May 2023). It is an open-core LLM engineering platform for tracing, evaluations, prompt management, datasets and a playground. Copyright on the root LICENSE now reads ClickHouse, Inc. for 2023-2026; the public handbook still describes the product as open source with a commercial enterprise layer. Self-host options include Docker Compose for local or single-VM use and Kubernetes with Helm for production.

What it does well

• Full LLMOps loop in one place. Tracing, prompt versioning, datasets, scores (including LLM-as-a-judge and human labels) and a playground that can open from a bad trace.
• Clear self-host architecture. Langfuse Web (UI and API) and Langfuse Worker (async processing), with PostgreSQL for transactional data, ClickHouse for observations, Redis or Valkey for cache and queues, and S3-compatible blob storage for raw events and multi-modal attachments. Traces are written to object storage first, then ingested into ClickHouse, which protects against short database outages.
• Deployment paths that match team size. Docker Compose for a quick try; Helm and cloud templates for production. An optional Langfuse AI Gateway (beta) can sit in front of model providers and record traffic.
• SSO and hardening knobs. Email and password auth is on by default; you can disable password auth and use OIDC or SAML providers. Docs cover headless init, data masking options, and outbound URL hardening for LLM connections and webhooks.
• OpenTelemetry path. Self-hosted Langfuse documents observability via OpenTelemetry and an OTLP ingestion route, so teams already on OTEL can feed the same backend.

What to check before you commit

• Licence is open-core, not “all MIT”. The root LICENSE says content outside the ee/ paths (including web/src/ee/, worker/src/ee/, and packages/shared/src/server/ee/) is available under the MIT Expat licence. Those ee/ trees use a separate Langfuse Enterprise licence. Project docs state that core capabilities such as tracing, evaluations, prompt management, experiments, annotation and the playground are MIT, while enterprise modules such as SCIM, extended audit logging and data retention policies need a commercial licence when you self-host. Read both LICENSE files yourself.
• More moving parts than a single container. Postgres, ClickHouse, Redis and blob storage each need backups, upgrades and monitoring. Compose is fine to learn; production guidance points at Kubernetes.
• Sign-up defaults. Email and password sign-up is enabled by default on self-hosted instances. Disable open sign-up or move to SSO before the instance is reachable beyond trusted operators. Use headless initialisation if you want a known admin without clicking through the UI.
• Masking and retention. Server-side ingestion masking and some retention controls are documented as enterprise or separately configured features. Decide client-side redaction and retention policy before prompts with PII start landing in ClickHouse and object storage.
• Outbound LLM calls. Playground and evaluation features may call an LLM API you configure. Treat those keys and allowlists like any other production secret.

Arize Phoenix

Arize Phoenix is Arize AI’s open-source AI observability and evaluation platform. The repository was created in November 2022. It focuses on tracing, evaluations, datasets, experiments, a playground, prompt management and, more recently, an in-product assistant and a remote MCP server. Instrumentation is OpenTelemetry-based through the related OpenInference project (Apache-2.0). For a managed product, Arize offers Arize AX; the same OpenTelemetry and OpenInference instrumentation is described as working with both.

What it does well

• Fast local start. pip install arize-phoenix and phoenix serve, or uvx arize-phoenix serve, give a working UI. Docker images and a Helm chart cover shared deployments.
• OpenTelemetry and OpenInference first. Traces use OTLP (HTTP on the UI port path, gRPC on 4317 in the usual container layout). Framework and provider auto-instrumentation is documented for many agent and LLM stacks.
• Eval and experiment workflow. Datasets, experiments, annotations (including newer DECISION and AGENT kinds in v21) and prompt versioning support a “change, measure, keep” loop rather than only live debugging.
• Auth that finally defaults toward safer binds. From v21.0.0, unset PHOENIX_ENABLE_AUTH means loopback stays open for local work, while non-loopback binds (including Docker’s ::) enable authentication. Helm values enable auth by default. OAuth2 and OIDC options are documented for team deployments.
• Retention control. PHOENIX_DEFAULT_RETENTION_POLICY_DAYS defaults to 0 (keep forever). You can set a positive day count so old traces are removed on a schedule, and override per project in the UI.

What to check before you commit

• Licence is Elastic License 2.0 (ELv2), not Apache or MIT. You may use, copy and modify Phoenix subject to ELv2 limits. You may not offer it to third parties as a hosted or managed service that exposes a substantial set of its features. You also must not circumvent licence-key functionality or strip licence notices. Internal self-host for your own company is the usual fit; a multi-tenant SaaS built on Phoenix needs legal review against ELv2.
• Storage choice matters early. Default local mode can use SQLite in a working directory. Shared or durable deployments should use PostgreSQL (PHOENIX_SQL_DATABASE_URL or the discrete Postgres variables) and a persistent volume plan.
• Docker and auth. The container image binds non-loopback and therefore expects authentication and API keys for exporters unless you explicitly set PHOENIX_ENABLE_AUTH=false. Do not “turn auth off” on a shared network to save five minutes.
• Secrets and `PHOENIX_SECRET`. Without PHOENIX_SECRET, Phoenix can sign tokens with a key stored in the database; anyone with a database copy can then decrypt saved credentials. Set a strong secret in a vault, and know that changing it makes previously saved credentials unreadable.
• Default admin password. Docs describe a built-in admin (admin@localhost) whose initial password defaults to admin and must be changed before a normal session. Change it on first boot.
• Product telemetry. PHOENIX_TELEMETRY_ENABLED defaults to collecting basic web analytics (page views and UI interactions). Trace data is stated as not collected. Set it to false if your policy requires no outbound analytics.

OpenLIT

OpenLIT is an open-source, OpenTelemetry-native platform aimed at tracing, evaluating, guarding and improving AI agents and the “harness” around them (tools, prompts, context, cost and GPU monitoring). The repository was created in January 2024. We checked whether OpenLIT was weak or abandoned before choosing a third tool; as of early November 2026 it remains clearly active (2.1.1 in late October, ongoing commits), self-hostable, and Apache-2.0, so it stays as the third option in this post.

What it does well

• OpenTelemetry as the spine. SDKs initialise with a short call (openlit.init() in Python) and export OTLP. The platform now runs a first-party OTLP receiver on gRPC 4317 and HTTP 4318, so you do not need a separate collector container for the default Compose path.
• Light self-host stack. Docs describe OpenLIT (dashboard, APIs, OTLP receiver, realtime engine) plus ClickHouse for otel_* storage. Docker Compose also runs NATS JetStream for live span signals used by threshold alerts. Helm charts cover Kubernetes. Existing ClickHouse can be reused.
• Broader agent harness features. Beyond traces, the project documents evaluations, guardrails, prompt and context management, cost tracking and optional GPU collectors. That is useful when the problem is not only “what did the model say” but also “which tool and which prompt version caused the failure”.
• Licence clarity for the community edition. The root LICENSE is Apache License 2.0. Docs state the community edition is free to self-host with no licence key and no per-trace fee.

What to check before you commit

• Change the default login immediately. Quickstart docs list default credentials user@openlit.io / openlituser for a fresh Compose deploy. Treat that like any other default password: change it before the port leaves localhost.
• NATS passwords in Compose. Installation docs warn that Compose ships placeholder NATS passwords and that you should set NATS_RECEIVER_PASSWORD and NATS_ENGINE_PASSWORD before anyone else can reach the deployment.
• Enterprise extras. Organisation-wide governance features such as custom roles, audit logs, event alerts and notification destinations are documented as OpenLIT Enterprise and marked in the docs. Confirm which screens you need before you assume every control is in the community build.
• Prompt and completion capture. SDK docs state that prompt and completion content is captured on spans by default (capture_message_content=True). Turn that off, truncate with max_content_length, or use documented guardrail redaction when traces must not hold raw customer text. Do not rely on a single-message redaction path for multi-turn chats without reading the current SDK notes.
• Smaller community than the first two. Star counts and release cadence are healthy enough for this shortlist, but operational docs and third-party runbooks are thinner than Langfuse or Phoenix. Budget time for ClickHouse operations yourself.

Security and privacy notes

Whichever tool you pick, these points matter more than features:

• Traces hold secrets by default. Prompts and completions often contain customer data, internal identifiers and sometimes API keys pasted into messages. Assume stored traces are as sensitive as a production database dump.
• Close open registration. Langfuse enables email and password sign-up by default. Phoenix now enables auth on non-loopback binds in v21, but local defaults and explicit PHOENIX_ENABLE_AUTH=false can still leave a wide-open UI. OpenLIT ships a known default user. Create admins first, then lock sign-up or move to SSO.
• API keys for ingestion. SDKs and gateways authenticate with project keys. Scope them per environment, rotate them, and never embed production keys in mobile apps or public front ends.
• Multi-tenant boundaries. If several product teams share one instance, confirm project or organisation isolation, who can see whose traces, and whether admin roles can read everything. Do not share one project key across unrelated apps.
• Retention and backups. Infinite retention is a privacy bug waiting to happen. Set a retention window, encrypt backups, and test deletes. Remember object storage and ClickHouse cold storage may outlive the UI delete button.
• Client-side redaction first. Prefer masking PII before export when the debugger does not need the raw value. Server-side masking, where offered, is a second line, not the only line.
• Network exposure. Put the UI behind HTTPS and your SSO or VPN. OTLP ports belong on a private network. Do not publish a fresh Compose stack on the public internet to “try it for a day”.
• Vendor analytics. Phoenix documents optional product telemetry separate from your traces. Turn it off when policy requires. Review each project’s outbound connections the same way you would for any admin console.

When to pick which

• Pick Langfuse when you want a full self-hosted LLMOps suite (traces, prompts, datasets, scores, playground) with MIT-licensed core features, you can run Postgres plus ClickHouse plus Redis and object storage, and you accept that some governance features sit behind an enterprise licence.
• Pick Arize Phoenix when you want OpenTelemetry and OpenInference-centric tracing with a strong eval and experiment workflow, a quick local start that grows into Postgres-backed self-host, and ELv2 is acceptable (internal use, not offering Phoenix itself as a multi-tenant SaaS).
• Pick OpenLIT when you want an Apache-2.0, OTEL-native stack with a lighter Compose footprint (OpenLIT plus ClickHouse, plus NATS for alerts), care about agent harness concerns such as guardrails and GPU or cost views, and you are ready to harden default credentials and content-capture settings on day one.
• Pilot only one first. Instrument one service, prove that traces answer a real incident question, then expand.

A small example

This is an invented example, not a real company. Consider an 80-person fintech product team in Ahmedabad building a support assistant that summarises KYC documents and ticket history for agents. The first pilot answered well in demos, then a wrong summary reached a customer and nobody could show the exact prompt, retrieved chunks or tool results from that minute.

A calm plan would be: run one observability backend on an internal subdomain behind their existing OIDC login, with OTLP only on the private network. Point the assistant (and the company AI gateway) at that backend with a per-environment API key. Turn off open sign-up, change every default password, and set prompt content capture to redact Aadhaar-like fields and account numbers before export. Keep raw traces for 30 days in hot storage and 90 days encrypted offline, with access limited to the platform team and two support leads. Use a small labelled dataset of past tickets to score prompt changes before each release. After two weeks of notice, refuse to debug production AI issues that have no trace id.

Practical checklist

• Write down the jobs you need: live traces, evals, prompt versions, datasets, cost views, or only debugging.
• Read each LICENSE file yourself (Langfuse MIT plus ee/, Phoenix ELv2, OpenLIT Apache-2.0 plus Enterprise extras).
• Create the first admin, change default passwords, then close or control sign-up; prefer SSO.
• Put UI and OTLP on private networks with HTTPS; pin image tags.
• Decide prompt and completion retention, redaction and who may open a trace.
• Prefer client-side PII masking for fields you never need in the clear.
• Back Postgres, ClickHouse or SQLite and object storage; test a restore.
• Scope ingestion API keys per app and environment; rotate on a schedule.
• Link traces to your gateway and inference layers so cost and model identity stay consistent.
• Pin versions, read breaking notes (especially Phoenix v21 auth), and schedule upgrades.

Closing thought

Teams ship AI features without traces because the demo works and the dashboard can wait. Revisit that habit. Learn what these three tools actually store, under which licence, and with which auth defaults. Unlearn “the provider console is enough”. Practise one traced path in staging, then apply the same standard to production. The next wrong answer is much easier to fix when you can see the chain that produced it.