Langfuse

Connect a Langfuse project so CloudThinker can investigate LLM traces, costs, evaluation scores, and prompt versions

Connect a Langfuse project so CloudThinker can tell you why an LLM call failed, what a trace cost, and how a prompt version scores. The connection uses a project-scoped API key pair (a public key and a secret key) and works with Langfuse Cloud and self-hosted Langfuse.

Prerequisites

• A Langfuse Cloud or self-hosted project that is receiving traces from your application.
• Permission to create API keys in that project.
• Your Langfuse host: the base URL you sign in to, such as https://cloud.langfuse.com (EU) or https://us.cloud.langfuse.com (US), or your self-hosted domain.
• For self-hosted Langfuse: the instance serves /api/public/mcp over HTTPS and accepts connections from the internet. Behind a firewall, allow CloudThinker’s egress IPs. These docs don’t publish CloudThinker’s egress IP list: ask your CloudThinker contact, or contact us. See Reaching private services for other options. CloudThinker rejects plain http:// hosts.
• The Admin role in your CloudThinker workspace, which can add and edit connections. See Roles.

Setup

1. Create an API key pair in Langfuse

   Sign in to Langfuse, open the project you want CloudThinker to read, and go to Project → Settings → API Keys. Click Create new API keys.

   Langfuse shows a Public Key (starts with pk-lf-) and a Secret Key (starts with sk-lf-). Copy both now. Langfuse shows the secret key only once.

2. Open the Langfuse connection in CloudThinker

   In CloudThinker, open More → Workspace settings → Connections, stay on the Builtin Connections tab, and select Langfuse.

3. Enter the host and keys

   Field	What to enter	Example
   LANGFUSE_HOST	Base URL of the Langfuse instance where the project lives, with no path. A host without https:// gets it added.	https://cloud.langfuse.com
   LANGFUSE_PUBLIC_KEY	The project’s public key	pk-lf-...
   LANGFUSE_SECRET_KEY	The project’s secret key	sk-lf-...

   A key pair works only on the host and in the project that issued it. To read a second project, add a second Langfuse connection with that project’s keys.

4. Connect

   Click Connect. CloudThinker tests the connection by opening a session with Langfuse’s MCP server at /api/public/mcp.

   Success state: the Langfuse connection shows a Connected status. Run the verify prompt below to confirm the keys read your project.

Verify the connection

Summarize my Langfuse project: recent traces, error count, models in use, and prompts

Success state: CloudThinker lists a sample of traces from the last 7 days, recent observations, prompts, and datasets from this project. A project with no traces returns empty lists, not an error. A 401 means the keys are wrong; see Troubleshooting.

Required permissions

A Langfuse key pair belongs to one project and reaches that project’s traces, scores, datasets, and prompts. Choosing the project is how you limit what CloudThinker sees.

The key pair can also write to that project, for example scores, comments, and annotation queue items. CloudThinker writes only when you ask for a specific change, and every write goes through Approvals.

Every change an agent makes goes through your workspace’s Approvals first. On the default Auto · Rules + AI · Ask me mode, AI judgment can let a safe change run without asking — and an Allow Rule can too, once CloudThinker turns on Rule enforcement for your workspace. Choose Manual to approve every change yourself.

Create a key pair used only by CloudThinker, separate from the keys your SDKs send traces with. You can then revoke it without breaking ingestion.

Agent capabilities

Capability	What CloudThinker does
Trace investigation	Ranks recent failed or slow observations and correlates errors, latency, and model use
Conversation tracing	Rebuilds one conversation or session as an ordered flow of model and tool calls, with errors marked
Usage reports	Totals tokens, cost, and latency for a time window, broken down by model and session
Quality review	Reviews scores, datasets, annotation queues, comments, and prompt versions, and points out evaluation gaps
Changes on request	Writes such as adding a score or comment, or changing an annotation queue item, only when you name the target and the change. It never writes during an investigation, report, or review. The call goes through Approvals.

Example prompts

Which Langfuse traces failed in the last hour, and what do the errors have in common?
Break down LLM cost and latency by model for the last 24 hours in Langfuse
Which prompt versions have the lowest evaluation scores this week?

Mention a time window in each request. Trace and usage queries are bounded, and a short window returns a focused answer faster.

Troubleshooting

401 Unauthorized

The public or secret key is wrong, was deleted, or the two keys come from different projects. Copy both keys from the same project again, check for extra spaces, and update the connection.

Connection fails against Langfuse Cloud

Each Langfuse Cloud region keeps its data apart, so a key pair only works on the host that issued it. Set LANGFUSE_HOST to the base URL you sign in to, for example https://cloud.langfuse.com for EU or https://us.cloud.langfuse.com for US.

Every request returns 404

LANGFUSE_HOST has a path after the domain. Set it to the base URL only. A 404 on one trace or dataset means that ID doesn’t exist in this project.

Self-hosted Langfuse returns 503 or never answers

Check that the instance serves /api/public/mcp over HTTPS, that your proxy preserves the public Host header, and that no firewall blocks CloudThinker from the host. To allowlist CloudThinker, see Reaching private services.

A search returns nothing

An empty answer is not a broken connection. Widen the time window a step at a time, and confirm the project is receiving traces from your application.

Security

• Least privilege — grant only the permissions the agents need for your use case; start read-only and widen later.
• Read-only by default — use read-only credentials unless you want agents to make changes through this connection.
• Rotate credentials — rotate keys and tokens on your normal schedule; CloudThinker picks up the new value when you update the connection.
• Revoke on offboarding — remove the credential at the provider when you delete a connection or a teammate leaves.

• One project per key pair: the key pair reaches the whole project, so connect only the project CloudThinker should read.
• HTTPS only: CloudThinker refuses a host that isn’t HTTPS, so keys never travel in plain text.

Related

LangSmith Connection

Investigate LLM runs and evaluations in LangSmith

Approvals

Control which connection writes run without asking you