Skip to content

nopii.mdx

Latest commit

聽

History

History
175 lines (129 loc) 路 6.26 KB

nopii.mdx

File metadata and controls

175 lines (129 loc) 路 6.26 KB
Raw
title NoPII Integration
sidebarTitle NoPII
logo /images/integrations/nopii_icon.svg
description Trace NoPII privacy-proxied LLM calls in Langfuse. Server-side spans from NoPII and client-side application traces appear together in a single trace view via W3C traceparent.

NoPII Integration

In this guide, we'll show you how to use Langfuse to trace NoPII privacy-proxied LLM calls and connect them to your application traces end-to-end.

What is NoPII? NoPII is a hosted privacy proxy that exposes OpenAI- and Anthropic-compatible endpoints. PII in outbound prompts is replaced with deterministic vault tokens before requests reach the underlying LLM, and the original values are restored in the response. The LLM provider only ever sees tokenized placeholders, never raw PII.

What is Langfuse? Langfuse is an open source LLM engineering platform that helps teams trace LLM calls, monitor performance, and debug issues in their AI applications.

NoPII has built-in Langfuse support. When enabled in the NoPII admin console, every proxied request emits server-side spans (sanitize, llm-call, desanitize) directly to your Langfuse project. PII never appears in those spans, only tokenized content. Combined with W3C traceparent propagation, your application's traces and NoPII's server-side traces appear together as a single trace in Langfuse.

Get started

  1. Enable Langfuse in the NoPII admin console at app.nopii.co (NoPII pushes its own traces to your Langfuse project).
  2. Install the dependencies:
pip install langfuse openai anthropic python-dotenv
  1. Set environment variables. NoPII identifies your tenant from the underlying provider key (via a one-way hash), so no separate NoPII credential is required:
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_HOST=https://us.cloud.langfuse.com
# Other Langfuse data regions: 馃嚜馃嚭 EU https://cloud.langfuse.com, 馃嚡馃嚨 Japan https://jp.cloud.langfuse.com, 鈿曪笍 HIPAA https://hipaa.cloud.langfuse.com

Example 1: OpenAI via NoPII

Point the OpenAI client at NoPII with base_url, then trace the call with Langfuse's @observe() decorator. Inside the generation, read Langfuse's active trace and observation IDs and forward them as a W3C traceparent header. NoPII attaches its server-side sanitize / llm-call / desanitize spans as children of that generation, so both sides appear together in a single Langfuse trace.

import os

from dotenv import load_dotenv
from langfuse import Langfuse, observe
from openai import OpenAI

load_dotenv()

client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    base_url="https://api.nopii.co",
)

langfuse = Langfuse(
    public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
    secret_key=os.environ["LANGFUSE_SECRET_KEY"],
    host=os.environ.get("LANGFUSE_HOST", "https://us.cloud.langfuse.com"),
)

PROMPT = (
    "Summarize the customer record for John Smith. "
    "His SSN is 234-56-7891 and his email is john.smith@acme.com."
)


@observe(as_type="generation")
def call_llm(prompt: str) -> str:
    # Reuse Langfuse's active trace/observation IDs so NoPII's server-side spans
    # land under the same trace.
    trace_id = langfuse.get_current_trace_id()
    span_id = langfuse.get_current_observation_id()
    traceparent = f"00-{trace_id}-{span_id}-01"

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        extra_headers={"traceparent": traceparent},
    )
    langfuse.update_current_generation(
        model="gpt-4o",
        usage_details={
            "input": response.usage.prompt_tokens,
            "output": response.usage.completion_tokens,
        },
    )
    return response.choices[0].message.content


@observe()
def customer_lookup(prompt: str) -> str:
    return call_llm(prompt)


result = customer_lookup(PROMPT)
print(result)

langfuse.flush()

Example 2: Anthropic via NoPII

Same pattern with the Anthropic SDK. NoPII's Anthropic-compatible endpoint accepts the same base_url override; the bare Anthropic SDK appends /v1/messages for you.

import os

import anthropic
from dotenv import load_dotenv
from langfuse import Langfuse, observe

load_dotenv()

client = anthropic.Anthropic(
    api_key=os.environ["ANTHROPIC_API_KEY"],
    base_url="https://api.nopii.co",
)

langfuse = Langfuse(
    public_key=os.environ["LANGFUSE_PUBLIC_KEY"],
    secret_key=os.environ["LANGFUSE_SECRET_KEY"],
    host=os.environ.get("LANGFUSE_HOST", "https://us.cloud.langfuse.com"),
)

PROMPT = (
    "Draft a follow-up note for patient Maria Garcia (DOB: 03/15/1985). "
    "Her SSN is 321-54-9876 and her email is maria.garcia@gmail.com."
)


@observe(as_type="generation")
def call_llm(prompt: str) -> str:
    # Reuse Langfuse's active trace/observation IDs so NoPII's server-side spans
    # land under the same trace.
    trace_id = langfuse.get_current_trace_id()
    span_id = langfuse.get_current_observation_id()
    traceparent = f"00-{trace_id}-{span_id}-01"

    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
        extra_headers={"traceparent": traceparent},
    )
    langfuse.update_current_generation(
        model="claude-sonnet-4-20250514",
        usage_details={
            "input": response.usage.input_tokens,
            "output": response.usage.output_tokens,
        },
    )
    return response.content[0].text


@observe()
def patient_followup(prompt: str) -> str:
    return call_llm(prompt)


result = patient_followup(PROMPT)
print(result)

langfuse.flush()

In both examples, the LLM only ever sees tokenized placeholders, but the response your application receives contains the restored original PII. In Langfuse, NoPII's server-side spans show the sanitized content; your application's spans show what your code actually saw.

Learn more