How to Instrument Multi-Step AI Agent Workflows for Span-Level Error Tracking

Quick answer

For a quick setup, install the LangSmith Python SDK with pip install -U langsmith, then enable tracing in the application’s runtime configuration with LANGSMITH_API_KEY and LANGSMITH_TRACING=true. Keep the key out of source code.

Use @traceable on the request handler, workflow steps, and tools, giving each a clear name and a run type such as chain or tool. Make nested calls within the active tracing context so the trace shows their parent-child relationship.

Inputs and outputs are captured by default. If a tool payload might contain secrets or sensitive information, transform it with process_inputs or process_outputs before it is logged.

When a tool should be marked as failed, let its exception escape the traced function; catch it higher up only if the workflow is meant to recover.

After a test invocation, inspect the run records for a shared trace_id, the expected parent_run_id, intended inputs and outputs, and an error on a deliberately failed tool run.

Step-by-step guide

Prerequisites and starting state

Start with a Python application that already uses LangChain; this guide instruments that application rather than setting up a complete agent framework. Before editing workflow functions, confirm you can run the application in the environment where you plan to enable tracing.

Have a LangSmith API key ready and identify how the application receives its runtime configuration. Keep the key in configuration rather than embedding it in the agent’s source code; the next step enables tracing with environment variables and installs the Python SDK.

Sources: OpenAI integration — Docs by LangChain; langsmith-sdk: Python tracing helpers.

Install the LangSmith Python SDK and enable tracing

Install the LangSmith Python SDK in the same environment that runs your LangChain application:

pip install -U langsmith

Then configure tracing in the application’s runtime environment before starting it. Set LANGSMITH_API_KEY to your key and LANGSMITH_TRACING to true; keep the key in environment or secret-management configuration, not in your agent’s source code.

Restart the application after changing its environment, then run a request. The SDK can send trace data to LangSmith when tracing is enabled.

If no trace appears, confirm the running process received both variables and that the API key is valid. The OpenAI integration’s separate install command is not needed here: it installs that integration, not the LangSmith SDK or a complete agent framework.

Sources: OpenAI integration — Docs by LangChain; langsmith-sdk: Python tracing helpers.

Add named spans to workflow and tool functions

Decorate the request handler, workflow steps, and tools you want to inspect. Give each a readable name and use a run type such as chain for workflow functions and tool for tool functions.

from langsmith import traceable

@traceable(name="handle_request", run_type="chain")
def handle_request(question):
    return plan_and_execute(question)

@traceable(name="plan_and_execute", run_type="chain")
def plan_and_execute(question):
    return lookup_tool(question)

@traceable(name="lookup_tool", run_type="tool")
def lookup_tool(query):
    return search(query)  # Replace with your existing tool call.

Keep nested traced calls within the active tracing context, as shown here, so the tool run is recorded as a child of the workflow that called it. Adapt the names and function bodies to your application, then run a request and inspect the trace for the named runs and their parent-child relationship.

Sources: OpenAI integration — Docs by LangChain; langsmith-sdk: Python tracing helpers.

Control which tool inputs and outputs are recorded

LangSmith records traced function inputs and outputs by default. Before relying on those records, check whether tool arguments or results contain credentials, personal data, or sensitive payloads.

Pass process_inputs and process_outputs callbacks to @traceable to transform data before it is logged. For example, adapt the callbacks to your tool’s argument and result fields, and return only the information you need for debugging:

def safe_inputs(inputs):
    return {"query": "[REDACTED]"}

def safe_outputs(outputs):
    return {"status": outputs.get("status")}

@traceable(
    name="lookup_tool",
    run_type="tool",
    process_inputs=safe_inputs,
    process_outputs=safe_outputs,
)
def lookup_tool(query):
    return search(query)  # Replace with your existing tool call.

Run the workflow and inspect the tool run’s recorded fields. Confirm that useful debugging context remains and sensitive values are absent; if a field is still exposed, adjust its callback before using tracing with real data.

Sources: OpenAI integration — Docs by LangChain; langsmith-sdk: Python tracing helpers.

Preserve tool failures in span error records

Let an exception that should mark a tool call as failed escape the decorated tool function. LangSmith records the exception on that run and re-raises it, allowing the calling workflow to decide what to do.

Catch it higher in the workflow only when the agent is meant to recover, such as by returning a fallback; otherwise, let it continue upward. For a quick check, trigger a controlled tool failure and inspect its run in the trace: the tool span should show the error, while the workflow can handle the propagated exception if recovery is intended.

If an expected error is missing, check that the tool function does not catch the exception before it reaches the tracing wrapper. Also check that the exception type has not been configured to suppress its error field.

Sources: OpenAI integration — Docs by LangChain; langsmith-sdk: Python tracing helpers; Read Shared Dataset Examples With Runs — Docs by LangChain.

Verify trace structure and recorded errors

1. Run a test request through the instrumented workflow, then inspect its trace in the LangSmith project. Start with the root run and follow the tool run beneath it.

2. Check that the root and tool runs have the same trace_id, and that the tool’s parent_run_id matches its parent run’s id. Review the inputs and outputs fields to confirm they contain useful debugging context and no data you meant to redact.

3. Trigger a controlled tool exception and inspect that tool run’s error field. If the child run is missing or has no parent, check that the nested call ran within the active tracing context. If the expected error is absent, check that the exception escaped the traced function and that its type is not configured to suppress error recording.

Sources: OpenAI integration — Docs by LangChain; langsmith-sdk: Python tracing helpers; Read Shared Dataset Examples With Runs — Docs by LangChain.

Troubleshoot missing context or misleading error records

If a tool run is missing from its expected parent-child chain, first check that the nested traced function ran within the active tracing context. Calls made in separate threads or processes may need explicit context propagation; the SDK guidance cited here does not provide those setup steps, so check your framework’s context-propagation guidance before moving work across execution boundaries.

If a run contains sensitive inputs or outputs, update its process_inputs or process_outputs callback to remove or transform those fields, then inspect a new trace to confirm the unwanted data is absent. For a missing error, check that the exception escapes the traced tool function and that its type has not been configured to suppress the error field.

The SDK records and re-raises exceptions by default, so catch one higher in the workflow only when recovery is intended.

Sources: OpenAI integration — Docs by LangChain; langsmith-sdk: Python tracing helpers; Read Shared Dataset Examples With Runs — Docs by LangChain.

Next steps

If your Python LangChain application needs clearer debugging for multi-step runs, this approach is a practical starting point: name the important workflow and tool spans, keep them in the active trace, and decide deliberately what data each span records. I’d treat the trace as useful only after checking both its structure and its privacy boundaries.

Run a normal test request, then a controlled tool failure. Confirm that the tool run shares the root’s trace, points to its parent, contains only intended input and output fields, and shows the expected error.

If the chain breaks across threads or processes, or your deployment needs region-specific tracing settings, consult the relevant setup guidance before relying on this configuration.