Bulwark vs LangSmith: different jobs, commonly confused
LangSmith, built by the LangChain team, is a framework-agnostic platform for tracing, evaluating and debugging LLM applications, and it is very good at that. Bulwark is a governance layer: agent identity, scope enforcement, human approval gates and a tamper-evident evidence trail. The overlap is small. Teams generally adopt LangSmith to make agents work well and Bulwark to prove agents were allowed to do what they did.
Short answer
Not substitutes. If you are choosing between them you are probably solving two problems and have only recognised one. LangSmith for quality, Bulwark for authority and evidence.
What LangSmith is built to do
LangSmith gives deep visibility into an LLM application: every model call, tool invocation and intermediate reasoning step, captured as a trace you can inspect. On top of that sits an evaluation system — offline datasets, LLM-as-judge and code-based evaluators run against real production traces — plus prompt versioning and cost tracking across the workflow.
This is a mature product aimed squarely at the engineering problem of making agents work, and we have no interest in pretending otherwise. If your agent gives bad answers and you do not know why, this is the category of tool you want.
What Bulwark is built to do
Bulwark sits at the decision point in front of consequential actions rather than inside the model call. It gives each agent a registered identity with a named owner, carries the human principal the agent is acting for alongside it, enforces scope on data and tools, routes defined actions to human approval, and writes every decision to an HMAC hash-chained trail with a standalone verifier.
It has nothing useful to say about whether your agent's answers are good. That is not the question it was built to answer.
Where the confusion comes from
Both products record what agents do, so both look like "logging" from a distance. The difference is what the record is for and therefore how it is built.
A trace exists to help an engineer reconstruct a failure this week. It is rich, large, sampled under load, retained for a debugging window, and stored mutably because nobody is adversarial about it. A decision record exists to demonstrate to a sceptical third party what was permitted years ago. It is narrow, complete, retained for the obligation period, and hash-chained because the reader does not have to trust the operator.
Optimising one store for both sets of requirements means doing both badly.
When LangSmith alone is enough
Plenty of teams do not need a governance layer, and it is worth being direct about which:
- Agents that only read public or low-sensitivity data.
- Internal tooling with no external compliance obligations.
- Prototypes and experiments not yet in front of customers.
- Agents with no ability to take irreversible action.
When you need governance as well
The signals that the observability layer has stopped being sufficient:
- An enterprise security questionnaire asks how you restrict agent data access.
- An auditor asks who approved a specific automated action, and when.
- Your agents can move money, delete records or send external communications.
- You are in scope for the EU AI Act, ISO 42001 or SOC 2 covering AI systems.
- You cannot produce a current list of every agent running and who owns it.
Running both
There is no integration conflict. LangSmith instruments the model calls; Bulwark gates the consequential actions. An agent can be fully traced and fully governed at once, and in a regulated environment it usually should be.
Side by side
| LangSmith | Bulwark | |
|---|---|---|
| Category | LLM observability and evaluation | AI agent governance and evidence |
| Built for | Engineers debugging and improving agent quality | Security and compliance proving agent authority |
| Strongest at | Tracing, evals, prompt versioning, cost attribution | Identity, scope enforcement, approvals, verifiable evidence |
| Enforcement | Observes; not designed to block traffic | Deterministic gate on defined consequential actions |
| Record integrity | Mutable trace store | HMAC hash chain with standalone verifier |
| Identity captured | Calling application or service | Agent plus the human principal it acted for |
| Answers "why is my agent bad?" | Yes, this is the core use case | No, and it does not try to |
| Answers "who authorised this action?" | Not its purpose | Yes, as a single verifiable decision receipt |
Frequently asked
Can LangSmith traces satisfy an auditor?
They can support an audit as corroborating detail, but they are not built as evidence. The gaps an auditor tends to probe are sampling, retention windows shorter than the obligation period, and a mutable store that cannot demonstrate the record was not altered.
Does Bulwark replace LangSmith?
No, and we would rather you did not try. Bulwark captures decision records, not full traces, and has no evaluation capability. If you remove your observability tooling you lose the ability to debug agent quality.
Do the SDKs conflict?
No. They instrument different layers — one wraps model calls, the other sits in front of consequential actions — and are commonly deployed in the same application.
We use Langfuse, Arize or Braintrust instead. Does this comparison hold?
Broadly yes. The distinction being drawn is between the observability category and the governance category rather than about any single vendor. The specific feature sets differ, but the retention, sampling, integrity and enforcement gaps are properties of what observability tooling is for.
Try the governance half
Agent registry, scope enforcement, human approvals and a hash-chained evidence trail you can verify without trusting us. Free Developer tier, 100,000 governed decisions a month, no card required.
Related
- AI governance vs AI observability: what the difference actually isObservability tells you what your AI did and how well. Governance tells you what it was permitted to do and proves it. Why teams need both.
- Why sending agent logs to your SIEM is not AI governanceA SIEM stores what happened. Governance records why an action was permitted. What the difference means when an auditor asks about an AI decision.