Trace Features & Overview

Trace is an AI agent builder that turns process maps into working automations. You design a flow on the canvas, and Trace breaks it into steps, selects the right agent for each step, and executes tools with structured inputs and clear outputs. Agents call your SaaS APIs and databases, run functions, and use retrieval-augmented generation to ground answers in current data. Variables and memory carry context across steps so decisions stay consistent, while policies handle routing, retries, and fallbacks without manual babysitting.

Ship with production controls from day one. Version prompts and tools, run evals on real datasets, and set safety, cost, and latency thresholds. Insert human approvals where judgment matters, route tasks to queues with SLAs, and track every run with logs and metrics. Compare releases, roll back fast, and deploy agents to web, chat, email, or a clean API. Start free and scale as flows grow. Core Features

• Full prompt tracing: Record prompts, inputs, tool calls, outputs, tokens, and latencies per interaction. Inspect tree views for agents and chains, jump to slow steps, and correlate errors with exact parameters to fix issues fast.
• Dataset evals and regression tests: Create labeled datasets from production conversations, then run batch evals across prompts and models. Compare pass rates and rubric scores, promote the best variant, and prevent quality drift before releases.
• Prompt versioning and A/B: Version every system and user prompt with clear diffs and metadata. Route traffic between variants, collect ratings and metrics, and roll forward or back with a click when a change underperforms.
• Cost and quota analytics: Track token usage, per-request cost, cache hit rates, and provider quotas in real time. Set budgets and alerts by environment so experiments never surprise finance or throttle production.
• Guardrails and policies: Add PII redaction, profanity filters, and allowlists at the request boundary. Block risky outputs, log incidents for review, and export redaction spans so downstream stores never see sensitive content.
• Feedback and human review: Capture thumbs, rubrics, and annotator notes in the UI or via SDK. Link feedback to traces and datasets, then retrain or rewrite prompts using concrete failure examples from your own users.
• Monitoring and alerting: Define SLAs for latency, failure rates, and safety flags. Receive actionable alerts with trace links, last successful inputs, and diffed prompt versions so on-call engineers resolve issues quickly.
• Tool and retrieval visibility: Trace function calls and RAG steps with inputs, retrieved chunks, and scores. Spot irrelevant context, tune chunk sizes or rerankers, and verify that tools return the fields your prompts expect.
• Offline and real-time pipelines: Stream events from SDKs for instant dashboards or batch-ingest logs from warehouses. Backfill historical runs to baseline metrics and keep one view across staging and production.
• SDKs and APIs: Use lightweight client libraries to log events, attach metadata, and push eval results. Webhooks notify CI and analytics tools on eval complete, model switch, or guardrail trigger.
• Collaboration and RBAC: Organize projects by environment, restrict secrets, and assign roles for product, data, and engineering. Notes, saved views, and shareable links keep reviews tight during rollouts.
• Data residency and compliance: Choose regions, set retention, and export everything to your warehouse. Enterprise options add SSO, audit logs, and customer-managed keys for stricter programs.

Supported Platforms / Integrations

• Client SDKs for JavaScript, Python, and backend frameworks
• OpenAI, Anthropic, Google, Mistral, and compatible providers
• Vector stores and search APIs for RAG pipelines
• Data warehouses and BI tools for exports
• CI/CD hooks and webhooks for eval automation
• Issue trackers and incident channels for alerts

Use Cases & Applications

• Product teams improving answer quality for chat and agents
• Platform teams standardizing safety, logging, and governance
• Data teams running batch evals before prompt or model changes
• Support and ops measuring cost, latency, and failure hot spots

Pricing

• Free: developer seat, basic tracing, 7-day retention
• Pro: $49 per user per month, longer retention, dataset evals, alerts
• Team: $149 per month per project, prompt A/B and advanced RBAC
• Enterprise: contact sales, SSO, audit logs, EU residency, CMEK

Why You’d Love It

• Gives a single view of prompts, costs, quality, and incidents
• Turns production runs into datasets that lift accuracy
• Adds guardrails and alerts that protect users and budgets

Pros & Cons

Pros

• Clear traces across prompts, tools, and retrieval
• Practical eval workflows tied to real production data
• Useful cost, latency, and safety monitoring for releases

Cons

• Annotation and dataset prep add upfront work
• Best value shows at moderate to high traffic levels

Conclusion Trace helps you ship AI features you can measure and trust. You watch prompts in the wild, evaluate changes with real datasets, and keep budgets and safety in check. Teams move faster because quality, cost, and risk live in one workflow.

SEO Tags AI observability, LLM evals, prompt testing, prompt versioning, A/B testing, token cost analytics, RAG tracing, tool call monitoring, guardrails, PII redaction, SDK for logging, model comparison, human feedback, CI evals