
Building trust in forecasting agents.
From sensitive-data controls and adversarial evaluations to reliable execution and operational visibility.
PRODUCT MISSION
Help sellers use AI agents for real forecasting work without losing trust in their data or the experience.
Problem & goal
A forecasting assistant needed seller-specific context to answer questions about the right products. That same context introduced risk: sensitive identifiers could leak, a request could try to cross seller boundaries, or conversation data could be retained beyond its intended use.
Production agents also faced a second problem. Questions covering many products could take several minutes and fail mid-stream as the request crossed the interface, runtime, and tools.
The goal: establish evidence for safe data handling, keep long-running work dependable, and give the team a way to understand failures.
Solution
My work grew across three connected areas: security readiness for a tenant-aware assistant, reliability fixes across production agents, and shared observability for the tools they called.
I worked on input validation and sensitive-data handling, leakage and access-boundary evaluations, timeout and streaming behavior, and tool-level metrics and session traces. Each addressed a different part of the trust problem.
User journey
My role
- 01
Start with the data boundary.
I mapped sensitive seller context through requests, validation, tools, memory, and storage. I implemented input-validation and prompt/data-handling controls, worked through leakage and access-boundary test cases, and reviewed retention policy and configuration.
- 02
Turn security questions into evaluation evidence.
I reviewed a teammate’s automated adversarial-test integration for the forecasting assistant. In separate agent evaluation work, I raised the per-evaluator acceptance threshold to 0.9. These results supported security and production-readiness review.
- 03
Carry that attention into production.
I traced long-running failures and implemented timeout, heartbeat, session-isolation, and streaming-parser fixes across four agents. I also built reusable tool metrics and session tracing so the team could investigate what happened within a request.
This case brings together security-readiness and production reliability work across related agent initiatives.
Decisions
Follow sensitive data through the whole workflow.
The assistant needed seller context to retrieve relevant forecasts. I followed where that context entered, how it was used, and where it could persist. The evaluation work covered three practical risk categories:
- Prompt injection
- Attempts to redirect the agent away from its intended instructions or manipulate how it used context.
- Sensitive-data exposure
- Leakage cases checking whether responses revealed protected identifiers or other sensitive context, including PII concerns.
- Seller boundaries
- Access-boundary cases checking behavior when a request tried to reach another seller’s data.
Trade-off: the system needed enough context to answer correctly, while every additional copy introduced exposure and retention considerations. I paired runtime controls with evaluation evidence and retention review.
Support long work without leaving users disconnected.
I traced failures across the response stream, runtime, and downstream tools. Aligning timeout budgets, heartbeat behavior, and session handling addressed the request as a connected experience.
Trade-off: longer execution budgets allow demanding tasks to finish, but users still need timely feedback and failures must remain visible. Increasing a timeout alone would leave those needs unresolved.
Make new tools observable by default.
I added success-rate and latency instrumentation to the shared tool abstraction, plus a reusable wrapper for tools registered through another path. Session traces exposed which tools ran and in what order.
For alerting, I replaced an unsuitable dynamic search approach with a metrics-query alarm that could cover the changing tool fleet. That avoided maintaining a separate static alarm for every new tool.
Impact
Security readiness and diagnosis
The security work supplied runtime controls and evaluation evidence for the organization’s security and data-protection review. The observability work established tool-level measurement and end-to-end session visibility as part of production readiness.
Uptime is the reported availability outcome. Security and observability outcomes are described through the controls, evaluation work, and diagnostic capabilities delivered.
How I would measure continued trust
Task success: completed, useful answers by workload size, plus time to first response and final answer.
Security guardrails: leakage and unauthorized-access failures by test category, with failed cases reviewed before release decisions.
Operational health: per-tool failures and latency, interrupted sessions, and time to identify the failing step.
Proposed follow-up measures. Availability, answer quality, and data protection need their own evidence.
What I took away
Trust is built across the product lifecycle. Data handling, adversarial evaluations, runtime behavior, and operational visibility all shape whether an agent is ready for real use.