Satya Prakash Solanki · Abu Dhabi, UAE
Satya Prakash Solanki, AI Assurance Architect & Product Engineering Lead: Trustworthy Agentic AI, from architecture to production.
AI Assurance Architect and Product Engineering Lead. I design how agentic systems are evaluated, secured, governed and kept reliable, and I engineer the products that take them to production.
- Agentic AI Assurance
- Evaluation, security, governance and reliability
- Product Engineering
- Architecture, agentic platforms and delivery
Who I am
Making AI provable, not just possible.
I have spent my career on one question: can this system be trusted? Today I architect how agentic and LLM systems are evaluated, attacked, guarded, observed and governed, across healthcare, finance, education, and oil and gas, and on enterprise and government programmes across the UAE and GCC.
Experience at
Practice aligned to
- OWASPLLM Top 10
- NISTAI RMF
- MITREATLAS
- OWASPTop 10
- UAEIA / NESA
- DoHADHICS
Agentic AI assurance
Six disciplines, one assurance system.
Trustworthy AI is not one control. Select a discipline to see what it covers.
One
Assurance
system
01 / 06
Security
Attack AI systems before adversaries do.
- Prompt-injection, jailbreak and exfiltration testing
- Risk categorisation mapped to OWASP LLM Top 10 and MITRE ATLAS
- SAST, SCA and DAST in a secure SDLC
02 / 06
Reliability
Agents that behave the same way on the thousandth run.
- Failure-mode analysis and fallback design
- Agent SLOs, retries and determinism testing
- Drift and regression detection
03 / 06
Assurance
Measured results decide what ships.
- Evaluation pipelines as release gates
- Golden datasets and LLM-as-a-judge
- Groundedness, faithfulness, hallucination scoring
04 / 06
Governance
Know which models are in use, how risky each is, and what evidence backs it.
- Use-case risk tiering
- Model inventory and model cards
- Audit trails aligned to NIST AI RMF
05 / 06
Compliance
Controls and evidence mapped to OWASP, NIST, MITRE and UAE frameworks.
- OWASP LLM Top 10 and NIST AI RMF mapping
- UAE IA / NESA and ADHICS alignment
- Evaluation evidence and audit trails
06 / 06
Trust
Behaviour you can trace, explain and rely on.
- Guardrails for PII, topic scope and grounding
- Agent tracing with Langfuse, Arize and OpenTelemetry
- Token, cost and latency telemetry; human-in-the-loop review
AI security
Security is designed in, not bolted on.
I lead adversarial testing of LLM and agentic systems, with a full application security practice underneath. Risks are categorised against the OWASP LLM Top 10, NIST AI RMF and MITRE ATLAS.
SAST
Semgrep · Checkmarx
SCA
Dependency risk
DAST
OWASP ZAP · Burp Suite
API security
Postman · Nuclei
Container & IaC
Image and config scanning
Penetration testing
Metasploit · nmap · OpenVAS
Red-team coverage map
OWASP Top 10 for LLM Applications
- LLM01Prompt injectionUntestedTested
- LLM02Sensitive information disclosureUntestedTested
- LLM03Supply chainUntestedTested
- LLM04Data and model poisoningUntestedTested
- LLM05Improper output handlingUntestedTested
- LLM06Excessive agencyUntestedTested
- LLM07System prompt leakageUntestedTested
- LLM08Vector and embedding weaknessesUntestedTested
- LLM09MisinformationUntestedTested
- LLM10Unbounded consumptionUntestedTested
Agentic AI
Agentic systems, built and secured.
Agents call tools, read data and act on their own. I build them, and I design the layers every request passes through, so autonomy never outruns control.
Select a layer to see the capability behind it.
- then
- then
- then
- then
Layer 1 of 5
Agentic AI Development
Multi-agent systems built with LangGraph, CrewAI, Agno and MCP, including testing, data-validation, bug-logging and notification agents in production use.
- LangGraph
- CrewAI
- Agno
- MCP
- LangSmith
Layer 2 of 5
AI Gateways
One controlled entry point for model and agent calls, with identity, access and token-cost visibility.
- Identity
- Access control
- Token cost
Layer 3 of 5
AI Shield
Runtime defence for agents: prompt-injection detection, tool-call validation and limits on excessive agency.
- Prompt injection
- Tool-call validation
- Excessive agency
Layer 4 of 5
AI Guardrails
Input and output controls for PII, toxicity, topic scope and grounding, with false-positive rates tuned against live traffic.
- PII redaction
- Topic scope
- Grounding
Layer 5 of 5
Secure AI Systems
Hardened RAG, MCP and tool integrations, with least-privilege access and tenant isolation.
- RAG
- MCP security
- Tenant isolation
Cross-cutting
AI Governance · Responsible AI Execution
Use-case risk tiers, model inventory and evaluation evidence carried through delivery, with tracing, an audit trail and human-in-the-loop checks for bias and toxicity.
- NIST AI RMF
- Langfuse
- OpenTelemetry
- Audit trail
Product engineering
From architecture to shipped product.
Assurance is strongest when it is built in. I lead product engineering for AI platforms, so the systems I assure are systems I have helped design, build and deliver.
- Architect
- Build
- Assure
- Ship
- Operate
01
Architect
Platform and system design: a two-plane Kafka architecture for real-time and batch evaluation, multi-tenant security with Keycloak identity and tenant isolation, and the platform’s quality architecture.
02
Build
Hands-on in Python, TypeScript and Java. Agentic systems, evaluation pipelines and data platforms on PySpark, Kafka, Trino and Airflow, built and run in production.
03
Deliver
Roadmaps presented to executive leadership, solution architecture and estimates for enterprise and government bids, including infrastructure sizing and token-cost forecasting.
Why it matters
AI that can’t be proven can’t be scaled.
Boards, regulators and customers now ask the same question: how do you know it works, and how do you know it is safe? Assurance turns that question into evidence, so organisations can move faster with less risk.
Release gate · agent v2.3
PASS| Metric | Result | Threshold |
|---|---|---|
| Faithfulness | 0.94 | ✓ |
| Context precision | 0.91 | ✓ |
| Prompt-injection bypass | 0 / 120 | ✓ |
| PII leakage | 0 | ✓ |
| p95 latency | 1.8 s | ✓ |
- years in quality and AI assurance
- 13+
- fewer production hallucination incidents
- 40%
- engineers mentored; onboarding time cut by 50%
- 6
- Best Employee of the Year, 2024 and 2025
- 2×
The journey
Thirteen years, one question. The systems kept getting bigger.
Every role has asked the same thing: can this system be trusted? What changed is the system, from a single service to agents that act on their own, and the proof it takes to answer.
Read the full story2012 – 2015
Prove the service
Naukri.com · QA Engineer
- The system
- High-traffic recruitment, real estate and education services and their APIs.
- The proof
- API automation with Rest Assured and Serenity BDD, and nightly Jenkins regression.
Days → hourstest feedback time; Best QA Engineer, twice
2015 – 2018
Prove the product
Fliplearn · Senior QA Engineer
- The system
- An e-learning platform and school marketplace on web and mobile.
- The proof
- Shift-left testing with product teams; 200+ scenario cross-browser and mobile suites.
−30%post-release defects, with under 2% escaping
2018 – 2022
Prove the platform
Saal.ai · Automation → Senior QA
- The system
- AI-driven enterprise products for healthcare, education, finance, and oil and gas.
- The proof
- Automation frameworks built from scratch, Playwright, Cypress and Appium in CI/CD, and JMeter and k6 performance work.
3 days → 4 hregression cycle time
From 2022
Prove the model
Saal.ai · AI Assurance Architect
- The system
- Big data pipelines, LLMs and retrieval-augmented systems.
- The proof
- Evaluation pipelines as release gates, golden datasets, and an LLM-powered data validation framework.
−40%production hallucination incidents
Today
Prove the agent
Saal.ai · AI Assurance Architect & Product Engineering Lead
- The system
- Autonomous agents, and the AI platforms they run on.
- The proof
- Red-teaming, guardrails, governance and reliability engineering, while architecting and building the products themselves.
2×Best Employee of the Year, 2024 and 2025
What carried over
The tools changed every few years. The habits that make software trustworthy did not.
01
Doubt
Assume the system is wrong until the evidence says otherwise.
02
Edge cases
The failures that matter live at the edges: the injected prompt, the drifting model, the thousandth run.
03
Evidence
Release decisions are made on measured results against agreed thresholds.
Selected work
Problems worth solving.
From practiceReliability
How do you know an AI agent is reliable once it is in production?
Architecting a reliability and observability platform that evaluates agent behaviour in real time and in batch, so failures surface before users find them.
2evaluation planes, real-time and batch, kept separate
From practiceEvaluation
Can you prove a retrieval-augmented system is telling the truth?
An evaluation pipeline across retrieval, chunking, embedding and generation, enforced as release gates, that cut production hallucination incidents by 40%.
40%fewer hallucination incidents in production
From practiceData Quality
What if most manual data checks could validate themselves?
An LLM-assisted validation framework for big data pipelines that automated more than 70% of manual ETL checks.
70%+of manual ETL checks automated
Quality engineering & toolkit
Hands-on across the full testing stack.
Functional and non-functional testing across web, mobile, API and data platforms, now extended to LLMs and agents. These are the tools I build, test and ship with.
- Functional
- Functional, regression, smoke, integration and cross-browser
- Non-functional
- Performance, load, security, penetration and data quality
- Platforms
- Web, mobile, API, data pipelines and LLM-based agents
Web, mobile & API automation
08- Playwright
- Selenium
- Cypress
- Appium
- Rest Assured
- Postman
- TestNG
- Serenity BDD
Performance & load
03- JMeter
- k6
- LoadRunner
Application security
08- OWASP ZAP
- Burp Suite
- Semgrep
- Checkmarx
- Nuclei
- Metasploit
- nmap
- OpenVAS
AI evaluation & red-teaming
04- DeepEval
- RAGAS
- Promptfoo
- LangSmith
Agentic AI
06- LangGraph
- LangChain
- CrewAI
- Agno
- MCP
- Dify
Observability
04- OpenTelemetry
- Langfuse
- Arize
- NVIDIA NeMo
Languages
05- Python
- Java
- TypeScript
- JavaScript
- Node.js
Data & databases
08- Kafka
- PySpark
- Trino
- Airflow
- Great Expectations
- PostgreSQL
- MongoDB
- MySQL
Cloud & DevOps
08- AWS
- Azure
- Docker
- Kubernetes
- Jenkins
- GitHub Actions
- Keycloak
- Jira
Speaking topics
Clear thinking on AI you can prove.
Practical, evidence-led talks for leaders and engineers on evaluating, securing and governing AI.
- 01Release gates for LLM and agentic systems
- 02Red-teaming AI agents: prompt injection, tool poisoning and excessive agency
- 03AI governance in the GCC that speeds decisions up
- 04Reliability engineering for autonomous agents in production
Insights
Notes from the field.
Evaluation7 min read
Beyond accuracy: evaluating agentic AI layer by layer
A practical framework for evaluating agents per layer, choosing metrics and thresholds, calibrating LLM judges against humans, and catching regressions across prompt and model changes.
Read
Performance7 min read
TPC-H benchmarking with Trino · Part 4
Turning a query engine benchmark into a performance regression test
Baselines, noise, run counts, statistical comparison and per-query thresholds for catching Trino performance regressions in CI.
Read
Performance9 min read
TPC-H benchmarking with Trino · Part 3
Reading Trino query plans to find bottlenecks in TPC-H derived queries
How to read EXPLAIN and EXPLAIN ANALYZE in Trino, and what fragments, exchanges, join distribution, dynamic filters and statistics tell you.
Read
Principles to practice
Shipping AI that has to be trusted?
I work with leadership and engineering teams on agentic AI assurance, security and governance, and on the architecture and engineering of AI products, including solution design for enterprise and government programmes across the GCC. I am also open to senior leadership roles.