Multi-Agent AI for Salesforce Data Workflows

Revolutionizing Enterprise Data Workflows with a Multi-Agent AI System

This article details Informatica's approach to building CLAIRE, a multi-agent AI system designed to automate and streamline complex enterprise data management workflows within their Intelligent Data Management Cloud (IDMC).

Historically, tasks like data discovery, governance, quality assessment, and pipeline orchestration could take up to three months of manual effort. Informatica's CLAIRE system addresses this by employing a multi-agent architecture, reducing these workflows to mere days with a reported 90% task success rate.

The Mission: Unified and Automated Data Management

Informatica's core mission was to create a multi-agent AI system that transforms enterprise data management by making complex workflows intuitive and automated. This is achieved by embedding CLAIRE directly across IDMC, orchestrating capabilities such as discovery, classification, data quality, and pipeline execution through a unified interface.

The system features specialized agents designed to interpret user intent, operate across heterogeneous systems, and execute multi-domain workflows. To enhance user interaction, two models were introduced:

• CLAIRE GPT: A generalist interface for cross-domain requests.
• Specialized Copilots: Embedded within individual products for contextual assistance.

This architecture predates the common industry term "agents" and was driven by the inherent complexity of enterprise data workflows that demanded autonomous, coordinated execution.

Challenges with Unified Systems and Single-Agent AI

Scaling unified systems for enterprise data workflows proved unworkable due to the fragmented nature of discovery, governance, data quality, and pipeline orchestration. These stages often operate across disconnected systems with no single execution layer, leading to fragile chains, latency, and context loss during handoffs.

Single-agent AI systems encountered limitations in handling enterprise workflows due to their inability to simultaneously manage:

• Context-heavy reasoning: Difficulty in maintaining and applying complex contextual information across multiple steps.
• Diverse toolchains: Inability to reliably select and utilize different tools for distinct tasks (e.g., profiling, statistical analysis, code execution).
• Multi-step execution: Problems with consistency and accuracy when performing sequential operations.

Early iterations exposing multiple tools to a single agent resulted in incorrect tool selection, exceeded context limits, and inconsistent outputs. Specialization into multiple agents with focused responsibilities was identified as the necessary path forward.

Designing the Multi-Agent Architecture

The multi-agent system design centered on three core challenges:

1. Orchestration: An orchestration agent acts as the control plane, responsible for intent detection, plan generation, and routing execution to specialized agents.
2. Planning: A planning layer allows the orchestration agent to generate high-level plans that users can review and modify, balancing adaptability with predictability.
3. Agent Specialization: Each specialized agent operates within a constrained context and optimized toolset, enhancing accuracy and efficiency (e.g., a data quality agent handling profiling, rule recommendation, and cleansing).

Deterministic tool routing logic was implemented to ensure agents invoked the correct tools based on intent and context, stabilizing execution.

Execution Reliability and Dependency Management

Coordinating multiple agents introduced distributed system challenges:

• Execution Reliability: Ensuring consistent and accurate outputs across chained workflows involving numerous model calls.
• Dependency Management: Managing the intricate dependencies where the output of one agent serves as the input for another.
• Failure Propagation: Preventing single errors from cascading and causing complete workflow failure.

To mitigate these, Informatica implemented:

• Validation Checkpoints: Intermediate steps that validate inputs and outputs.
• Strict Data Contracts: Defined interfaces between agents.
• Guardrails: Mechanisms to detect anomalies early.
• Adaptive Planners: To handle dynamic user modifications and recompute dependencies on the fly.

Beyond interactive workflows, the system supports background and headless agent execution for scheduled or event-driven tasks like data quality assessments.

Context Modeling and Semantic Understanding

Interpreting open-ended enterprise data requests required robust context modeling and semantic understanding. The system addresses this through a semantic layer that performs:

• Entity Resolution: Identifying and linking related entities.
• Intent Decomposition: Breaking down user intent into actionable steps.
• Metadata Enrichment: Augmenting data with relevant contextual information.

This ensures each agent receives precise context, avoiding issues related to too much or too little context affecting performance and accuracy.

Production Readiness: Accuracy, Grounding, and Reliability

Preparing the system for production involved rigorous validation of accuracy, grounding, and reliability. Key strategies included:

• Agent-Specific Validation Frameworks: Moving beyond generic AI metrics to tailored evaluations for each agent's function (e.g., code execution correctness for cleansing, deduplication accuracy for rule generation).
• Outcome Metrics: Measuring the successful completion of agent actions end-to-end, rather than just model token correctness.
• LLM-as-a-Judge Scoring: Utilizing AI for evaluation.
• Reflection-Based Validation: For generated code.

These efforts led to significant improvements, with the data quality agent achieving a 90% task success rate, 98% grounding accuracy, and a 1% hallucination rate. Reliability was further bolstered by validation layers, guardrails, and horizontally scalable infrastructure.

Key Takeaways

• A multi-agent AI architecture effectively decomposes complex, multi-stage enterprise data workflows.
• Agent specialization enhances accuracy, efficiency, and reliability compared to single-agent systems.
• Robust orchestration, planning, and inter-agent communication are critical for managing dependencies and preventing failure propagation.
• Dedicated semantic layers and context modeling are essential for interpreting open-ended user intents.
• Agent-specific outcome metrics and rigorous validation frameworks are paramount for production readiness and demonstrable AI performance.