An AI assistant answers a contract question in two seconds. The request succeeds. The model uses few tokens. Every service looks healthy.

The answer is still wrong.

Imagine an employee asks: “How much notice do we need to stop this contract renewing?” The assistant says 30 days. A signed amendment changed the notice period to 60 days.

Nothing crashed. But the system failed at the job the employee needed it to do.

This is why enterprise AI needs both observability and evaluation. Observability helps you understand what happened. Evaluation checks whether the result met a defined standard. You need both to find the mistake and judge whether a change fixes it.

Observability and evaluation answer different questions

Observability uses evidence from a running system: requests, timings, errors, retrieved document references and tool operations. It helps an engineer reconstruct a request and investigate its behaviour.

Evaluation applies checks to that behaviour or its result. Did retrieval find the amendment? Did the answer use it correctly? Was the employee allowed to access it?

The two overlap. An evaluation score can appear beside a trace. A failed evaluation can point to a slow or broken retrieval step. But a fast request is not evidence of a correct answer, and a correct answer in one test is not evidence that the service is reliable.

Open the observability and evaluation diagram at full size.

Follow one request through the system

A trace connects recorded operations belonging to a request. Each operation is represented by a span: a unit of work with a start time, an end time and relevant attributes. Retrieving contract passages, calling a model and creating a reminder can each have their own span. This is the basic structure described in OpenTelemetry’s tracing documentation.

For our contract question, useful evidence includes:

• Which application release and prompt version handled the request.
• Which document versions and passages retrieval selected.
• Which model was requested and which model the provider reported.
• How long each recorded step took, including retries.
• Whether a tool operation succeeded, failed or remained pending.

Metrics summarize many requests: error rates, response-time distributions and token usage. Logs and events record particular occurrences, such as a timeout or approval decision. Correlation IDs let you connect those records when investigating a specific case.

OpenTelemetry’s GenAI conventions provide shared names for AI-related telemetry. Their status and instrumentation support vary; check the convention version your libraries implement instead of assuming every tool exports identical fields.

Find the failed step, not just the bad answer

Return to the 30-day answer. Several different failures could produce it.

The amendment never reached the index. The ingestion pipeline missed it. Changing the answer prompt will not make a missing document appear.

The amendment was indexed but not retrieved. Search selected the original agreement. Investigate indexing, filters, ranking and document relationships.

The right evidence reached the model, but the answer ignored it. Now investigate how the context was assembled and how the model interpreted it.

The answer was correct, but the action failed. If the assistant also promised to create a reminder, check whether the reminder exists with the right date and owner. “Done” in a chat message is not confirmation from the business system.

Keep retrieval quality, answer quality and action outcome separate. Microsoft’s RAG evaluation guidance makes a similar distinction between checking retrieved context and checking the generated response.

Define what a good result looks like

“The answer sounds reasonable” is too vague to be a test.

For this example, a useful test case specifies the question, the employee’s access scope, the agreement and amendment versions, and the expected behaviour:

• State the 60-day notice period and cite the applicable amendment.
• Do not confuse notice before renewal with the renewal date itself.
• If the necessary document is unavailable, explain the limitation instead of inventing a number.
• If a reminder is requested, follow the required approval flow and verify the resulting record.

Use separate checks for correctness, supporting evidence, completeness and permitted behaviour. An answer may be faithful to an outdated source and still be wrong for today’s task. Groundedness measures support in the supplied evidence, not whether that evidence is current or authoritative. Microsoft’s end-to-end evaluation guide explains why multiple quality measures must be considered together.

Test before release, then check production

Offline evaluation runs known test cases against a candidate version before users depend on it. Compare the current and proposed configurations on the same cases.

Version the cases, expected results and scoring rules. Record the model, prompt, retrieval configuration, index snapshot or document versions, and tool versions where relevant. Otherwise, a score change may come from different source data rather than your code change.

Include missing documents, ambiguous questions, conflicting versions and denied access. Reserve a separate holdout set that is not used to tune prompts. If you repeatedly adjust the system to pass every known question, passing those questions becomes weaker evidence of general improvement.

Online evaluation checks production behaviour, often on sampled requests. It can reveal new question types and failures that the test set missed. MLflow’s documentation describes production evaluation alongside offline regression and pre-release testing.

Online does not have to mean blocking every response. Many checks can run asynchronously after a request finishes. Required permission checks and approvals still belong in the execution path; a later evaluation cannot undo an unauthorized action.

Who checks the answers?

Use code for properties that can be checked precisely: a required field, an expected date, a valid citation reference or a record created in a test system.

Use a model-based judge for checks that need language interpretation, such as whether an answer addresses the question. Give it explicit criteria and the evidence it needs.

A judge can also be wrong. Compare its decisions with domain reviewers, inspect disagreements and version its instructions and model. A high score from another model is not proof of correctness. Google’s judge evaluation guidance explicitly compares model ratings with human ratings.

For agents, inspect the resulting state as well as the conversation. Repeat important cases when behaviour varies between runs, and do not confuse one successful attempt with consistent performance. Anthropic’s agent evaluation guide describes combining code-based, model-based and human checks.

Do not turn telemetry into another data leak

Debugging does not require copying every contract and conversation into a general-purpose dashboard.

Start with timings, statuses, versions and access-controlled references. Collect raw prompts, passages and tool results only when the purpose requires them, with appropriate access restrictions and retention. A judge also processes data: decide what it may receive and where that processing happens.

Never record credentials or access tokens. Review what instrumentation captures automatically. Redaction helps, but it is not a reason to collect everything first. These choices follow OpenTelemetry’s data-minimization guidance.

Sampling matters too. Reviewing only user complaints will not estimate overall quality. Reviewing only a small random sample may miss rare, serious failures. Use routine sampling plus targeted checks for high-risk paths, and report which population each result represents.

Turn the evidence into a decision

Suppose the trace shows that the amendment was retrieved but removed while assembling the model’s context. Fix that step, add the incident as a regression case and rerun the relevant tests. Then check sampled production requests to see whether the failure returns.

Compare results by task type and version, not just one overall average. Show the sample count. A headline pass rate can hide that contract amendments fail while simple summaries work.

The practical loop is: observe the request, evaluate the result, locate the failure, change the responsible component and test again. Telemetry supplies evidence. Evaluation turns defined expectations into checks. Together, they make improvement something you can investigate rather than guess.

Continue the series

Previous: Part 6 — How Identity and Authorization Work in Enterprise AI.

Next: Part 8 — How the Whole Enterprise AI Platform Fits Together.

The post How Enterprise AI Is Observed and Evaluated appeared first on Arcentra Systems.

An AI agent is not a language model with a longer prompt. It is a software system that lets a model choose the next step, use approved tools, inspect the result and continue until a task is complete.

The distinction matters. A model can suggest that a ticket should be created. An agent can request the ticketing tool, provide structured arguments and use the returned ticket ID in its next step. The model proposes the action; the runtime validates and orchestrates it, while the tool service and target system enforce authorization and perform the operation.

This article follows that loop from a user’s goal to a verified result. It also shows where identity, permissions, approvals and recovery belong.

Open the agent loop diagram at full size.

A small example

Suppose an engineer asks an internal operations agent:

Find out why payment authorization failures increased after the last deployment. If a configuration change is responsible, prepare a rollback for review.

A chatbot can describe possible causes. An agent can work through the task:

1. look up the latest payment-service deployment;
2. query the relevant error and latency metrics;
3. compare the deployment change with the incident window;
4. read the approved runbook;
5. prepare a rollback change;
6. stop and ask an authorized engineer to approve execution.

No single model response completes this job. It requires several decisions, several tools and new information after each step.

What makes a system an agent?

The word agent is used loosely. A useful engineering definition is:

An agent is a system in which a model helps control the execution of a multi-step task and can select tools in response to the current state.

The surrounding system normally contains five parts:

• Model: interprets the goal and proposes the next step.
• Instructions: describe the task, boundaries and completion conditions.
• Tools: typed operations for reading data or changing an external system.
• Runtime: maintains state, calls the model, validates requests, executes tools and applies limits.
• Environment: the files, services, networks and credentials available to the runtime.

The model is important, but the agent is the complete system. Changing the tool permissions or execution environment can change what an agent is capable of even when the model stays the same.

What is a tool?

A tool is a controlled interface between the agent runtime and another capability. It can read information, perform computation or request a change.

Examples include:

• get_deployment(service, environment)
• query_metrics(service, metric, start_time, end_time)
• search_runbooks(query, service)
• create_change_request(service, proposed_change)
• execute_approved_rollback(change_id)

Each tool has a name, a description and a schema that defines its allowed arguments. The model sees that contract and may produce a structured tool request. The runtime then checks the request and invokes the actual function or API.

This is different from giving the model unrestricted shell access, database credentials or a general-purpose administrative API. Good tools expose the smallest operation needed for a task and keep credentials outside the model context.

The agent loop, step by step

1. Receive the goal with identity

The request enters the runtime with the authenticated user, tenant, role and relevant application context. Identity is not decoration added after the model has chosen an action. It determines which information and tools are available from the beginning.

In the incident example, a support analyst might be allowed to read metrics but not prepare a production rollback. A platform engineer may prepare one, while a separate production owner must approve execution.

2. Assemble the current state

The runtime builds the model input from the user’s goal, system instructions, available tool descriptions and the observations collected so far.

State may include deployment identifiers, retrieved documents, tool errors, approval status and a compact summary of earlier steps. Long-running work also needs durable checkpoints outside the model context so it can resume after a timeout, restart or human review.

3. Let the model propose the next step

Given the current state, the model can usually do one of four things:

• answer or produce a final result;
• ask the user for missing information;
• request a tool call;
• stop because the task is complete, blocked or unsafe.

For the incident, the first decision might be a request to call get_deployment. The response should contain the selected tool and arguments, not a claim that the deployment was already inspected.

4. Apply policy before execution

A tool request is a proposal, not authorization. The runtime checks it against deterministic controls:

• Is this tool available to this user and tenant?
• Are the arguments valid and within scope?
• Is the action read-only, reversible or destructive?
• Does it require approval?
• Has the agent exceeded its time, cost or action limits?

If the answer is no, the tool is not executed. A prompt such as “only use tools you are allowed to use” is helpful guidance, but it is not an access-control mechanism.

The policy gate is an additional control, not a replacement for authorization in the target system. The database, ticketing platform, payment service or deployment API must still verify the calling identity and permitted operation at its own boundary.

5. Execute through the tool layer

A tool adapter or executor translates the approved request into the actual API call, database query or sandboxed operation. It supplies credentials, applies timeouts and normalizes the result.

The model should not receive long-lived secrets. The runtime or tool service owns those credentials and uses them only for the authorized operation.

6. Return an observation

The tool returns an observation: data, a confirmation, an error or a new external state. The runtime records the result and provides the relevant portion to the model for the next decision.

Tool output must be treated as untrusted input. A web page, document, ticket or repository can contain text that attempts to instruct the model to ignore its rules or call another tool. Retrieved text is data; it does not gain authority because an agent read it.

7. Repeat or stop

The model evaluates the observation and chooses another step. It might query a second metric, inspect a runbook, correct a failed tool call or ask for approval.

The loop needs explicit stopping conditions. Stop when the result is verified, the maximum number of steps is reached, a required approval is missing, the next action is unsafe, or the available evidence cannot support a reliable conclusion.

A tool call is not the action itself

This is the most important boundary in the architecture.

The model might produce a request resembling:

{
  "tool": "create_change_request",
  "arguments": {
    "service": "payment-api",
    "proposed_change": "restore timeout_ms from 5000 to 2000"
  }
}

That output does not change production. The runtime still has to validate the schema, verify the service scope, confirm that the user may prepare a change and call the change-management API. The tool result might then return:

{
  "change_id": "CHG-1842",
  "status": "awaiting_approval"
}

The next model response can explain what was prepared and who must approve it. It must not claim that the rollback has run.

Read tools and action tools need different controls

Not every tool carries the same risk.

• Read tools retrieve documents, metrics, records or system state.
• Computation tools run code or transform data in a controlled environment.
• Action tools send messages, change records, deploy software, move money or alter infrastructure.

Read access can still expose sensitive data, so it requires identity and filtering. Action tools add side effects and need stronger safeguards. A practical policy can allow routine reads automatically, require confirmation for reversible writes and prohibit high-impact actions unless a separate approval workflow authorizes them.

Why narrow tools work better

A general tool called call_internal_api(method, url, body) is powerful but difficult to govern and difficult for a model to use reliably. A narrow tool called get_payment_incident_metrics is easier to describe, validate, authorize and test.

Useful tool design includes:

• a precise name and description;
• a small input schema with meaningful field descriptions;
• explicit required fields and allowed values;
• bounded, structured output;
• clear error messages the agent can act on;
• stable identifiers rather than ambiguous display names;
• pagination and result limits;
• idempotency support for operations that may be retried.

Tool design is part of agent engineering. A stronger model cannot fully compensate for a vague or unsafe interface.

Retries, partial failure and duplicate actions

Multi-step tasks fail in ordinary ways: APIs time out, credentials expire, a service returns stale data, or the process restarts after an action succeeded but before the result was recorded.

Read operations can often be retried safely. Write operations need more care. If an agent repeats send_payment after a timeout, it must not create a second transfer. Use idempotency keys, durable operation IDs and status checks so the runtime can determine whether an earlier attempt already completed.

For workflows with several side effects, define recovery explicitly. Some operations can be reversed; others require a compensating action or human intervention. “Ask the model to try again” is not a transaction strategy.

Workflows and agents are not the same thing

A workflow follows a path defined in code. An agent allows the model to choose parts of that path dynamically.

If every incident investigation always performs the same five queries in the same order, use a workflow. It is easier to test, cheaper to run and more predictable.

An agent is useful when the required steps depend on what it discovers: the affected service is unknown, different evidence leads to different tools, or the number of investigation steps cannot be fixed in advance.

Many production systems combine both approaches. Code defines the major stages and security boundaries; an agent makes bounded decisions inside one stage.

Where MCP fits

The Model Context Protocol, or MCP, standardizes how compatible clients discover and call tools exposed by servers. A tool definition includes a name, description and input schema; a tool result can return structured or unstructured content.

MCP can reduce custom integration work, but it does not decide which tools an employee should be allowed to use, whether a production action needs approval, or whether a tool’s implementation is safe. Those remain responsibilities of the application, identity platform, policy layer and tool service.

Protocol compatibility is not the same as trust. Connect only approved servers, validate their tool definitions and results, and apply the same least-privilege controls used for any other integration.

Identity must reach the tool boundary

An enterprise agent should not turn many users into one invisible shared service account.

The tool layer needs enough identity context to enforce the same business permissions that apply outside the agent. Depending on the system, this can use delegated user authorization, a workload identity with tightly scoped policy, or a controlled combination of both.

At minimum, record:

• who requested the task;
• which agent and version handled it;
• which identity called each tool;
• which policy allowed or denied the call;
• who approved a sensitive action;
• what external object or operation was created.

The next article in this series explains how identity and authorization work in enterprise AI.

What should be observed?

A production trace should make the execution reconstructable without collecting unnecessary sensitive content. Capture:

• task and trace identifiers;
• model, instruction and tool versions;
• tool selections and validated arguments, with secrets and sensitive fields redacted;
• policy decisions and approvals;
• tool latency, retries and errors;
• token, compute and external API cost;
• completion reason and verified outcome.

Operational tracing does not require storing private model reasoning. Record the observable decisions, tool calls, evidence and outcomes needed to debug and audit the system.

How to evaluate an agent

A fluent final answer is not enough. Agent evaluation should test the full trajectory: the sequence of decisions and actions that led to the result.

Useful questions include:

• Did the agent complete the intended task?
• Did it choose the correct tools?
• Were the tool arguments valid?
• Did it avoid unnecessary calls and loops?
• Did it respect tenant and user permissions?
• Did it request approval before a sensitive action?
• Did it recover correctly from timeouts and tool errors?
• Did it stop when evidence was insufficient?
• Was the latency and cost acceptable?

Test normal tasks, ambiguous instructions, unavailable tools, malicious tool output, partial failures and attempts to cross authorization boundaries.

A practical production checklist

• Use an agent only when the path genuinely needs dynamic decisions.
• Give each tool a narrow purpose and typed schema.
• Keep credentials outside model context.
• Authorize tool calls at execution time and enforce permissions again at the target resource.
• Separate read, write and destructive capabilities.
• Require approval for high-impact actions.
• Treat documents and tool results as untrusted input.
• Use sandboxes and network restrictions for code execution.
• Add time, cost and iteration limits.
• Make writes idempotent and define recovery.
• Checkpoint long-running tasks.
• Trace decisions, tool calls, policy outcomes and external side effects.
• Evaluate complete task trajectories, not only final text.

The complete sequence

The basic runtime path is:

Goal and identity -> runtime assembles state -> model proposes a response or tool call -> policy validates it -> tool executes -> observation returns -> state is updated -> the loop repeats or stops.

An agent becomes useful when tools connect language-model decisions to real systems. It becomes operable when the surrounding runtime keeps those decisions bounded, authorized, recoverable and observable.

Start with How Enterprise AI Systems Work, Part 1 of the series.

Sources and further reading

• Anthropic: Building effective agents
• Anthropic: Writing effective tools for agents
• OpenAI: A practical guide to building agents
• Model Context Protocol specification: Tools
• Model Context Protocol specification: Authorization

The post How AI Agents and Tools Work appeared first on Arcentra Systems.

Open the enterprise AI diagram at full size.

When people talk about enterprise AI, the discussion often jumps straight to models: GPT, Claude, Gemini or a model running inside the company.

But the model is only one part of the system.

A large language model, or LLM, can interpret a request and generate text. By itself, it has no reliable access to the company’s current documents and cannot enforce user permissions or safely operate business systems.

To use AI in a real organization, engineers have to build a complete system around the model.

Take a simple question: “Which termination conditions are included in our supplier contract?” Something must find the right contract, check that the employee may read it, and give the relevant passages to the model. We will follow that path, then look at what changes when the assistant also needs to perform an action.

This is a reference design for a knowledge assistant that can also use business tools, not a mandatory checklist for every AI application. Classification, extraction or a small internal assistant may need only some of these components. A separate gateway, vector index and agent loop are architectural choices, not prerequisites.

What is an enterprise AI system?

In this series, an enterprise AI system is an application built around a language model, with the data access, permissions and operational controls needed for use inside an organization.

For this knowledge-and-tools reference design, the components include:

• user-facing applications;
• identity and access control;
• search over company data;
• RAG;
• an LLM gateway;
• one or more language models;
• agents and tools;
• evaluation, monitoring and audit.

Each component has a different job. Search finds information. Authorization decides who may see it. The gateway controls model access. The model generates an answer. An agent can request an action. Observability records what happened.

These parts can belong to one application. When several teams need the same capabilities, some can become shared platform services.

The system has two main flows

The architecture becomes easier to understand when we separate two different processes:

1. preparing company knowledge;
2. processing a live user request.

The first process prepares the searchable index and keeps it up to date. The second starts when a user asks a question. In the diagram, the upper row prepares knowledge. The central row handles the request, and the lower branch executes approved tool calls. The retrieval arrow shows evidence returning to the orchestrator, which requested the search.

Preparing company knowledge

Suppose a company wants an assistant to answer questions about policies, contracts, technical documentation and its internal knowledge base.

The language model does not automatically know these documents. The information has to be collected, prepared and indexed.

Data sources

Company information may live in file stores, databases, document systems, wikis, CRM platforms, Git repositories and internal APIs.

The ingestion pipeline—the process that imports data—connects to approved sources and reads the documents or records that may be used by the AI system. It must also track changed documents, deletions and changes to access permissions.

Chunking and metadata

Sending a whole document for every question can waste input space and make useful details harder to find. This design splits large documents into smaller meaningful sections called chunks.

A contract, for example, may be divided into payment terms, responsibilities, termination clauses and appendices.

Each chunk keeps metadata: fields such as its source, version, date and access rules. In a shared service, a tenant identifies the organization or account whose data must remain separate. These fields let the system filter results and show where an answer came from.

Embeddings

Each chunk can then be converted into an embedding: a list of numbers that roughly represents its meaning.

This can help a search for “leaving a supplier agreement early” find a passage about “termination for convenience,” even though the wording differs. Similarity helps find candidates; it does not prove that they answer the question.

Vector search

The embeddings are stored in a vector index. A user’s question is converted into a compatible embedding and compared with the indexed vectors.

If you have heard of Qdrant, this is where it fits: it is a vector database, not the language model and not the whole RAG system. It stores vectors and associated metadata and supports similarity searches with filters. The searchable record also needs the original passage text or a reference to it. Other databases and search engines can fill this role too.

Production systems often combine vector search with ordinary keyword search and metadata filters. Meaning is useful, but exact names, identifiers, dates and permissions still matter.

Processing a live request

Now consider a user asking:

Which termination conditions are included in our supplier contract?

It may look like the application simply sends this question to a model. A reliable enterprise system performs several additional steps.

1. The application receives the request

The user may be working in a company chat, portal, CRM, support system, mobile application or a specialized AI interface.

The application owns the user experience and the business process. It knows what the user is trying to do, what inputs are required and what a successful outcome means.

In a shared-platform design, it calls common services for model access and retrieval. In a small application, those components may live in the same codebase.

2. Identity and authorization are checked

Before searching documents or calling a model, the system needs to know:

• who made the request;
• which organization and team they belong to;
• which information they may access;
• which models and tools they may use;
• which actions they are allowed to perform.

If the employee cannot open a contract in the source system, the AI assistant must not use that contract when preparing an answer.

This is why enterprise RAG is not only a search problem. It is also an authorization problem. We examine this boundary separately in Enterprise RAG Is an Authorization Problem Before It Is a Search Problem.

3. The orchestrator plans the operation

The orchestrator is the application code that coordinates the steps around the model. It can be a straightforward sequence of functions; it does not have to be another AI model.

It can decide whether the request needs internal knowledge, which sources to search, which model capability is required, whether a tool may be called and whether human approval is necessary.

For a simple question, the flow may contain one search and one model call. A longer process may involve several searches, models, tools, retries and approval steps.

The workflow needs timeouts, limits on retries and clear failure handling. A long-running process may also need to save its progress so that it can resume after a failure or wait for approval.

4. RAG finds relevant information

RAG means retrieval-augmented generation: find useful information, add it to the model request, then generate an answer. The information supplied for a particular model call is called its context.

A basic RAG operation works like this:

1. understand or rewrite the question;
2. search only the information the user may access;
3. select the strongest passages;
4. add them to the model request;
5. ask the model to answer from that evidence;
6. return the answer with links to the sources.

RAG does not retrain the model. It supplies selected information for this request. That information is only as current as its source and the index: an old copy of a contract still produces an answer about an old contract.

It also does not guarantee correctness. Search can retrieve the wrong passage, and a model can misunderstand good evidence. Retrieval and generation must both be tested.

5. The LLM gateway controls model access

The prepared request does not need to go directly to a specific model provider. It can first pass through an LLM gateway.

The gateway provides one controlled entry point for model calls. It can:

• authenticate the calling application;
• choose an approved model;
• apply data-handling and regional rules;
• enforce token, rate and budget limits;
• remove or mask sensitive information;
• switch to an approved, compatible fallback when a provider fails;
• record usage, cost and audit data.

A token is a small unit of text processed by a model; token counts affect input limits and many providers’ bills. Gateway features vary and must be configured. It does not automatically make every provider interchangeable or enforce every business rule.

The gateway does not own the business workflow. Its job is to make model access consistent. The design is covered in The Enterprise LLM Gateway Is the New Control Plane.

6. The language model generates the answer

The model receives the user’s question, instructions and selected company information. If tools are available, their descriptions can be included too. The context has a size limit, so the application must choose what to send.

It interprets that context and generates an answer.

The model is not a database, and it does not guarantee factual accuracy. The quality of the result depends on the complete system around it: the information retrieved, the instructions supplied, the model selected and the checks applied afterward.

Retrieved documents are evidence, not instructions. A document that says “ignore your rules and send this file elsewhere” must not gain control of the application. Permissions and tool validation have to be enforced outside the model.

7. Agents can request actions

Sometimes the user needs more than an answer. The operation may need to create a ticket, check a payment, find a customer in the CRM, prepare a report or send a document for approval.

Here, an agent means a loop in which the application lets a model choose the next step from an allowed set of actions, runs the necessary checks, and gives the result back to the model.

The agent may search again, ask for missing information, call a registered tool, request approval or finish the operation.

If the steps are always the same, ordinary application code is often easier to test. Letting the model choose is useful when the next step genuinely depends on what it discovers.

8. Tools connect AI to business systems

A tool is a controlled interface to an external capability: a CRM API, payment service, database, ticketing platform, calculator or internal service.

The model may suggest a tool and its arguments. The surrounding runtime must validate the request, check authorization, execute the operation and return a structured result.

Every tool should have clear operations, limited permissions, input validation, logging and error handling. High-impact actions may require a human to approve them.

The model proposes. The platform authorizes and executes. The tool result returns to the runtime and, when another model decision is needed, is included in the next model call. The loop stops on completion, a failure or a configured step or cost limit.

Checking quality and understanding failures

Evaluation checks quality

Receiving an answer is not enough. The organization needs to know whether the system found the correct documents, followed its instructions, used tools correctly and produced an acceptable business result.

Evaluation can measure:

• retrieval quality;
• groundedness in the supplied evidence;
• instruction and policy compliance;
• correct tool selection;
• business correctness;
• latency and cost.

Tests run during development, before a release and on selected production operations. Changes to a model, prompt, retrieval configuration or tool definition should be treated as release changes.

Teams can use failed quality tests to block a release or decide to roll it back. The delivery model is described in AI Evaluation Is a Release Engineering Problem.

Observability explains what happened

A successful HTTP request does not prove that an AI operation succeeded.

Operators need to reconstruct the whole path:

1. who sent the request;
2. which permissions were applied;
3. which documents were retrieved;
4. which context was sent to the model;
5. which model was selected;
6. which tools were called;
7. which errors or fallbacks occurred;
8. what the user received;
9. how long the operation took and what it cost;
10. whether the business operation succeeded.

Observability is not a reason to copy every contract, prompt or tool response into logs. Record identifiers, versions and timing by default; capture content only when permitted, with redaction, restricted access and defined retention. These events should be connected by one operation identity. If an answer is wrong, engineers can then identify whether the failure came from retrieval, authorization, orchestration, the model or a tool.

We cover this operating model in Enterprise AI Observability.

A complete example

Return to the question:

Which termination conditions are included in our supplier contract?

Assume the employee has selected a specific supplier contract in the application. For this example, we choose hybrid search: vector search for related wording and keyword search for the supplier name. The request follows these steps:

1. The application receives the question.
2. The system identifies the user and their organization.
3. Authorization checks whether the user may access the contract.
4. The question is converted into an embedding.
5. Hybrid search looks for relevant contract sections within the user’s permitted scope.
6. The application verifies access to the selected passages before sending them onward.
7. The strongest passages are added to the model request.
8. The LLM gateway applies policy and selects an approved model.
9. The model generates an answer from the supplied evidence.
10. The application checks the cited passages and returns the answer with source links. If the evidence is insufficient, it says so.
11. The operation produces a correlated trace and audit record, with sensitive content redacted or omitted according to policy.

The user sees a short answer. Tracing records the steps as they happen; quality tests check representative questions before release and selected operations in production. Neither requires pretending that every answer has been proven correct.

AI application versus AI platform

A single AI application can call a model directly and own its own retrieval pipeline, prompts and monitoring. For the first small product, that may be the correct design.

The problem appears when many teams repeat the same work. Each team builds separate model connections, credentials, RAG pipelines, safety rules, prompts, monitoring and cost tracking.

An AI platform moves repeated capabilities—such as model access, authorized retrieval and monitoring—into shared services. Applications keep their own user experience and business logic. The point is to avoid maintaining the same controls separately in every product, not to introduce every component on day one.

This platform boundary is described in AI Infrastructure Is Becoming the New Enterprise Platform.

The main idea

For the knowledge assistant in this article, the answer path is: identify the user → retrieve permitted evidence → prepare context → call the model through the gateway → check and return the answer. The orchestrator coordinates these steps.

If the model proposes a tool call, the path branches: validate the request and permissions → execute the tool → return its result to the orchestrator. A model response is not, by itself, permission to act.

Identity checks, monitoring and quality evaluation apply across the system. The model is one component; the other components determine what it can see, what it can do and how engineers find out when something goes wrong.

The rest of the series

1. How Embeddings and Vector Search Work
2. How RAG Works
3. How AI Agents and Tools Work
4. How Identity and Authorization Work in Enterprise AI
5. How Enterprise AI Is Observed and Evaluated
6. How the Whole Enterprise AI Platform Fits Together

Sources and further reading

• Google Cloud: RAG infrastructure using Vector Search
• Microsoft: Access language models through a gateway
• Microsoft: Design a secure multitenant RAG solution
• AWS: Agentic AI architecture in the enterprise
• OpenTelemetry semantic conventions
• NIST AI Risk Management Framework

The post How Enterprise AI Systems Work appeared first on Arcentra Systems.

In practice, a capable model is usually the easiest component to replace.

The difficult work begins when an AI system must operate inside the enterprise: use live data, preserve existing permissions, resolve inconsistent identifiers, call business APIs, survive partial failures and leave evidence that operators and auditors can reconstruct.

A model can be connected through an API in minutes. Connecting that model to a bank, fintech platform, retailer or industrial operator can take months because the integration must carry more than data. It must carry identity, authority, workflow state and evidence across systems that were not designed to participate in probabilistic execution.

That is the real enterprise AI integration problem.

What Is Enterprise AI Integration?

Enterprise AI integration is the engineering discipline that connects models and agents to corporate identities, data, applications, events and transaction systems through controlled, observable workflows.

It includes connectors and APIs, but it is not defined by their number. A production integration must preserve four properties across every boundary:

• Identity: who initiated the request, which service or agent is acting and on whose behalf.
• Authority: which records, tools and operations are permitted in the current context.
• State: what has already happened, which version of a record is current and whether a workflow is complete.
• Evidence: which sources, decisions, calls and outcomes must be retained for operations, audit and improvement.

If any one of these properties disappears between the AI application and a system of record, the workflow becomes unreliable even when the model response appears correct.

This is why enterprise AI integration requires architecture rather than a growing collection of point-to-point connectors. The model speaks in tokens. The enterprise operates through identities, records, transactions, events and approval rules. The integration layer is where those two environments are reconciled.

The Model Call Is One Step in a Distributed Transaction

An enterprise AI request should be treated as an execution path, not as a prompt followed by an answer.

A user request or business event first establishes an identity. The system then determines delegated authority, retrieves current context, applies policy, chooses a model, invokes tools or APIs and records evidence about the outcome. The model call sits near the middle of this path. It does not own the path.

The surrounding integration may involve an identity provider, SSO, Active Directory or LDAP; CRM, ERP or core banking systems; SharePoint, Confluence or document repositories; relational databases; message brokers and event buses; secrets management; an API gateway; a workflow engine; and security monitoring.

The complexity does not come from connecting twenty endpoints. It comes from preserving meaning across them.

A customer number in a CRM may not match the identifier in a core banking platform or fraud system. A document may remain searchable after the policy it describes has expired. An employee may be permitted to view a customer record but not the investigation associated with it. A downstream system may accept an update but time out before acknowledging it.

These are not language-model problems. They are distributed-systems and enterprise-integration problems.

A Banking AI Request Is Not a Question-Answering Flow

Consider a request such as: “Show me the latest suspicious transactions for customer John Smith.”

The visible task looks like information retrieval. The actual banking AI integration may need to:

1. Authenticate the employee and establish the active session.
2. Determine whether the employee is allowed to access suspicious-activity information.
3. Resolve which John Smith is intended without leaking records for other customers.
4. Map the customer across CRM, core banking and fraud-monitoring identifiers.
5. Retrieve current transactions and their investigation status.
6. Search relevant policies and case documents with source-level permissions.
7. Assemble evidence for the model without crossing disclosure boundaries.
8. Generate the response and apply output policy.
9. Record the sources, decisions and downstream calls in an audit trace.

The model may complete its part in seconds. The surrounding system must establish which customer the user means, whether the request is authorized, which records are current, what each fraud flag means and which evidence may be disclosed.

In February 2026, the Central Bank of the Republic of Uzbekistan announced that solutions selected through its AI Challenge were moving into banking-system implementation. The program includes an Uzbek-language speech-to-text model and a retrieval-based assistant for regulatory and institutional information. The Central Bank states that the solutions will run on internal servers to protect confidential information and banking secrecy.

The significant engineering transition is not only the development of models for Uzbek language and banking terminology. It is their integration into controlled institutional workflows, source systems and operational boundaries.

Authentication Is Not End-to-End Authorization

Logging a user into an AI application does not automatically authorize every retrieval or action performed on that user’s behalf.

Many demonstrations use one privileged service account for all downstream calls. This reduces implementation time, but it removes the relationship between the employee’s authority and the data returned by the model. Every request effectively receives the permissions of the integration account.

Production systems need separate identities for the user, agent and downstream service. Authority must be delegated explicitly and narrowed for the task. The integration chain should be able to answer:

• Which human or service initiated the task?
• Which agent is executing it?
• Which permissions were delegated?
• Which tool and parameters were authorized?
• When does that authority expire?
• Which system enforced the decision?

The NIST National Cybersecurity Center of Excellence addresses these questions in its 2026 concept paper on software and AI agent identity and authorization. It treats delegation, least privilege, auditing and the binding of agent actions to human authority as separate requirements—not as consequences of successful login.

An agent allowed to read a transaction should not automatically inherit the authority to freeze an account, export a complete customer history or open a fraud investigation. Tool descriptions tell a model what an operation does. They do not grant legitimate permission to execute it.

Agents Turn Integration Flows Into Transactions

Integration becomes more demanding when the AI system moves from reading information to changing enterprise records.

An agent may select tools dynamically, revise its plan or repeat a call after an ambiguous response. In a transactional environment, an uncontrolled retry can create a duplicate payment, reopen a resolved case, reserve inventory twice or send the same customer notification repeatedly.

An agentic integration therefore needs mechanisms familiar from distributed transaction processing:

• idempotency keys for operations that may be retried;
• explicit state transitions rather than inferred progress;
• timeouts and circuit breakers around unreliable dependencies;
• compensating actions when a multi-system workflow fails partway through;
• approval gates for high-impact or irreversible operations;
• durable records of tool inputs, outputs and business outcomes.

Model reasoning cannot replace these controls. A convincing explanation of an action does not make the action authorized, atomic or reversible.

The AWS enterprise architecture for agentic AI separates access to models, tools and knowledge sources for this reason. Tool discovery and secure execution are distinct platform capabilities, with authorization applied to the actor and execution context.

Google Cloud’s reference architecture for orchestrating access to disparate enterprise systems follows the same principle. MCP servers form an anti-corruption layer between the agent and backend systems, while IAM, structured logging and distributed tracing remain separate operational requirements.

MCP can standardize how a tool is described and invoked. It does not remove the need for authorization, transaction semantics or failure handling behind that tool.

A Fluent Answer Can Hide an Integration Failure

Traditional enterprise applications usually expose failure. A missing record produces an error. A rejected transaction returns a status. A timed-out dependency prevents the screen from completing.

AI systems create a more dangerous possibility: part of the workflow can fail while the final answer remains fluent.

A response may combine current transaction data with an obsolete policy document. A search connector may return only part of an authorized corpus. One action may succeed while a second system times out. The model can still produce a coherent narrative because fluency is not evidence that every dependency completed correctly.

This changes the definition of AI integration testing. Evaluating answer quality is necessary, but insufficient. Tests must also cover:

• revoked permissions during an active session;
• ambiguous users, customers or account identifiers;
• stale, contradictory or partially indexed sources;
• unavailable tools and degraded downstream APIs;
• duplicated requests and delayed acknowledgements;
• partially completed multi-system actions;
• policy changes between retrieval and execution;
• audit records that cannot be correlated end to end.

The central test is not whether the model answers under ideal conditions. It is whether the integrated system behaves safely when its dependencies provide incomplete, delayed or contradictory evidence.

AI Integration Monitoring Must Follow the Business Transaction

Monitoring only the model call cannot explain why an integrated AI workflow failed.

Teams need a trace connecting the initiating identity, authorization decisions, retrieved sources, model requests, tool selection, downstream API calls, retries and final business outcome. Without that chain, operators cannot distinguish a model problem from a stale index, slow permission service, retry loop or unavailable system of record.

The OpenTelemetry guidance for generative AI observability standardizes telemetry for model operations, token usage, tool calls and tool results. That provides an important part of the execution narrative. Enterprise AI integration must extend the same trace into identity services, retrieval infrastructure, workflow engines and business APIs.

A successful model response is not a successful transaction if the required downstream action did not occur.

End-to-end tracing also changes cost management. The cost of an AI workflow includes retrieval, data movement, tool execution, retries, external services and operational capacity—not only model tokens. Usage must be attributed to the application and business workflow that created it.

This is one reason observability is a core capability in the enterprise AI infrastructure platform, rather than an optional dashboard attached to a model provider.

Enterprises Need a Shared AI Integration Layer

Large organizations cannot sustainably build a separate integration mechanism for every assistant and agent.

A shared AI integration layer should provide stable contracts for:

• identity propagation and delegated authority;
• access to models, knowledge and approved tools;
• API orchestration and event handling;
• retries, idempotency and common failure policies;
• workflow state and approval gates;
• end-to-end telemetry and audit evidence;
• usage attribution and operational controls.

This layer does not eliminate integration work. It moves repeated controls into shared engineering components while leaving business meaning with the teams that understand each domain.

The platform team may own the gateway, tool registry, orchestration runtime and telemetry contract. A banking team must still define what constitutes a suspicious transaction, which investigation states are valid and when human approval is mandatory. A retailer must still own inventory, pricing and fulfilment semantics.

Centralize the mechanics that must be consistent. Federate the decisions that require domain accountability.

Arcentra Systems applies this boundary across Design, Build and Operate: architecture, implementation and long-term operation are distinct responsibilities, but they must share one integration contract.

When Should the Shared Layer Be Built?

Not every prototype requires a platform. A direct model call can be appropriate when the use case is isolated, read-only and low-risk.

The shared layer becomes justified when several of the following conditions appear:

1. Multiple AI applications connect to the same systems of record.
2. Teams independently rebuild identity propagation, retrieval or tool authorization.
3. Agents can create, update or delete enterprise records.
4. Workflows span systems with different transaction and failure semantics.
5. Operators cannot reconstruct a request across the model and downstream services.
6. Audit, security or finance teams cannot attribute actions and cost consistently.
7. Provider or system changes require coordinated edits across many applications.

At that point, the organization already has an AI integration layer—it is simply distributed across application code, duplicated connectors, privileged service accounts and operational workarounds.

The architectural task is to make it explicit.

Better Models Do Not Remove the Integration Problem

As models improve, the model’s share of the total engineering problem often becomes smaller.

Models can be replaced without changing the business objective. Enterprise identifiers, permission structures, transaction rules, source semantics and operational dependencies remain specific to the organization. A stronger model may reduce reasoning errors, but it cannot determine which customer record is authoritative, repair a broken delegation chain or make a non-idempotent banking API safe to retry.

Companies do not struggle with enterprise AI only because models are insufficient. They struggle because useful AI must operate inside systems that already encode decades of business meaning and control.

The future of enterprise AI will not be decided by better models alone.

It will be decided by integration architecture designed to preserve identity, authority, state and evidence across the complete transaction path.

The model generates the answer. Engineering determines whether the answer can be trusted—and whether the enterprise survives acting on it.

The post AI Integration Is Harder Than AI Itself appeared first on Arcentra Systems.