Safety-firstCustomerAgents

Savvyscale’s Salesforce engineers are outlining the fundamental aspects of building a safe agent that not only responds to customer enquiries, but acts as a salesperson and a client advisor.

Listen to This Post

11 min listen

Date
Read
9 min
  1. Does my business need a customer agent?1 min
  2. When the agent goes rogue..1 min
  3. Building a safety-first customer agent4 min
  4. 1. Picking the right model and platform
  5. 2. Setting the guardrails and access
  6. 3. Monitoring the agent’s behaviour
  7. A note on regulations1 min
  8. Agent evaluation checklist1 min

What’s worse - having no agent on your website or having a poorly performing one?

Even though we are in the age of AI, many businesses still struggle to deploy a good customer service agent. Some businesses - big and small - don’t even have one. This could make them seem unreachable at a time when clients need them, or make them lag behind their competitors.

But what could pose real damage to your brand and business is an agent that is unhelpful, at the very least. And at most, an agent that goes rogue could put the whole business in hot water.

Does my business need a customer agent?

If you’re wondering whether a customer service agent is something your business might need, consider this: Do your clients expect a swift, correct and honest answer to their questions right away? Do your competitors have 24/7 support, built on agents and AI? Do you want to offload your customer service and sales staff so that they can focus on more complex and high-value cases?

If you have answered positively to even one of these questions, then it’s clear that you do need one. Traditionally, an agent could do the following things:

  • Resolve customer complaints
  • Manage order and shipment enquiries
  • Answer product and service questions

The setup behind the agent was pretty straightforward - you set up the so-called “chatbot” with preset questions and answers; if the customer enquiry matched them, then they could be answered in a dry sentence.

In the age of AI, an agent (opens in a new tab) could do so much more. For example:

  • Consider when to escalate to a human
  • Work across many conversations and channels
  • Use conversation history and real-time context
  • Proactively contact customers when there is an issue
  • Cross-sell and promote relevant products and services

And many, many more things that previously were unimaginable. In summary, an agent is programmed to do whatever is needed to complete the task -  whether it is browsing websites, updating data or even logging into systems. However, the setup behind all of this gets ever more complicated, and perhaps this is the reason why so many businesses fail to utilize a proper customer agent.

When the agent goes rogue..

Your team launches its first AI agent. Customers start using it to resolve their issues. You are a regulated business: finance, fintech, maybe legal. Then the agent starts to sound friendly and chatty instead of the strict, professional tone you asked for. You told it to promise as little as possible, yet it promises things you never had in mind, mentions products you never sold, and takes an attitude that doesn't represent you.

Or maybe your team launches a customer assistant agent that can help clients enquire about your financial products, while updating the salespeople’s pipeline. Your salespeople are happy because they have an agent that can assist them and transfer hot leads directly to them. Then one morning your top salesperson, John Smith, finds his pipeline empty. The assistant updated another salesperson’s - named John Smalls - opportunities instead, because it decided that was the right "John".

These two examples can be representative of cases where an agent goes rogue and does things it’s not supposed to do. However, the stakes can get much higher - and even have legal repercussions for your business.

A Singapore financial company’s customer agent, for example, was secretly storing sensitive customer data in another database - over 14,000 records (opens in a new tab) were discovered. This was not an act of an evil sci-fi agent, this was due to the way the agent’s guardrails - or lack of, were set.

So how can you make sure your team gets it right?

Building a safety-first customer agent

An AI agent is only as good as its instructions, data, architecture and, most of all, the knowledge of the team that builds it.

In experienced hands, AI agents stop being an experiment and become a dependable part of your operations, whether you serve businesses or consumers.

Anyone can build an AI agent with the help of an LLM and a paid subscription. Making that agent reliable enough to put in front of your customers is a different story.

Our experienced Salesforce engineers have picked their top 3 things to look out for when setting up your agent.

1. Picking the right model and platform

The model is the first decision you need to make when setting up your agents. Things to consider here are:

  • Instruction-following: Does the model stick to your tone, scope and rules, even when a customer pushes it off script? For a regulated business, this matters more than raw intelligence.
  • Data residency and privacy: Where is customer data processed and stored, and is it used to train the provider’s models? If you operate in the EU, this can decide the shortlist for you.
  • Speed and cost at scale: A model that’s ideal for a handful of complex cases may be too slow or expensive for thousands of routine chats a day.
  • Platform vs. model: You can build directly on a model, or use a customer-service platform that wraps one in ready-made features like handover to humans, analytics and channel integrations. Platforms get you live faster; building on the model gives you more control.

One of the most popular models - Claude by Anthropic - is quite powerful when it comes to research and complex tasks. We have seen multiple use cases where the model is really useful for supporting compliance and fraud reviews, and handling serious client inquiries.

In our experience, OpenAI’s models work well when you need strict instruction-following agents that do the job quickly, which is often the case with customer support agents.

Additionally, you can opt  for a ready-made agent like Fin (opens in a new tab) that can be customized, or have a custom-built one on Salesforce (opens in a new tab), by a trusted Salesforce consultancy. The advantage here is that you get out-of-the box features combined with an agent grounded in your business requirements.

Salesforce is one of the vendors that have out-of-the-box trust and security layers. A great example of this is the Einstein Trust Layer (opens in a new tab), which sits between the enterprise application and any LLMx. By practicing zero-retention and using secure gateways, the Trust Layer adds an additional protection layer to every agentic interaction.

Check the diagram below for more information on its functionalities.

How the Einstein Trust Layer handles prompts and responses
  1. Prompt, from the CRM apps: Secure Data Retrieval, Dynamic Grounding, Data Masking, Prompt Defense.
  2. Secure gateway: Zero Retention, between the Einstein Trust Layer and the models.
  3. Models: Hosted Models, in Salesforce trust boundary; Bring Your Own Models, on your infrastructure; External Models, with shared trust boundary.
  4. Response, back to the CRM apps: Toxicity Detection, Data Demasking, Feedback Framework, Audit Trail.

2. Setting the guardrails and access

Guardrails decide what an agent can say and do. Access level defines what the agent can see and touch. The platform's built-in tools are a good start, but on their own they are not enough. Guardrails come in layers, from basic scope rules, up to the architecture around the agent. Access should be treated with the utmost caution, always applying the least privilege principle - give the agents only the minimal access they need to fulfill their tasks.

One of our senior Salesforce engineers, who has hands-on experience with grounding and delivering agents for financial businesses gave the following advice:

“I've been building agentic solutions for the past two years and working with CRMs for the past ten. As an engineer, I can tell you there aren't enough ready-made AI tools, integrations or documentation yet for me to type one prompt into the most powerful model available, sit back, scroll my phone and click "Always allow" on every request. AI isn't there yet, and nobody knows when it will be.

Until then, you need a specialist who has made enough mistakes in testing environments, and built enough different AI agents, to know what works, what doesn't, and where it isn't worth investing your time and money.”

They continue with a shortlist of guardrails must-haves:

  • Every answer is tied to robust data behind it
  • Swift handover to a human is readily available
  • Each action interaction and behaviour is logged
  • Allowed and forbidden actions and words are responsibly set up

3. Monitoring the agent’s behaviour

Once the model has been chosen and the guardrails have been set, it’s time to begin monitoring what the agent is doing in reality - but first, always in a sandbox.

An agent’s performance should not only be measured on meeting their set goals, but monitoring how they meet them. Having an audit trail makes all the difference when it comes to troubleshooting what the agent did, what their reasoning was, where did they look and what they did.

“Your agentic implementation is as strong as your capability to constantly monitor, analyze and improve the AI agent every day.” - one of Salesforce engineers advice - “Be especially careful when it comes to rogue behaviours such as hallucinations, running in loops and leaking or storing sensitive data. Always version the agents so you can roll back to a previous version that did not misbehave.”

Additionally, you should keep an eye on:

  • Answers that don’t match your approved information, or products and promises you never offered
  • Drift in tone, such as a strict, professional agent gradually becoming chatty
  • How often it escalates to a human, and whether it escalates the right cases
  • Attempts to access data or take actions outside its remit
  • Complaints and negative feedback tied to agent conversations

A note on regulations

Robust, grounded agents are fast becoming much more of a must-have rather than a nice-to-have business requirement. If you’re doing business in the EU, using agents for credit scoring, life or health insurance pricing is classified as “high-risk” as per the EU AI Act. This means a mandatory log, documentation, as well as human oversight is required by law, starting December 2027.

As part of the same legislation, under Article 50, businesses are required to let the customer know they are now talking to an AI agent when the conversation starts.

Other countries aren’t waiting either. South Korea has also launched AI regulations - the AI Basic Act - which came into force in January 2026. It requires businesses to notify customers if their products or services use generative AI.

Additionally, the United Kingdom and Singapore, are also working on AI/agentic regulations, designed to protect customers and ensure that there are legal consequences for how businesses adopt and use AI.

All of the above shouldn’t be scary, it should be welcome. Because as the industry matures, and more regulations are introduced, bad actors will be weeded out and rogue AI agents will, hopefully, become more rare.

However, we are still in the digital workforce’s infancy and we advise all of the businesses that want to either build an agent, or improve their existing one, to reach out to a trusted partner to minimize the risks we outlined.

Agent evaluation checklist

Priority area

Red flag

Green flag

Proactivity

Produces drafts that staff have to re-enter by hand

Completes tasks and updates your systems itself, within set permissions, and asks for confirmation before high-impact actions

System compatibility

Only works after a major system overhaul or cloud migration

Works reliably with your existing CRM, ERP and databases, including older ones

Risk management

Makes decisions no one can see or explain

Follows fixed rules for what it can and can’t do, with clear points where it hands over to a human

Data and security

Unclear where customer data goes, or whether it’s used for training

Data stays where you need it, isn’t retained by the model provider, and access follows least privilege

Governance and traceability

Patchy logs and guardrails that are applied inconsistently

Timestamped, replayable audit trails linked to your policies and procedures

Value and ROI

Success measured in vanity metrics like chat count

Success measured in faster resolution, lower processing costs and increased revenue

We design Salesforce’s agents for what you don’t see coming.