What This Update Actually Is
Batch Testing is a structured QA tool built directly into HubSpot's Customer Agent. Admins can create a set of test questions, run the agent against all of them at once, and review every response in a single results view.
For each response, reviewers mark it "Looks good" or "Looks bad." Marking something bad immediately surfaces a "Coach agent" button and an optional notes field. That means you can fix the agent on the spot or leave a note for a teammate who holds coaching permissions.
The loop closes when you re-run the test after coaching and confirm the improvement. It's not a one-time audit. It's a repeatable pre-deployment review cycle.
Why HubSpot Shipped This
Here's the real problem. Before this update, there was no structured way to validate Customer Agent behavior before pushing changes live. Admins were essentially flying blind.
The internal frustration is real and familiar. You coach the agent, cross your fingers, and find out if it worked when a real customer gets a bad answer. That's not a quality process. That's a gamble.
HubSpot shipped Batch Testing to give admins confidence before rollout and a structured feedback loop for continuous improvement. For teams running AI agents at any real volume, that's the missing piece.
This pairs naturally with HubSpot's recent move to automatically classify Help Desk and Customer Agent conversations by topic. That update surfaces the themes. Batch Testing lets you turn those themes into test questions and validate agent performance against them.
How to Use It Step by Step
- Create your test set: Add questions manually or let HubSpot generate them from themes. Themes are automatically surfaced from recent Inbox or Help Desk conversations. A topic needs at least 5 conversations on a similar subject before it appears as a theme option.
- Select your segment (optional): If you have access to Customer Agent Segments, scope the test to a specific audience. This validates that the agent performs correctly for a particular group, not just in general.
- Run the test: Customer Agent generates a response to every question in your set. Depending on set size, this completes in one batch.
- Review responses: For every Q&A pair, mark it "Looks good" or "Looks bad." Don't rush this step. The quality of your review directly drives the quality of your coaching.
- Coach and fix: When you mark a response bad, a "Coach agent" button appears alongside an optional notes field. Fix the agent's knowledge or behavior immediately, or write a note so a teammate with coaching permissions can handle it later.
- Re-run and confirm: Run the same test set again after coaching. Verify the previously bad responses now pass before you push anything live.
What It Touches in Your HubSpot Strategy
This update ripples further than it looks. Here's where it connects across your portal.
Service Hub is the obvious home. If your team runs Customer Agent as a first-response layer before tickets reach a human, Batch Testing is now part of your agent governance process. Any change to knowledge, behavior, or segments should run through a batch test before going live.
Content Hub and Marketing Hub portals that use Customer Agent for self-service content discovery or onboarding support need the same discipline. The agent doesn't know which content is stale or which product details changed. Batch Testing catches those gaps.
Key Takeaway
The theme-generation feature ties directly to your live Inbox and Help Desk data. The questions your agent struggles with in production are now the same questions that appear in your test sets. That's a tight feedback loop that most teams haven't had access to before.
For RevOps leaders, think about this as change management for AI. Any time someone edits the agent's knowledge base, changes a routing rule, or modifies a segment, that's a deployable change. Batch Testing gives you a pre-deployment checklist built for AI instead of code.
The notes field also matters operationally. Not everyone who reviews responses has coaching permissions. Notes create a hand-off document between the reviewer and the coach. That's a small feature with real team workflow implications.
Key Takeaway
Segment-scoped testing is a big deal for teams with differentiated audiences. Testing the agent against your enterprise segment separately from your SMB segment means you're not assuming one configuration works for every customer type.
If you're thinking carefully about where AI judgment ends and human judgment begins, we've written directly about that tension in "When a Tool Stops Being a Tool". Batch Testing is a direct answer to that question. It keeps the humans accountable for what the agent says.
Who Should Care Most
Not every team needs this on day one. Here's who should move fast.
- Service Hub admins who manage Customer Agent as a first-line support layer and need a quality gate before every configuration change.
- RevOps leaders who want a repeatable deployment process for AI changes, not a one-off test-and-hope approach.
- Marketing and content teams using Customer Agent for self-service in high-traffic Help Centers, where a bad AI response reaches a lot of humans before anyone notices.
- Growing companies on Professional or Enterprise tiers across Service Hub, Content Hub, Marketing Hub, Sales Hub, Data Hub, or Smart CRM, since Customer Agent is available across all of them.
- Teams using Customer Agent Segments who need to validate performance per audience type, not just at the global agent level.
George's Take
I've been inside a lot of portals where the Customer Agent was configured once and never touched again. Not because it was perfect, but because nobody had a safe way to test changes. Admins were scared to break something that was barely working. Batch Testing removes that fear. It gives you a sandbox with real questions, real answers, and a direct path to fix what's wrong before any customer sees it. That's not just a feature. That's a change in how responsibly teams can run AI.
“The teams who flourish with AI aren't the ones who deploy the fastest. They're the ones who built a review process before they ever went live.”
If you're already leaning into HubSpot's AI tools across your portal, it's also worth checking out the On-Demand AI Summary for the Notification Sidebar. That update shows how HubSpot is layering AI summaries into the daily workflow for the humans managing the portal, not just the customers interacting with it.
If your team is running Customer Agent and you want to build a proper QA and coaching process around it, that's exactly the kind of work we do at Sidekick Strategies. Book a strategy call and let's look at your portal together.
Frequently Asked Questions
What is HubSpot Customer Agent Batch Testing?
Batch Testing is a tool inside HubSpot's Customer Agent that lets admins create a set of test questions, run the AI agent against all of them at once, review every response, mark bad answers, coach the agent directly from the results, and re-run the test to confirm improvements before any changes go live.
Who has access to Customer Agent Batch Testing?
Customer Agent is available on Professional and Enterprise tiers of Content Hub, Data Hub, Marketing Hub, Sales Hub, Service Hub, and HubSpot Smart CRM. It's also available via HubSpot Credits. The segment-scoping feature within Batch Testing requires access to Customer Agent Segments.
How does HubSpot generate test questions automatically?
HubSpot surfaces themes from recent live conversations in your Inbox or Help Desk. A theme appears as a question-generation option when at least 5 conversations cover a similar topic. Admins can select those themes to auto-populate test questions, or add questions manually.
What happens when I mark a Batch Test response as 'Looks bad'?
Marking a response bad immediately surfaces a 'Coach agent' button and an optional notes field. Admins can fix the agent's knowledge or behavior right from the results view, or write a note for a teammate who has coaching permissions to address later in bulk.
Can I test the agent for a specific customer segment?
Yes, if your portal has access to Customer Agent Segments, you can scope a batch test to a specific segment. This lets you validate that the agent performs correctly for a particular audience type rather than testing only at the global agent level.
How is Batch Testing different from just chatting with the agent to test it manually?
Manual testing is slow, inconsistent, and doesn't scale. Batch Testing runs the full question set in one pass, keeps a structured record of every response, and ties coaching actions directly to the results. It creates a repeatable pre-deployment review cycle instead of an informal one-off check.
