Why we built our own AI call summaries instead of buying one
UDBOARD Team·November 3, 2026·Behind the Build
The easy version
The easy version of AI call summaries is a wrapper. You transcribe the call, stuff the transcript into a prompt, and ask a language model to write a summary. It takes about two weeks to ship and it works well enough that you can demo it.
We shipped that version first. Within a few weeks, the feedback from internal testing was consistent: the summaries were accurate but not useful.
A useful summary for a sales call looks different from a useful summary for a support call. A useful summary for a discovery call — where you're trying to understand a prospect's situation — looks different again. The generic "here is what was discussed" output was fine if you wanted to remember the call. It wasn't useful if you wanted to know what to do next.
That was the starting point.
What the off-the-shelf version misses
Generic AI summaries have a few consistent failure modes on real small-business calls:
They don't know what "a good call" looks like for your team. A generic model doesn't know that for your business, a qualified lead is someone who asked about pricing and mentioned they currently use a competitor. It can't extract that signal if it doesn't know to look for it.
They miss action items. A customer mentions in passing that they'll "follow up next Tuesday." A generic summary buries that in the body text. A useful summary surfaces it as an action item with a date.
They occasionally hallucinate names. This one is particularly bad. A summary that gets a customer's name wrong is worse than no summary — it erodes trust in the whole feature.
They're the same for every call type. You don't want the same format for a 2-minute inbound call from an existing customer asking a quick question and a 45-minute discovery call with a new prospect.
What we built instead
The core insight was: the template drives the output, not the model.
A great model with a vague prompt produces vague summaries. A solid model with a well-designed template that tells it exactly what to extract, in what format, for what purpose, produces useful output.
We built a template system with four call types:
- Inbound lead call — who called, what they're looking for, what their current setup is, what the next step is, and whether they're a qualified lead
- Sales call — prospect details, objections raised, commitments made, next actions with dates
- Support call — issue description, resolution status, follow-up needed, customer sentiment
- Discovery call — business context, pain points, decision timeline, recommended next action
The template tells the model exactly what fields to extract. The output is structured — not a paragraph, but a set of named fields that UDBOARD can display, filter on, and use to drive automation ("create a follow-up task if action items are not empty").
We also added a confidence-based name verification step. If the model isn't confident about a name, it returns the phone number rather than guessing — which is less impressive to read but more trustworthy to act on.
What we learned
It took four months, not two weeks. The two-week version works for a demo. The four-month version works in production, on real calls, with real variation in audio quality, accents, background noise, and call length.
The transcription quality matters as much as the summary quality. A great summarisation prompt can't rescue a bad transcript. We spent more time on the transcription pipeline than on the summarisation layer.
Support calls are harder than sales calls. A sales call has a relatively predictable structure. A support call can be anything. The support template took three iterations before the output was consistently useful.
Small teams have less tolerance for errors than enterprise teams. If you're a 500-person company, a few bad summaries out of thousands of calls is a rounding error. If you're a 5-person team and the AI gets your best customer's name wrong, that's a serious problem. We built for the 5-person team.
What's next
The current system uses one set of templates per call type across all workspaces. Later this year we're building per-workspace template customisation — so a legal firm's discovery call template extracts different fields from a SaaS company's discovery call template.
We're also building a feedback loop. When a team member edits an AI summary, that edit is a signal about what the model got wrong. We're not using that signal yet. We will be.
If you want to see the summaries in action on your own calls, the easiest way is to try it: