A lot of what gets sold as "AI automation" right now is a template someone copied, with a $5,000 setup fee and $1,000 a month stuck on top. It works fine in the demo. Then a customer types something the tool didn't expect and the whole thing falls over.

I read a thread this week from a business owner who was tired of exactly that, and I think the frustration is fair. I've spent over 20 years implementing business systems like SAP, Infor, and JD Edwards, and most of that work was handling bad data and odd cases. So here are the five questions I'd ask any vendor before paying them anything. That includes me.

1. Can you show me the logs and the error handling?

Every automation fails sometimes. A service goes down, or a customer gives an address in a format nobody planned for. If the person selling you the tool says it rarely makes mistakes, I'd end the call, because they probably haven't run it on real, messy input.

What you want to see is a log, which is just a running list of what the tool did and when, like the history file a program keeps. Ask to see a failed run in it. Then ask what happens when the tool gets confused. The right answer is that it stops and hands the case to a person by text or email, with the original message attached.

When I build something, we test it on your own real examples before you pay for it, and I make sure the confusing ones are in the pile.

2. Do I own the accounts, or do you?

Some vendors build everything inside their own accounts on Zapier, a CRM, or whatever service they use. It's like having your company's files saved only on their computer. If you stop paying the monthly fee, they can switch it off, and your booking or quoting system goes with it.

You want it built in your own accounts, with your own logins and your own keys. If you part ways, another developer should be able to open it up and keep it running without starting over.

I put this in the contract. You own every account and file, and at the end you get a written handover that lists each account, what it does, and how to turn it off.

3. What tools are you actually connecting?

Phrases like "proprietary AI" or "custom neural network" often cover something much simpler. Ask the vendor to share their screen and walk through it step by step. Very often it's three boxes: a form sends some text to a chatbot service, and the service sends an answer back.

There's nothing wrong with that. Some jobs really are that simple. But it takes an afternoon to set up, and it shouldn't be priced like custom software.

When I show a client a workflow, I name each program, what it receives, and what it passes along. If the job doesn't need AI at all, I say so. What you're paying me for is knowing which steps to build and testing them against your real cases, not the three boxes.

4. Where is my data going?

If you handle medical records, accounting figures, or client contact details, you need to know which outside websites see that information. Some vendors send it through whatever third-party service is convenient, without much thought about who else can read it.

This is where I think a local setup makes a real difference. A local model is an AI program that runs on your own computer or server, like any other program you'd install, so your records never leave your office. For a clinic or an accounting firm, that's a different offer from a cloud service.

Sometimes a cloud service is the right call. When it is, I name it, list exactly what gets sent to it, and you decide before anything is built. You also get a one-page map of every place your data travels.

5. Can you write out my process on paper first?

If nobody can describe a task step by step on paper, a chatbot isn't going to fix it. It will just repeat the confusion faster.

So this should be the first thing any vendor does. Sit down with the person who does the task today and write out each step, each decision, and each exception. Who gets the quote request? What happens when the customer leaves out their phone number? What does your bookkeeper do when an invoice doesn't match?

Sometimes the answer is to fix the process first and automate it later. I'd rather tell you that than build something on top of a mess.

Where I'd start

I don't ask new clients to sign a long retainer. I start with a small pilot: one task, a fixed price, a written scope, and a test on your real examples. You keep every file and account whether we continue or not.

If one small piece works, you can decide what to automate next with real results in front of you. If you want to talk through a task you're thinking about, you can book a free call with me.

← Back to Articles