Every automation vendor can show you something impressive.
The demo is the easiest artifact to produce in this industry. It runs on data that was chosen because it works, in an environment nobody else is using, against a process that has been quietly simplified for the purpose. None of that is dishonest, and none of it is evidence.
The decision you are making is who to be tied to for the next three years, and how hard it will be to leave. Almost nothing in a sales process is designed to answer that.
These are the questions that do.
Ask Who Owns the Result
The single most useful question in a vendor evaluation:
If we ended this relationship tomorrow, what exactly would we have?
Acceptable answers are specific: the source code in a repository you control, documentation of how it runs, credentials held in your accounts, and infrastructure billed to you. Anything vaguer than that is worth pressing on.
The specific things to establish before signing:
Where does the code live, and is the repository owned by your organization or theirs?
Whose accounts hold the credentials and API keys the automation uses?
Who is the infrastructure billed to, and can that be transferred without a rebuild?
Is there anything proprietary in the delivery that you would lose access to?
A shop that builds on your infrastructure, in your repository, with your credentials, has made leaving easy on purpose. That is a signal about how they expect to keep your business.
Ask What Happens When Their Best Person Leaves
Small shops carry key-person risk, and pretending otherwise helps nobody. The question is whether they have designed around it.
Useful things to ask:
How many people at your shop could support this after launch?
What does the documentation cover, and can we see an example from another project?
If we call in eight months with a broken integration, who picks up?
The answer you want is boring. Documentation exists, more than one person understands the system, and support is a defined arrangement rather than a favor.
Ask for a Reference That Went Badly
Every vendor has a reference list of clients who are happy. That list is a filter output, not a sample.
The more revealing request:
Tell me about a project that did not go the way you expected, and what you changed afterward.
A shop that has been doing this for a while has one. The answer tells you how they behave when a project goes sideways, which is the only vendor behavior that really matters, because the smooth projects do not need a good vendor.
Be suspicious of a spotless record. It usually means either a short history or a selective memory.
Notice Whether They Ever Tell You No
This is the strongest signal available, and it shows up early.
A shop that scopes you down is a shop that expects to be there in year two. A shop that agrees with every idea in the discovery call is optimizing for the contract.
Things a good vendor will say to you, unprompted:
That process is too low-volume to be worth automating.
That one has too many exceptions. Write the rules down first, then call us.
You should buy that rather than build it.
Start with the boring one, not the one that is annoying you.
When we scoped the settlement claim automation for Class Action Settlement House, the work we took on was not the most interesting thing they did. It was the highest-volume, most repetitive, most rule-bound thing they did: staff pulling each claim's details and typing every field into a settlement portal, one record at a time. That work consumed 80% to 85% of the team's time. It was also boring, which is exactly why it was the right first project.
The result was roughly 13,500 hours of manual processing eliminated and about $450,000 saved. The process underneath it was well chosen.
A vendor who only ever agrees with you is not helping you choose.
Watch How They Talk About Completeness
Ask what share of cases the automation will handle end to end.
If the answer is 100%, one of two things is true: the process is unusually clean, or nobody has looked at it closely. The second is far more common.
The claim-filing system we built fills roughly 90% of every form automatically. The remaining 10% is the honest part of the number, and the human review step at the end of each batch was designed in from the start rather than added after something went wrong.
I plug in the URL and it fills almost 90% of the fields. It is such an enormous timesaver, so that we focus more on selling and filing than on the other stuff.
Lee, Class Action Settlement House
A vendor who volunteers the incomplete number is telling you they have examined the process. A vendor who promises totality is telling you they have not.
Ask What Year Two Costs
Automation is an asset with a carrying cost: integrations break when the other side changes, credentials expire, and the process evolves.
A vendor who has not thought about this has quoted you a launch rather than a system. Ask directly what ongoing support looks like, what it costs, and what is specifically excluded. The full accounting is in What an Automation Project Actually Costs.
A Short Checklist
Before signing, you should be able to answer all of these:
Ownership: we hold the code, credentials, and infrastructure.
Continuity: more than one person can support it, and documentation exists.
Honesty: they told us something we did not want to hear.
Completeness: we know what share of cases are handled and what happens to the rest.
Year two: we know what maintenance costs and what it covers.
Exit: we know what leaving would involve, and it is not a rebuild.
If a vendor makes any of these hard to pin down during the sales process, that is information. It does not get easier after the contract is signed.
A Simple Rule of Thumb
Hire the shop that tries to make the project smaller.
The one expanding scope in the first conversation is describing the invoice they would like to send. The one narrowing it is describing the system you will still be running in three years.
Final Thoughts
Demos are easy and ownership is hard, so evaluate on ownership.
The best sign you have found the right partner is that the project got smaller and more specific during the sales process, not larger.
If you are evaluating vendors for an automation project and want a second read on the scope, we are happy to look at it with you. Book a short consult.