AI vendors are easy to be impressed by in a demo. This checklist helps you judge them on what matters after you sign. Use it for chatbots, agents, copilots and custom builds.

1. The basics

  1. What exactly does the product or service do, and what does it not do?
  2. Which customers like us can we speak to?
  3. How long has this exact product been live with paying customers?
  4. Is the AI built by you, or do you use another company's model? Which one?

2. Your data

  1. Where is our data stored and processed?
  2. Is our data used to train any model, yours or a third party's?
  3. Who at your company can see our data?
  4. How long do you keep it, and how do we delete it?
  5. What happens to our data if we leave?

3. Accuracy and safety

  1. How do you measure accuracy, and what is it on tasks like ours?
  2. What happens when the AI is wrong or unsure?
  3. Can it take actions on our systems? Which ones need human approval?
  4. How do you protect against prompt injection and misuse? (Ask them to explain it simply.)
  5. Can we see logs of what the AI did and why?

4. Security and compliance

  1. Which security certifications or audits do you hold? Ask to see them.
  2. How do you control access and permissions?
  3. How do you support our duties under the DPDP Rules?
  4. Have you had a security incident, and how was it handled?

5. Cost and contract

  1. What are all the costs: set-up, licences, usage, support, extra features?
  2. What is the cost at double our expected volume?
  3. What is the notice period and exit process?
  4. Who owns the work and the outputs?

6. Support and improvement

  1. Who supports us, in which hours and languages?
  2. How do we report and fix wrong answers?
  3. How often does the product change, and how do we hear about it?

How to score

Mark each answer clear, vague or missing. Three or more vague answers on data, security or cost is a reason to pause. Missing answers on data location or training use should stop the deal until resolved.

Red flags

  • "Our AI is 100% accurate."
  • Refusing to say which model they use
  • No written answer on data use
  • Pressure to sign before a pilot
  • No way to export your data

Run a pilot first

Agree a short pilot with a clear measure, a fixed price and a written exit. Compare two vendors on the same ten real tasks.

Get help

Browse verified partners on SARVO or tell us what you need. Related guides: AI agent development cost in India, custom AI vs off-the-shelf and questions to ask before hiring a technology partner.

FAQ

Do I need all 25 questions for a small tool? No. For low-risk tools, focus on data (5 to 9) and cost (19 to 22).

Should I ask for security reports? Yes. A serious vendor will share certifications or audit summaries, sometimes under an agreement.

Who should be in the room? Someone from the business team, someone from IT and someone who handles legal or compliance.