Anti-hype checklist: 7 questions before buying "AI for engineering"

"AI for engineering" is the buzzword of the year. And a good part of what gets sold under that label does not survive 7 questions.

This is the checklist I give any engineering manager before signing. Every question comes from decisions I made building agents that write to production data. This is not theory: it is scar tissue.

The 7 questions

1. What happens when the model hallucinates? Not "if": when. If the vendor has no concrete technical answer (validation, verification, rollback), they have a problem that is about to become yours.

2. Does the agent write straight to production, or is there a dry-run with confirmation first? A serious agent shows what will change before touching anything, then waits for your approval.

3. Which permissions does it run with? If the answer is "an admin account", you should run. The agent must inherit each user's permissions, not bypass them with an all-powerful service account.

4. Does it validate against your system's real schema, or trust whatever the model believes? LLMs invent properties and states with total confidence. Without validation against the real schema, those inventions end up in your data.

5. Which operations are forbidden? An agent without hard limits is not finished. Explicit red lines are a sign of serious engineering, not weakness.

6. Where is the audit trail? Every operation must be traceable: who asked for it, what changed, when. No log, no control.

7. Can you show me a case where your answer was "this does not need AI"? Whoever never says it sells AI, not solutions. It is the question that separates consultants from salespeople the fastest.

Use it in your next meeting

Save this list for your next vendor meeting. The first six questions evaluate the product. The seventh evaluates the person. And if you want a second opinion on a proposal sitting on your desk, let's talk.