Ask every AI services vendor: can you give me a named production reference with before/after metrics? How long from kickoff to first live system? Who is accountable if it doesn't perform? These three questions eliminate most vendors who are actually consulting firms with an AI label.
67% of enterprise AI projects fail to reach production. The most common reason is not technical - it is that companies hired an AI consulting firm and received a strategy document instead of a working system. In 2026, the market for "AI services" includes three very different categories: AI consulting firms (strategy), AI platform companies (tools you operate), and AI services companies (they build and run the system). The buying decision is different for each.
This guide gives you 8 specific questions to ask before signing any AI services engagement. The questions are designed to separate vendors who are accountable for production outcomes from those who are accountable only for deliverable documents.
Question 1: Can you give me a named production reference I can call?
This is the single most important question. Not a case study PDF, not a testimonial quote - a person at a company who has a live AI system in production today, who will take your call and answer specific questions.
The right answer: "Yes, here is [Name] at [Company]. The system went live on [date] and here are the metrics." A vendor who hedges - "We're in conversations with reference customers", "Our clients prefer confidentiality" - is telling you they do not have a deployed production system they can point to.
- How long from kickoff to first live investigation in production?
- What is the before/after on the key metric (investigation time, escalation rate, MTTR)?
- Which production systems is the AI connected to?
- What happens when the system makes a mistake - how is it corrected?
- If you had to do it over, what would you ask differently during the sales process?
Question 2: How long from kickoff to first live AI system in production?
A focused workflow deployment - one use case, one set of systems - should be live in production in 2-6 weeks. If the vendor says 3-6 months before anything is in production, they are describing a consulting engagement, not an AI services engagement.
The week-by-week plan matters. Week 1 should end with: stack audit complete, integration plan documented. Week 2 should end with: read-only system connections live, first AI investigations running on real data. Weeks 3-4: playbooks tuned, team trained, system handed over. If the vendor's plan has discovery phases extending past week 2 without live system connections, ask why.
Question 3: Who is accountable if the system doesn't perform as expected?
This question surfaces the accountability model. An AI services company should have a clear answer: we monitor the system's output, we have a defined improvement cycle, and here is the SLA on response time when issues are flagged. A consulting firm's accountability ends at delivery.
Ask specifically: "If the AI misclassifies a ticket category and sends a wrong diagnosis to support - what happens next? Who catches it? How fast?" The answer to this reveals whether the vendor has thought about production operations or just the initial deployment.
Question 4: What specific systems will you connect to, and can you confirm compatibility before we sign?
A credible AI services vendor should be able to name the specific integrations for your stack before the contract is signed. For B2B support investigation, this means: your log database (ClickHouse, BigQuery, Redshift), bug tracker (Linear, Jira), billing system (Stripe, Recurly), code repository (GitHub, GitLab), and monitoring (Datadog, Sentry, New Relic).
Vague answers like "we integrate with all major tools" or "we'll assess during discovery" indicate the vendor has not deployed this system before in a stack like yours. Ask for a compatibility checklist and a written confirmation that the key integrations are achievable within the timeline.
Question 5: How is the system priced, and what are you paying for if it stops working?
Usage-based or outcome-based pricing aligns vendor incentives with your results. A vendor who charges the same fixed monthly fee whether the system runs 100 investigations or zero has no production incentive.
Ask: "If the system has a two-week outage due to an integration breaking, do we continue paying?" The answer reveals whether the pricing model assumes production accountability. Red flags: high upfront project fees with no performance SLA, per-seat pricing with no throughput guarantee, contracts that require you to operate the system yourself after delivery.
Question 6: What does your forward-deployment model look like?
"Forward deployment" is a specific operating model: a vendor engineer works inside your stack from day one, connecting to your production systems rather than building in a sandbox. It is different from a consulting model (they work in their office, deliver artifacts) and a platform model (they give you tools to operate yourself).
Ask: "Who from your team will be in our systems in week one, and what access will they need?" A concrete answer - "an engineer with read-only ClickHouse and Linear access from day 3" - is a good sign. "We will begin integration planning in week two of discovery" is not.
Question 7: What is your data security and access model?
Production AI systems connect to your live customer data. The security model matters as much as the AI capability. Minimum acceptable standards for a US B2B engagement in 2026:
- Read-only by default. No write or delete access without explicit per-action human approval.
- Data not used for training. Customer data queried during investigation is not used to train models or shared with third parties.
- Encryption in transit and at rest. TLS 1.2+ and AES-256 minimum.
- SOC 2 compliance or in progress. For US enterprise buyers, this is a minimum procurement requirement.
If a vendor cannot answer these questions in writing before the engagement begins, treat it as a blocker. Do not accept "we'll address security during implementation."
Question 8: Build vs buy - have you modeled the comparison for our specific case?
A credible AI services vendor should be able to run this comparison for you, not hide from it. The honest build-vs-buy analysis for a focused workflow:
| Factor | In-house hire | AI services company |
|---|---|---|
| Time to first production system | 6-12 months | 2-4 weeks |
| First-year cost | $210-310K (salary + recruiting) | Usage-based, typically $24-120K/year |
| Production accountability | Internal - your team's problem | Vendor stays accountable |
| When hire makes more sense | 5+ workflows, AI core to product roadmap, internal team to support the infrastructure | |
If the vendor tells you build-vs-buy always favors services, they are not giving you a fair analysis. If they refuse to engage with the comparison at all, that is also a red flag. The honest answer is: services is faster and cheaper for a single focused workflow; hire makes sense when AI becomes core to your product.
The Altor answer to each of these questions
Production reference: Portkey AI. Contact available on request. 45 min → 2 min investigation. 200+ tickets in production.
Time to production: 14 days from kickoff to first live AI investigation.
Accountability: Altor monitors system output and runs tuning cycles. SLA on response to production issues.
Integrations: ClickHouse, Linear, Stripe, GitHub, Datadog, Slack. Pre-scoped before signature.
Pricing: Usage-based per investigation event. No charge during verified outages.
Forward deployment: Altor engineer in your stack by day 3. Read-only access only.
Security: Read-only by default. No training on customer data. TLS 1.2+, AES-256. SOC 2 Type II in progress.
Build vs buy: We will model this for your specific volume before you decide.
FAQ
What is the most important question to ask an AI services company?
Ask for a production reference - a customer you can call who has a live AI system in production, with measurable outcomes. If they cannot provide one, the engagement will end with strategy documents and a plan, not a working system.
How long should an AI services engagement take to reach production?
A focused workflow deployment should be live in production within 2-6 weeks. Engagements that require 3-6 months before any production system exists are consulting engagements with an AI label, not AI services.
What is the difference between AI consulting and AI services?
AI consulting delivers a strategy document - roadmap, architecture plan, implementation guidelines. AI services delivers a production system running on your data. The deliverable, accountability, and timeline are fundamentally different.
How should AI services be priced?
Usage-based or outcome-based pricing aligns the vendor's incentives with your results. Fixed project fees with no production accountability often indicate a consulting model.
Should I hire AI engineers or use an AI services company?
For a single high-cost workflow, an AI services company typically delivers faster ROI: 14 days to first production system vs 6-12 months for a hire. For 5+ workflows or AI core to your product roadmap, building in-house makes more sense long-term.
Bring these questions to the Altor demo
We will answer every one of them on the call - with the reference contact, the Portkey metrics, and the build-vs-buy model for your specific ticket volume and stack.
Book a Demo (US Hours)Compare Altor to alternatives → · Calculate your investigation ROI → · Altor home