Deploying an AI chatbot is more than connecting a model API to a chat interface. This production-readiness checklist helps you compare environments, estimate operating costs, validate reliability controls, and decide whether a prototype is ready for real users.
Overview
A local prototype proves that a conversation flow can work. Production deployment must prove that the system remains secure, predictable, available, understandable, and affordable when usage changes. Before launch, review five areas: application controls, model behavior, infrastructure, user experience, and ongoing operations.
The right deployment target depends on your risk and traffic profile. A private staging environment may be enough for internal testing, while a public chatbot needs stronger authentication, abuse protection, observability, privacy controls, and rollback procedures.
| Environment | Primary purpose | Controls to verify | Cost decision |
|---|---|---|---|
| Local development | Build and debug the chat flow | Environment variables, mock data, basic error handling | Use representative test volumes rather than production traffic |
| Staging | Validate integrations and release candidates | Test authentication, rate limits, logging, moderation, and rollback | Use production-like settings where practical, but cap usage |
| Private production | Serve a controlled group of users | Access control, auditability, privacy review, support path, monitoring | Estimate usage by user group and peak periods |
| Public production | Serve external users at unpredictable volume | Abuse prevention, availability plan, incident response, accessibility, analytics | Model normal, peak, and abnormal usage separately |
Do not treat hosting selection as a substitute for application design. A managed platform can reduce infrastructure work, but your team still owns prompt behavior, secrets, access rules, data handling, and release decisions. For a broader model-selection process, see How to Choose an LLM for Your Application.
How to estimate
Estimate chatbot readiness and operating cost with a repeatable worksheet rather than a single optimistic forecast. Start by defining one planning period, such as a month, and record the assumptions used in that estimate.
- Estimate conversations: multiply expected active users by average conversations per user during the period.
- Estimate turns: multiply conversations by average user-assistant turns per conversation.
- Estimate tokens: separate input tokens from output tokens. Include system instructions, retrieved context, tool results, conversation history, and the model response.
- Apply provider pricing inputs: use the current input and output rates supplied by your selected provider or deployment option. Keep these as editable spreadsheet cells because they can change.
- Add non-model costs: include application hosting, databases, object storage, retrieval infrastructure, monitoring, authentication, email or notification services, and any moderation or gateway services.
- Model scenarios: create low, expected, peak, and abuse or failure scenarios. Do not use the expected case as your capacity limit.
A simple model-cost formula is:
Estimated model cost = (input tokens ÷ billing unit × input rate) + (output tokens ÷ billing unit × output rate)
Then calculate total operating cost as model cost plus fixed infrastructure costs and variable supporting-service costs. If the chatbot uses retrieval or tools, estimate those calls separately. A response that triggers a search, database query, or external API may cost more and may also add latency or create another failure point.
For a chatbot that serves several workflows, calculate each workflow independently. Customer support, document search, account lookup, and agentic task execution usually have different prompt sizes, tool calls, and response lengths. This makes the estimate more useful than one blended average.
Inputs and assumptions
Record the assumptions behind both the launch checklist and the estimate. At minimum, capture the following:
- Traffic: active users, conversations per user, turns per conversation, peak concurrency, and expected growth.
- Prompt size: system instructions, conversation history, retrieved passages, tool schemas, and other repeated context.
- Response size: typical and maximum output length, including structured responses returned to the application.
- Model routing: which requests use the primary model, a lower-cost model, a fallback model, or a non-model response.
- Data retention: whether conversations are stored, for how long, and which fields require redaction or access restrictions.
- Availability target: acceptable downtime, timeout behavior, retry limits, and the user-facing fallback when the model or a dependency is unavailable.
- Security: secret storage, authentication, authorization, tenant isolation, input validation, and protection against prompt injection or tool misuse.
- Safety: moderation rules, blocked actions, escalation paths, and review procedures for uncertain or harmful requests.
- Observability: latency, error rate, token usage, response completion rate, fallback rate, user feedback, and cost by workflow.
Keep provider-specific details in a separate configuration record. Review model identifiers, SDK commands, context limits, rate limits, deployment regions, privacy controls, and pricing inputs before each release. Avoid placing secrets in source code, client-side bundles, screenshots, or shared prompt examples.
Conversation storage deserves special attention. Store only what the product needs, define deletion behavior, and restrict access to logs. If the chatbot uses retrieval, establish a maintenance owner for the knowledge base. The RAG Knowledge Base Maintenance workflow provides a useful recurring review pattern.
For reliability, test representative prompts rather than only successful demonstrations. Include ambiguous questions, long conversations, empty inputs, unsupported requests, adversarial instructions, tool failures, timeouts, and malformed structured output. A prompt testing and evaluation framework can turn these cases into repeatable release checks.
Worked examples
Consider a hypothetical internal support chatbot. The team expects 200 active users in a planning period. Each user is expected to start 8 conversations, with 6 turns per conversation. The estimate is therefore 1,600 conversations and 9,600 turns.
Suppose a typical turn contains 2,000 input tokens and produces 500 output tokens. The planning volume would be approximately 19.2 million input tokens and 4.8 million output tokens before accounting for unusually long sessions or retries. The team should then insert the current provider rates into the formula rather than hard-code a price in the article or application documentation.
Next, separate the workflow mix. If half of the turns are simple policy questions, one-third use retrieval, and the remainder call an account system, each category should receive its own token and dependency assumptions. Retrieval turns may include additional context; account actions may require authentication, function validation, audit logging, and a more conservative timeout.
Now add a peak scenario. If a company meeting, product launch, or support incident could temporarily increase traffic, calculate the number of simultaneous requests and verify the provider and application rate limits. The key question is not only “Can the model answer?” but also “What does the user see when capacity is exhausted?” Prepare a clear retry message, queueing behavior, or human escalation path.
For a public chatbot, add an abuse scenario. Automated requests, repeated long prompts, oversized histories, and repeated tool calls can change the cost profile quickly. Set request limits, maximum input and output sizes, per-user or per-tenant quotas, and alerts for unusual spend or traffic. These controls should be tested in staging, not added after the first incident.
Finally, define a rollback. Keep the previous prompt and model configuration available, use versioned deployment artifacts, and document who can disable a feature or route traffic to a fallback. The LLM prompt versioning workflow and LLM application CI/CD guide can help formalize these release steps.
When to recalculate
Recalculate the estimate and repeat the launch checklist whenever a key input changes. That includes a new model, a new provider, a changed prompt, a larger retrieval context, a new tool integration, a different retention policy, or a change in user volume. Recheck after pricing inputs, rate limits, context limits, SDK behavior, or deployment requirements change.
Use a practical review rhythm:
- Before launch: complete security, privacy, reliability, accessibility, cost, and rollback sign-off.
- During staging: compare measured token usage and latency with the original assumptions.
- After the first release: review real workflow mix, fallbacks, errors, user feedback, and spend.
- Monthly: inspect production health, quota usage, failed requests, unresolved safety cases, and cost by feature.
- Quarterly: review provider options, pricing inputs, model quality, retention settings, and operational ownership.
- After a major model or integration change: rerun evaluations, security tests, cost estimates, and rollback drills.
Finish each review with a short decision record: what changed, which assumptions were updated, what tests passed, and who owns the next action. A chatbot is ready for production when its failure modes are known and managed—not when its demo looks convincing. Use this checklist before every launch, model change, and meaningful traffic increase, then keep the worksheet with the deployment documentation so the next review starts with evidence rather than guesses.