Integrating large language models into production is deceptively tricky. The demo works beautifully; the production system surfaces edge cases you never anticipated. The three biggest challenges we've encountered are prompt reliability (small wording changes cause wildly different outputs), cost management (token usage adds up fast at scale), and latency (streaming responses help, but cold starts on serverless can be painful). Our current stack: OpenAI GPT-4o for reasoning tasks, GPT-4o-mini for classification and summarisation, LangChain for orchestration, and a Redis cache for repeated queries. This combination keeps costs predictable while maintaining quality.