When we started building Opswake, the obvious choice was GPT-4. Everyone was using it. The documentation was excellent. The community was huge.
We used it for 6 weeks. Then we switched to Claude. Here's exactly why.
The problem with GPT-4 for customer-facing conversations
GPT-4 is brilliant at many things. But when you're building an AI that talks to Indian SMB customers on WhatsApp — real people asking real questions about dental appointments, property listings, coaching fees — you need something specific: an AI that follows instructions precisely and doesn't improvise.
GPT-4 improvises. A lot.
We had a real estate client whose AI agent started quoting property prices that didn't exist in the knowledge base. GPT-4 was "helpfully" estimating based on context. That's great for a chatbot demo. It's a disaster when a customer shows up expecting a price that was never real.
Why Claude works better for this use case
Claude (Anthropic's model) is trained with a much stronger emphasis on following explicit instructions. When we tell it "only answer from the provided knowledge base, say you don't know if the answer isn't there" — it actually does that. Consistently.
Three months of production data across 50+ businesses:
Hallucination rate on Claude: under 2%
Hallucination rate on GPT-4 (same prompts): 11%
For a business owner whose AI is talking to their customers, that difference is everything.
The cost argument
Claude Sonnet is also meaningfully cheaper per token than GPT-4 Turbo for comparable quality. At our volume — processing hundreds of thousands of WhatsApp messages monthly — this compounds fast.
Where GPT-4 is still better
Coding tasks. Complex multi-step reasoning. Creative generation. If we were building a coding assistant or a research tool, GPT-4 would be the call. But for structured, instruction-following, customer-facing conversations? Claude wins for our use case.
The lesson
Don't pick an LLM based on benchmarks or hype. Pick it based on your specific failure modes. For us, hallucination in customer conversations was the critical failure mode. That drove the decision.
If you're building something AI-powered and need help picking the right model for your use case, that's exactly the kind of scoping conversation we have with clients before writing a line of code.