Here's a request every AI team gets sooner or later.
A team builds a customer-support chatbot on one LLM provider. It works well, the client is happy, and it runs in production for six months.
Then the client sends a short message: "The bills are getting high. Can we use a cheaper model for simple questions and keep the expensive one for hard ones? And can we add a second provider as a backup when the first one goes down?"
A developer says it will take a day.
Then they search the codebase. The provider name is checked with if provider == "openai": in 30 different files. The API key and model name are passed by hand into almost every function. Two months ago, a new developer added a second provider and named his function generate_text(), while everyone else used generate(). One branch was never updated, and it only failed in production, at night, for one specific type of question.
The "one day" change took two weeks.
Nothing was wrong with the model. Nothing was wrong with the prompt. The problem was how the code was organized, in a lot of very ordinary Python.
This is the third post in our AI & LLM series at CreoWis. In the first post, we looked at working with large data without running out of memory. In the second post, we looked at handling errors and files so your pipeline doesn't crash. This time, we'll look at object-oriented programming (OOP): how to structure your code so that your AI app can grow and change without breaking.
Why AI Engineers Should Care About This
Because almost everything in an AI app gets replaced at some point:
LLM providers change their prices, models, and limits every few months
Clients ask for a backup provider, or a cheaper model for simple tasks
You swap the embedding model, the vector database, or the PDF parser
New developers join and add features in their own style
If your code is a long list of functions passing the same five values around, every one of these changes touches every file.
There are two more reasons.
First, every library you use is built with classes. The LLM SDKs, LangChain, LlamaIndex, the vector database clients: when you write client.chat(...), you're using an object. If you can't read classes, you can't read the code you depend on.
Second, AI coding assistants are very good at producing code that works today. They are much less careful about code that is easy to change in six months. OOP gives you the tools to do it.
Designing for future changes is your job, not the AI's.
The Code Before OOP
Let's start with how most chatbots are first written. Here's a stand-in for the LLM call, so every example in this post runs on your machine:
def call_llm(model, messages):
# stand-in for a real API call (Part 2 covers timeouts and errors)
return f"[{model}] reply to: {messages[-1]['content']}"
Now a chatbot that remembers the conversation for each user:
model = "small-model"
history_user_1 = []
history_user_2 = []
def ask(model, history, question):
history.append({"role": "user", "content": question})
reply = call_llm(model, history)
history.append({"role": "assistant", "content": reply})
return reply
ask(model, history_user_1, "What's your refund policy?")
ask(model, history_user_2, "Please update my billing address to 12 Lake Road.")
It works. But look at what you have to keep matched up by hand: the right model, the right history list, for the right user. With two users, that's easy. With 2,000 users, you'll keep histories in a dictionary, pass them around between functions, and one day someone passes history_user_2 into the call for user 1.
Now user 1's question goes to the LLM together with user 2's billing address. That's not just a bug. That's a data leak.
Nothing in this code stops that mistake. The data (the history) and the behaviour (asking a question) are separate things, and you are the only one keeping them together.
Classes and Objects: Keeping Data and Behaviour Together
The idea of OOP is simple: put the data and the functions that use it into one unit.
class ChatSession:
def __init__(self, user_id, model):
self.user_id = user_id
self.model = model
self.history = []
def ask(self, question):
self.history.append({"role": "user", "content": question})
reply = call_llm(self.model, self.history)
self.history.append({"role": "assistant", "content": reply})
return reply
session_1 = ChatSession("user_1", "small-model")
session_2 = ChatSession("user_2", "small-model")
session_1.ask("What's your refund policy?")
session_2.ask("Please update my billing address to 12 Lake Road.")
Now you don't pass the model or the history anywhere. You just tell session 1 to ask a question, and session 1 already knows its own model and its own history. It can't use session 2's history by accident.
A few words you'll see everywhere:
Class: the blueprint (ChatSession). Writing it doesn't create anything yet, the same way defining a function doesn't run it.
Object (also called an instance): a real thing created from the class by calling it, ChatSession(...). Each call creates a new, separate object.
Method: a function defined inside a class (ask).
Instance variable: a value stored on the object (self.history).
You've been using objects all along. a = 2 gives you an object of type int. ChatSession(...) gives you an object of type ChatSession. You're now creating your own types.
__init__: The First-Time Setup
Think about switching on a new phone. It asks for your name, language, Wi-Fi and email before you can use it. Apps come later.
__init__ is that setup step for an object. Python runs it automatically right after the object is created. The values you put in ChatSession("user_1", "small-model") are delivered to __init__'s parameters, exactly like function arguments.
You'll often hear __init__ called "the constructor". Strictly speaking, a hidden method called __new__ creates the empty object, and __init__ fills in its starting data. In an interview, saying both is the safe answer.
self: "Which Object Am I?"
Here's the mistake almost everyone makes once:
class ChatSession:
def __init__(self, user_id, model):
print(f"Session ready for {user_id}")
def ask(self, question):
return call_llm(model, [{"role": "user", "content": question}])
session = ChatSession("user_1", "small-model")
session.ask("Hi")
# NameError: name 'model' is not defined
model is a local variable inside __init__. When __init__ finishes, it's gone. The ask method can't see it, even though both live in the same class.
This is what self is for. When you call session.ask("Hi"), Python passes the object itself as the first argument. These two lines do exactly the same thing:
session.ask("Hi")
ChatSession.ask(session, "Hi") # session goes into self
That's why you write self in every method but never pass it yourself. The dot does it for you.
So self.model = model means: take the local value and store it on the object, where every other method can reach it. self.model and model are two different variables. One lives as long as the object; the other disappears when __init__ ends.
One rule saves a lot of pain: create every attribute in__init__, even if it starts empty.
class ChatSession:
def __init__(self, user_id, model):
self.user_id = user_id
self.model = model
self.history = []
self.last_reply = None # empty for now, but it exists
def ask(self, question):
self.history.append({"role": "user", "content": question})
self.last_reply = call_llm(self.model, self.history)
self.history.append({"role": "assistant", "content": self.last_reply})
return self.last_reply
def summary(self):
return f"{self.user_id}: {len(self.history)} messages, last reply: {self.last_reply}"
If self.last_reply were only created inside ask, calling summary() on a brand-new session would crash with AttributeError: 'ChatSession' object has no attribute 'last_reply'. This one shows up a lot in AI apps, where a dashboard calls summary() before the user has asked anything.
The Four Pillars of OOP
Everything from here builds on classes and self. OOP is usually described with four "pillars":
Pillar
One-line idea
In an AI app
Inheritance
A child class reuses a parent's code
Every LLM client shares the same retry logic
Encapsulation
Signal which data is internal
The API key is marked as "don't touch"
Polymorphism
Same method name, different behaviour
generate() works for every provider
Abstraction
Hide complexity, and enforce a required shape
Python refuses a provider that misses generate()
None of these add new logic. They decide how you structure the logic, so that a team can change it safely. We'll also cover composition, which isn't one of the four pillars but is used in real projects more than inheritance.
Let's build an LLM provider layer step by step, one pillar at a time.
Inheritance: Write Shared Logic Once
Every LLM client needs the same things: an API key, a model name, a timeout, and retry logic for rate limits. In Part 2 we wrote that retry logic as a function. Copying it into every provider class means fixing every copy when it needs a change.
Inheritance lets you write it once in a parent class, and every child class gets it for free.
Here are the error classes from Part 2 again:
class LLMProviderError(Exception):
"""Base error for anything that goes wrong with the LLM provider."""
class RateLimitError(LLMProviderError):
pass
class ProviderUnavailableError(LLMProviderError):
pass
Look at class RateLimitError(LLMProviderError). That's inheritance. You already used it in Part 2: RateLimitErroris a kind of LLMProviderError, so except LLMProviderError catches it too.
Now the parent class for every LLM client:
import time
class BaseLLMClient:
def __init__(self, api_key, model, timeout=30, retries=3):
self.api_key = api_key
self.model = model
self.timeout = timeout
self.retries = retries
def generate_with_retry(self, prompt):
for attempt in range(self.retries):
try:
return self.generate(prompt)
except (RateLimitError, ProviderUnavailableError):
if attempt == self.retries - 1:
raise
time.sleep(2 ** attempt) # wait 1 s, then 2 s...
And a child class for one provider:
class OpenAIClient(BaseLLMClient): # (BaseLLMClient) means "inherits from"
def generate(self, prompt):
# the real SDK or HTTP call goes here
return f"[OpenAI {self.model}] {prompt}"
client = OpenAIClient(api_key="sk-test", model="small-model")
client.generate_with_retry("Summarize this ticket")
# '[OpenAI small-model] Summarize this ticket'
OpenAIClient never defined __init__ or generate_with_retry. It got both from the parent. It only had to write the part that is actually different: how to talk to its provider.
The super() Bug
Now one provider needs an extra setting, an organization ID. So the developer gives the child its own __init__:
Why? When a child defines its own __init__, Python runs the child's and not the parent's. The parent's setup never happened, so self.retries, self.model and the others were never created.
The fix is super(), which means "my parent":
class OpenAIClient(BaseLLMClient):
def __init__(self, api_key, model, organization, **kwargs):
super().__init__(api_key, model, **kwargs) # run the parent's setup first
self.organization = organization # then add your own
def generate(self, prompt):
return f"[OpenAI {self.model}] {prompt}"
A few things worth knowing:
super() does not create a second object. There's only one object; super() just lets it use the parent's version of a method.
If the child has no __init__ at all, the parent's runs automatically. You only need super().__init__() when you write your own.
A child can override any parent method by defining one with the same name. The child's version wins.
When a Class Has Two Parents
Python also allows a class to inherit from more than one parent:
class CacheMixin:
def describe(self):
return "Caches responses"
class LoggingMixin:
def describe(self):
return "Logs every call"
class SmartClient(CacheMixin, LoggingMixin):
pass
SmartClient().describe()
# 'Caches responses'
print(SmartClient.__mro__)
# (<class 'SmartClient'>, <class 'CacheMixin'>, <class 'LoggingMixin'>, <class 'object'>)
Both parents have describe(). Python picks one using the MRO (Method Resolution Order): the class itself, then its parents from left to right, then upward, ending at object, the parent of every class in Python. Swap the order of the parents and the other one wins.
It works, but it's easy to get confused about which method runs. Our advice: avoid multiple inheritance unless you really need it. Most of the time, the next tool is the better choice.
Composition: When "Is-A" Is the Wrong Question
Here's a design we see often. A team builds a RAG pipeline and makes it inherit from the LLM client, because it needs generate_with_retry:
class RAGPipeline(OpenAIClient):
...
It works on day one. But now the RAG pipeline is an OpenAI client. When the client asks for a different provider, you have to change the parent class itself. And the pipeline has inherited everything from the client, including methods and settings it should never use.
Inheritance gives you the whole package. A good test is the IS-A question:
"OpenAIClient is an LLM client." ✅ Inheritance fits.
"A RAG pipeline is an OpenAI client." ❌ It isn't. A RAG pipeline has a retriever and has an LLM.
When the answer is HAS-A, use composition: build small separate pieces, and pass them in as objects.
class KeywordRetriever:
def __init__(self, documents):
self.documents = documents
def search(self, question, k=2):
# a real retriever would use embeddings and a vector database
words = set(question.lower().split())
ranked = sorted(
self.documents,
key=lambda doc: len(words & set(doc.lower().split())),
reverse=True,
)
return ranked[:k]
class RAGPipeline:
def __init__(self, retriever, llm):
self.retriever = retriever # the pipeline HAS a retriever
self.llm = llm # and HAS an LLM
def ask(self, question):
context = "\n".join(self.retriever.search(question))
prompt = f"Answer using only this context:\n{context}\n\nQuestion: {question}"
return self.llm.generate_with_retry(prompt)
Now look at what you can do without touching RAGPipeline at all:
docs = ["Refunds are processed within 7 days.", "Support is open 9am to 6pm."]
retriever = KeywordRetriever(docs)
pipeline_a = RAGPipeline(retriever, OpenAIClient("sk-test", "small-model", "org-123"))
pipeline_b = RAGPipeline(retriever, OpenAIClient("sk-test", "large-model", "org-123"))
Cheap model for one pipeline, expensive model for the other. Later you can swap the keyword retriever for a vector database retriever in the same way. The pipeline doesn't care, as long as the piece has a search() method.
Inheritance
Composition
Relationship
IS-A (OpenAIClient is an LLM client)
HAS-A (pipeline has an LLM)
How
class Child(Parent)
Pass objects into __init__, store them on self
What you get
Everything from the parent
Only the pieces you choose
Coupling
Tight: a change in the parent affects every child
Loose: swap a piece without changing the class
This is why you'll often hear "favour composition over inheritance". Use inheritance for real IS-A relationships, like a family of LLM clients. Use composition to put those pieces together.
Encapsulation: Signals, Not Locks
Our base class stores the API key as self.api_key. Anyone can read it, change it, or log it. Let's at least tell other developers that it's internal.
Python has three naming levels:
Written as
Called
Message to other developers
self.model
Public
Free to use anywhere
self._api_key
Protected (1 underscore)
Internal detail. Don't touch from outside
self.__call_count
Private (2 underscores)
Strictly for this class. Python renames it
class BaseLLMClient:
def __init__(self, api_key, model, timeout=30, retries=3):
self.model = model
self.timeout = timeout
self.retries = retries
self._api_key = api_key # protected: "please don't touch"
self.__call_count = 0 # private: name gets mangled
def _record_call(self):
self.__call_count += 1 # inside the class, the short name works
Here's the important part. None of this is security. In Java, private really blocks access. In Python, it's a naming convention. Python's philosophy is "we are all consenting adults":
client = BaseLLMClient("sk-live-123", "small-model")
client._api_key # 'sk-live-123' still readable
client.__call_count # AttributeError
client._BaseLLMClient__call_count # 0 still reachable
For double-underscore names, Python quietly renames the attribute to _ClassName__name. This is called name mangling, and its real purpose is to stop a child class from accidentally overwriting the parent's value. If OpenAIClient also had a __call_count, it would become _OpenAIClient__call_count, and the two wouldn't clash.
Now a real problem we've seen. Someone adds a debug line while chasing a bug:
The underscore didn't hide anything. The API key is now in your logs, and logs get shipped to monitoring tools, shared in tickets, and pasted into chats.
So treat encapsulation as a message to developers, and handle secrets properly:
Read keys from environment variables or a secret manager, never write them in code.
Give classes that hold secrets a __repr__ that hides them, so print(client) and most logs stay safe.
Real access control happens in your API layer and database, not in underscores.
Polymorphism: Same Method, Different Provider
Back to our opening story. The original code looked something like this:
def answer(provider, question):
if provider == "openai":
... # 50 lines, written by developer 1
elif provider == "claude":
... # 50 lines, written by developer 2, different names
elif provider == "local":
...
And this if/elif block was copied into 30 files.
Polymorphism ("many forms") means: same method name, different behaviour depending on the object. The team agrees on one rule: every provider class has a generate(prompt) method. What happens inside can be completely different.
class OpenAIClient(BaseLLMClient):
def generate(self, prompt):
return f"[OpenAI {self.model}] {prompt}"
class ClaudeClient(BaseLLMClient):
def generate(self, prompt):
return f"[Claude {self.model}] {prompt}"
class LocalModelClient(BaseLLMClient):
def generate(self, prompt):
return f"[Local {self.model}] {prompt}"
def answer(llm, question):
return llm.generate_with_retry(question) # doesn't care WHICH client it gets
answer() has no if at all. It only needs an object with generate(). This is called duck typing: "if it walks like a duck and quacks like a duck, it's a duck." Python doesn't check the type, only whether the method exists.
You've seen this in built-ins too. len("abc") and len([1, 2]) both work, each in its own way.
Choosing the Provider From Config
In a real app, the provider name arrives as a string, from an environment variable or a config file. A dictionary turns that string into the right class:
Notice we store the classes, not objects. Nothing is created until a provider is actually needed, so you don't load clients (or open connections) you never use.
Here's how the registry function is often written:
def get_answer(provider_name, question):
try:
llm = LLM_REGISTRY[provider_name](os.environ["LLM_API_KEY"], "small-model")
return llm.generate_with_retry(question)
except:
return f"Provider {provider_name} is not supported."
Now the new developer adds his provider:
class GeminiClient(BaseLLMClient):
def generate_text(self, prompt): # not "generate"
return f"[Gemini {self.model}] {prompt}"
LLM_REGISTRY["gemini"] = GeminiClient
get_answer("gemini", "Hi")
# 'Provider gemini is not supported.'
Gemini is supported. It's right there in the registry. The real problem is an AttributeError from the misnamed method, and the bare except: turned it into a message that lies. It would also hide a missing API key, a rate limit, and every other real error. (Part 2 has the full story on why bare except: is dangerous.)
A safer version only handles the case that really means "not supported":
def get_llm(provider_name, model):
llm_class = LLM_REGISTRY.get(provider_name)
if llm_class is None:
raise ValueError(f"Unsupported LLM provider: {provider_name}")
return llm_class(os.environ["LLM_API_KEY"], model)
Now a typo shows up as a real error. But it still only shows up when someone calls Gemini, which could be in production, at night, for one specific type of question. We want to catch it much earlier.
Abstract Classes: Make Python Enforce the Contract
You can't run a knowledge-transfer session for every new developer, and documentation gets skipped. What you need is a blueprint that forces every provider class to have the required methods.
That's what an abstract class does:
from abc import ABC, abstractmethod
class BaseLLMClient(ABC): # inherit ABC: this is now a blueprint
def __init__(self, api_key, model, timeout=30, retries=3):
self.model = model
self.timeout = timeout
self.retries = retries
self._api_key = api_key
@abstractmethod
def generate(self, prompt): # every child MUST write this
pass
def generate_with_retry(self, prompt): # a normal method: inherited for free
for attempt in range(self.retries):
try:
return self.generate(prompt)
except (RateLimitError, ProviderUnavailableError):
if attempt == self.retries - 1:
raise
time.sleep(2 ** attempt)
def __repr__(self):
return f"{type(self).__name__}(model={self.model!r}, api_key='***')"
Now try the new developer's class again:
class GeminiClient(BaseLLMClient):
def generate_text(self, prompt):
return f"[Gemini {self.model}] {prompt}"
GeminiClient("sk-test", "small-model")
# TypeError: Can't instantiate abstract class GeminiClient
# without an implementation for abstract method 'generate'
(The exact wording changes a little between Python versions.)
Python refuses to even create the object. The developer finds the mistake the first time they run their own code, not in production. The processes_payment-style typo, the generate_text mismatch, the forgotten method: all caught immediately.
A few rules:
You can't create an object of the abstract class itself. BaseLLMClient(...) raises a TypeError. It's only a blueprint.
It's a minimum, not a maximum. A child must implement generate(), but it can add as many extra methods as it likes, such as stream() or count_tokens().
Normal methods in the abstract class, like generate_with_retry, are inherited as usual.
This is also the second meaning of abstraction. The first is simply hiding complexity: whoever calls pipeline.ask() doesn't need to know about retries, prompts or retrievers. The second is defining what must exist while leaving how to each class. Abstraction and polymorphism work together: the abstract class guarantees the shared method name, so the polymorphic code that calls it is safe.
That @ line on top of the method is a decorator, a function that wraps another function to change how it behaves. It deserves its own post, so for now: the @abstractmethod line marks the method below it as required.
Putting It All Together
Here's the full provider layer, with every idea from this post in one place:
import os
import time
from abc import ABC, abstractmethod
class LLMProviderError(Exception):
"""Base error for anything that goes wrong with the LLM provider."""
class RateLimitError(LLMProviderError):
pass
class ProviderUnavailableError(LLMProviderError):
pass
# Abstraction + inheritance: one blueprint, shared retry logic
class BaseLLMClient(ABC):
def __init__(self, api_key, model, timeout=30, retries=3):
self.model = model
self.timeout = timeout
self.retries = retries
self._api_key = api_key # encapsulation: internal, never printed
@abstractmethod
def generate(self, prompt):
pass
def generate_with_retry(self, prompt):
for attempt in range(self.retries):
try:
return self.generate(prompt)
except (RateLimitError, ProviderUnavailableError):
if attempt == self.retries - 1:
raise
time.sleep(2 ** attempt)
def __repr__(self):
return f"{type(self).__name__}(model={self.model!r}, api_key='***')"
# Polymorphism: same method name, different provider
class OpenAIClient(BaseLLMClient):
def generate(self, prompt):
# the real SDK or HTTP call goes here (see Part 2 for timeouts)
return f"[OpenAI {self.model}] {prompt}"
class ClaudeClient(BaseLLMClient):
def generate(self, prompt):
return f"[Claude {self.model}] {prompt}"
LLM_REGISTRY = {
"openai": OpenAIClient,
"claude": ClaudeClient,
}
def get_llm(provider_name, model):
llm_class = LLM_REGISTRY.get(provider_name)
if llm_class is None:
raise ValueError(f"Unsupported LLM provider: {provider_name}")
return llm_class(os.environ["LLM_API_KEY"], model)
# Composition: the pipeline HAS a retriever and HAS an LLM
class KeywordRetriever:
def __init__(self, documents):
self.documents = documents
def search(self, question, k=2):
words = set(question.lower().split())
ranked = sorted(
self.documents,
key=lambda doc: len(words & set(doc.lower().split())),
reverse=True,
)
return ranked[:k]
class RAGPipeline:
def __init__(self, retriever, llm):
self.retriever = retriever
self.llm = llm
def ask(self, question):
context = "\n".join(self.retriever.search(question))
prompt = f"Answer using only this context:\n{context}\n\nQuestion: {question}"
return self.llm.generate_with_retry(prompt)
# Wiring it up from config
provider = os.environ.get("LLM_PROVIDER", "openai")
retriever = KeywordRetriever([
"Refunds are processed within 7 days.",
"Support is open 9am to 6pm.",
])
simple_questions = RAGPipeline(retriever, get_llm(provider, "small-model"))
hard_questions = RAGPipeline(retriever, get_llm(provider, "large-model"))
print(simple_questions.ask("How long do refunds take?"))
Now go back to the client's request from the start of this post:
Cheaper model for simple questions, expensive one for hard ones: two pipelines, two model names. No other code changes.
A new provider: one new class with a generate() method, and one line in the registry. If someone names the method wrong, Python refuses to create the object.
Switching providers: change LLM_PROVIDER in the environment. Zero code changes.
The API key: read from the environment, marked internal, and hidden when the client is printed.
The two-week change is now an afternoon.
Quick Recap
Classes and objects keep data and behaviour together, so one user's chat history can't end up in another user's request.
__init__ sets up the object automatically. Create every attribute there, even if it starts empty.
self is the object the method was called on. obj.method(x) is the same as Class.method(obj, x).
Inheritance (IS-A) puts shared logic like retries in one place. If the child has its own __init__, call super().__init__().
Composition (HAS-A) builds a pipeline from swappable pieces. Prefer it whenever the answer to "is it a…?" is no.
Encapsulation in Python is a naming convention, not security. Keep secrets in environment variables, and keep them out of print and logs.
Polymorphism gives every provider the same method name, so one function and one registry handle all of them, with no if/elif chains.
Abstract classes make Python enforce that contract, so a misnamed method fails on day one instead of in production.
What's Next
Our pipeline is now clean, safe, and easy to change. But it still has one big problem.
Run it over 5,000 documents and it handles them one at a time. Each LLM call spends a second or two waiting for a response over the network, and during that wait your CPU does almost nothing. 5,000 calls, one by one, can take hours.
In the next post, we'll look at how Python does more than one thing at a time: threads, processes andasyncio. We'll cover what CPU cores really do, the difference between concurrency and parallelism, what Python's GIL is, and which tool to use for LLM calls versus heavy computation.
Until then, open the last AI project you worked on and look for the if provider == ... checks. Each one is a place where a class could be doing the job for you.
The model is rarely the problem. Most of the time, it's the very ordinary Python around it.
If you're building an AI product and running into problems like these, whether it's a codebase where switching LLM providers means touching dozens of files, a prototype that gets harder to change with every new feature, or AI-generated code that works today but nobody wants to extend, let's talk. This is exactly the kind of work we do at CreoWis.
We at CreoWis believe in sharing knowledge publicly to help the developer community grow. Let’s collaborate, ideate, and craft passion to deliver awe-inspiring product experiences to the world.
This article is crafted by Syket Bhattachergee, a passionate developer at CreoWis. You can reach out to him on LinkedIn,X/Twitter and follow his work on the GitHub.
