Microservices

What are microservices?

Old-school systems cram every feature into one "monolith" — touch one part and the whole thing shakes. Microservices do the opposite: split the system into small, independent services, each doing one job — a user service, a chat service, a vector-search service — each developed, deployed and scaled on its own.

Why AI systems love them

AI components are wildly different
Model inference, data cleaning and the frontend API all need different compute and tech stacks. Splitting them up lets you optimize each one.
Independent scaling
If inference is under pressure, add machines to inference alone — no need to scale the whole system.
Fault isolation
One service goes down, the rest keep running, instead of the whole app collapsing.

What does it cost?

More complexity
More services means more network calls, logging, monitoring and service discovery. Operations get noticeably harder.
Consistency is trickier
Data spreads across services, so cross-service operations need care — you can't just do everything in one transaction like a monolith.

When to use it

As the team grows, modules couple up, or you need independent scaling and frequent deploys, microservices often pay off. But for a small project with a few people, keeping it simple may be the better deal.

Bottom line: microservices split the elephant into many small boxes — each service is small and independent, so one failure doesn't sink the whole and one change doesn't force a full redeploy.