Case study · Freelance mission
Stabilising a data API as usage grows
During a freelance mission, I was hired to stabilise a heavily used data API with slow response times and errors during periods of heavy traffic.
I proposed, designed and implemented API versioning. This made it possible to introduce sub-resources progressively without forcing a breaking change on applications that were already connected.
The system at a glance
The API exposed several linked data repositories and already used Varnish to cache responses. This diagram summarises the parts of the platform involved in the mission.
Simplified view of the platform involved in the mission.
The mission
The API exposed varied data from several business domains across linked repositories. Many responses depended on relationships and joins between that data, and the API served a large number of applications.
The aim was to stabilise the platform so it could support rapid, sustained growth as more applications connected to the API.
The situation when I joined
The API was slow to respond, and the applications using it encountered errors during periods of heavy traffic.
Varnish was already in place. The main issue was cache invalidation: a change in one locale could flush every cached response for that repository across all locales. Because repositories were linked, a safe strategy also had to account for the dependent responses that could be affected by the same change.
The diagnosis
I used PostgreSQL EXPLAIN to inspect the slowest queries and understand how the database was executing them. This exposed several issues in the way data was queried and indexed across the many relationships in the model.
I also analysed how API Platform and Varnish handled responses. Locale headers and Gedmo translations affected the results, while links between repositories made it difficult to identify exactly which cached entries a change could invalidate.
The work completed
A complete rewrite of the API cache library
I rewrote the library used to cache API Platform responses and introduced dependency tags tied to the data and relationships used by each response. The logic separated dependency identification, propagation across repositories, and invalidation of the affected tags. Deep traversal also accounted for indirect dependencies, so only cache entries affected by a change were invalidated.
API versioning and sub-resources
I proposed and implemented API versioning so the response model could evolve and include sub-resources without breaking existing applications and consumers. This gave the teams a controlled way to extend the API while keeping existing integrations running.
Database optimisation
I improved the most expensive PostgreSQL queries and added or adjusted indexes based on the execution plans observed during the analysis.
Dependency-aware invalidation
When data changed, the system traced direct and indirect dependencies between repositories and selected only the tags for affected responses.
Data changes
A change is made in a repository or locale.
Dependencies are detected
The logic follows direct and indirect links between repositories.
Tags are targeted
Tags attached to affected responses are selected.
The cache stays consistent
Only the necessary Varnish entries are invalidated.
The outcome
The goal was not to reach a specific traffic number. It was to stabilise the platform so it could absorb rapid, sustained growth as more applications connected to the API. Over time, monthly traffic grew from 7 to 55 million calls, with much faster response times; the errors seen before the improvements no longer recurred.
API versioning made it possible to add sub-resources progressively while existing applications kept working.
My role
As Lead PHP / Symfony Engineer, I led the diagnosis and technical decisions, then carried the stabilisation work through implementation.
Facing a similar performance problem?
If an API is becoming slower or harder to evolve as its usage grows, I can help identify what is limiting it and build a safer path forward.
Discuss a project