Technology
Go services that hold their latency under load
Go is quick to write and quick to run, until goroutine leaks, GC pressure and loose contracts pile up. We measure the services under real load, fix what the profiles show and give your teams a template for the next service.
- Duration
- 3 to 10 weeks per stage
- Model
- Measure, fix, standardise
- Output
- Profiled services and a service template
Signs you need this
Each of these shows up in runtime metrics, profiles or incident notes.
- Goroutine count climbs between deploys and only drops when the pod restarts.
- p99 latency spikes line up with garbage collection, or memory grows until the container is killed for exceeding its limit.
- Requests keep running after the client has gone, because the context is not passed down to database and downstream calls.
- A change to a protobuf message breaks a consumer nobody knew about, or old field numbers have been reused.
- Every service handles logging, configuration, health checks and shutdown differently, and new services start as a copy of the last one.
- Nobody has looked at a CPU or heap profile from production, so optimisation starts from guesses.
- A rewrite in Go is proposed for speed, but nobody has measured where the time actually goes.
Fast by default is not the same as fast under load
Go gives you cheap goroutines, a small runtime and quick start-up. The same features make some failures quiet. A goroutine blocked on a channel nobody reads never raises an error; it just holds memory. A handler that ignores its context keeps working for a client that left seconds ago. Under light load, none of this is visible.
The obvious fixes are more replicas, higher memory limits or a rewrite of the slow part. They buy time and hide the cause. We start from evidence instead: CPU, heap, goroutine and block profiles from production, execution traces for latency, and runtime metrics over several days. The fix is usually small and specific, such as a bounded worker pool, a context deadline, a reused buffer or a GOMEMLIMIT that matches the container.
This service covers the engineering inside Go services. When latency comes from the wider system, such as the database or the way services call each other, Performance and Scalability looks across the whole stack.
What is included, and what is not
Included
- Production profiling with pprof and execution traces, with the endpoints kept private
- Goroutine leak and context cancellation review across handlers, workers and clients
- GC and memory tuning:
GOGC,GOMEMLIMITand allocation hot spots - gRPC and protobuf contract rules, versioning policy and breaking-change checks in CI
- Timeouts, retries with backoff and concurrency limits between services
- A service template with configuration, structured logging, health checks and graceful shutdown
- Benchmarks that show whether moving a hot path into Go pays off
Not included
- Rewriting whole systems in Go without a measured reason
- Front-end, mobile or desktop development
- Cluster and infrastructure design
- On-call support for the services after handover
How the work runs
- Weeks 1 to 2Measure. With read-only access to code, dashboards and profiling endpoints, we collect CPU, heap and goroutine profiles from production and map latency per endpoint. Findings are ranked by user impact.
- Weeks 2 to 6Fix the hot spots. Leaks, missing cancellation, lock contention and allocation-heavy paths are fixed in pull requests your team reviews. Each fix comes with a benchmark or profile that shows the change.
- Weeks 5 to 10Set contracts and the template. Protobuf rules and breaking-change checks go into the pipeline. The patterns that worked become a service template, so the next service starts from it rather than from a copy.
- ThroughoutKeep access narrow. Access is read-only by default. Any write access needed to ship fixes is removed when the stage ends.
What you get
All outputs belong to you: code, reports, diagrams and their source files, kept in your repositories.
log/slog logging, health checks, graceful shutdown and OpenTelemetry.When this is not the right service
- Teams that want Go because it is popular, with no latency or cost problem to solve. A well-run service in another language is usually cheaper to keep.
- Problems that come from service boundaries or chatty calls between services. Distributed Systems Design is the better starting point.
- Organisations looking only for extra Go developers. We fix, standardise and hand over; your team owns the services.
Frequently asked questions
Which Go versions do you work with?
Supported releases, which are the two most recent major versions. If a service is older, upgrading it is usually the first step, because newer releases often improve the runtime and garbage collector with no code change.
Is it safe to profile in production?
Yes, with care. Short CPU and heap profiles add little overhead, and the endpoints stay off the public network. We agree the schedule and scope with your team before collecting anything.
Should we move a slow Python or Java service to Go?
Only if measurement says so. Often the time goes to I/O, queries or serialisation, which a rewrite would not fix. When a CPU-bound hot path is the problem, moving that one path can pay off, and we benchmark it before and after.
How do you handle protobuf changes across teams?
With rules that tools can check: never reuse field numbers, reserve removed ones, add rather than change, and version the package when a real break is needed. Breaking-change checks in CI stop mistakes before they merge.
What does our team do during the work?
Your engineers review every change and join the profiling sessions, so reading a profile becomes a team skill. For longer work, an embedded architect works with the team 2 to 3 days a week.
Start with a short technical conversation
Thirty minutes. You describe your Go services, and the architect who would run the work tells you where we would start.