I Built a Rate Limiter in Go. This Is What I learnt.

One day I will be in an interview for a Senior position. Everything will be going swimmingly and then they ask the following — “I would like you to build a rate limiter”. I will freeze up completely. I will start perspiring from places I didn’t think possible. Imposter syndrome will smack me in the face like the cold bitch she is.

I will panic. I will scream. I will run from the room a blathering mess never to touch another piece of software in my life and turn into a goose farmer. No imposter syndrome can touch me there.

I decided it was time to do something about it. I would face my fear head on and build an actual rate limiter. Deep down I know it’s the unknown that really scares me so the best thing I can do is get to know it intimately.

What Is a Rate Limiter?

According to Google:

Actually something practical that can be and is used in the real world.

Why Build It In Go?

Go is a language I am somewhat familiar with. I don’t use it on a professional level (yet) but I aim to in the future. A few reasons why I like it:

• Syntax is close to Javascript/Typescript — The language I am most familiar with.
• Standard library seems really good.
• It’s easier than Rust in my opinion. (Still can’t get my head around ownership, borrowing and lifetimes).

What Are The Different Approaches for a Rate Limiter?

There are different algorithms you can use when building a rate limiter.

• Token Bucket — You have a bucket that holds tokens that accumulate at a fixed rate. Each request consumes one token; if the bucket is empty, then the request is denied (or queued).
• Leaky Bucket — Another bucket but this time requests go into the bucket (queue) and then they leave the bucket at a fixed rate (this is the leaking part). If the bucket is full the request is denied, otherwise it’s accepted.
• Fixed Window Counter — The window here is time. Count the number of requests within a given time window and if it goes over the limit then reject. The counter will get reset at the end of the given window (every minute for example).
• Sliding Window Log — Here a timestamp is stored for each request. When a request comes in, remove any old timestamps from the current window and if there is space, accept the new request.

There are more I discovered on my research but for the sake of this article I am only showing these.

The approach I chose to go with was the “Token Bucket”. This was for a couple of reasons:

• It seemed to be the most popular approach. I like sticking with the masses in these kind of exercises.
• It is a flexible approach when it comes to rate limiters. It allows short burst of requests.
• It’s apparently used by the Big boys — AWS, GCP, NGINX etc so I’m in good company.

Full Disclaimer: I learnt how to do this using both ChatGpt free chatbot and the newish Opus 4.5 model from Claude. I tried both models and asked many questions throughout to get a comprehensive learning as possible. For me this is one of the best usages of AI currently.

Planning Out The Code

Let’s start getting into the implementation.

I started with a basic project structure in Go

ratelimiter/
├── go.mod
├── limiter/
│ ├── limiter.go
│ └── limiter_test.go
└── main.go

There are two things to consider in this approach:

1. The average rate of requests — eg 10 per second
2. The “burst” rate — these are short spikes of traffic that go beyond the average rate of requests but simulate more realistically the nature of API usage on the web. We need to allow these but only up to a certain capacity — eg 50 tokens worth

With this in mind I started to lay out some code.

Laying Out The Code

When it comes to laying out code for something new I like to take the “sculpture approach”. Make the basic shape first and then go around and fill in the details.

So what would I need here?

• A struct to house the different fields we will need for this rate limiter
• A function to create a new “token bucket”
• A function to control whether a request should be allowed
• A function to control whether a burst of requests are allowed
• A function to refill the bucket based on time elapsed since the last refill

With that in mind I built out the struct first.

type TokenBucket struct {
 capacity       int64         // Maximum number of tokens
 tokens         int64         // Current number of tokens (scaled by precision)
 refillRate     int64         // Tokens added per second
 lastRefill     time.Time     // Last time tokens were added
 nanosPerToken  int64         // Nanoseconds needed to generate one token
}

const precision int64 = 1_000_000 // Needed to avoid floating point issues later on

const precision int64 = 1_000_000 // Needed to avoid floating point issues later on

The nanosPerToken was used internally for precision to avoid floating point issues that I run into when running some tests but I don’t want to worry too much about that now.

Next up we want the function to create a new “token bucket”:

func NewTokenBucket(capacity float64, refillRate float64) *TokenBucket {
 capInt := int64(capacity)
 rateInt := int64(refillRate)
 
 // Calculate nanoseconds per token: 1 second = 1e9 nanoseconds
 var nanosPerToken int64
 if rateInt > 0 {
  nanosPerToken = int64(time.Second) / rateInt
 }

 return &TokenBucket{
  capacity:      capInt * precision,
  tokens:        capInt * precision, // Start with a full bucket
  refillRate:    rateInt,
  lastRefill:    time.Now(),
  nanosPerToken: nanosPerToken,
 }
}

With this approach we can create different rate limiters for different use cases. If we want a really strict rate limiter we reduce the capacity and reduce the refillRate. If we want a very relaxed rate limiter we can make these values large.

One thing I am really loving about Go in the testing. They make it so easy to set up tests that it almost feels wrong not to. I set up a basic test to make sure this was working.

func TestNewTokenBucket(t *testing.T) {
 tb := NewTokenBucket(10, 1)

 tokens := tb.Tokens()
 if tokens != 10 {
  t.Errorf("expected tokens 10, got %d", tb.tokens)
 }
}

Adding The Logic For The Rate Limiter

Let’s add the logic for refilling the bucket next:

func (tb *TokenBucket) refill() {
 if tb.refillRate <= 0 {
  return
 }

 now := time.Now()
 elapsed := now.Sub(tb.lastRefill).Nanoseconds()

 // Calculate tokens to add: (elapsed_nanos / nanos_per_token) * precision
 tokensToAdd := (elapsed * precision) / tb.nanosPerToken
 
 if tokensToAdd > 0 {
  tb.tokens += tokensToAdd

  // Cap at capacity
  if tb.tokens > tb.capacity {
   tb.tokens = tb.capacity
  }

  tb.lastRefill = now
 }
}

This does the following:

• Checks the refillRate is more than 0
• Finds out the elapsed time since the last refill
• Finds out from that how many tokens to add
• If this number of tokens is greater than 0 it checks it won’t go over the capacity and if it does it sets it to the capacity
• Updates the lastRefill to now

Next up we want to implement the logic to see whether we allow a request or not:

func (tb *TokenBucket) Allow() bool {
 tb.refill()

 if tb.tokens >= precision { // Do we have at least 1 token (1_000_000 units)?
  tb.tokens -= precision // Subtract a token
  return true
 }

 return false
}

What this function is doing:

• Calling the refill function first
• Then seeing if we have at least 1 token in the bucket
• If we do delete a token from the bucket tokens and return true — request accepted
• If we don’t then return false — request denied

We have very similar logic for allowing n requests too, in the case of a “burst” of requests

func (tb *TokenBucket) AllowN(n int) bool {
 tb.refill()

 required := int64(n) * precision
 if tb.tokens >= required {
  tb.tokens -= required
  return true
 }

 return false
}

• Again start off by calling the refill function
• Work out how many tokens are being requested
• If the number of tokens in the bucket are equal or more than the requested number of tokens then return true — request accepted
• Otherwise return false — request denied

Test For The Rate Limiter Functionality

Like I mentioned earlier I love the testing functionality in Go. Here are the basic tests I wrote to make sure the functionality of the rate limiter was working:

// Test #1
func TestAllow_InitialBurst(t *testing.T) {
 tb := NewTokenBucket(5, 1)

 for i := 0; i < 5; i++ {
  allowed := tb.Allow()
  if !allowed {
   t.Errorf("request %d should have been allowed", i+1)
  }
 }

 if tb.Allow() {
  t.Errorf("6th request should have been denied")
 }
}

// Test #2
func TestAllow_Refill(t *testing.T) {
 // Create a bucket with capacity 2, refill rate 10/second
 tb := NewTokenBucket(2, 10)

 // Use up all tokens
 tb.Allow()
 tb.Allow()

 // Should be denied now
 if tb.Allow() {
  t.Error("should be denied when empty")
 }

 // Wait for refill (100ms should give us 1 token at 10/sec)
 time.Sleep(110 * time.Millisecond)

 // Should be allowed now
 if !tb.Allow() {
  t.Error("should be allowed after refill")
 }
}

// Test #3
func TestAllowN(t *testing.T) {
 tb := NewTokenBucket(10, 1)

 // Request 5 tokens - should succeed
 if !tb.AllowN(5) {
  t.Error("should allow 5 tokens")
 }

 // Request 6 more - should fail (only 5 left)
 if tb.AllowN(6) {
  t.Error("should deny 6 tokens when only 5 available")
 }

 // Request 5 - should succeed
 if !tb.AllowN(5) {
  t.Error("should allow 5 tokens")
 }

 // Now empty
 if tb.AllowN(1) {
  t.Error("should deny when empty")
 }
}

These tests test the following

• Test #1 — Makes sure a new bucket can be created and the Allow function is working as expected.
• Test #2 — Testing the Refill function and making sure requests are allowed through when they should be and blocked when not.
• Test #3 — Tests that the AllowN function also behaves as it should and allows n requests through or not depending on n and the capacity.

What I Learnt Building This Rate Limiter

I learnt many things during this experience and I think it confirms my belief that practical experience is the best way (for me at least) to learn.

• Writing tests in Go is easy and the language almost encourages it. Tests also proved so useful in this experience catching edge cases.
• The standard library has so much baked in.
• Deciding what should and should not be in a struct for me is still really hard to decide up front. I definitely need more practice with this.
• You need time up front to really understand the problem to enable you to translate that into concrete functionality that will be needed in the feature.
• I learnt about the different approaches one can take to build out a rate limiter.
• I learnt some syntax stuff in GO around functions. Regular functions vs Methods. How the second function in the example below with the part in the parentheses before the function name is called the receiver. I won’t get into this now but I went down so cool rabbit holes.

// Regular function - stands alone
func DoSomething(limiter *RateLimiterMiddleware) {
    // ...
}

// Method - attached to a type
func (m *RateLimiterMiddleware) DoSomething() {
    // ...
}

Conclusion

Overall thanks to this experience of actually building out a Rate Limiter I realised that it’s just like any other problem I have faced. When you think about the actual problem, break it down into smaller pieces and start to understand the details it doesn’t become so scary anymore.

This is what I love about programming. You start with something you really don’t understand. You spend time with, trying things out, asking questions, poking at it to see what happens. Then at a certain point you do understand and it’s no longer mysterious.

It does take time but that’s ok. The effort and time you spend are totally worth it at the end of the day as it brings clarity to your understanding of software.