Skip to content
BranchesTags

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
backend
backend
 
 
README.md
README.md
 
 

Repository files navigation

Rate Limiter Service

.NET 9 Redis PostgreSQL Docker Architecture Pattern

This project is a centralized Rate Limiting and Abuse Detection microservice built with .NET 9. Unlike classic in-app rate limiter mechanisms, this is a standalone structure serving all microservices externally, effectively acting as a "traffic police" for the entire system.

For concurrency management, Redis atomic operations were preferred. Clean Architecture was adopted as the architectural pattern, and workflows were separated according to CQRS principles. This structure enhances maintainability and facilitates future system expansion.

Why I Built This?

I wanted to move beyond simple CRUD applications and tackle distributed system challenges. My main goals were:

  • To implement Clean Architecture (Onion) without over-engineering.
  • To handle high-concurrency traffic using Redis (Atomic Increments) instead of exhausting the database.
  • To decouple business logic using MediatR and Pipeline Behaviors.
  • To implement security patterns like Abuse Detection (Automatic Banning).

Technologies

  • .NET 9 (Web API - Minimal APIs)
  • MediatR (CQRS Pattern)
  • FluentValidation (Request Validation)
  • Entity Framework Core (PostgreSQL)
  • StackExchange.Redis (Distributed Caching & Counters)
  • Docker Compose

Architecture & Design Decisions

graph TD
    classDef client fill:#f9f,stroke:#333,stroke-width:2px,color:black;
    classDef api fill:#3498db,stroke:#2980b9,color:white;
    classDef app fill:#e67e22,stroke:#d35400,color:white;
    classDef infra fill:#27ae60,stroke:#2ecc71,color:white;
    classDef domain fill:#f1c40f,stroke:#f39c12,color:black;
    classDef db fill:#95a5a6,stroke:#7f8c8d,color:white;

    Client([Client / Microservice]):::client

    subgraph "Rate Limiter Service"
        direction TB

        subgraph "API Layer (Presentation)"
            Endpoints[Minimal API Endpoints]:::api
            GlobalEx[Global Exception Handler]:::api
        end

        subgraph "Application Layer (Core)"
            Pipeline["MediatR Pipeline
            (Logging & Validation Behaviors)"]:::app
            Handlers["Command & Query Handlers"]:::app
            Interfaces["Interfaces
            (IRedisService, IApplicationDbContext)"]:::app
        end

        subgraph "Domain Layer (Core)"
            Entities["Entities
            (RateLimitRule, BlacklistIp)"]:::domain
        end

        subgraph "Infrastructure Layer"
            RedisImpl[RedisService Implementation]:::infra
            EfCoreImpl[EF Core DbContext]:::infra
        end
    end

    subgraph "Docker Infrastructure"
        RedisDB[(Redis Cache)]:::db
        PostgresDB[(PostgreSQL DB)]:::db
    end

    Client -->|HTTP Request| Endpoints
    Endpoints -->|Sends Command| Pipeline
    Pipeline --> Handlers
    Handlers --> Interfaces
    Handlers --> Entities

    Interfaces -.->|Implemented By| RedisImpl
    Interfaces -.->|Implemented By| EfCoreImpl

    RedisImpl --> RedisDB
    EfCoreImpl --> PostgresDB

    Endpoints -.->|Catch Errors| GlobalEx
Loading 
/src
├── RateLimiter.API            # Presentation Layer (Minimal API)
├── RateLimiter.Application    # Business Logic (MediatR, Validators)
├── RateLimiter.Domain         # Enterprise Logic (Entities)
└── RateLimiter.Infrastructure # External Concerns (EF Core, Redis)

1. Caching Strategy (Cache-Aside)

Performance is critical for a rate limiter. Going to PostgreSQL for every request creates a bottleneck.

  • Whitelists & Rules: I used the Cache-Aside pattern. The application first checks Redis. If data is missing, it fetches from the DB and caches it for a specific period.
  • Rate Counters: Runs entirely in-memory (Redis).

2. Concurrency Control

To prevent race conditions where multiple requests might slip through simultaneously, I used Redis INCR (Atomic Increment). This ensures the counter works correctly even under heavy load.

3. Cross-Cutting Concerns (AOP)

Instead of polluting handlers with validation and logging logic, I implemented MediatR Pipeline Behaviors.

  • ValidationBehavior: Intercepts the request, validates DTOs, and throws 400 Bad Request if invalid.
  • LoggingBehavior: Automatically logs the entry and exit of every command/query.

4. Configuration Management

Hardcoded values are bad practice. All limits (e.g., MaxFailuresAllowed, BanDuration) are managed via appsettings.json (appsettings.Development.json) using the Options Pattern.

Setup & Getting Started

You don't need to manually install PostgreSQL or Redis. I added a docker-compose file for the infrastructure.

1. Prerequisites

  • .NET 9 SDK
  • Docker Desktop

2. Spin up Docker

Run the following command in the root directory to start Postgres and Redis:

docker-compose up -d

3. Database Migrations

Apply EF Core migrations to create the tables:

cd RateLimiter.API
dotnet ef database update

4. Run the Application

dotnet run

The API will be active at http://localhost:5207 (or a similar port).

API Endpoints (Examples)

1. Create Rule

Defines how many requests are allowed for a specific endpoint.

POST /api/rules
Content-Type: application/json

{
  "endpoint": "/api/payment",
  "maxRequests": 100,
  "windowSeconds": 60
}

2. Check Request

Microservices call this endpoint to ask: "Should I allow this IP to pass?"

POST /api/ratelimit/check
Content-Type: application/json

{
  "ipAddress": "192.168.1.50",
  "endpoint": "/api/payment"
}

Response:

{
  "isAllowed": true,
  "requestsRemaining": 99,
  "resetTime": "2025-12-24T15:30:00Z"
}

3. Report Abuse

This endpoint is intended for other services (e.g., Auth Service) to report suspicious activities. For example, if an IP address makes consecutive failed password attempts, this endpoint is called. If the threshold (e.g., 5 times in 10 minutes) is exceeded, the system automatically bans the IP.

POST /api/security/report-failure
Content-Type: application/json

{
  "ipAddress": "192.168.1.50",
  "reason": "Brute force attempt"
}

Future Improvements (v2)

  • Sliding Window Algorithm: Currently using Fixed Window counter. Switching to Sliding Window (via Redis Sorted Sets or Lua scripts) provides smoother limiting.
  • Dashboard: A simple UI to view banned IPs and active rules could be useful.

 

Rate Limiter Service

.NET 9 Redis PostgreSQL Docker Architecture Pattern

Bu proje, .NET 9 kullanılarak geliştirilen merkezi bir Rate Limiting ve Abuse Detection mikroservisidir. Uygulama içi kullanılan klasik rate limiter mekanizmalarından farklı olarak, tüm mikroservislere dışarıdan hizmet veren bağımsız bir yapıdadır. Bu sayede sistem genelinde trafiği kontrol eden bir “trafik polisi” gibi çalışır.

Concurrency yönetimi için Redis atomic operations tercih edilmiştir. Mimari olarak Clean Architecture yaklaşımı benimsenmiş, iş akışları ise CQRS prensiplerine göre ayrıştırılmıştır. Bu yapı, hem sürdürülebilirliği artırır hem de sistemin ileride genişletilmesini kolaylaştırır.

Neden Bunu Yaptım?

Basit CRUD uygulamalarının ötesine geçip distributed system zorluklarını ele almak istedim. Ana hedefler şunlardı:

  • Over-engineering yapmadan Clean Architecture (Onion) uygulamak.
  • Veritabanını yormak yerine Redis (Atomic Increments) kullanarak yüksek concurrency trafiğini yönetmek.
  • Business logic'i MediatR ve Pipeline Behaviors kullanarak decouple etmek.
  • Abuse Detection (Otomatik Banlama) gibi güvenlik patternlerini uygulamak.

Teknolojiler

  • .NET 9 (Web API - Minimal APIs)
  • MediatR (CQRS Pattern)
  • FluentValidation (Request Validation)
  • Entity Framework Core (PostgreSQL)
  • StackExchange.Redis (Distributed Caching & Counters)
  • Docker Compose

Mimari & Tasarım Kararları

graph TD
    classDef client fill:#f9f,stroke:#333,stroke-width:2px,color:black;
    classDef api fill:#3498db,stroke:#2980b9,color:white;
    classDef app fill:#e67e22,stroke:#d35400,color:white;
    classDef infra fill:#27ae60,stroke:#2ecc71,color:white;
    classDef domain fill:#f1c40f,stroke:#f39c12,color:black;
    classDef db fill:#95a5a6,stroke:#7f8c8d,color:white;

    Client([Client / Microservice]):::client

    subgraph "Rate Limiter Service"
        direction TB

        subgraph "API Layer (Presentation)"
            Endpoints[Minimal API Endpoints]:::api
            GlobalEx[Global Exception Handler]:::api
        end

        subgraph "Application Layer (Core)"
            Pipeline["MediatR Pipeline
            (Logging & Validation Behaviors)"]:::app
            Handlers["Command & Query Handlers"]:::app
            Interfaces["Interfaces
            (IRedisService, IApplicationDbContext)"]:::app
        end

        subgraph "Domain Layer (Core)"
            Entities["Entities
            (RateLimitRule, BlacklistIp)"]:::domain
        end

        subgraph "Infrastructure Layer"
            RedisImpl[RedisService Implementation]:::infra
            EfCoreImpl[EF Core DbContext]:::infra
        end
    end

    subgraph "Docker Infrastructure"
        RedisDB[(Redis Cache)]:::db
        PostgresDB[(PostgreSQL DB)]:::db
    end

    Client -->|HTTP Request| Endpoints
    Endpoints -->|Sends Command| Pipeline
    Pipeline --> Handlers
    Handlers --> Interfaces
    Handlers --> Entities

    Interfaces -.->|Implemented By| RedisImpl
    Interfaces -.->|Implemented By| EfCoreImpl

    RedisImpl --> RedisDB
    EfCoreImpl --> PostgresDB

    Endpoints -.->|Catch Errors| GlobalEx
Loading 
/src
├── RateLimiter.API            # Presentation Layer (Minimal API)
├── RateLimiter.Application    # Business Logic (MediatR, Validators)
├── RateLimiter.Domain         # Enterprise Logic (Entities)
└── RateLimiter.Infrastructure # External Concerns (EF Core, Redis)

1. Caching Strategy (Cache-Aside)

Bir rate limiter için performans kritiktir. Her istekte PostgreSQL'e gitmek darboğaz yaratır.

  • Whitelists & Rules: Cache-Aside pattern kullandım. Uygulama önce Redis'i kontrol eder. Veri yoksa DB'den çeker ve belirli bir süre için cache'ler.
  • Rate Counters: Tamamen in-memory (Redis) çalışır.

2. Concurrency Control

Birden fazla isteğin aynı anda aradan sızdığı race condition durumlarını önlemek için Redis INCR (Atomic Increment) kullandım. Bu, ağır yük altında bile sayacın doğru çalışmasını sağlar.

3. Cross-Cutting Concerns (AOP)

Handler'ları validasyon ve loglama mantığıyla kirletmek yerine MediatR Pipeline Behaviors uyguladım.

  • ValidationBehavior: İsteği yakalar, DTO'ları doğrular ve geçersizse 400 Bad Request fırlatır.
  • LoggingBehavior: Her command/query'nin giriş ve çıkışını otomatik olarak loglar.

4. Configuration Management

Hardcoded değerler kötü bir pratiktir. Tüm limitler (örn: MaxFailuresAllowed, BanDuration) appsettings.json (appsettings.Development.json) üzerinden Options Pattern kullanılarak yönetilir.

Kurulum & Başlangıç

PostgreSQL veya Redis'i manuel kurmanıza gerek yok. Altyapı için bir docker-compose dosyası ekledim.

1. Gereksinimler

  • .NET 9 SDK
  • Docker Desktop

2. Docker'ı Ayağa Kaldırın

Postgres ve Redis'i başlatmak için kök dizinde şu komutu çalıştırın:

docker-compose up -d

3. Veritabanı Migration'ları

Tabloları oluşturmak için EF Core migration'larını uygulayın:

cd RateLimiter.API
dotnet ef database update

4. Uygulamayı Çalıştırın

dotnet run

API http://localhost:5207 (veya benzer bir port) adresinde aktif olacaktır.

API Endpoints (Örnekler)

1. Kural Oluşturma (Create Rule)

Belirli bir endpoint için kaç isteğe izin verildiğini tanımlar.

POST /api/rules
Content-Type: application/json

{
  "endpoint": "/api/payment",
  "maxRequests": 100,
  "windowSeconds": 60
}

2. İstek Kontrolü (Check Request)

Mikroservisler bu endpoint'i çağırarak sorar: "Bu IP'nin geçişine izin vereyim mi?"

POST /api/ratelimit/check
Content-Type: application/json

{
  "ipAddress": "192.168.1.50",
  "endpoint": "/api/payment"
}

Cevap:

{
  "isAllowed": true,
  "requestsRemaining": 99,
  "resetTime": "2025-12-24T15:30:00Z"
}

3. Şüpheli Davranış Bildirimi (Report Abuse)

Bu endpoint, diğer servislerin (örn: Auth Service) şüpheli durumları bildirmesi içindir. Örneğin; bir IP adresi üst üste hatalı şifre denemesi yaptığında bu endpoint çağrılır. Eşik değer (örn: 10 dakikada 5 kez) aşılırsa sistem IP'yi otomatik olarak banlar.

POST /api/security/report-failure
Content-Type: application/json

{
  "ipAddress": "192.168.1.50",
  "reason": "Brute force attempt"
}

Gelecek Geliştirmeler (v2)

  • Sliding Window Algorithm: Şu an Fixed Window sayacı kullanılıyor. Sliding Window'a geçiş (Redis Sorted Sets veya Lua scriptleri ile) daha pürüzsüz bir limitleme sağlar.
  • Dashboard: Banlanan IP'leri ve aktif kuralları görmek için basit bir UI faydalı olabilir.

About

Centralized traffic control service for microservices. Implements Rate Limiting and Abuse Detection using .NET 9, MediatR, and Redis with Cache-Aside pattern.

Topics

abuse-detectionatomic-operationsclean-architecturecqrsdockerdotnet-9mediatrmicroservicespostgresqlrate-limiterredis

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages