Self Hosted SSO with Rust and SQLite to Save you 10,000 USD

Optimizing a Rust and SQLite Web Service for 10,000 Concurrent Users on 4GB RAM, 4vCPUs Server

In the world of software architecture, fashion comes and goes. Microservices promise infinite scalability but often deliver staggering complexity. Serverless offers convenience but can obscure costs and control. Amidst this churn, the humble monolithic application has seen a resurgence which is a testament to the power of simplicity, unified logic, and rapid development.

But what happens when that simplicity collides with the brutal reality of internet scale?

This article is a data-driven journey, a case study in pushing a “simple” architecture to its absolute limits. We will take a well-architected but naive monolithic web service (built with Rust , the Axum framework , and SQLite in its high-concurrency Write-Ahead Log (WAL) mode ) and subject it to a punishing load of 10,000 virtual users.

What follows is not just a story of success, but a chronicle of failure, diagnosis, and iterative refinement. We will peel back the layers of our application, discovering how a single, overloaded endpoint can trigger a catastrophic cascade failure. We will then apply fundamental principles of high-performance computing (decoupling, batching, and the separation of concerns) to methodically eliminate one bottleneck after another. Each fix will reveal a new, more subtle performance barrier, leading us from the high-level abstractions of our application code down to the fundamental physical constraints of filesystem I/O.

This is a long, technical journey. By the end, you will not only understand the surprising scalability of a well-tuned monolith but also possess a generalized framework for analyzing and solving the most stubborn performance problems in your own systems.

A Correct but Naive Monolith

Every performance tuning story begins with a baseline: a system that is functionally correct but untested under duress. Our system is a multi-tenant Single Sign-On (SSO) service designed to provide authentication for a suite of products.

The Core Architecture:

• Language & Framework: Rust with the Axum web framework, running on the Tokio async runtime. This stack was chosen for its performance, memory safety, and excellent concurrency primitives.
• Database: A single SQLite database file, operating in WAL mode. This choice prioritizes simplicity of deployment and management (no separate database server required) while WAL mode allows for concurrent reads even while writes are happening, a crucial feature for web applications.
• Key Feature: The service provides authentication via SSO (GitHub, Google, Microsoft) and an OAuth 2.0 Device Authorization Grant for CLI or desktop applications.

The “hot path,” the endpoint that will receive the most traffic in our load test, is the one that initiates the Device Flow. A client application calls this endpoint to get a code for the user to enter on another device. This involves a simple database write to store the generated codes.

Here is the initial, perfectly reasonable implementation of the handler and the database interaction:

// in handlers/auth.rs

pub async fn device_code(
    State(state): State<AppState>,
    Json(req): Json<DeviceCodeRequest>,
) -> Result<Json<DeviceCodeResponse>> {
    // 1. Validate the incoming request against the database (a read).
    // ... validation logic ...

    // 2. Call a service to create the codes and write them to the database.
    let device_code = DeviceFlowService::create_device_code(
        &state.pool,
        &req.client_id,
        &req.org,
        &req.service,
    ).await?;

    // 3. Return the generated codes to the client.
    Ok(Json(DeviceCodeResponse {
        device_code: device_code.device_code,
        user_code: device_code.user_code,
        // ... other fields
    }))
}

// in auth/device_flow.rs

pub async fn create_device_code(
    pool: &SqlitePool,
    client_id: &str,
    org_slug: &str,
    service_slug: &str,
) -> Result<DeviceCode> {
    // ... code generation logic ...

    // A single, straightforward database INSERT statement.
    sqlx::query(
        r#"
        INSERT INTO device_codes (id, device_code, user_code, ...)
        VALUES (?, ?, ?, ...)
        "#,
    )
    .bind(...)
    .execute(pool)
    .await?;

    // ... return the created object
}

This code is clean, correct, and follows best practices for an async Rust web service. Each request is handled in its own async task, and the database call is properly .awaited. For a service with a few hundred users, this would likely perform flawlessly for years.

Load Testing with k6

To simulate our 10,000-user scenario, we turn to k6 , a powerful, open-source load testing tool.

Load Distribution:

• 70% - Subscription checks: GET /api/subscription (most common operation)
• 20% - Device flow writes:
  • POST /auth/device/code (creates device codes - INSERT operations)
  • POST /auth/token (polls for authorization)
• 10% - Batch user reads: GET /api/user (3 concurrent requests)

We define success by a few key metrics:

• Throughput: The total number of successful requests per second.
• Median Latency (p50): The response time that 50% of users experience. This tells us how the “typical” user feels.
• Tail Latency (p99): The response time for the 99th percentile. This is crucial as it represents the experience of your unluckiest users and is often the first indicator of systemic problems. Our goal is a p(99) under 30 seconds.
• Failure Rate: The percentage of requests that result in an error.

The First Result

After running the test for about ten minutes, the results are in, and they are abysmal. Bluntly put, a catastrophic failure.

A p99 latency of nearly a minute is unacceptable. A 14% failure rate means the service is fundamentally broken under load. The logs from the server tell the story.

The server logs are flooded with a single, ominous warning at around 200 concurrent users:

sso-server | WARN sqlx::query: slow statement: execution time exceeded alert threshold ... summary="INSERT INTO device_codes..." ... elapsed=2.24s

Diagnosis

The first issue is Write Contention indicated by the slow statement log. While SQLite’s WAL mode allows many readers to proceed while a writer is active, it strictly serializes all write operations. Only one write can happen at a time. At 200 concurrent users, the requests to INSERT into the device_codes table are arriving much faster than SQLite can process them. They form a queue, and the database becomes the bottleneck.

A secondary effect could be an Async Runtime Thread Starvation. Our Axum server runs on the Tokio async runtime, which uses a small, fixed number of OS threads (in our case, 4) to concurrently run many thousands of async tasks. When a request handler calls .await on the slow database query, it yields control, but the underlying database connection is still busy. As more and more requests get stuck waiting for the database, all 4 of the runtime’s worker threads become occupied managing these blocked I/O operations. The runtime has no free threads left to poll new incoming network connections. The server is effectively, completely blocked.

This is a complete systemic failure, all originating from a single, contended database write.

In an async system, a single slow, blocking operation on a hot path doesn’t just slow down requests, it can consume the entire runtime and trigger a total service outage.

Decoupling with an Actor Model

The core problem is that our web handlers, whose job is to handle HTTP requests as quickly as possible, are directly tied to the performance of our database writer. To solve this, we must decouple them.

Don’t Block the Event Loop.

This is a foundational rule of high-performance asynchronous programming. The threads that handle network I/O must remain free to accept new work. Any long-running or potentially blocking operation (like a contended database write) should be offloaded from these critical threads.

The solution is to implement a variation of the Actor Model . We will create a dedicated, asynchronous background task whose only job is to write to the database. Our web handlers will no longer write to the database directly. Instead, they will send a “write command” as a message over an async channel to this writer task.

This has two immediate benefits:

1. Sending a message on a channel is an extremely fast, non-blocking operation. The web handler is freed up to handle the next incoming request.
2. The writes are naturally serialized by the single receiving task, which perfectly matches SQLite’s single-writer limitation.

The Implementation:

We use a Tokio MPSC (Multi-Producer, Single-Consumer) channel .

1. We define the message and spawn the writer in main.rs:

// A message type to send to our writer task.
// It includes a `oneshot` channel for the writer to send the result back.
pub enum DbRequest {
    CreateDeviceCode {
        // ... fields
        responder: oneshot::Sender<Result<DeviceCode>>,
    },
}

// The writer task itself.
async fn db_writer_task(pool: SqlitePool, mut rx: mpsc::Receiver<DbRequest>) {
    while let Some(req) = rx.recv().await {
        match req {
            DbRequest::CreateDeviceCode { responder, ... } => {
                let result = DeviceFlowService::create_device_code(&pool, ...).await;
                // Send the result back to the waiting handler.
                let _ = responder.send(result);
            }
        }
    }
}

// In `main()`
let (tx, rx) = mpsc::channel::<DbRequest>(1024);
tokio::spawn(db_writer_task(pool.clone(), rx));

// The channel sender `tx` is added to the shared application state.
let app_state = AppState { db_tx: tx, ... };

2. And in handlers/auth.rs:

pub async fn device_code(
    State(state): State<AppState>,
    Json(req): Json<DeviceCodeRequest>,
) -> Result<Json<DeviceCodeResponse>> {
    // ... validation logic ...

    // Create a `oneshot` channel to receive the response.
    let (tx, rx) = oneshot::channel();
    let db_request = DbRequest::CreateDeviceCode { responder: tx, ... };

    // Send the message. This is non-blocking and returns instantly.
    state.db_tx.send(db_request).await?;

    // Asynchronously wait for the response from the writer task.
    // While waiting, this task yields and the worker thread is free for other work.
    let device_code = rx.await??;

    Ok(Json(DeviceCodeResponse { ... }))
}

The Second Result

We run the k6 test again. The results are a night-and-day difference, but a new, more subtle problem emerges.

This is a fascinating result. Our median latency has plummeted from 162ms to an incredible 0.7ms! This proves the decoupling worked exactly as intended. The web handlers are no longer blocked. They fire their message and return, leading to lightning-fast responses for the “typical” request. However, our p99 latency is still terrible, and the failure rate is still high.

Diagnosis

We solved the thread starvation problem, but the root cause (slow writes) remains. We’ve simply moved the queue. Instead of requests queueing up and blocking the web server, they are now queueing up inside the MPSC channel, waiting for the single writer task to process them.

Imagine a restaurant where 100 waiters (the web handlers) can instantly take orders and give them to a single manager (the writer task) who has the only credit card machine (the database). The waiters are now incredibly efficient, but the manager has a giant, growing pile of receipts to process one by one. A customer whose order is at the bottom of that pile is going to wait a very long time.

The bottleneck has moved from the web handlers to the dedicated writer task. It is still performing thousands of individual, high-overhead database transactions, and it simply can’t keep up.

Decoupling a bottleneck doesn’t eliminate it; it moves it. True optimization requires addressing the root cause of the slowness, not just shuffling the queue.

The Power of Transactional Batching

Our writer task is performing a “chatty” operation. For every single request, it initiates a new transaction, performs one INSERT, and commits. This transactional overhead, when repeated thousands of times per second, is immense.

Batch, Don’t Chatter.

This is a cornerstone of database performance. The cost of one transaction that inserts 200 rows is orders of magnitude less than the cost of 200 separate transactions that each insert one row. By batching our writes, we can dramatically increase the throughput of our writer task.

The strategy is to modify the writer task to be a “batch processor.” It will wait for a message, but then, instead of processing it immediately, it will greedily consume as many messages as are waiting in the channel (up to a limit) and execute them all inside a single database transaction.

The Implementation:

We update the db_writer_task in main.rs to incorporate batching logic.

const BATCH_SIZE: usize = 256;
const BATCH_TIMEOUT: std::time::Duration = std::time::Duration::from_millis(5);

async fn db_writer_task(pool: SqlitePool, mut rx: mpsc::Receiver<DbRequest>) {
    let mut batch = Vec::with_capacity(BATCH_SIZE);

    loop {
        // Wait for a message, but with a timeout.
        let msg = tokio::time::timeout(BATCH_TIMEOUT, rx.recv()).await;

        match msg {
            Ok(Some(req)) => {
                batch.push(req);
                // If the batch is full, process it.
                if batch.len() >= BATCH_SIZE {
                    process_batch(&pool, std::mem::take(&mut batch)).await;
                }
            }
            Err(_) => { // Timeout elapsed
                // If there's anything in the batch, process it.
                if !batch.is_empty() {
                    process_batch(&pool, std::mem::take(&mut batch)).await;
                }
            }
            Ok(None) => break, // Channel closed
        }
    }
    // ... process any remaining items
}

async fn process_batch(pool: &SqlitePool, batch: Vec<DbRequest>) {
    // Start a single transaction.
    let mut transaction = pool.begin().await.unwrap();

    // Dynamically build a single multi-row INSERT statement.
    // e.g., INSERT INTO ... VALUES (...), (...), (...), ...
    // ... logic to build the query and bind all parameters ...
    
    match large_insert_query.execute(&mut transaction).await {
        Ok(_) => {
            // Send success back to all waiting handlers.
        }
        Err(e) => {
            // Send the error back to all waiting handlers.
        }
    }

    // Commit the single transaction.
    transaction.commit().await.unwrap();
}

This logic is robust. It processes a batch either when it’s full or when a short timeout (5ms) expires, ensuring good latency even under lighter loads.

The Third Result

We run the test again. The results show another massive leap forward.

We’ve finally met our goal! The p99 latency is now under 30 seconds. Throughput has increased by another 20% to over 1 million requests. The median remains incredibly fast. Batching was clearly the right approach. However, The p99 is still quite high, and the failure rate hasn’t improved. We’ve increased the writer’s capacity, but there could still a lingering source of high latency.

Diagnosis

The system is now far more efficient, but we’ve overlooked a detail in our writer task. Looking back at the process_batch logic, what does it actually do? It receives a batch of requests and then, inside the single-threaded writer task, it loops through them to generate the data for the database.

// Inside the single-threaded writer task...
for req in &batch {
    // This work is happening sequentially on the writer's thread!
    let id = uuid::Uuid::new_v4().to_string();
    let device_code = DeviceFlowService::generate_device_code();
    let user_code = DeviceFlowService::generate_user_code();
    // ...
}

While our writer task is busy generating 256 UUIDs and user codes, it’s not interacting with the database. It’s performing synchronous, CPU-bound work, and during that time, the MPSC channel is filling up again. We have accidentally serialized all our CPU-bound code generation onto a single thread.

A dedicated I/O task must be ruthlessly focused on I/O. Offloading CPU-bound work to an I/O task can inadvertently re-serialize your application logic and become a new bottleneck.

True Separation of Concerns

The solution is now crystal clear. We need a true separation of concerns.

Separate CPU-Bound and I/O-Bound Work.

The work of generating codes is CPU-bound and perfectly parallelizable. The web handlers, running in parallel on Tokio’s multi-threaded runtime, are the ideal place for this. The writer task is I/O-bound and must be dedicated to that alone.

The fix is to shift the responsibility. The web handlers will generate all the necessary data. The message sent over the channel will contain the fully-formed data, ready for insertion. The writer task will be simplified to doing nothing but binding this pre-generated data to its query.

The Implementation:

1. The handler in handlers/auth.rs to do the work:

pub async fn device_code(
    State(state): State<AppState>,
    Json(req): Json<DeviceCodeRequest>,
) -> Result<Json<DeviceCodeResponse>> {
    // ... validation ...

    // Perform CPU-bound work here, in the parallel web handler.
    let id = Uuid::new_v4().to_string();
    let device_code = DeviceFlowService::generate_device_code();
    let user_code = DeviceFlowService::generate_user_code();

    // The message now contains the data, not the request to create it.
    let db_request = DbRequest::CreateDeviceCode {
        id,
        device_code: device_code.clone(),
        user_code: user_code.clone(),
        // ...
    };

    // Send and wait for confirmation.
    state.db_tx.send(db_request).await?;
    let _ = rx.await??;

    Ok(Json(DeviceCodeResponse { device_code, user_code, ... }))
}

The writer’s process_batch function no longer needs to generate anything. It simply receives a batch of messages and binds the data they contain. This makes the writer itself much faster and more focused.

The Fourth Result

We run the test one last time, expecting a final, dramatic improvement. The results are… almost identical. A WALL!

The p99 latency saw a minor improvement, as did the failure rate, but this is not the breakthrough we hoped for. The architectural change was correct (it is a better design) but it did not solve the primary remaining source of latency.

This is perhaps the most important result of our entire journey. When a sound architectural improvement yields no significant performance gain, it means you have stopped optimizing your application and have finally found the bottleneck in the underlying platform.

Confronting the Physical Limits

While running the fourth load test, this was logged at around 500 concurrent users:

sso-server | WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="PRAGMA wal_checkpoint(TRUNCATE);" ... elapsed=5.89s

This log is now the only “slow query” warning.

The Final Bottleneck

The Write-Ahead Log is a marvel. When we INSERT, SQLite just quickly appends the change to the end of the -wal file. This is very fast. However, that data must eventually be moved from the -wal file into the main .db database file. This process is called a checkpoint.

Under our extreme load, the -wal file could grow to an enormous size in a matter of seconds and end up slowing I/O. Our background task, running PRAGMA wal_checkpoint(TRUNCATE) every 10 seconds, has a monumental task. To safely move that data, it must acquire an exclusive write lock on the database.

For the 5–6 seconds it takes for the filesystem to perform this massive I/O operation, the entire database is locked. Nothing else can write to it.

This is the source of our tail latency.

1. Our db_writer_task runs, processing batches and writing to the WAL file at lightning speed.
2. Every 10 seconds, the checkpoint task kicks in and locks the database for 6 seconds.
3. During this 6-second “stall,” our db_writer_task is completely blocked. When it tries to begin a transaction, it simply waits.
4. Meanwhile, the MPSC channel, which has a buffer of 16,384 messages, continues to fill with tens of thousands of requests from the hyper-efficient Axum handlers.
5. When the checkpoint finishes and releases the lock, the writer task is faced with a colossal backlog. The requests that arrived during the stall are the ones that experience the 25-second latency. It could easily be mitigated by setting the wal_checkpoint(PASSIVE) but that might end up in a runaway -wal file size.

We have successfully optimized our Rust code to the point where it is no longer the bottleneck. The system is now entirely bound by the physical I/O speed of the disk and the fundamental operating principles of SQLite.

The goal of performance tuning is to eliminate application-level bottlenecks until your performance is dictated by the known, physical limits of your hardware and platform. Reaching this wall is a form of success.

Finding Balance and the True Limit

Our journey seemed to end at a wall. At 10,000 virtual users, we had optimized our application code to its apparent limit, yet a high tail latency and a persistent 12–15% failure rate remained. The logs pointed to a final, stubborn bottleneck: the heavy I/O cost of SQLite’s wal_checkpoint operation. However, a crucial observation was made related to the client machine running the test was itself under immense strain. This sparked a new hypothesis: what if 10,000 VUs wasn’t just the server’s bottleneck, but a point where the entire test environment began to break down?

To isolate the server’s true capabilities, we conducted two final, decisive experiments.

Finding the Sustainable Limit at 6,000 Users

First, the test was re-run with the load capped at a more conservative 6,000 VUs. The results were not an incremental improvement; they were a breakthrough, revealing the system’s true performance envelope.

Test Results at 6,000 Virtual Users (4-vCPU):

The change was dramatic. The failure rate vanished, confirming that the 10k VU test was indeed causing client-side resource exhaustion as a symptom of the server’s high tail latency. The p99 latency was halved to a respectable 12 seconds. Most surprisingly, the overall throughput increased by over 20%. This is classic behavior for an overloaded system: reducing the load to a sustainable level can paradoxically increase its efficiency. We had found the “performance cliff”—the point beyond which the system’s stability rapidly degrades. At 6,000 VUs, our application was stable, reliable, and performing at its peak.

The Paradox

This discovery led to an even more intriguing question: what if the 4-vCPU server was, in fact, architecturally unbalanced? We had four powerful “producer” threads (the Axum handlers) flooding a single “consumer” (the writer task, limited by disk I/O).

To test this, we ran the 6,000 VU test one last time against our most constrained environment yet: a tiny 1-vCPU server with only 1GB of RAM. The results were astonishing and cemented our final understanding of the system.

On a server with a quarter of the CPU and RAM, the service became perfectly stable. While throughput dropped by a predictable 25%, the failure rate was nonexistent. The paradox was clear: starving the application of CPU resources had made it more reliable.

The reason is system balance. On the 1-vCPU machine, the CPU itself became the bottleneck. The web handlers (producers) were forced to compete for CPU time with the writer task (the consumer). This acted as a natural governor, implicitly rate-limiting the producers to a pace the I/O subsystem could comfortably handle. The massive queue in our channel never formed, the WAL file grew less aggressively, and the checkpoint stalls became less disruptive. The bottleneck gracefully shifted from volatile I/O back to a more predictable and stable CPU limit.

This final experiment is a perfect illustration of the Theory of Constraints . By over-provisioning our CPU relative to our disk, we created an imbalance that led to instability. The smaller, more balanced server, while slower, was fundamentally more resilient. It also makes a powerful statement about the efficiency of Rust, whose low overhead allowed this complex application to run so effectively on a single core.

The Million-Dollar Monolith

Our journey began with a simple question: how far can a “simple” monolithic architecture be pushed? We started with a clean, correct Rust and SQLite service that failed catastrophically under minimal load. Through a rigorous, data-driven process of optimization (diagnosing a cascade of failures and methodically applying fundamental engineering principles) we transformed it into a highly-tuned service whose performance is no longer limited by its code, but by the physical constraints of its environment.

The final, stable results, even on a modest 1-vCPU server, speak for themselves:

• Peak Throughput: A sustained ~1,700 requests per second.
• Median Latency (p50): A near-instant 2.3 milliseconds.
• Tail Latency (p99): A stable and predictable 16 seconds.
• Reliability: A virtually perfect 99.999% success rate under a peak load of 6,000 virtual users.

This success represents a powerful foundation for a real business. By modeling typical user behavior (assuming an active user makes a request every 10 seconds during peak hours), we can confidently translate these metrics into a tangible user capacity:

This monolith can comfortably serve between 500,000 and 2,000,000 Monthly Active Users.

The most profound implication of this journey is not just technical, but financial.

A Practical Cost Analysis

The modern developer is presented with compelling hosted platforms like Auth0 , Clerk , and the auth components of BaaS providers like Supabase . These services used to offer immense value by providing speed, convenience, security expertise, and a rich feature set out of the box before AI assisted development started.

Today, you don’t need to go through 10 Google search results to find an obscure internet forum with a solution to your errors, you get the solution in 5 seconds.

Such hosted auth solutions, therefore, no longer offer much value (unless if you consider great marketing appeal) and the worst trade-off is cost at scale.

Let’s conduct a direct, apples-to-apples cost comparison for a business with 500,000 Monthly Active Users (MAU); a user base our monolith can easily support.

• Cost of Our Monolith: The 1-vCPU, 1GB RAM server that delivered our stable results can be provisioned from a cloud provider for max $5 per month**. Including backups and data transfer, a generous, all-in operational cost would be **~$10 per month.

• Cost of Hosted Alternatives:

  • Supabase/Firebase Auth: These services offer generous free tiers (typically 50,000 MAU). Beyond that, they charge per user. At 500,000 MAU, the cost would be approximately $1,350 per month.
  • Clerk.dev: Their “Pro” plan, aimed at scaling applications, is priced per MAU. At 500,000 MAU, the cost would be $10,000 per month.
  • Auth0/Okta: These enterprise-grade platforms offer more complex features, and their pricing reflects that. A plan supporting our feature set (multi-tenancy, custom domains, device flow) for 500,000 MAU would almost certainly be well over $15,000 per month.

The financial reality is stark.

This is not a minor difference; it is a strategic one. At scale, our simple, efficient monolith offers a 100x to 1000x cost advantage over its managed counterparts. This represents over $100,000 per year in saved capital that can be reinvested into core product development, marketing, or hiring.

Of course, this analysis does not mean managed services are without merit. For an early-stage startup where time-to-market is the only priority, or for teams without the expertise to manage security and infrastructure, the subscription cost is a valid and often wise investment in speed and risk reduction.

Our journey proves there is another way. By choosing an efficient modern stack like Rust, embracing the robust simplicity of SQLite, and applying rigorous engineering principles, we can build systems that are not just performant, but economically transformative. This investigation has been a powerful reminder that true scalability is not achieved by prematurely adopting complex, distributed systems, but by deeply understanding the limits of a simple one.

The decision to invest in engineering craftsmanship is not just a technical choice; it’s one of the most significant financial decisions a business can make.

repo: https://github.com/drmhse/sso