Caching Service Tokens in Gravitee with a Loopback API

Every request to one of our business APIs made an HTTP Callout to an authentication server. In our measurements, a request served from the new cache was roughly 781 milliseconds faster than a cold request that still made that call.

The fix was an internal Auth Loopback API. We put Gravitee’s response Cache policy in front of the auth server, pointed the existing callout at the loopback, and let the business API keep parsing the same JSON response as before.

The implementation was small. Finding it took a day because several more direct approaches ran into boundaries in the gateway version we were using. Those failed attempts mattered: they showed us which extension points were actually available without an administrative configuration change.

Why the direct approaches failed

Each attempt began with a reasonable model of the gateway. The problem was that our model did not match the policy or sandbox contract.

The loopback was not a way around the Cache policy. It was a way to give that policy the kind of value it was built to cache: an upstream HTTP response.

The loopback request path

The final path worked like this:

1. The client calls the business API.
2. The business API makes an HTTP Callout to http://localhost:8082/loopback.
3. The Auth Loopback API checks its Cache policy.
4. On a hit, the loopback returns the cached auth response.
5. On a miss, the loopback calls the auth server and returns the response, which the policy caches.
6. The business API extracts access_token from that JSON.
7. It sets the backend Authorization header and continues to the backend.

This added a local HTTP hop to the common path, but removed a slower remote auth call. It also kept the change local to the gateway: the business APIs still received the response shape they already understood.

What we measured

We compared a cold request, which still had to reach the auth server, with a subsequent request served from the cache:

These two observations are not a benchmark across workloads, but they were enough to confirm the mechanism and the size of the delay in our request path. After the change, auth traffic followed cache misses rather than business request volume:

before: auth requests = business API requests
after:  auth requests = loopback cache misses

The cold request remained slow. That was an accepted trade-off because the configured cache window was eight hours in this deployment. Restarts, deployments, and expiry could still produce a cold request, so the latency did not disappear; it moved off the usual path.

What the failures clarified

Policy names are not contracts

“Cache” can mean response caching, arbitrary key-value storage, or several other things depending on the product and version. In the Gravitee version we had, its useful contract was response caching. Once we designed around that behavior rather than the label, the loopback followed naturally.

The sandbox boundary was part of the design

The Groovy restrictions ruled out convenient forms of shared state. An administrator might have been able to extend the whitelist, but that would have added a production configuration dependency and more privilege to the script. We did not test that route because the loopback worked within the existing boundary.

OAuth2 compatibility is specific

Our auth endpoint used familiar OAuth2 vocabulary, but that did not make it compatible with the native resource. The useful question was not whether the endpoint looked OAuth2-like; it was whether its request and response contract matched what the resource required. Ours did not.

The cache key is a security decision

We used one static key, service-token-general, because this deployment used one service credential and one permission set across the affected APIs.

That key would be unsafe if the token varied by tenant, user, backend, scope, environment, or region. A cache hit must never cross an authorization boundary. In a system with those dimensions, the key needs to include them explicitly, for example:

service-token:{environment}:{backend}:{scope}

The cache scope matters too. A response shared at API scope may be visible across consumers, while application scope partitions responses by application. The right choice follows the credential and authorization model, not convenience.

Check the current policies before using this pattern

The loopback solved a constraint in an older Gravitee installation. It should not be the default for a new deployment.

Gravitee’s current Cache policy documentation still describes caching upstream response content, status, and headers. Gravitee also provides a Data Cache policy for arbitrary key-value operations. Its documentation includes the service-token flow directly: read the token, call the auth endpoint only on a miss, then store the new value.

I would now choose in this order:

1. If the native OAuth2 resource matches the auth server, use it and its built-in caching.
2. If Data Cache is available, evaluate it for the token flow.
3. If only response caching is available, consider the loopback pattern.
4. If none matches the requirement, decide explicitly between a custom policy and an administrative gateway change.

Production checks and rollback

Before shipping the loopback, we needed to answer a short set of operational questions:

• Is the cache TTL shorter than the token’s usable lifetime, with enough margin for clock and network delay?
• Does the response condition prevent authentication failures from being cached?
• Does the cache key include every dimension that changes authorization?
• Is the loopback reachable only from the intended gateway path?
• Is the cache local to one gateway node or shared across nodes?
• Can logs distinguish a hit, a miss, and an auth-server failure?
• Is cold-cache latency acceptable after expiry, restart, and deployment?

Rollback stayed simple: point the HTTP Callout back to the auth server. That restored the original latency but removed the loopback from the request path, without requiring a backend change.

The pattern was useful because it matched the gateway we actually had. On a current Gravitee installation, Data Cache may make the extra API unnecessary. On the older installation, the loopback gave the response Cache policy the job it could perform and left the sandbox boundary intact.