Jurisdictional Inference Routing: Complying with the EU AI Act at the Proxy Level

The EU AI Act introduces tiered obligations for high-risk AI systems, including transparency, human oversight, and risk management. For organizations serving LLM inference to users in the EU, this means ensuring that inference requests from EU citizens are processed under compliant conditions. One pragmatic approach is to route inference requests based on the jurisdiction of the end user at the proxy level. This article walks through a concrete implementation of jurisdictional inference routing using a lightweight Go proxy that inspects request origin and routes to appropriate inference endpoints.

Why Proxy-Level Routing?

Embedding compliance logic directly into the inference server can be brittle and hard to audit. A separate proxy layer decouples compliance from inference, making it easier to update rules without touching the model serving infrastructure. It also allows you to enforce routing before any data reaches the model, reducing the risk of non-compliant processing.

Architecture Overview

The system consists of three components:

1. Inference Proxy – A Go service that receives incoming LLM requests, inspects the client IP, looks up its jurisdiction via a GeoIP database, and forwards the request to the appropriate inference endpoint.
2. GeoIP Database – A local MaxMind GeoLite2 database for mapping IPs to countries.
3. Inference Endpoints – Two or more endpoints: one for EU requests (e.g., running on EU-hosted infrastructure with full compliance logging) and one for non-EU requests (e.g., using a cheaper, less-compliant provider).

Implementation Details

Step 1: Proxy Service in Go

We'll use maxminddb-golang to read the GeoIP database. The proxy listens for HTTP requests, extracts the client IP from headers (X-Forwarded-For or X-Real-IP), looks up the country, and routes accordingly.

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"net"
	"net/http"
	"net/http/httputil"
	"net/url"
	"strings"

	"github.com/oschwald/geoip2-golang"
)

var (
	euEndpoint, _ = url.Parse("https://eu-inference.example.com")
	globalEndpoint, _ = url.Parse("https://global-inference.example.com")
	db              *geoip2.Reader
)

// EU country codes (ISO 3166-1 alpha-2)
var euCountries = map[string]bool{
	"AT": true, "BE": true, "BG": true, "HR": true, "CY": true, "CZ": true,
	"DK": true, "EE": true, "FI": true, "FR": true, "DE": true, "GR": true,
	"HU": true, "IE": true, "IT": true, "LV": true, "LT": true, "LU": true,
	"MT": true, "NL": true, "PL": true, "PT": true, "RO": true, "SK": true,
	"SI": true, "ES": true, "SE": true,
}

func getClientIP(r *http.Request) string {
	if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
		parts := strings.Split(xff, ",")
		return strings.TrimSpace(parts[0])
	}
	if xri := r.Header.Get("X-Real-IP"); xri != "" {
		return xri
	}
	ip, _, _ := net.SplitHostPort(r.RemoteAddr)
	return ip
}

func routeRequest(w http.ResponseWriter, r *http.Request) {
	clientIP := getClientIP(r)
	ip := net.ParseIP(clientIP)
	if ip == nil {
		http.Error(w, "Invalid client IP", http.StatusBadRequest)
		return
	}

	record, err := db.City(ip)
	if err != nil {
		log.Printf("GeoIP lookup failed for %s: %v", clientIP, err)
		// Default to global endpoint
		reverseProxy(globalEndpoint, w, r)
		return
	}

	country := record.Country.IsoCode
	if euCountries[country] {
		reverseProxy(euEndpoint, w, r)
	} else {
		reverseProxy(globalEndpoint, w, r)
	}
}

func reverseProxy(target *url.URL, w http.ResponseWriter, r *http.Request) {
	proxy := httputil.NewSingleHostReverseProxy(target)
	proxy.ServeHTTP(w, r)
}

func main() {
	var err error
	db, err = geoip2.Open("GeoLite2-City.mmdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	http.HandleFunc("/", routeRequest)
	log.Fatal(http.ListenAndServe(":8080", nil))
}

Step 2: Deploying the Proxy

Build the binary and run it behind a reverse proxy like Nginx that terminates TLS and sets X-Forwarded-For correctly. Ensure the GeoIP database is updated regularly (e.g., via a cron job).

Step 3: Configuring Inference Endpoints

The EU endpoint should run on infrastructure within the EU, with full logging of prompts and responses for transparency, and a mechanism for human oversight (e.g., random sampling by a compliance officer). The global endpoint can be a more cost-effective provider, but must still adhere to basic data protection.

Compliance Considerations

• Transparency: For EU requests, the proxy can inject a header indicating that the request is subject to EU AI Act compliance. The inference server can then log the interaction for auditing.
• Risk Management: If you identify a high-risk use case (e.g., credit scoring), you can route those requests to a separate endpoint with additional safeguards.
• Data Sovereignty: By routing to EU-hosted endpoints, you ensure that personal data does not leave the EU without adequate protections.

Performance Overhead

The GeoIP lookup adds ~1-2 ms per request. The reverse proxy adds negligible latency (typically <1 ms). Total overhead is under 5 ms, which is acceptable for most LLM applications.

Lessons Learned

• IP spoofing: Relying on client IP is imperfect; consider requiring authentication tokens for critical compliance paths.
• Database updates: MaxMind free databases are updated monthly. Automate updates to avoid routing based on stale data.
• Fallback behavior: When GeoIP fails, default to the most restrictive endpoint (EU) to err on the side of caution.
• Testing: Use a VPN to test routing from different EU countries.

Conclusion

Jurisdictional inference routing at the proxy level is a practical, low-overhead way to comply with the EU AI Act's territorial scope. By decoupling compliance from inference, you maintain flexibility and auditability. The implementation described here is production-ready and can be extended with additional rules (e.g., based on user role, model type, or risk category).

For a production deployment, consider adding metrics, health checks, and a configuration file to manage endpoints without recompiling.