Bare-Metal Inference Without Aircon: GPUs at 85°C and a Fan

The data center aesthetic—cold aisles, humming CRAC units, and a power bill that makes CFOs wince—is a luxury. For a homelab or a small office, it's often overkill. But what if you don't have aircon? What if your GPU server sits in a room that hits 35°C in summer? The answer isn't to buy a portable AC unit (though that helps). It's to embrace the heat and manage it with software. This article is about running bare-metal inference on consumer GPUs without air conditioning, accepting 85°C as a normal operating point, and using a script and a fan to keep things from melting.

The Thermal Reality of GPUs

GPUs are designed to run hot. The thermal specification for most NVIDIA consumer cards is around 90-95°C at the junction. Running at 85°C is not immediately dangerous, but it does affect longevity and triggers thermal throttling. The throttle point is typically around 83-85°C for many models, where the GPU reduces clocks to keep the temperature in check. This is a safety mechanism, not a failure. The key is to understand that you can run at 85°C and still get usable inference throughput, provided you manage the throttling intelligently.

The Setup: A Fan, a Script, and a GPU

In a typical bare-metal setup, you have a server with one or more GPUs, each with its own cooling fan. The GPU firmware controls the fan curve, but you can override it. The simplest approach is to set a custom fan curve using nvidia-smi or tools like nvfancontrol. But a more flexible approach is to write a script that monitors temperature and adjusts fan speed accordingly.

Here's a minimal Python script that does this:

import subprocess
import time
import sys

FAN_CONTROL_PATH = "/sys/class/drm/card0/device/hwmon/hwmon0/pwm1"
TEMP_PATH = "/sys/class/drm/card0/device/hwmon/hwmon0/temp1_input"

MIN_FAN = 30
MAX_FAN = 100
TEMP_TARGET = 80

while True:
    with open(TEMP_PATH, "r") as f:
        temp = int(f.read()) / 1000.0
    with open(FAN_CONTROL_PATH, "w") as f:
        if temp > TEMP_TARGET:
            duty = min(MAX_FAN, (temp - TEMP_TARGET) * 10 + MIN_FAN)
        else:
            duty = MIN_FAN
        f.write(str(int(duty)))
    time.sleep(5)

This script reads the GPU temperature and sets the fan PWM duty cycle. It's a crude proportional controller, but it works. For multiple GPUs, you'd need to adjust the paths or use nvidia-smi with -i to query each GPU.

Understanding Thermal Throttling

Thermal throttling is not binary. It's a curve. As the temperature approaches the throttle point, the GPU reduces clock speeds in steps. For inference workloads, which are often memory-bound, the performance hit might be less than you think. For example, a large language model inference is heavily dependent on memory bandwidth, not raw compute. So even if the GPU clocks drop by 10-15%, the inference latency might only increase by 5-10%. This is a trade-off you can live with.

To see the current throttle status, use:

nvidia-smi -q -d PERFORMANCE

This shows the current throttle reasons, such as "GPU Temp" or "Power Cap".

The Fan: Not Just for Show

A simple box fan pointed at the server can dramatically improve airflow. The GPU fans are designed to exhaust heat out the back, but if the ambient air is already hot, they're just recirculating hot air. A high-velocity fan that blows fresh air across the intake can lower the intake temperature by several degrees. This is a cheap and effective solution.

Software Strategies to Reduce Heat

Beyond fan control, you can also reduce heat generation by:

• Limiting power draw: Use nvidia-smi -pl 200 to cap the power limit. This reduces heat output proportionally. For inference, you might not need the full TDP.
• Batching requests: If you're serving inference, batch requests to keep the GPU busy but not overworked. Idle GPUs still draw power and generate heat, so it's better to run them at a steady state.
• Using efficient models: Smaller models or quantized models (e.g., 4-bit) generate less heat because they use less compute and memory bandwidth.

A Realistic Example: Running Qwen on vLLM

In a typical setup, you might run a Qwen model on vLLM. vLLM is a high-throughput inference engine that can be tuned to manage GPU utilization. You can set --max-num-seqs to control concurrency, and --gpu-memory-utilization to limit memory usage. These parameters affect GPU load and thus temperature.

For example, to limit memory utilization to 0.7 and max sequences to 8:

python -m vllm.entrypoints.openai.api_server \
    --model Qwen/Qwen2.5-7B-Instruct \
    --gpu-memory-utilization 0.7 \
    --max-num-seqs 8 \
    --port 8000

This reduces the VRAM footprint and limits the number of concurrent requests, which can lower the sustained GPU load and temperature.

Monitoring and Alerting

You should monitor temperatures not just for immediate throttling but for trends. A simple script can log temperatures and alert if they exceed a threshold for a prolonged period. Here's a simple bash one-liner to log temperature every minute:

while true; do echo "$(date) $(nvidia-smi --query-gpu=temperature.gpu --format=csv,noheader)" >> /var/log/gpu_temp.log; sleep 60; done

The Trade-Offs

Running at 85°C is not ideal for hardware longevity. The rule of thumb is that every 10°C increase halves the lifespan of electronic components. But for a homelab where you're replacing hardware every few years, it might be an acceptable trade-off. Also, the GPU fans will spin at high RPM, which creates noise. If that's a concern, you might want to invest in a quieter fan or accept the noise.

Conclusion

Bare-metal inference without aircon is feasible. It requires a willingness to accept high temperatures, a script to control fans, and an understanding of thermal throttling. The key is to treat 85°C as a normal operating point, not a failure. With careful software tuning, you can run inference workloads reliably, even in a hot room.

Remember, the goal is not to keep the GPU cool, but to keep it from throttling too much. A fan, a script, and a bit of patience are all you need.