ComfyUI on demand: systemd socket activation for a GPU container

Even with nothing to do, my ComfyUI container held 0.5 to 2.8 GB of VRAM on a GPU where my language models were fighting for every gigabyte. I wanted image generation available at any moment from my chat UI, but I did not want to pay for it while nobody was drawing anything.

The answer was twenty lines of systemd: a socket that listens on ComfyUI’s port, starts the container on the first connection, and lets it stop after 10 idle minutes. Together with a small guard that makes room for image jobs and swaps to a lighter model when a big LLM is awake, image generation now costs zero VRAM when idle.

Contents · 9 sections

TL;DR #

• systemd socket activation plus systemd-socket-proxyd turns any HTTP service that cannot itself be socket-activated into an on-demand one.
• StopWhenUnneeded=yes on the backend unit makes it stop automatically when the proxy exits after its idle timeout.
• Monitoring that probes the port every few seconds keeps waking it. Point health checks at something else.
• A guard process polls every second: when an image job is queued and a big LLM is loaded but idle, it unloads the LLM first. Without that, a forced VRAM eviction made one image take 113 seconds instead of about 24.
• When a big LLM is awake, the guard switches the chat UI’s image model from SDXL to SD1.5, which needs about 4 GiB instead of about 9.

Background: one GPU for everything #

My home lab runs on one 32 GB AMD GPU. It serves an always-on 9B language model, embedders and a reranker, an on-demand 27B model, vision models for OCR, and the desktop. Image generation is a guest on that card.

ComfyUI runs in a ROCm container built on PyTorch for ROCm. I chose it over the older Stable Diffusion web UI for its active development, its integration with Open WebUI, and its ROCm support; the old UI’s checkpoints were reused. Measured on this card:

• SDXL, 1024x1024, 25 steps: 17 seconds warm, 32 to 40 seconds cold;
• SD1.5, 512x768: about 9 seconds;
• card peak during an SDXL job with the language models resident: 30.8 of 31.9 GiB.

That last number is the problem. About 1 GiB of headroom. I tried the obvious flags first:

• --reserve-vram 5 had no effect on ROCm: peak 31.3 GiB, models stayed resident;
• --lowvram worked but took 235 seconds per image warm: unusable;
• --normalvram plus an explicit POST /free after 90 idle seconds was the best of the three.

Still, a ComfyUI process sitting idle kept 0.5 to 2.8 GB. The only way to make that zero is for the process not to exist.

Socket activation for a service that does not support it #

systemd can listen on a port on behalf of a service and start the service when a connection arrives. Normally the service must accept the listening socket from systemd. ComfyUI does not, and neither do most containers.

systemd-socket-proxyd bridges the gap. It is a tiny proxy that does accept the socket from systemd and forwards each connection to a real address. The chain is:

1. comfyui-proxy.socket listens on the public port.
2. The first connection starts comfyui-proxy.service, which runs systemd-socket-proxyd to the backend port.
3. The proxy Requires= the backend, so comfyui.service (the container) starts.
4. After 10 minutes with no connections, the proxy exits.
5. Nothing needs the backend any more, and StopWhenUnneeded=yes stops it.

The units, sanitised:

# comfyui-proxy.socket
[Unit]
Description=ComfyUI on demand - starts ComfyUI on the first connection

[Socket]
ListenStream=127.0.0.1:<public-port>
FreeBind=true

[Install]
WantedBy=sockets.target

# comfyui-proxy.service
[Unit]
Requires=comfyui.service comfyui-proxy.socket
After=comfyui.service

[Service]
ExecStart=/usr/lib/systemd/systemd-socket-proxyd --exit-idle-time=10min 127.0.0.1:<backend-port>

# comfyui.service (the container)
[Unit]
After=docker.service
Requires=docker.service
StopWhenUnneeded=yes

[Service]
Type=simple
Restart=on-failure
# docker stop on idle exits 143 (SIGTERM) or 137 (SIGKILL after the timeout): a clean stop
SuccessExitStatus=143 137 130
ExecStartPre=-/usr/bin/docker rm -f comfyui
ExecStart=/usr/bin/docker run --rm --name comfyui \
  --device /dev/kfd --device /dev/dri \
  -p 127.0.0.1:<backend-port>:<container-port> \
  -v <models>:/store <comfyui-rocm-image> \
  --listen 0.0.0.0 --port <container-port> --normalvram
# hold the first connection until ComfyUI actually answers
ExecStartPost=/bin/sh -c 'for i in $(seq 1 120); do \
  curl -sf http://127.0.0.1:<backend-port>/system_stats >/dev/null && exit 0; sleep 1; done; exit 1'
ExecStop=/usr/bin/docker stop -t 30 comfyui

Details that matter:

• ExecStartPost with a readiness loop means the proxy’s After=comfyui.service really waits for a working ComfyUI, not just a started container. The first request is slow (a cold start), but it succeeds.
• SuccessExitStatus=143 137 130: docker stop makes the container exit with a signal status. Without this line, every idle stop is logged as a failure and Restart=on-failure would bring it straight back.
• FreeBind=true lets the socket bind to an address that is not up yet at boot.

Find whatever keeps waking it #

The first day, ComfyUI never stopped. A monitoring agent had an HTTP check and a port check on ComfyUI’s port, every few seconds. Each check was a connection, each connection reset the proxy’s idle timer, and the first one after a stop started the container again.

I removed those checks. Anything that monitors an on-demand service must check something that does not wake it: the systemd unit state, or the socket unit.

The guard: make room before the image job allocates #

On-demand start solved idle VRAM. It did not solve the other direction: a big language model sitting idle in VRAM when an image job arrives.

That happened the same week. With the 27B model (about 19 GB) idle in VRAM, an image request from the chat UI made the driver evict the model’s memory to system RAM. The image took 113 seconds instead of about 24, and the GPU firmware logged MES failed to respond to INVALIDATE_TLBS, the same eviction path that had deadlocked the driver earlier that day.

So a small Python guard runs permanently beside ComfyUI (as its own service, deliberately not bound to the ComfyUI unit, which now stops and starts all the time). Every second it does this:

def make_room():
    # ComfyUI starting means a request is on its way: act before it allocates
    starting = systemctl_is("comfyui.service") == "activating"
    if not starting:
        q = comfy_queue()                 # GET <backend>/queue; refused = no job
        if not q or not (q["queue_running"] or q["queue_pending"]):
            return
    for model in big_models_loaded():     # router GET /models, status loaded/loading
        if recently_used(model, seconds=20):
            continue                      # never cut off a model that is working
        router_unload(model)              # POST /models/unload {"model": model}

Three rules in that loop came from real failures:

• Poll the backend port, never the public one. Polling the public port would wake ComfyUI.
• Treat “activating” as a job. The connection that started ComfyUI is an image request; free memory before it loads a checkpoint.
• Never unload a busy model. “Busy” means a slot is processing right now (a long prompt can take minutes without a log line) or the router proxied a request to it in the last 20 seconds. If in doubt, assume busy: the image job then runs slower beside the model, which is better than killing someone’s answer.

The second rule: a smaller image model while a big LLM is awake #

Sometimes the big model is busy and cannot be unloaded. For that case the guard also switches the chat UI’s image model:

• big model loaded or loading: SD1.5 at 512x768 (about 4 GiB, about 9 seconds);
• otherwise: SDXL at 1024x1024.

One catch: Open WebUI reads its configuration table only at startup, so changing the database does nothing to a running instance. The guard uses Open WebUI’s admin API instead (GET then POST of the images configuration), with an API key stored in a root-only environment file. A model set by hand that is not one of the two profiles is left alone.

Measured: the big model woke at 13:17:24, the image model switched to SD1.5 at 13:17:33; after the big model went to sleep, SDXL was restored.

Image editing on the same setup #

Later I added image editing through ComfyUI with Qwen-Image-Edit (about 29 GB of model files including the text encoder and a Lightning LoRA). On this card it takes about 3 minutes cold and 1.5 to 2 minutes warm. When the 27B is busy and cannot be unloaded, ComfyUI can only load part of the edit model and slows to about 25 seconds per step, which is the guard’s “run beside it” case made visible.

Checklist #

• If a GPU service holds memory while idle, make the process go away, not just its models.
• Use a .socket unit plus systemd-socket-proxyd --exit-idle-time for services that cannot accept a socket from systemd.
• Put StopWhenUnneeded=yes on the backend, and a readiness loop in ExecStartPost.
• Add SuccessExitStatus for the exit codes of a normal container stop.
• Remove monitors that connect to the on-demand port; check unit state instead.
• Run the VRAM guard as its own service, polling the backend, never the public socket.
• Unload idle big models before an image job allocates; never unload a busy one.

Frequently asked questions #

Can I socket-activate a Docker container with systemd? #

Yes, through a proxy. A .socket unit listens on the port, its service runs systemd-socket-proxyd to the container’s real port, and that service Requires= the container’s unit. With --exit-idle-time on the proxy and StopWhenUnneeded=yes on the container unit, it also stops by itself.

Does --reserve-vram work for ComfyUI on AMD ROCm? #

In my tests it had no effect: peak VRAM and resident models were the same with and without it. --lowvram reduced memory but made SDXL about six times slower. An idle-time /free call, and stopping the process entirely when idle, worked better.

Why does my on-demand service never stop? #

Something keeps connecting to it. Every connection resets the idle timer. Look for health checks, port scanners, dashboards and uptime monitors pointing at that port.

How do I stop image generation from evicting my LLM? #

Unload the LLM before the image job allocates, if it is idle, and use a smaller image model while it is busy. Watching the ComfyUI queue and the service’s “activating” state once a second was fast enough in my setup.

The lesson #

The cheapest VRAM is the VRAM held by a process that is not running. systemd already has the pieces to make almost any service start on demand and stop when idle. The work is in the edges: what wakes it, what counts as a clean stop, and who gets the memory when two workloads want it at once.