Zed Editor + Docker Agent + ACP: coding with a local agent plugged into llmman

In two minutes

Today I'm showing you how to plug Zed, my code editor, into an agent that runs locally, thanks to the ACP protocol.

The agent in question is Docker Agent. And for the language model, I'm not going to call a cloud API: I'm going to point Docker Agent at llmman, which serves my local model behind an OpenAI-compatible API.

I wrote an article on how to use llmman: k33g.org/p/20261001-llmman.

Here's a summary of my setup for this article:

Of course, you can use other LLM engines instead of llmman: Docker Model Runner, Ollama, or any other server that speaks the OpenAI API will work just fine. More on that at the end of the article.

Quick reminder on the 4 building blocks used

Zed Editor

Zed is a code editor that supports the ACP (Agent Client Protocol) protocol. Concretely, that means you can plug any ACP-compatible agent directly into Zed's agent panel, and chat with it just like you would with a built-in agent (for example the Claude Code extension in VSCode).

Documentation: zed.dev/acp/editor/zed

ACP: Agent Client Protocol

ACP is an open protocol (on the same model as LSP, but for agents) that standardizes communication between a code editor (the ACP client) and a code agent (the ACP server). The agent runs as a separate process, "speaks JSON-RPC" over stdin/stdout, and since it's a standard, the editor doesn't need to know the "inner workings" of the agent to use it.

Documentation: agentclientprotocol.com and zed.dev/acp

Docker Agent

Docker Agent is a code agent runtime (even multi-agent) configurable via a simple YAML file: you describe your providers (how to connect to the model(s)), your agents (instructions, model used, allowed tools) and your models. It knows how to expose these agents through several interfaces, including... ACP. That's exactly what I need to bridge Zed and my local models.

llmman

llmman is the tool that serves my local models behind a single API compatible with OpenAI (but also Ollama and Anthropic). It relies on llama.cpp as the inference engine and manages models as OCI images (Docker Hub, Hugging Face, private registry...). For this article, it'll be the server that Docker Agent queries to get the model's responses.

I talked about it recently over here: k33g.org/p/20261001-llmman, to explain how to install and use it (a kind of first contact).

Setting up all the components

Install Docker Agent (on macOS)

brew install docker-agent

For other platforms, check the documentation: docker.github.io/docker-agent

Start llmman as an LLM server

If it's not already done (see my dedicated article for the installation):

llmman serve

Then you download a model (from Hugging Face for example) and check that everything went well:

llmman pull huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M
llmman list

Configure Docker Agent to talk to llmman

Here's the configuration file I use, llmman.docker-agent.yaml:

providers:
  llmman:
    api_type: openai_chatcompletions
    base_url: http://127.0.0.1:17434/v1

agents:

  root:
    model: mellum
    description: A local code agent, powered by a small LLM.
    instruction: |
      Your name is Riker. You are a developer expert.
      You act as a mentor and coding partner. 

      # Tool discipline

      Built-in tools:
      - shell: use the `shell` tool to explore the project, read files, or run commands.
        Chain commands as long as it is useful, then give a clear final answer.

      # Style

      Be concise. Prefer a small, correct, compilable example over prose.

    toolsets:
      - type: shell

models:
  mellum:
    provider: llmman
    model: huggingface.co/jetbrains/mellum2-12b-a2.5b-instruct-gguf-q4_k_m:Q4_K_M

The important bit is base_url: http://127.0.0.1:17434/v1: that's the address where llmman exposes its OpenAI-compatible API. Everything else (agent, instructions, tools) is standard Docker Agent configuration.

Now that we have a model and a way to connect to it via an API and Docker Agent, let's make the connection with Zed.

Declare the agent in Zed

In Zed's settings (settings.json), we add an agent_servers section that contains a sub-section pointing to the docker-agent serve acp command with the path to our YAML file:

"agent_servers": {
  "🤖 Docker Agent (llmman)": {
    "type": "custom",
    "command": "docker-agent",
    "args": [
      "serve",
      "acp",
      "/Users/k33g/kDrive/Rickub/demo-docker-agent-acp/llmman.docker-agent.yaml"
    ],
    "env": {}
  }
}

Of course, adapt the path to your own Docker Agent configuration file.

Let's go

Open Zed's agent panel, pick "🤖 Docker Agent (llmman)" from the list of available agents, and chat with it like with any code agent: it explores your project, reads files, runs shell commands, all while running entirely locally, on your own model:

The agent asks whether you want to run a command:

What if I don't want to use llmman?

Nothing forces you to. The only contract between Docker Agent and the model server is an OpenAI-compatible API. So in the providers file, you can replace base_url with any server that exposes this type of API:

• Docker Model Runner, which, like llmman, relies on llama.cpp internally.
• Ollama — with its OpenAI-compatible API.
• ... or any other runtime that knows how to expose this same kind of endpoint.

To sum up

• Zed speaks ACP to Docker Agent.
• Docker Agent speaks OpenAI API to a model server, here llmman, which relies on llama.cpp.
• Everything runs locally.

There you go, it's a simple setup to put in place, and it lets you keep full control over your model while enjoying Zed's developer/user experience. Feel free to ask questions 🙂