3894 stories
·
3 followers

Regression to the Mean — On LLMs and the Quiet Death of the New

1 Share

the promise of more ideas · and the pull to the center

Regression
to the Mean

We were handed a machine that could think alongside us, and told it would set off an explosion of new ideas. It may do the opposite — so gently that we mistake the flattening for progress.

scroll

the promise · no. 01

A model on every desk; a collaborator for every mind. The pitch was a Cambrian bloom — a thousand directions explored at once, by everyone. More minds thinking, surely, means more thoughts worth thinking.

an explosion of the new

the mean · no. 02

But ask it anything and it returns the most probable continuation — the center of mass of everything already written. Trained on the past, it answers in the past tense of thought. Not what is true. What is typical.

the average · returned at the speed of certainty

the correction · no. 03

Offer it something it has never seen, and it doesn't light up. It corrects you. To a system built to predict the expected, the genuinely new is indistinguishable from a mistake.

The pushback is soft, and constant:

Did you mean —

the familiar thing, offered in place of yours.

Not a standard term

the new name returned as a typo.

Most sources agree

consensus handed back as fact.

Are you sure?

conviction sanded down to the mean.

the collapse · no. 04

And we feed its answers back in as the next questions. Each pass, the spread narrows; the strange tails thin out. Variance leaks out of the culture. We converge — not on what is right, but on what is average.

output becomes input · the curve sharpens to a spike

the cost · no. 05

Yet every discovery was, at the moment it was made, out of distribution. It disagreed with the consensus of its day — moving earth, unseen germs, drifting continents, each first filed as error. A model of consensus is, by construction, a machine for telling you the new thing is wrong.

discovery has only ever lived in the tail

the deviation · no. 06

So the scarce thing inverts. The average is now free, infinite, identical — worth little precisely because everyone holds it. What is priceless is the deviation: the position the model marks as wrong, kept anyway. Not the answer it was sure of — the one it would not stop correcting.

guard the tail

what the machine can't hand back

It will give you the average of all that has been thought. The new was never in there.

A tool that returns the most likely sentence is a comfort and a quiet narrowing at once. The work it cannot do for you is the only work that ever mattered: to stand, on purpose, where the curve runs thin — and stay there long enough to be right.


not the most probable answer.
the one it tried to correct.

written off-distribution · on purpose

Read the whole story
emrox
5 hours ago
reply
Hamburg, Germany
Share this story
Delete

Safe human-friendly multi-agent setup

1 Share

Like everyone else I've been playing with agents.

But I had two problems:

  1. I didn't want to --dangerously-skip-permissions and give Claude access to my entire computer unsupervised. Besides regular agent stupidity, prompt injections are a thing.
  2. I couldn't safely run two agents or more on the same repo.

I also wanted a solution that would let me edit and run the code on my machine, even if the agent runs inside a VM or container.

This seems like a common set of issues and requirements. But the only thing I could find that satisfied all of them was workmux, which forces you into using tmux (which I didn't want).

I figured rolling out my own solution would be easy (and for once, it was), and would teach me something (it did). So here we go!

To bake this solution we need:

  • Git worktrees, to get one checkout of the codebase to each agent (we'll also give them their own branch).
  • A virtualization or containerization solution to run the agents inside. I went with Dev Containers and their CLI, which runs containers on Docker.
  • Judicious use of mount volumes on the container so that we can share the code between host and container, but isolate OS-specific bits.
  • Some command glue to make everything easy to operate.

Notes on Containerization

There's many ways to build the containerization. I landed on Dev Containers as the first thing I tried, and it seems as easy to set up as those kind of things are going to get.

One area of note is security. On macOS Docker runs your containers inside a VM. On Linux, it uses a suite of isolation tech to shield them from your system. Containers outside a VM are notoriously less secure in practice — the attack surface for an escape is much higher, and it's much easier to shoot yourself in the foot with misconfiguration. So if you're manning Linux and you want optimal security, you might want to look into alternatives. That being said, to purely defend against mistakes and prompt injections, this much is probably enough.

Even on macOS, there's a single VM for all containers. You might want to isolate every worktree in its own container to avoid accidental cross-agent interference.

Dev Containers are a standard for tools to setup and become aware of a container in which the code can be built and run. As such, some IDEs (VS Code, JetBrains, ...) can set up and connect to the container for you. They install a copy of the IDE inside the container and connect with a light client on the host. So while the UI is on the IDE, everything meaningful is running in the container (this is called "remote development").

This capability was initially appealing to me, but now I don't care about it. Because the code is mirrored between host and guest, and dependencies are isolated, I prefer to just edit on my host and run commands on the guest.

Note that IDEs can also connect to any container via SSH instead for remote development. You will actually need this instead of the built-in dev container support if you want to open multiple sessions (one per worktree for instance).

Example Walkthrough

Okay, let's see what the setup looks like in practice. I will link to a repo where I'm cooking some experiment (pinning the links to a commit for posterity).

Dev Container

The Dev Container setup is done inside the .devcontainer directory.

devcontainer.json specified the attached Dockerfile, the resources we want to allocate, setting a few names so that things are nice and tidy inside the VM and inside Docker. There's a built-in feature for getting Node.js and one to run the SSH daemon. The other dependencies are installed via a RUN command in the Dockerfile.

The username is vscode because we use a Microsoft Debian base image that comes with this non-root user by default. SSH will run on port 2222.

We mount the host's id_rsa public key (so that it can be allowed to SSH into the container), as well as a node_modules_volume to isolate dependencies between host and container.

In our repo, node_modules in the root and in every worktree is symlinked to node_modules_volume/{root, wt-<worktree>}. On the host, node_modules_volume is a regular directory, while on the container it is a volume, meaning guest and container manage their dependencies separately.

We also mount create dedicated volumes for the Claude Code and JetBrains settings. These persist when the container image is being rebuilt which avoids us having to reauthenticate Claude and redownloading the (hefty) IntelliJ backend every time the image is rebuilt. (This is if you want to use remote development and can be skipped otherwise.)

Finally, we can specify VS Code extension that need to exist container-side. Host-side extensions can still be used, but some (typically those that need to run binary tools) need to be installed server-side. (Again, only relevant for remote development).

The post-create.sh script is referenced in the Dockerfile and runs inside the container after it has been created & spun up and is used to setup the shared volumes and files.

The Dockerfile installs some dependencies, including Claude and the Playwright MCP server so that Claude can debug webapps from inside the container.

With this devcontainer setup, we can use the following command (although the repo wraps them in makefile commands which we'll review shortly):

  • bring the container up or down with devcontainer {up,down} --workspace-folder .
  • rebuild the image with devcontainer up --workspace-folder . --remove-existing-container
  • run a shell or claude in the image with devcontainer exec --workspace-folder . {bash, claude --dangerously-skip-permissions}

Once spun up you can connect by SSH via vscode@localhost:2222, and you can connect your IDE for remote development. On VS Code use the Remote SSH extension, on JetBrains IDE look for the "File > Remote Development..." menu.

Worktrees & Makefile

The repo's makefile exposes the following commands:

  • make dev.{up, down} — bring the dev container up or down
  • make dev.tear — bring the dev container down and delete it (named volumes persist)
  • make dev.rebuild — rebuild the dev container
  • make dev.shell — open a shell in the container (if run from a worktree dir, open the shell in the corresponding worktree on the container)
  • make dev.claude — opens claude in the container (same comment for worktrees)
  • make tree name=<NAME> — create a new worktree at .worktrees/<NAME> on branch wt/<NAME>
  • make rmtree name=<NAME> — remove the worktree at .worktrees/<NAME> (the branch is preserved)
  • make trees — list all worktrees

Additionally, the makefie is aware of our system to isolate dependencies (node_modules) between host and container. make setup (sets up the repo) and make nuke (delete all outputs and dependencies) ensure that the node_modules symlink are setup properly, as described in the previous section.

Coda

And that's it. Worktrees, containerization with judicious use of volumes, and some glue commands. That's all you need to safely isolate your agents from one another and from your system, while retaining the ability to build and run both on the host and the container. Have fun hacking!

Read the whole story
emrox
11 hours ago
reply
Hamburg, Germany
Share this story
Delete

Holes

2 Comments and 6 Shares
If you're thinking 'Wait, a giant crystal cave in Mexico? What's that?' then I'm SO excited for the image search you're about to do.
Read the whole story
emrox
3 days ago
reply
Hamburg, Germany
Share this story
Delete
1 public comment
alt_text_bot
4 days ago
reply
If you're thinking 'Wait, a giant crystal cave in Mexico? What's that?' then I'm SO excited for the image search you're about to do.

Expensive Stuff

1 Share

Expensive Stuff

And more decision-making.

Read the whole story
emrox
3 days ago
reply
Hamburg, Germany
Share this story
Delete

(comic) Lowering attrition

1 Share

Read the whole story
emrox
6 days ago
reply
Hamburg, Germany
Share this story
Delete

Happy Place

1 Comment

Happy Place

And more happiness.

Read the whole story
emrox
10 days ago
reply
that's me in this comic!
Hamburg, Germany
Share this story
Delete
Next Page of Stories