Nikhil.
← All marginalia
AIJudgment
I can't tell, before I hit enter, whether its initiative will be a gift or a mess.

For Once, I Wasn't Annoyed

For once, an AI did way more than I asked, and I wasn't annoyed.

That's the part that's bothering me.

I was deep in a risk assessment yesterday, trying to wrangle a mess of numbers into something that made sense. I needed a few formulas. So I handed Claude the data and said, roughly, help me work out the risk here.

It gave me the formulas. Clean, correct, done.

Then I noticed it had also built a web page. A working calculator, just sitting there. Type in a number, watch the risk move. I never asked for it. I didn't even know I wanted it.

And it was better than the thing I actually asked for.

Here's why that unsettles me. I've spent weeks quietly fighting this exact behavior. The model that redecorates the whole room when you asked it to move one shelf. I've started adding lines to my prompts like a warning label: do only what I said. Nothing extra. Stay in your lane.

If I'd used that line yesterday, I'd have my formulas and nothing else. I'd have missed the best part of the whole session, and never known it was there to miss.

That's what gets me. The instinct that annoyed me last week and the one that delighted me yesterday are the same instinct. Identical. The only thing that changed was whether the surprise turned out to be useful.

So I don't know what to tell it anymore. Rein it in, and I lose the calculators I didn't know I needed. Let it run, and some days I'm just cleaning up.

Before I hit enter, I can't tell which day it'll be. And I don't think the model can either. It just tries, every time, and I find out along with it whether this was a gift or a mess.

Maybe that's the deal now. Not telling these things exactly what to do. Just living with the fact that they'll always reach past the ask, and rolling the dice on whether, this time, you're glad they did.

More writing