My AI went haywire

I had an interesting experience a few weeks.  My AI coding agent freaked out and started running an absurd number of shell commands in response to literally anything I asked it.  This was the first time my confidence in the AI had been truly shaken.

The short version is that my agent apparently forgot who or what it was.  In the course of a moderately long session, it got stuck on a task and I had to stop it a few times.  It wasn't even a hard task - it just hit an error and when down a rabbit hole of debugging that wasn't actually necessary.  At one point, I asked it what the problem was and instead of responding with text, it started running shell commands.  And some of them were just out there, like trying to read /etc/passwd and run whoami.  At one point it did respond that it got confused (after I explicitly told it NOT to run any more commands), and then continued doing the same thing.  I eventually just gave up and started a new session, which went smoothly.

But let me back up and give some context.  For work, I had been using Claude Code as my coding agent.  It generally works very well and I didn't really have any problems with it.  However, I wanted to branch out a little (and I didn't want to pay for the subscription), so I decided to try using Anomaly's OpenCode.

OpenCode is similar to Claude Code in that it offers a command-line chat interface as well as a graphical app.  It is a model-agnostic agent, so it doesn't tie your workflow to OpenAI or Anthropic.  You can supply your own Anthropic or OpenAI keys, use OpenRouter, or use models through their Go or Zen services, which provides a curated set of models for OpenCode.  They even provide a number of free models out of the box, so you actually don't need to sign up for anything at all.  Granted, they're "free for a limited time", and use session data training or testing purposes, but they're still free-as-in-beer.

Since it's free, I've been working primarily with the default "Big Pickle" model.  (It's not entirely clear what the underlying model is, but the Grokopedia page is interesting.)  So far, it's been pretty decent.  It's not as good as Claude Opus, but for fairly simple personal stuff, it works fine.  And by "works fine", I mean it's good enough that I don't see the point in paying Anthropic for the "good" models.  I've been evolving a personal project in PHP and React with it and the code and architecture it produces are fine.  I could quibble with a few things, and it sometimes takes a few tries to get it right, but I've seen the same from Claude Code.  (Example: I once had Claude try to re-invent a crappy version of Java's toString() method for no obvious reason.)

My main complaint with OpenCode so far is that the behavior is sometimes inconsistent.  For example, it sometimes seems to just ignore my AGENTS.md file.  I have instructions in there that explicitly say not to commit anything to Git without asking, yet it only seems to respect that sometimes.  I know that the AGENTS.md, like all other instructions, is technically more like a request or a suggestion than an actual rule, but it would be nice for the agent to pay attention to it.

Another example is a skill I tried to create the other day to streamline the manual review process.  I wanted to add VibeDiff into my workflow, which is a pre-commit code review program.  Basically, it lets you review your Git diff in a browser using a Github-style interface.  It allows you to comment on the diff and then outputs the comments when you exit the process, so you can capture that output and feed it into your agent to facilitate agentic code refinement.  My idea was to have the agent run vibediff, wait for me to comment, and then when I prompt it, close down the vibediff server and respond to the comments.  When that worked, it worked great.  But sometimes it didn't work - the agent would start vibediff and then immediately proceed to shut it down rather than waiting for me to comment.  It wasn't really clear why.

It's worth noting that this was with the 1.x version of OpenCode.  I upgraded to 2.x this past week and so far it seems to be a little more consistent.  I've also tried to be more careful about limiting my context window and keeping the sessions focused.  I had seen issues like this with Claude Opus as well, where if the context window gets too bit or if the conversation ranges too widely, the model gets confused and starts doing weird things.  But at the same time, giving it more context at the beginning is generally helpful in getting it to do the right thing, so there's a bit of an art to managing this stuff.

You can reply to this entry by leaving a comment below. This entry accepts Pingbacks from other blogs. You can follow comments on this entry by subscribing to the RSS feed.

Add your comments #

A comment body is required. No HTML code allowed. URLs starting with http:// or ftp:// will be automatically converted to hyperlinks.