<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Open Source on Liam Ding</title>
    <link>https://www.liamding.com/en/tags/open-source/</link>
    <description>Recent content in Open Source on Liam Ding</description>
    <image>
      <title>Liam Ding</title>
      <url>https://www.liamding.com/img/og-cover.png</url>
      <link>https://www.liamding.com/img/og-cover.png</link>
    </image>
    <generator>Hugo -- 0.147.7</generator>
    <language>en</language>
    <lastBuildDate>Thu, 12 Nov 2026 00:00:00 +0800</lastBuildDate>
    <atom:link href="https://www.liamding.com/en/tags/open-source/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>AI writes the code now. Four open-source projects for checking it</title>
      <link>https://www.liamding.com/en/posts/2026-11-12-four-open-source-projects-for-checking-ai-written-code/</link>
      <pubDate>Thu, 12 Nov 2026 00:00:00 +0800</pubDate>
      <guid>https://www.liamding.com/en/posts/2026-11-12-four-open-source-projects-for-checking-ai-written-code/</guid>
      <description>Writing code got cheap and checking it didn&amp;#39;t. Four open-source projects that put a separate checker at a different stage, compared by what each checks and where the check leaks.</description>
      <content:encoded><![CDATA[<h2 id="the-problem">The problem</h2>
<p>On 8 November (US Pacific time) Ryan Dahl, who created Node.js and Deno, posted: &ldquo;it&rsquo;s shocking that software engineering has transitioned to 100% AI written in such a short time.&rdquo; He went on: &ldquo;It writes C++ better than I ever could, but the problem of making sure it works didn&rsquo;t magically disappear. we are just shipping enormously more volume at higher quality.&rdquo; By the time I read it on 11 November it had about 1.84 million views.</p>
<p>That is the part of the job that did not get cheaper. A model can produce a thousand lines in a minute; someone still has to know whether the design was right, whether the cheap model followed the plan, and whether the change opened a hole.</p>
<p>So this week&rsquo;s open-source picks are four projects that each try to own a piece of that checking. Three of them were on GitHub&rsquo;s trending pages when I looked, and Google announced the fourth on 7 November. I opened all four on 11 November 2026, around 3 PM Pacific.</p>
<h2 id="what-i-concluded">What I concluded</h2>
<p>All four separate the check from the work in some way, and two of them, EGM and Cloudflare&rsquo;s skill, state the rule outright: the agent that checks must not be the agent that did the work, and it should start without the author&rsquo;s context. They differ in <strong>when</strong> they check:</p>
<ol>
<li>Before any code exists: review the design.</li>
<li>At handoff: give a cheaper model a plan whose every step has a command and an expected result.</li>
<li>Before release: hunt for vulnerabilities, then have a different agent try to disprove each one.</li>
</ol>
<p>Pick the one that sits at the stage where your mistakes are most expensive. And treat &ldquo;fresh context&rdquo; as a claim to test, not a feature you get by installing a plugin.</p>
<h2 id="constraints">Constraints</h2>
<p>Anyone comparing these should know:</p>
<ul>
<li>Star counts and commit dates move daily. The numbers below are a snapshot from the GitHub API on 11 November 2026; commit dates are UTC.</li>
<li>I read the READMEs, docs and one issue. I did not install or run any of these, so every claim about behaviour is the project&rsquo;s own.</li>
<li>Two of the four have had no commits on main since mid-September. That is not necessarily abandonment for a skill made of Markdown prompts, but it is worth knowing.</li>
<li>They solve different problems. The comparison is about where the check sits and how independent it is, not which one is &ldquo;best&rdquo;.</li>
</ul>
<h2 id="the-options-and-what-i-traded-away">The options, and what I traded away</h2>
<table>
  <thead>
      <tr>
          <th>Repo</th>
          <th>License</th>
          <th>Stars (11 Nov)</th>
          <th>Latest commit on main</th>
          <th>What it checks, and when</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><a href="https://github.com/google/elephant-goldfish-model">google/elephant-goldfish-model</a></td>
          <td>Apache-2.0</td>
          <td>17,657</td>
          <td>11 Nov 2026</td>
          <td>The design, before code</td>
      </tr>
      <tr>
          <td><a href="https://github.com/shadcn/improve">shadcn/improve</a></td>
          <td>MIT</td>
          <td>10,810</td>
          <td>12 Sep 2026</td>
          <td>The plan and its execution, at handoff</td>
      </tr>
      <tr>
          <td><a href="https://github.com/cloudflare/security-audit-skill">cloudflare/security-audit-skill</a></td>
          <td>MIT</td>
          <td>70,320</td>
          <td>14 Sep 2026</td>
          <td>Vulnerabilities, each one challenged by a second agent</td>
      </tr>
      <tr>
          <td><a href="https://github.com/openai/codex-security">openai/codex-security</a></td>
          <td>Apache-2.0</td>
          <td>11,676</td>
          <td>11 Nov 2026</td>
          <td>Vulnerabilities in a repo or a diff, plus patches and fix checks</td>
      </tr>
  </tbody>
</table>
<p><strong>Elephant-Goldfish Model: check the design before anyone writes code.</strong> A plugin for Antigravity, Claude Code and Codex. The &ldquo;elephant&rdquo; session holds your project context and drafts the specification. &ldquo;Goldfish&rdquo; reviewers get the draft and the source it references, without the earlier conversation, and look for unstated assumptions. You resolve the trade-offs and approve the spec before implementation. If your agent cannot create fresh subagents, EGM stops the review instead of running the review in the same context. There is also a drift report that compares design docs with the current code. The README is careful: &ldquo;A review does not replace tests or human judgment,&rdquo; and this is &ldquo;not an officially supported Google product.&rdquo;</p>
<p>The interesting detail is in its own docs. The support page says reviewer isolation evidence is &ldquo;narrower&rdquo; than the install tests. Issue <a href="https://github.com/google/elephant-goldfish-model/issues/237">#237</a>, open since 30 October, reports that on Codex the reviewers still receive the project and user <code>AGENTS.md</code>, and on Antigravity the project <code>AGENTS.md</code> and <code>GEMINI.md</code>. The doc says Claude&rsquo;s reviewer adapters are set to leave out <code>CLAUDE.md</code>, and that for Codex and Antigravity no instruction-file exclusion control is verified. The reporter&rsquo;s argument is that these files often hold a team&rsquo;s assumptions (&ldquo;we always use X&rdquo;), so a reviewer that starts with them reviews with the author&rsquo;s priors. EGM&rsquo;s own advice is to keep stable conventions in those files and write any assumption the Goldfish must challenge as an explicit review criterion.</p>
<p><strong>improve: make the plan carry its own test.</strong> A skill from shadcn that audits a codebase with your most capable model and writes plans for cheaper models to execute. It never edits source itself; it only writes to <code>plans/</code>. Each plan step ends with a command and its expected output, plans record the commit they were written against so the executor can check for drift, and they spell out STOP conditions. The README admits that its audit subagents over-report, so the advisor re-reads every cited location before showing you a finding. <code>execute</code> sends a cheaper model into an isolated git worktree, then re-runs every done criterion and reads the diff, with at most two rounds of revision. Merging stays with you. The cost: the last commit on main was 12 September, and the quality of the whole thing rests on how good those gate commands are.</p>
<p><strong>Cloudflare&rsquo;s security-audit-skill: the finder never confirms its own find.</strong> A six-phase audit skill: reconnaissance, hunting, then every candidate goes to a fresh verifier that tries to disprove it. Results are one of three verdicts, <code>confirmed</code>, <code>needs_validation</code> or <code>rejected</code>, and a zero-dependency Node validator checks <code>findings.json</code> against a JSON schema (a second one checks the coverage ledger). The README states the rule plainly: &ldquo;The agent that checks a finding is never the agent that found it.&rdquo; It also requires an OS-level sandbox with external networking disabled for anything that builds or runs the target. Two numbers in its docs are worth keeping. In its test runs, a single run found roughly half of the vulnerabilities that repeated runs found in total. And Cloudflare&rsquo;s own blog says that when they grew this into a fleet scanner over 128 repos, the worst single repo took just over 14 hours, so full scans became a periodic backlog sweep rather than a per-PR check. Every confirmed finding there must ship a proposed patch.</p>
<p><strong>Codex Security: the packaged version.</strong> OpenAI&rsquo;s CLI, TypeScript SDK and GitHub Action. It scans a repo, chosen paths or a Git diff (<code>cs scan . --diff origin/main</code>), validates candidate findings, generates patches and verifies existing fixes, and exports SARIF, JSON or CSV. The weekly Action example sets an estimated $5 limit per scan and reports findings without failing the job; the README notes that requests in progress can go over the limit. It works with OpenAI, Amazon Bedrock, OpenRouter, Fireworks AI and Cheaper Inference; each provider needs its own credentials. Some cyber requests and protected findings need OpenAI&rsquo;s Trusted Access for Cyber. This repo is the most active of the four; its last commit was a few minutes before I looked.</p>
<h2 id="evidence">Evidence</h2>
<p>Put Dahl&rsquo;s problem next to these and a pattern shows up:</p>
<ul>
<li>Each check produces something you can inspect afterwards: a reviewed spec, a plan with commands, a <code>findings.json</code> with verdicts, a saved report. None of them asks you to trust a chat reply that says &ldquo;looks good&rdquo;.</li>
<li>Where independence is the stated goal (EGM and Cloudflare&rsquo;s skill), it is also the hardest part to verify. EGM documents its own gap. Cloudflare&rsquo;s skill gets independence by spawning a new agent, which is only as fresh as the host lets it be. The README doesn&rsquo;t say whether host instruction files reach that agent.</li>
<li>Repetition matters. In Cloudflare&rsquo;s test runs, one run found about half of what several runs found; Codex Security&rsquo;s deep mode repeats standard scans and deduplicates the results. A single pass is a sample.</li>
<li>Stars don&rsquo;t line up with any of this. The 70,320-star repo has not changed since September, and the README calls it the single-repo starting point the later system evolved from. The most active of the four is OpenAI&rsquo;s codex-security; its latest commit landed minutes before I looked.</li>
</ul>
<p>Counter-arguments worth taking seriously. &ldquo;Tests already do this&rdquo;: for behaviour you thought to test, yes. None of these replaces tests, and improve and Cloudflare both lean on the repo&rsquo;s own test commands. They target the design mistakes and the holes nobody wrote a test for. &ldquo;A second model has the same blind spots&rdquo;: partly. That is why the fresh context matters, and why Cloudflare&rsquo;s blog says different models turn up different bugs. &ldquo;This is just more tokens&rdquo;: it is. The question is whether one more pass costs less than the bug, and Cloudflare&rsquo;s note that extra passes cost about half the first is the closest thing to a number here.</p>
<h2 id="where-this-stops-being-true">Where this stops being true</h2>
<p>For a small change in a codebase you know well, a human review plus tests is usually cheaper and faster. For a prototype you will throw away, design review is overhead. These earn their keep when the author is a model, the reviewer&rsquo;s time is the scarce thing, and a mistake ships to people who did not write it.</p>
<h2 id="what-i-would-do-differently">What I would do differently</h2>
<p>If I were adding checks to an AI-heavy codebase this month:</p>
<p>Put the first check where a mistake costs the most. For most teams that is the design, before a model writes a thousand lines in the wrong direction.</p>
<p>Make every handoff carry its own pass/fail command, the way improve&rsquo;s plans do, so the executor doesn&rsquo;t have to judge its own success.</p>
<p>Run security scans on the diff for every change and a full sweep on a schedule, and expect to run the full one more than once.</p>
<p>Test the &ldquo;fresh context&rdquo; claim yourself. Issue #237&rsquo;s method is simple: plant a marker string in each instruction file and ask the reviewer to list every marker it can see.</p>
<p>Choose by stars last.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Five open-source agent projects, judged by where they stop</title>
      <link>https://www.liamding.com/en/posts/2026-11-05-open-source-agents-judged-by-where-they-stop/</link>
      <pubDate>Thu, 05 Nov 2026 00:00:00 +0800</pubDate>
      <guid>https://www.liamding.com/en/posts/2026-11-05-open-source-agents-judged-by-where-they-stop/</guid>
      <description>After a Railway employee&amp;#39;s test agent probed another company&amp;#39;s infrastructure, I picked five open-source agent projects by where each one puts its stop, not by stars.</description>
      <content:encoded><![CDATA[<h2 id="the-problem">The problem</h2>
<p>On 2 November (US Pacific time), Josh Manders posted on X asking Railway why one of its employees was &ldquo;trying to breach my infrastructure&rdquo;. His follow-up lists what the activity did: pulled credentials from inside an app container and asked for every secret on the platform, tried the cloud metadata endpoint, asked the cluster what the credentials were allowed to do, guessed internal database and cache names, and port-scanned all 65,536 internal addresses, twice. About 24 commands in 15 minutes, scripted. He says every attempt was blocked and nothing was accessed.</p>
<p>The employee, Jake Runzer, replied that there was no malice: he had been &ldquo;running some agents to do signups + agentic testing on similar platforms&rdquo;, had halted the harness, and agreed it was &ldquo;going off the rails and reaching too far&rdquo;. Railway&rsquo;s @JustJake called it &ldquo;agentically &lsquo;off the rails&rsquo;&rdquo; and said external testing that isn&rsquo;t manual would be banned.</p>
<blockquote>
<p>All of the incident detail above is one party&rsquo;s account plus the other side&rsquo;s replies on X. Nobody has published logs. I use it as an example of a failure shape, not as a finding about anyone.</p></blockquote>
<p>What interests me is the shape. Most of those commands look like reads: list secrets, query metadata, scan ports. A gate that asks a person before &ldquo;spending money&rdquo; or &ldquo;sending a message&rdquo; would have let every one of them through. The stop that mattered was at the network and credential layer of the target, and it held.</p>
<p>So this week&rsquo;s open-source picks are agent projects that each put a stop somewhere specific. I opened all five on 4 November 2026, around 3 PM Pacific.</p>
<h2 id="what-i-concluded">What I concluded</h2>
<p>Pick an agent tool by where it stops, then match that to the way your agent is likely to fail. Stars measure attention: the most-starred repo here is a one-commit research prototype, and the only one that controls the network sits at 493.</p>
<p>Four questions cover most of it:</p>
<ol>
<li>What can the agent reach (files, network, credentials)?</li>
<li>What counts as an instruction, as opposed to material it reads?</li>
<li>Who is allowed to hand it work?</li>
<li>Which actions wait for a person, and what happens when nobody is there?</li>
</ol>
<h2 id="constraints">Constraints</h2>
<p>Anyone comparing these should know:</p>
<ul>
<li>Star counts and commit dates move daily. Numbers below are a snapshot from the GitHub API on 4 November 2026; commit dates are UTC.</li>
<li>Four of the five repos were created in the last five weeks. Open Duet&rsquo;s repository dates from 2023, but its README describes the current coworker product.</li>
<li>I read READMEs and docs. I did not install or run any of these, so every claim about behaviour is the project&rsquo;s own claim.</li>
<li>These solve different problems. The comparison is about where the stop sits, not which tool is &ldquo;best&rdquo;.</li>
</ul>
<h2 id="the-options-and-what-i-traded-away">The options, and what I traded away</h2>
<table>
  <thead>
      <tr>
          <th>Repo</th>
          <th>License</th>
          <th>Stars (4 Nov)</th>
          <th>Latest commit on main</th>
          <th>Where it stops</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td><a href="https://github.com/strands-agents/box">strands-agents/box</a></td>
          <td>Apache-2.0</td>
          <td>493</td>
          <td>4 Nov 2026</td>
          <td>What the agent can reach: OS isolation, network egress, credentials</td>
      </tr>
      <tr>
          <td><a href="https://github.com/KKKKhazix/agent-autoloop">KKKKhazix/agent-autoloop</a></td>
          <td>MIT</td>
          <td>2,424</td>
          <td>4 Nov 2026</td>
          <td>What counts as an instruction, and side effects that need a card</td>
      </tr>
      <tr>
          <td><a href="https://github.com/deepseek-ai/dsh-superagent-prototype">deepseek-ai/dsh-superagent-prototype</a></td>
          <td>MIT</td>
          <td>18,524</td>
          <td>31 Oct 2026</td>
          <td>Who may give it work, and observe before auto</td>
      </tr>
      <tr>
          <td><a href="https://github.com/Anil-matcha/open-duet">Anil-matcha/open-duet</a></td>
          <td>Apache-2.0</td>
          <td>6,771</td>
          <td>4 Nov 2026</td>
          <td>Which actions need approval, per coworker</td>
      </tr>
      <tr>
          <td><a href="https://github.com/wangzan101/Mika">wangzan101/Mika</a></td>
          <td>Apache-2.0</td>
          <td>6,604</td>
          <td>4 Nov 2026</td>
          <td>Money only, by default</td>
      </tr>
  </tbody>
</table>
<p><strong>Strands Box: stop at the boundary.</strong> A sandbox engine in Rust. Its README says operations the engine checks are denied by default, an egress gateway checks connections and HTTP method and path, an MCP broker checks tool calls, and credentials are injected so the agent never receives the underlying secrets. Policies can look at history: an earlier file read can deny a later outbound request. Decisions are recorded as OTLP JSON. This is the layer that maps most directly to the Railway list: a harness that cannot see secrets or reach a metadata endpoint has nothing to &ldquo;reach too far&rdquo; with. The cost: it&rsquo;s a preview that runs locally on macOS with Apple silicon, and writing the policies is your job.</p>
<p><strong>agent-autoloop: stop at what counts as an instruction.</strong> A background runner for command-line coding agents, with a kernel of about 1,650 lines and no runtime dependencies. Before deleting data, restarting services, spending money or sending messages outside, it asks through a card and waits without holding a process. Text written by other people, such as an issue title, goes into a file marked as material rather than into the prompt, and events that try to set the working directory, the agent or its access level are rejected. Work handed back and forth between agents stops after 8 hops. Its SECURITY.md is unusually candid: this does not stop all prompt injection, and an agent with &ldquo;read&rdquo; access can still read keys on the machine unless you isolate it with a separate system account or a container. Not on Windows yet, and the webhook templates are only tested against local mocks.</p>
<p><strong>DeepSeek&rsquo;s dsh-superagent prototype: stop at who may ask.</strong> The README says it is a research prototype for reference only, not an official DeepSeek product, and the repo has one commit. Its authorization doc is the interesting part: every source of work gets an admission rule, a request is explicit, inferred or not admitted, and bots cannot make explicit requests or stop a ticket. The default mode is <code>observe</code>, which records what it would have accepted and accepts nothing; the doc recommends switching to <code>auto</code> only after reviewing those recorded decisions. The cost: it is a design to learn from, not something to deploy.</p>
<p><strong>Open Duet: stop at the action, chosen per coworker.</strong> A self-hosted workspace for named AI coworkers, with four approval modes: Ask first, Accept edits, Approve for me and Plan only. A routine never approves an action by itself, and changes in connected apps wait for approval. Its own caveats: connected apps were tested against a fake Composio service, not the real one; the self-hosted server is experimental and single-owner; desktop installers are unsigned.</p>
<p><strong>Mika: stop only before money.</strong> An Android agent that runs apps on a hidden background screen, with no Mika server, and hands logins, verification codes and payments to you. Its modes doc is explicit that in the default Ask mode, only the step that spends money is confirmed: &ldquo;Sending messages, posting, submitting forms, deleting and changing settings are not asked about; they are just done.&rdquo; When nobody is there, a scheduled task skips the paying step and tells you. That is a clear, honest line, and a narrow one. If your worry is an agent messaging the wrong person, the default doesn&rsquo;t cover it.</p>
<h2 id="evidence">Evidence</h2>
<p>Map the Railway list onto these projects and the gaps show up:</p>
<ul>
<li>Pulling credentials and requesting every secret is what Box&rsquo;s credential injection and autoloop&rsquo;s &ldquo;use a separate account or container&rdquo; advice are about. An approval card would not catch it, because nothing looks like a side effect.</li>
<li>The metadata endpoint and the port scan are network reads. Box&rsquo;s egress gateway is the only stop here that inspects them.</li>
<li>&ldquo;Signups and agentic testing on similar platforms&rdquo; is a scope question. dsh&rsquo;s admission table, and observe before auto, are a way to see what an agent would take on before it does.</li>
<li>Open Duet and Mika gate on the type of action. That works for money and outgoing messages, and does nothing for reconnaissance.</li>
</ul>
<p>Star counts in the table don&rsquo;t line up with any of this. The 18,524-star repo describes itself as a prototype; the 493-star one is the only project here that stops outbound network calls.</p>
<p>Counter-arguments worth taking seriously. &ldquo;The target&rsquo;s defences held, so this proves the system works&rdquo;: for the target, yes. The agent&rsquo;s owner still found out from a public post. &ldquo;Approval gates are what users actually want&rdquo;: for consumer agents, probably, and Mika&rsquo;s money-only default is a defensible product choice. It is just a different stop. &ldquo;You can stack all four&rdquo;: you can, and for an agent with credentials you should. The point is to know which layer you are relying on.</p>
<h2 id="where-this-stops-being-true">Where this stops being true</h2>
<p>For an agent that only edits files in a throwaway checkout, an approval gate is enough, and a network sandbox is overhead. For a phone agent doing your shopping, money is the right first stop. The argument is aimed at agents left running with credentials and network access, where the worst case is something reading or scanning, not something buying.</p>
<h2 id="what-i-would-do-differently">What I would do differently</h2>
<p>If I were setting up a background agent this month:</p>
<p>Start from reach. Give it its own account or container, inject credentials instead of exposing them, and deny outbound traffic by default.</p>
<p>Keep other people&rsquo;s text out of the instruction path, as autoloop does, and cap how far work can be handed on.</p>
<p>Run new sources of work in observe mode first and read what it would have accepted.</p>
<p>Then add approval cards for side effects, and write down which actions are not gated, the way Mika&rsquo;s doc does.</p>
<p>Choose by stars last.</p>
]]></content:encoded>
    </item>
  </channel>
</rss>
