Skip to content

Migrate hand-crafted agents (Gemmini, CUDA, trn-nki1/2, Saturn) to Bu…#50

Open
Bryce-Lim wants to merge 1 commit into
ucb-bar:mainfrom
Bryce-Lim:migrate-handcrafted-agents-to-builtllmagent
Open

Migrate hand-crafted agents (Gemmini, CUDA, trn-nki1/2, Saturn) to Bu…#50
Bryce-Lim wants to merge 1 commit into
ucb-bar:mainfrom
Bryce-Lim:migrate-handcrafted-agents-to-builtllmagent

Conversation

@Bryce-Lim
Copy link
Copy Markdown

Refs #31

What this does

Hand-ports the prompts from each LLMAgent subclass in autocomp/agents/
into .md / .yaml configs under autocomp/agent_builder/.built/, so they
can run through BuiltLLMAgent instead of needing custom subclasses.

This is step 1 of the linked issue. The legacy LLMAgent subclasses are
left wired up so existing experiments still work; step 2 (running the Agent
Builder for each backend and comparing against these hand-migrated prompts)
is a separate follow-up.

New built-agent directories

  • built:gemmini (gemm/conv, pe_dim != 4)
  • built:gemmini-admm (admm-multifunction, pe_dim == 4)
  • built:cuda
  • built:trn-nki1
  • built:trn-nki2
  • built:saturn

How to verify

for a in gemmini gemmini-admm cuda trn-nki1 trn-nki2 saturn; do
  python -m autocomp.agent_builder.run_agent_builder \
    --agent-name $a --inspect autocomp/agent_builder/.built/$a
done

Then a 1-iteration run with agent_name = "built:gemmini" vs.
agent_name = "gemmini" to spot-check that the prompts look comparable.

Things worth a closer look during review

  • trn-nki1/trn-nki2 architecture.md were partly drafted by an LLM
    helper; I caught and fixed wrong SBUF/PSUM sizes but the rest of those
    paragraphs deserve a once-over.
  • trn-nki2/isa_docs.md has fewer ### subsections (25) than
    trn-nki1/isa_docs.md (91). Could be a real difference in the v2
    generator's coverage or could be an over-aggressive trim — worth
    diffing against autocomp/agents/trn_nki2/nki_isa_generator.py.
  • Saturn's code_examples.md is intentionally a stub (the legacy Saturn
    agent didn't include ICL examples).
  • One small behavior fix in built:saturn/rules.yaml: the legacy Saturn
    rule still says Optimize the test() function, but the project renamed
    that entry point to solution() on 4/6/2026. The migrated copy says
    solution(). Revert if the legacy test() was intentional.

…iltLLMAgent format

Hand-ports the prompts from each LLMAgent subclass in autocomp/agents/ into
.md/.yaml configs under autocomp/agent_builder/.built/, so they can run
through BuiltLLMAgent instead of needing custom subclasses. Legacy agents
remain wired up. Sets up step 2 (Agent Builder vs hand-crafted comparison).

Refs: #<issue-number>
@Bryce-Lim
Copy link
Copy Markdown
Author

@charleshong3 did some work in the past week surrounding issue #31 , would be greatly appreciated if you could take a look!

@charleshong3 charleshong3 marked this pull request as ready for review May 1, 2026 21:27
@charleshong3
Copy link
Copy Markdown
Member

I think the Trainium and Saturn agents aren't needed due to them already having built agents. They could be useful for the research so maybe keep them on your fork, but I would remove them from this PR so we don't clutter the repo.

For the CUDA agent, maybe make the architecture and ISA docs specific to a particular GPU, for example the L40S (which we have in the lab)? Information that is generic across GPUs is probably already well-represented in the LLM pretraining data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants