Migrate hand-crafted agents (Gemmini, CUDA, trn-nki1/2, Saturn) to Bu…#50
Open
Bryce-Lim wants to merge 1 commit into
Open
Migrate hand-crafted agents (Gemmini, CUDA, trn-nki1/2, Saturn) to Bu…#50Bryce-Lim wants to merge 1 commit into
Bryce-Lim wants to merge 1 commit into
Conversation
…iltLLMAgent format Hand-ports the prompts from each LLMAgent subclass in autocomp/agents/ into .md/.yaml configs under autocomp/agent_builder/.built/, so they can run through BuiltLLMAgent instead of needing custom subclasses. Legacy agents remain wired up. Sets up step 2 (Agent Builder vs hand-crafted comparison). Refs: #<issue-number>
Author
|
@charleshong3 did some work in the past week surrounding issue #31 , would be greatly appreciated if you could take a look! |
Member
|
I think the Trainium and Saturn agents aren't needed due to them already having built agents. They could be useful for the research so maybe keep them on your fork, but I would remove them from this PR so we don't clutter the repo. For the CUDA agent, maybe make the architecture and ISA docs specific to a particular GPU, for example the L40S (which we have in the lab)? Information that is generic across GPUs is probably already well-represented in the LLM pretraining data. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #31
What this does
Hand-ports the prompts from each
LLMAgentsubclass inautocomp/agents/into
.md/.yamlconfigs underautocomp/agent_builder/.built/, so theycan run through
BuiltLLMAgentinstead of needing custom subclasses.This is step 1 of the linked issue. The legacy
LLMAgentsubclasses areleft wired up so existing experiments still work; step 2 (running the Agent
Builder for each backend and comparing against these hand-migrated prompts)
is a separate follow-up.
New built-agent directories
built:gemmini(gemm/conv, pe_dim != 4)built:gemmini-admm(admm-multifunction, pe_dim == 4)built:cudabuilt:trn-nki1built:trn-nki2built:saturnHow to verify
Then a 1-iteration run with
agent_name = "built:gemmini"vs.agent_name = "gemmini"to spot-check that the prompts look comparable.Things worth a closer look during review
trn-nki1/trn-nki2architecture.mdwere partly drafted by an LLMhelper; I caught and fixed wrong SBUF/PSUM sizes but the rest of those
paragraphs deserve a once-over.
trn-nki2/isa_docs.mdhas fewer###subsections (25) thantrn-nki1/isa_docs.md(91). Could be a real difference in the v2generator's coverage or could be an over-aggressive trim — worth
diffing against
autocomp/agents/trn_nki2/nki_isa_generator.py.code_examples.mdis intentionally a stub (the legacy Saturnagent didn't include ICL examples).
built:saturn/rules.yaml: the legacy Saturnrule still says
Optimize the test() function, but the project renamedthat entry point to
solution()on 4/6/2026. The migrated copy sayssolution(). Revert if the legacytest()was intentional.