Skip to content

Bootstrap .agents/skills/ directory structure - #9639

Closed
acstll wants to merge 4 commits into
elastic:mainfrom
acstll:add-test-eui-pr-in-kibana-skill
Closed

Bootstrap .agents/skills/ directory structure#9639
acstll wants to merge 4 commits into
elastic:mainfrom
acstll:add-test-eui-pr-in-kibana-skill

Conversation

@acstll

@acstll acstll commented May 6, 2026

Copy link
Copy Markdown
Contributor

Note

PR repurposed and description updated by Claude Code. The original skill file has been removed; the actual test-eui-pr-in-kibana skill will be added in a follow-up PR. This PR now only bootstraps the .agents/skills/ directory structure.


Summary

Important

There's a better fully-automated alternative of a workflow for testing an eui commit/branch in Kibana, that should be available soon (code). This is a fallback.

Note

Also, this workflow of manually packing tarballs and manually modifying package.json files is not something we want to have in the future, but since it's still needed useful, i thought it was an easy way to start adding skills to this repo! Needing to create this Kibana PR was the trigger…

Tip

We could also add a variant of this that triggers a gh workflow for making a snapshot release, and then open the PR in Kibana

Adds a test-eui-pr-in-kibana agent skill that codifies the "Testing local EUI in local Kibana" workflow documented in wiki/contributing-to-eui/testing/testing-in-kibana.md (link).

You can ask an agent something like "let's test this changes in Kibana" to trigger the skill. Which will then pack the tarballs and set it up in Kibana in a new branch, optionally pushing and creating a PR to run CI.

Layout mirrors how elastic/kibana organizes agent assets:

  • .agents/skills/ is the source of truth
  • .claude/skills and .codex/skills symlink to it

Also extends .gitignore to ignore user-specific Claude Code state (.claude/*.local.json, .claude/worktrees/) while keeping the skills folder committable.

API Changes

N/A

Screenshots

Screenshot 2026-05-06 at 15 11 53

Impact Assessment

Note: Most PRs should be tested in Kibana to help gauge their Impact before merging.

  • 🔴 Breaking changes — What will break? How many usages in Kibana/Cloud UI are impacted?
  • 💅 Visual changes — May impact style overrides; could require visual testing. Explain and estimate impact.
  • 🧪 Test impact — May break functional or snapshot tests (e.g., HTML structure, class names, default values).
  • 🔧 Hard to integrate — If changes require substantial updates to Kibana, please stage the changes and link them here.

Impact level: 🟢 None

Release Readiness

nothing to document, skills like this should be discoverable (?)

QA instructions for reviewer

(agent-generated, edited by me)

QA is a guided dry-run of invoking the skill.

Prereqs

  • Claude Code installed and authenticated (Cursor or Codex CLI also work
    via the .codex/skills symlink — pick whichever you use)
  • A local Kibana checkout with upstream remote pointing to
    git@github.com:elastic/kibana.git
  • An EUI PR to test against. Easiest: an open PR of yours, or any merged
    PR like #9630

Steps

  1. Check out this branch in your EUI repo (gh pr 9639).
  2. Verify the symlinks resolved — ls .claude/skills/ and ls .codex/skills/ should both show test-eui-pr-in-kibana/.
  3. Open Claude Code from the EUI repo root and prompt something like: Test EUI #9630 in Kibana (or whichever PR number you picked). The skill should auto-load.
  4. Let it run, expect background tasks…

What to verify

  • Skill auto-triggers on phrases like "test EUI #N in Kibana", "create Kibana test PR for EUI #N", or "stage EUI PR in Kibana"
  • EUI side:
    • Builds with the temp packages/eui/package.json edit and the edit is reverted at the end (git status clean afterwards)
    • If the PR touches eui-theme-common / eui-theme-borealis, the skill detects and offers to build them; otherwise it asks
  • Kibana side:
    • New branch named test-eui-{N}-{slug} created off fresh upstream/main
    • Tarball file is snake_case (eui_{version}_{N}.tgz)
    • package.json points @elastic/eui to file:./...tgz and adds @elastic/eui-theme-common (published version, unless built)
    • packages/kbn-dependency-ownership/src/rule.ts and src/dev/license_checker/config.ts each get an @elastic/eui-theme-common entry
    • Single commit, no push, no PR opened (skill stops at "branch ready" and asks)
  • Gotchas the skill should handle:
    • Uses Kibana's required Node version via nvm, not the active one
    • Stages files explicitly (no git add -A — your unrelated working
      tree edits should not be picked up)

Checklist before marking Ready for Review

Reviewer checklist

  • Approved Impact Assessment — Acceptable to merge given the consumer impact.
  • Approved Release Readiness — Docs, Figma, and migration info are sufficient to ship.

Adds a `test-eui-pr-in-kibana` agent skill that codifies the workflow
documented in `wiki/contributing-to-eui/testing/testing-in-kibana.md`:
build a local @elastic/eui tarball from a PR branch, stand up a Kibana
branch consuming it, and run `yarn kbn bootstrap` so Kibana CI can run
against the EUI PR before merging.

Layout mirrors how elastic/kibana organizes agent assets:
- `.agents/skills/` is the source of truth
- `.claude/skills` and `.codex/skills` symlink to it

Also extends `.gitignore` to ignore user-specific Claude Code state
(`.claude/*.local.json`, `.claude/worktrees/`) while keeping the skills
folder committable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@acstll acstll self-assigned this May 6, 2026
@acstll acstll added the skip-changelog Use on PRs to skip changelog requirement (Don't delete - used for automation) label May 6, 2026
Clarifies when to use published theme packages and documents theme tarball output naming plus workspace dependency caveats to avoid Kibana bootstrap failures.

Co-authored-by: Cursor <cursoragent@cursor.com>
@acstll
acstll marked this pull request as ready for review May 12, 2026 10:34
Copilot AI review requested due to automatic review settings May 12, 2026 10:34
@acstll
acstll requested a review from a team as a code owner May 12, 2026 10:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new agent “skill” document to codify the manual workflow for testing an EUI PR branch inside a Kibana checkout (building local tarballs, wiring Kibana deps, bootstrapping, and optionally opening a draft PR), and updates repo ignores for local Claude Code state.

Changes:

  • Add .agents/skills/test-eui-pr-in-kibana/SKILL.md describing the end-to-end EUI→Kibana staging workflow.
  • Update .gitignore to ignore user-local Claude Code state files/directories.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

File Description
.gitignore Ignores .claude/*.local.json and .claude/worktrees/ user-local state.
.agents/skills/test-eui-pr-in-kibana/SKILL.md Introduces a detailed skill spec for building EUI tarballs and staging them in a Kibana branch/PR.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .agents/skills/test-eui-pr-in-kibana/SKILL.md Outdated
Comment on lines +143 to +149
```bash
cd {KIBANA_PATH}
git status # must be clean (warn if not)
git fetch upstream main
git checkout main
git merge --ff-only upstream/main # if this fails, stop and ask
```

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in 02bd0d8

Comment on lines +201 to +208
Kibana enforces `.nvmrc`. Each Bash invocation gets a fresh shell, so
source nvm and use the right version in the same command:

```bash
cd {KIBANA_PATH} && \
source ~/.nvm/nvm.sh && nvm use > /dev/null && \
yarn kbn bootstrap --no-validate
```

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in 02bd0d8

Address Copilot review feedback: explicitly check that the Kibana
`upstream` remote points at elastic/kibana and that nvm is present
before assuming either, instead of failing on the first command.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@weronikaolejniczak

Copy link
Copy Markdown
Contributor

My 2 cents are that an agent is not the right tool here. A much more cost-efficient, deterministic approach is a scripted one, which we already have thanks to Tomasz. Each step here is a deterministic shell command. So a script is cheaper, more reliable (no hallucination risk) and easier to test than an LLM agent. Arguably, it's also an easier one to maintain as opposed to keeping the skill in check thanks to the feedback loop (similar issue as with hand-written docs, wiki). I'd prefer for this to turn into a script that you can run locally as well, and reuses as much of the piping as possible with the CI workflow. This way, if CI is failing, we can still use it locally, which is an argument behind this experiment I suppose?

But I did read every line and I don't see any obvious issues! I don't mind this being merged to main if it's useful for you specifically.

@acstll

acstll commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

@weronikaolejniczak thanks for sharing your thoughts!

A much more cost-efficient, deterministic approach is a scripted one, which we already have thanks to Tomasz.

i know! i'm mentioning this in the very first paragraph of the description above, but i do agree with your argument (another thing i don't like is that it depends on AI being available, which might not always be the case)

This way, if CI is failing, we can still use it locally, which is an argument behind this experiment I suppose?

yes, i think it's a nice-to-have fallback, we might need to reach out for this awful pack-a-tarball workflow still…

how about this: let's remove the skill from this PR, leave only the empty folder and the symlinks (this way we kinda bootstrap having skills in the repo), and then i open another PR with a slightly better approach that includes this same skill re-purposed as a fallback, and a one or two skills more that will activate Tomasz' workflow by default, so if somebody in the future asks an agent something like "let's test this in Kibana", it'll reply something like: "here's the recommended approach (…), should i trigger it? otherwise you can do B and C, etc." — in summary, replace this with a set of small, scoped skills

Adds a shared .agents/skills/ directory as the canonical home for
Claude Code and Codex agent skills in this repo, with symlinks from
.claude/skills and .codex/skills for tool discovery. Actual skills
will be added in follow-up PRs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@acstll acstll changed the title Add agent skill for testing EUI PRs in Kibana Bootstrap .agents/skills/ directory structure May 14, 2026
@elasticmachine

Copy link
Copy Markdown
Collaborator

💚 Build Succeeded

History

cc @acstll

@elasticmachine

Copy link
Copy Markdown
Collaborator

💚 Build Succeeded

History

cc @acstll

@acstll

acstll commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

@tkajtoch what would be the correct way to "trigger" the new workflow?

@weronikaolejniczak

Copy link
Copy Markdown
Contributor

@acstll maybe I misunderstood you, if so please let me know 🙏🏻 What I meant to say is: let's not use a skill for this at all. To me a nice-to-have fallback would be a bash script. Similar like what you did with the release! I'm not confident that using LLM is a good use of resources. It takes longer, it costs more and it's harder to control the output of. But I also want to highlight this: it's only my judgement. So if the rest of the team thinks there is value to having this skill, I don't have anything against merging it.

And if we do decide to write it, what I would advise is updating our wiki and in the skill, point to that wiki instead. This would greatly simplify maintenance by having one source of truth. As I read the instructions, they seem perfect for human contributors as well.

@acstll
acstll marked this pull request as draft May 18, 2026 10:36
@acstll

acstll commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

What I meant to say is: let's not use a skill for this at all. To me a nice-to-have fallback would be a bash script. Similar like what you did with the release!

i got that, and it makes sense… either a bash script, or updating the wiki and pointing to it, they're all valid options; in the skills "spec" there's room for scripts as well, so it could be a combination (i still see value in having a skill for "testing eui in kibana" whatever the "implementation" might be)

i had removed the skill in this PR in 1c72bdf and left only the empty folders to get things moving

i mainly want to experiment and share (after working in the kibana repo for months, i'm missing "stuff" in the eui repo that we could be benefiting from, in my very humble opinion), i'll close this now to think about it a bit more — i also don't want to focus on this too much

@acstll acstll closed this May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Use on PRs to skip changelog requirement (Don't delete - used for automation)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants