Skip to content

Mysql datasource - #276543

Merged
erikcurrin-elastic merged 30 commits into
elastic:mainfrom
erikcurrin-elastic:mysql-datasource
Sep 15, 2026
Merged

erikcurrin-elastic merged 30 commits into
elastic:mainfrom
erikcurrin-elastic:mysql-datasource

Conversation

@erikcurrin-elastic

@erikcurrin-elastic erikcurrin-elastic commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Implement a MySQL fetcher

Dependencies

New: mysql2 library (current version 3.20.0)
Purpose: Used to plug into the MySQL binary protocol
Justification: Absolutely. We do not want to reverse engineer the protocol
Alternatives explored: The only alternative is the Http bridge, but it is not maintained actively and not viable
Existing dependencies: There are other libraries that we use for Kibana connectors.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🔍 Preview links for changed docs

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the MySQL connector. The read-only enforcement, identifier backtick-quoting, and pool-caching logic are solid and well-tested. Left two inline comments on correctness gaps where allowed/valid input is mishandled: single-quote doubling corrupting bound LIKE parameters in searchRows, and the derived-table wrapping breaking SHOW/DESCRIBE/EXPLAIN queries that assertReadOnly and the skill text advertise as supported.

Generated by Claude Reviewer for issue #276543 · 110.3 AIC · ⌖ 7.87 AIC · ⊞ 4.2K

Comment thread src/platform/packages/shared/kbn-connector-specs/src/lib/generic_db_connector.ts Outdated
Comment thread src/platform/packages/shared/kbn-connector-specs/src/specs/mysql/mysql.ts Outdated
@erikcurrin-elastic erikcurrin-elastic added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:agent-builder labels Jul 7, 2026
erikcurrin-elastic and others added 4 commits August 24, 2026 15:58
Ports the MySQL connector spec from elastic#255310 (kderusso/mysql-datasource), squashed into a single commit to take over the PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Moves credentials from the unencrypted schema config into auth: basic
(ctx.secrets) so they are encrypted at rest, adds the missing skill
property, bounds free-text Zod inputs, switches the icon to the
standard SVG-file + EuiIcon pattern with the official brand mark, fixes
docs placement/ordering, and significantly expands test coverage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…eSql action

- Rewrites MySQL connector to use ctx.getClient('mysql') via the new
  multi-client architecture framework (ClientTypeSpec), replacing the
  previous singleton approach
- Adds MysqlClientTypeSpec in lib/clients/mysql.ts: decodes Basic auth
  credentials, calls networkSettings.ensureHostnameAllowed() before
  opening a pool, creates a mysql2 connection pool
- Registers mysql client type in lib/clients/index.ts ClientRegistry
- Adds explicit scope to all actions (read-only actions: 'read';
  executeSql: 'destroy') so the LLM knows which actions are safe
- Adds executeSql action for unrestricted SQL (INSERT/UPDATE/DELETE/DDL)
- Removes single-quote escaping from escapeLikePattern — prepared
  statements handle quoting
- Adds WidgetType.Number and NumberWidget to the form-generator widget
  system so numeric config fields (e.g. port) render as <input type="number">
- Fixes form-generator field_builder to coerce string→number before Zod
  validation for ZodNumber schemas, and adds a serializer in NumberWidget
  to submit numeric values rather than strings
- Updates MySQL public docs to document executeSql (marked preview 9.6+)
  and correct stale read-only claims

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Inline LIMIT as an integer in query and searchRows to avoid
  mysqld_stmt_execute param-count mismatch when LIMIT ? is bound
  via the binary protocol
- Extract runSql helper to remove duplicated pool-lease / execute /
  cast-rows boilerplate across all action handlers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Share one read-only SQL guard across MySQL, Snowflake, and BigQuery, restrict query to SELECT/WITH, and apply Kibana TLS settings on the MySQL client.

Co-authored-by: Cursor <cursoragent@cursor.com>
@erikcurrin-elastic
erikcurrin-elastic marked this pull request as ready for review August 25, 2026 19:51
@erikcurrin-elastic
erikcurrin-elastic requested review from a team as code owners August 25, 2026 19:51
@kibanamachine

kibanamachine commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Dependency Review Bot Analysis 🔍

Found 1 new third-party dependencies:

Package Version Vulnerabilities Health Score
mysql2 3.24.3 🔴 C: 0, 🟠 H: 0, 🟡 M: 0, 🟢 L: 0 mysql2

Self Checklist

To help with the review, please update the PR description to address the following points for each new third-party dependency listed above:

  • Purpose: What is this dependency used for? Briefly explain its role in your changes.
  • Justification: Why is adding this dependency the best approach?
  • Alternatives explored: Were other options considered (e.g., using existing internal libraries/utilities, implementing the functionality directly)? If so, why was this dependency chosen over them?
  • Existing dependencies: Does Kibana have a dependency providing similar functionality? If so, why is the new one preferred?

Thank you for providing this information!

kibanamachine and others added 2 commits August 25, 2026 19:59
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the MySQL connector. Two concrete concerns flagged inline: the query action's subquery wrapping breaks on trailing-semicolon input (common in agent-generated SQL, currently uncovered by tests), and the shared read-only guard refactor silently tightens Snowflake/BigQuery runQuery by applying WRITE_PATTERN where they previously used prefix-only checks. Identifier quoting, LIKE escaping, parameterization, allowedHosts enforcement, and TLS handling all look sound.

Generated by Claude Reviewer for #276543 · opus · 186 AIC · ⌖ 14.5 AIC · ⊞ 5.1K

Comment thread src/platform/packages/shared/kbn-connector-specs/src/specs/mysql/mysql.ts Outdated
Comment thread src/platform/packages/shared/kbn-connector-specs/src/lib/generic_db_connector.ts Outdated
Remove unused `_opts` parameter from the mysql.test.ts mock to fix the
@typescript-eslint/no-unused-vars lint error, and add mysql2 to renovate.json
packageRules so the dependency_ownership check finds an owner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jbudz jbudz mentioned this pull request Aug 26, 2026
17 tasks
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Comment thread src/platform/packages/shared/kbn-connector-specs/src/lib/clients/mysql.ts Outdated
…host lookup, and pool.query for parameterless statements

- WRITE_PATTERN: broaden UPDATE arm from UPDATE\s+\w+\s+SET to UPDATE\s+ and
  drop the trailing \b so schema-qualified (db.table) and backtick-quoted
  identifiers are no longer able to bypass the read-only guard; add test cases
  for both evasions
- clients/mysql: remove dead getCustomHostSettings('mysql://...') call —
  resolveCustomHosts only stores https: and smtp: URLs, so the lookup always
  returned undefined; simplify toMysqlSslOptions signature accordingly
- mysql spec: use pool.query() for parameterless SQL (SHOW, DESCRIBE, bare
  SELECT/INSERT in executeSql) and reserve pool.execute() for parameterized
  searchRows calls to avoid unnecessary prepared-statement caching

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One inline note on the shared read-only SQL guard: the DELETE alternative in WRITE_PATTERN misses MySQL's multi-table delete form, which could let the read-scoped query action perform a delete when the DB user has write privileges. Non-blocking, defense-in-depth. The rest of the connector (identifier quoting, bound LIKE params, TLS defaults, host allowlisting, prior review feedback) looks solid.

Generated by Claude Reviewer for #276543 · opus · 253.5 AIC · ⌖ 14.4 AIC · ⊞ 5.1K

Comment thread src/platform/packages/shared/kbn-connector-specs/src/lib/generic_db_connector.ts Outdated
…ic_db_connector.ts

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
const schema = z.number();

expect(() => getWidgetComponent(schema)).toThrow(
/No widget found for schema type: ZodNumber/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we should use another example now that we have the number, but still check that if it's a non implemented type, it throws as we expect it to throw

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have some checks already that cover this

erikcurrin-elastic and others added 2 commits September 14, 2026 13:16
… to isTool:false

isReadOnlySql now matches the original Snowflake/BigQuery prefix-only semantics —
WRITE_PATTERN and MySQL executable-comment rejection remain exclusively in assertReadOnly.
executeSql is isTool:false to keep destructive SQL out of agent tool calls, matching
the established Snowflake/BigQuery pattern. Updates tests to reflect both changes,
and adds a non-implemented-type throw assertion to the registry describe block.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@erikcurrin-elastic
erikcurrin-elastic requested review from a team as code owners September 14, 2026 18:54
@botelastic botelastic Bot added the Team:One Workflow Team label for One Workflow (Workflow automation) label Sep 14, 2026
@kibanamachine kibanamachine added the reviewer:scout Agentic PR Scout test review label Sep 14, 2026
erikcurrin-elastic and others added 2 commits September 14, 2026 16:14
upstream/main migrated from yarn to pnpm and added pnpm-lock.yaml;
CI does a test merge so it finds the lockfile from main but our
package.json declares mysql2@3.24.3 which was absent, causing
ERR_PNPM_OUTDATED_LOCKFILE on every job.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@kibanamachine

Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
shared-packages 4.6MB 4.6MB +9.9KB
Unknown metric groups

ESLint disabled line counts

id before after diff
@kbn/connector-specs 4 3 -1

shared async chunks total size

id before after diff
all 15.0MB 15.0MB +411.0B

shared chunk count

id before after diff
all 201 202 +1

shared chunks total size

id before after diff
all 7.0MB 7.0MB +3.2KB

Total ESLint disabled count

id before after diff
@kbn/connector-specs 4 3 -1

total optimizer output size

id before after diff
all 63.9MB 63.9MB +13.5KB

Test Failures

  • [job] [logs] Jest Tests #14 / CasesWebhookActionConnectorFields renders Step Validation Step 2 is properly validated
  • [job] [logs] Jest Tests #14 / CasesWebhookActionConnectorFields renders Step Validation Step 3 is properly validated
  • [job] [logs] Scout Lane #114 - serverless-observability_complete / default / local-serverless-observability_complete - Synthetics manage rules privilege - lets a read user with can_manage_rules enable default alerting
  • [job] [logs] Scout Lane #3 - stateful-classic / default / local-stateful-classic - Lens ESQL dashboard inline editing - should add a limit without changing the chart type or the color
  • [job] [logs] Scout Lane #25 - stateful-classic / default / local-stateful-classic - Synthetics manage rules privilege - lets a read user with can_manage_rules enable default alerting

History

@erikcurrin-elastic
erikcurrin-elastic merged commit cdc5611 into elastic:main Sep 15, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting connectors-v2 release_note:skip Skip the PR/issue when compiling release notes reviewer:scout Agentic PR Scout test review Team:agent-builder Team:One Workflow Team label for One Workflow (Workflow automation) v9.6.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants