Skip to content

PMM-12392 Collect processlist from performance_schema on newer MySQL#359

Merged
4nte merged 14 commits into
mainfrom
PMM-12392-collect-processlist-from-performance-schema
May 14, 2026
Merged

PMM-12392 Collect processlist from performance_schema on newer MySQL#359
4nte merged 14 commits into
mainfrom
PMM-12392-collect-processlist-from-performance-schema

Conversation

@4nte
Copy link
Copy Markdown

@4nte 4nte commented Apr 23, 2026 •
edited
Loading

JIRA: PMM-12392

Feature build

abbrevations:
I_S - information_schema
P_S - peformance_schema

On MySQL 8.0.22+ and 5.7.39+, we want to query
performance_schema.processlist instead of information_schema.processlist to avoid the global thread-list mutex taken by the information_schema variant. Older MySQL, MariaDB, and other flavors keep using information_schema.

Additionally, we check/probe if performance_schema is really enabled on the host database - if it's not, we fallback to information_schema.

5.7.38 vs 5.7.39 benchmark on stock (PMM 3.7.0) vs patch (PMM-12392)

connect_latency connect_latency

scrape_duration scrape_duration

5.7.39 is performing better, this matches our expectations since 5.7.39+ benefits from using P_S instad of I_S, while 5.7.38 is stuck with I_S.

MySQL 5.7.39 release notes

Got the percona/tests suite running again:

  • Extract the right binary from the tarball (env_prepare_test.go), the extractor was matching postgres_exporter, so mysqld_exporter never got unpacked and the suite failed before launching.
  • Drop --exporter.global-conn-pool (assets/test.exporter-flags.txt), flag was removed in PMM-12153, so the exporter exited immediately on launch.
  • Use docker compose instead of docker-compose (Makefile), the v1 binary is gone on current Docker installs.
  • Configure the test exporter the way production does (utils_test.go), replaced the legacy DATA_SOURCE_NAME env var with --mysqld.address / --mysqld.username flags and MYSQLD_EXPORTER_PASSWORD.
  • pinned --config.my-cnf=/dev/null so the suite ignores the developer's ~/.my.cnf and stays deterministic.
  • Skip blank lines when parsing the flags file (utils_test.go), trailing newlines in test.exporter-flags.txt were being passed to the exporter as empty argv entries, which it rejects.

@4nte
PMM-12392 Collect processlist from performance_schema on newer MySQL
0124cb9 
On MySQL 8.0.22+ and 5.7.39+, query
performance_schema.processlist instead of information_schema.processlist
to avoid the global thread-list mutex taken by the information_schema
variant. Older MySQL, MariaDB, and other flavors keep using
information_schema.
@it-percona-cla
Copy link
Copy Markdown

it-percona-cla commented Apr 23, 2026 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@ademidoff
Copy link
Copy Markdown
Member

ademidoff commented Apr 23, 2026

@4nte I guess you implemented the additional check but didn't update the description.

ademidoff
ademidoff reviewed Apr 23, 2026
View reviewed changes
Comment thread collector/percona_info_schema_process_list.go Outdated
@4nte
PMM-12392 Check performance_schema is enabled before querying it
464f8eb 
Query @@performance_schema at instance init and only query
from performance_schema.processlist when it's actually enabled.
Falls back to information_schema.processlist otherwise.

Hoist semver.MustParse calls
to package-level vars so they're parsed once instead of on every scrape
maxkondr
maxkondr reviewed Apr 24, 2026
View reviewed changes
Comment thread collector/instance.go Outdated
Comment thread collector/percona_info_schema_process_list.go Outdated
Comment thread collector/percona_info_schema_process_list.go
4nte added 7 commits April 25, 2026 11:32
@4nte
PMM-12392 Group processlist schema constants in single const block
1655e5f
@4nte
PMM-12392 Pass context to newInstance and version/perfSchema probes
85eb22a
@4nte
PMM-12392 Fix extraction of mysqld_exporter from tarball
0bc9b36 
postgres_exporter was extracted instead of mysqld_exporter
@4nte
PMM-12392 Drop unsupported exporter flag from benchmark
3da171e 
The --exporter.global-conn-pool flag was dropped in PMM-12153
@4nte
PMM-12392 Use docker compose instead of deprecated docker-compose
6d42a25
@4nte
PMM-12392 Pass test exporter credentials via flags instead of DATA_SO…
8d4a437 
…URCE_NAME

Replace the legacy DATA_SOURCE_NAME env var with --mysqld.address,
--mysqld.username and the MYSQLD_EXPORTER_PASSWORD env var, matching
how the exporter is configured in production.

Also pin --config.my-cnf to /dev/null so the test ignores the
developer's ~/.my.cnf and stays deterministic regardless of the
local environment.
@4nte
PMM-12392 Skip blank lines when parsing exporter flags file
20e2d38
@4nte 4nte marked this pull request as ready for review April 27, 2026 07:52
@4nte 4nte requested a review from a team as a code owner April 27, 2026 07:52
@4nte 4nte requested review from ademidoff and maxkondr and removed request for a team April 27, 2026 07:52
ademidoff

This comment was marked as duplicate.

Sign in to view

ademidoff
ademidoff approved these changes Apr 27, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@ademidoff ademidoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job 👍

ademidoff
ademidoff reviewed Apr 27, 2026
View reviewed changes
Comment thread collector/instance.go Outdated
4nte and others added 2 commits April 27, 2026 12:29
@4nte @ademidoff
PMM-12392 Drop unreachable sql.ErrNoRows check
a502077 
Co-authored-by: Alex Demidoff <alexander.demidoff@percona.com>
@4nte
PMM-12392 Remove unused errors import
81d38ae
JiriCtvrtka
JiriCtvrtka reviewed Apr 27, 2026
View reviewed changes
Comment thread collector/exporter_test.go
JiriCtvrtka
JiriCtvrtka reviewed Apr 27, 2026
View reviewed changes
Comment thread collector/percona_info_schema_process_list.go Outdated
*pProcesslistMinTime,
)
// Prefer querying performance_schema.processlist instead of information_schema.processlist to avoid negative perf consequences
// Supported by MySQL >=5.7.39 and >=8.0.22
Copy link
Copy Markdown

@JiriCtvrtka JiriCtvrtka Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Supported by MySQL >=5.7.39 and >=8.0.22
// Supported by Percona Server >=5.7.39 and MySQL >=8.0.22

Copy link
Copy Markdown
Author

@4nte 4nte Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supported by Percona Server >=5.7.39 and MySQL >=8.0.22

To me, this reads like MySQL is supported from 8.0.22, but not from 5.7.39.

Since Percona server and MySQL are both aligned version-for-version in regards to P_S.processlist support, perhaps this reads better:

Supported by Percona Server/MySQL >=5.7.39 and >=8.0.22

JiriCtvrtka
JiriCtvrtka reviewed Apr 27, 2026
View reviewed changes
Comment thread collector/percona_info_schema_process_list_test.go Outdated
JiriCtvrtka
JiriCtvrtka approved these changes Apr 27, 2026
View reviewed changes
Copy link
Copy Markdown

@JiriCtvrtka JiriCtvrtka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice PR 💯

@4nte 4nte merged commit cddd146 into main May 14, 2026
15 checks passed
@4nte 4nte deleted the PMM-12392-collect-processlist-from-performance-schema branch May 14, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ademidoff ademidoff ademidoff approved these changes

@maxkondr maxkondr maxkondr approved these changes

@JiriCtvrtka JiriCtvrtka JiriCtvrtka approved these changes

Assignees

@4nte 4nte

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@4nte @it-percona-cla @ademidoff @maxkondr @JiriCtvrtka