Skip to content

Commit 511a8c1

Browse files
daniil-bergdaniil-berg
committed
test: add acceptance tests for tagging (close #65)
1 parent 91897f6 commit 511a8c1

4 files changed

Lines changed: 257 additions & 3 deletions

File tree

‎exporter/prometheus/tests/behat/tag.feature‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
@tool @tool_monitoring @monitoringexporter_prometheus @javascript
2+
Feature: Exporting tagged metrics
3+
4+
Background:
5+
Given the following "users" exist:
6+
| username | firstname | lastname | email |
7+
| manager1 | Manager | One | manager1@example.com |
8+
And the following "role assigns" exist:
9+
| user | role | contextlevel | reference |
10+
| manager1 | manager | System | |
11+
12+
Scenario: Using the tag query parameter filters out metrics without the specified tags
13+
Given I am logged in as "manager1"
14+
# Enable multiple metrics.
15+
And I navigate to "Plugins > Admin tools > Monitoring > Overview" in site administration
16+
And I click on "Enable" "link" in the "courses" "table_row"
17+
And I click on "Enable" "link" in the "overdue_tasks" "table_row"
18+
And I click on "Enable" "link" in the "quiz_attempts_in_progress" "table_row"
19+
And I click on "Enable" "link" in the "user_accounts" "table_row"
20+
And I click on "Enable" "link" in the "users_online" "table_row"
21+
# Add tags to some of them.
22+
And I click on "Configure" "link" in the "user_accounts" "table_row"
23+
When I set the following fields to these values:
24+
| Tags | Foo, Bar |
25+
And I click on "Save changes" "button"
26+
Then "Overview of Available Metrics" "heading" should be visible
27+
When I click on "Configure" "link" in the "overdue_tasks" "table_row"
28+
And I set the following fields to these values:
29+
| Tags | Bar, Baz |
30+
And I click on "Save changes" "button"
31+
Then "Overview of Available Metrics" "heading" should be visible
32+
# Ensure the endpoint delivers all, when no tags are specified.
33+
When I call the Prometheus endpoint
34+
Then I should see "# HELP tool_monitoring_courses"
35+
And I should see "# TYPE tool_monitoring_courses gauge"
36+
And I should see "# HELP tool_monitoring_overdue_tasks"
37+
And I should see "# TYPE tool_monitoring_overdue_tasks gauge"
38+
And I should see "# HELP tool_monitoring_quiz_attempts_in_progress"
39+
And I should see "# TYPE tool_monitoring_quiz_attempts_in_progress gauge"
40+
And I should see "# HELP tool_monitoring_user_accounts"
41+
And I should see "# TYPE tool_monitoring_user_accounts gauge"
42+
And I should see "# HELP tool_monitoring_users_online"
43+
And I should see "# TYPE tool_monitoring_users_online gauge"
44+
# Called with the "Bar" tag, only the two metrics that carry that tag should be exported.
45+
When I call the Prometheus endpoint with the following query parameters:
46+
| tag | bar |
47+
Then I should see "# HELP tool_monitoring_overdue_tasks"
48+
And I should see "# TYPE tool_monitoring_overdue_tasks gauge"
49+
And I should see "# HELP tool_monitoring_user_accounts"
50+
And I should see "# TYPE tool_monitoring_user_accounts gauge"
51+
And I should not see "tool_monitoring_courses"
52+
And I should not see "tool_monitoring_quiz_attempts_in_progress"
53+
And I should not see "tool_monitoring_users_online"
54+
# Called with the "Bar" and the "Baz" tag, only one metric should be exported.
55+
When I call the Prometheus endpoint with the following query parameters:
56+
| tag | bar,baz |
57+
Then I should see "# HELP tool_monitoring_overdue_tasks"
58+
And I should see "# TYPE tool_monitoring_overdue_tasks gauge"
59+
And I should not see "tool_monitoring_courses"
60+
And I should not see "tool_monitoring_quiz_attempts_in_progress"
61+
And I should not see "tool_monitoring_user_accounts"
62+
And I should not see "tool_monitoring_users_online"
63+
# Called with all three tags, no metrics should be exported.
64+
When I call the Prometheus endpoint with the following query parameters:
65+
| tag | foo,bar,baz |
66+
Then I should not see "tool_monitoring_courses"
67+
And I should not see "tool_monitoring_overdue_tasks"
68+
And I should not see "tool_monitoring_quiz_attempts_in_progress"
69+
And I should not see "tool_monitoring_user_accounts"
70+
And I should not see "tool_monitoring_users_online"
71+
# Order should matter and the parameter should be case-insensitive.
72+
When I call the Prometheus endpoint with the following query parameters:
73+
| tag | bAr,Foo |
74+
Then I should see "# HELP tool_monitoring_user_accounts"
75+
And I should see "# TYPE tool_monitoring_user_accounts gauge"
76+
And I should not see "tool_monitoring_courses"
77+
And I should not see "tool_monitoring_overdue_tasks"
78+
And I should not see "tool_monitoring_quiz_attempts_in_progress"
79+
And I should not see "tool_monitoring_users_online"
80+
# The endpoint should respond with a 422 status code, when an invalid/non-existent tag is passed.
81+
When I call the Prometheus endpoint with the following query parameters:
82+
| tag | foo,quux |
83+
Then I should not see "tool_monitoring_courses"
84+
And I should not see "tool_monitoring_overdue_tasks"
85+
And I should not see "tool_monitoring_quiz_attempts_in_progress"
86+
And I should not see "tool_monitoring_user_accounts"
87+
And I should not see "tool_monitoring_users_online"
88+
And I should see "422"

‎tests/behat/admin_settings.feature‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
@tool @tool_monitoring @javascript
2-
Feature: Administering metrics via the admin panel.
2+
Feature: Administering metrics
33

44
Background:
55
Given the following "users" exist:
@@ -9,7 +9,7 @@ Feature: Administering metrics via the admin panel.
99
| user | role | contextlevel | reference |
1010
| manager1 | manager | System | |
1111

12-
Scenario: Viewing the metrics dashboard, enabling and disabling metrics.
12+
Scenario: Registered metrics are visible in the overview and disabled by default
1313
Given I am logged in as "manager1"
1414
And I navigate to "Plugins > Admin tools > Monitoring > Overview" in site administration
1515
Then "Overview of Available Metrics" "heading" should be visible
@@ -47,6 +47,7 @@ Feature: Administering metrics via the admin panel.
4747
And I should not see "tool_monitoring_user_accounts"
4848
And I should not see "tool_monitoring_users_online"
4949

50+
Scenario: Enabling/disabling metrics makes them appear/disappear in the Prometheus export
5051
Given I am logged in as "manager1"
5152
# Enable the `courses` and `users_online` metrics.
5253
When I navigate to "Plugins > Admin tools > Monitoring > Overview" in site administration
@@ -90,7 +91,7 @@ Feature: Administering metrics via the admin panel.
9091
And I should not see "tool_monitoring_user_accounts"
9192
And I should not see "tool_monitoring_users_online"
9293

93-
Scenario: Configuring users_online metric.
94+
Scenario: Changing a metric-specific config affects the Prometheus export
9495
Given I am logged in as "manager1"
9596
And I navigate to "Plugins > Admin tools > Monitoring > Overview" in site administration
9697
Then "Overview of Available Metrics" "heading" should be visible

‎tests/behat/behat_tool_monitoring.php‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
<?php
2+
// This file is part of Moodle - http://moodle.org/
3+
//
4+
// Moodle is free software: you can redistribute it and/or modify
5+
// it under the terms of the GNU General Public License as published by
6+
// the Free Software Foundation, either version 3 of the License, or
7+
// (at your option) any later version.
8+
//
9+
// Moodle is distributed in the hope that it will be useful,
10+
// but WITHOUT ANY WARRANTY; without even the implied warranty of
11+
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12+
// GNU General Public License for more details.
13+
//
14+
// You should have received a copy of the GNU General Public License
15+
// along with Moodle. If not, see <http://www.gnu.org/licenses/>.
16+
17+
/**
18+
* Definition of the {@see behat_tool_monitoring} class.
19+
*
20+
* @package tool_monitoring
21+
* @copyright 2025 MootDACH DevCamp
22+
* Daniel Fainberg <d.fainberg@tu-berlin.de>
23+
* Martin Gauk <martin.gauk@tu-berlin.de>
24+
* Sebastian Rupp <sr@artcodix.com>
25+
* Malte Schmitz <mal.schmitz@uni-luebeck.de>
26+
* Melanie Treitinger <melanie.treitinger@ruhr-uni-bochum.de>
27+
* @license https://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
28+
*
29+
* {@noinspection PhpIllegalPsrClassPathInspection}
30+
*/
31+
32+
use Behat\Step\Given;
33+
34+
require_once(__DIR__ . '/../../../../../lib/behat/behat_base.php');
35+
36+
/**
37+
* Behat steps definitions.
38+
*
39+
* @package tool_monitoring
40+
* @copyright 2025 MootDACH DevCamp
41+
* Daniel Fainberg <d.fainberg@tu-berlin.de>
42+
* Martin Gauk <martin.gauk@tu-berlin.de>
43+
* Sebastian Rupp <sr@artcodix.com>
44+
* Malte Schmitz <mal.schmitz@uni-luebeck.de>
45+
* Melanie Treitinger <melanie.treitinger@ruhr-uni-bochum.de>
46+
* @license https://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
47+
*/
48+
class behat_tool_monitoring extends behat_base {
49+
/**
50+
* Enables/disables the `metrics` tag area for `tool_monitoring`.
51+
*
52+
* @param string $state "enabled" or "disabled".
53+
* @throws coding_exception
54+
* @throws dml_exception
55+
*
56+
* {@noinspection PhpUnused}
57+
*/
58+
#[Given('/^the metrics tag area is "(?P<state>enabled|disabled)"$/')]
59+
public function the_tag_area_is(string $state): void {
60+
global $DB;
61+
$enabled = match ($state) {
62+
'enabled' => 1,
63+
'disabled' => 0,
64+
// This should not happen due to the RegEx group above, but this is a sanity check.
65+
default => throw new coding_exception("Invalid state: '$state'."),
66+
};
67+
$area = $DB->get_record(
68+
table: 'tag_area',
69+
conditions: ['itemtype' => 'metrics', 'component' => 'tool_monitoring'],
70+
strictness: MUST_EXIST,
71+
);
72+
core_tag_area::update($area, ['enabled' => $enabled]);
73+
}
74+
}

‎tests/behat/tag.feature‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
@tool @tool_monitoring @javascript
2+
Feature: Tagging metrics
3+
4+
Background:
5+
Given the following "users" exist:
6+
| username | firstname | lastname | email |
7+
| manager1 | Manager | One | manager1@example.com |
8+
And the following "role assigns" exist:
9+
| user | role | contextlevel | reference |
10+
| manager1 | manager | System | |
11+
12+
Scenario Outline: Tag options <should_or_should_not> be shown when the tag area is <tagarea_enabled_or_disabled> and usetags is set to <usetags>
13+
Given the following config values are set as admin:
14+
| usetags | <usetags> |
15+
And the metrics tag area is "<tagarea_enabled_or_disabled>"
16+
And I am logged in as "manager1"
17+
When I navigate to "Plugins > Admin tools > Monitoring > Overview" in site administration
18+
Then "Manage tags" "link" <should_or_should_not> be visible
19+
When I click on "Configure" "link" in the "users_online" "table_row"
20+
Then "Configure Metric" "heading" should exist
21+
And "Tags" "field" <should_or_should_not> be visible
22+
And I <should_or_should_not> see "Manage standard tags"
23+
Examples:
24+
| usetags | tagarea_enabled_or_disabled| should_or_should_not |
25+
| 0 | disabled | should not |
26+
| 0 | enabled | should not |
27+
| 1 | disabled | should not |
28+
| 1 | enabled | should |
29+
30+
Scenario: Tags can be added to metrics and the overview can be filtered by tags
31+
Given I am logged in as "manager1"
32+
And I navigate to "Plugins > Admin tools > Monitoring > Overview" in site administration
33+
# Add tags to one metric.
34+
And I click on "Configure" "link" in the "user_accounts" "table_row"
35+
When I set the following fields to these values:
36+
| Tags | Foo, Bar |
37+
And I click on "Save changes" "button"
38+
Then "Overview of Available Metrics" "heading" should be visible
39+
# Check that they show up next to it in the overview.
40+
And "Foo" "link" in the "user_accounts" "table_row" should be visible
41+
And "Bar" "link" in the "user_accounts" "table_row" should be visible
42+
# Add tags to another metric.
43+
When I click on "Configure" "link" in the "overdue_tasks" "table_row"
44+
And I set the following fields to these values:
45+
| Tags | Bar, Baz |
46+
And I click on "Save changes" "button"
47+
Then "Overview of Available Metrics" "heading" should be visible
48+
# Check that the right tags appear next to the right metrics in the overview.
49+
And "Bar" "link" in the "overdue_tasks" "table_row" should be visible
50+
And "Baz" "link" in the "overdue_tasks" "table_row" should be visible
51+
And "Foo" "link" should not exist in the "overdue_tasks" "table_row"
52+
And "Baz" "link" should not exist in the "user_accounts" "table_row"
53+
# Filter by a tag that the two metrics have in common.
54+
When I click on "Bar" "link"
55+
Then "Overview of Available Metrics" "heading" should be visible
56+
And "overdue_tasks" "table_row" should be visible
57+
And "user_accounts" "table_row" should be visible
58+
And "courses" "table_row" should not exist
59+
And "quiz_attempts_in_progress" "table_row" should not exist
60+
And "users_online" "table_row" should not exist
61+
# Additionally filter by a tag that only one of the metric carries.
62+
When I click on "Baz" "link"
63+
Then "Overview of Available Metrics" "heading" should be visible
64+
And "overdue_tasks" "table_row" should be visible
65+
And "user_accounts" "table_row" should not exist
66+
And "courses" "table_row" should not exist
67+
And "quiz_attempts_in_progress" "table_row" should not exist
68+
And "users_online" "table_row" should not exist
69+
# Remove that tag filter and add another.
70+
When I click on "× Baz" "link"
71+
Then "Overview of Available Metrics" "heading" should be visible
72+
And "overdue_tasks" "table_row" should be visible
73+
And "user_accounts" "table_row" should be visible
74+
And "courses" "table_row" should not exist
75+
And "quiz_attempts_in_progress" "table_row" should not exist
76+
And "users_online" "table_row" should not exist
77+
When I click on "Foo" "link"
78+
Then "Overview of Available Metrics" "heading" should be visible
79+
And "user_accounts" "table_row" should be visible
80+
And "overdue_tasks" "table_row" should not exist
81+
And "courses" "table_row" should not exist
82+
And "quiz_attempts_in_progress" "table_row" should not exist
83+
And "users_online" "table_row" should not exist
84+
# Reset all filters.
85+
When I click on "Show all metrics" "link"
86+
Then "Overview of Available Metrics" "heading" should be visible
87+
And "courses" "table_row" should be visible
88+
And "overdue_tasks" "table_row" should be visible
89+
And "quiz_attempts_in_progress" "table_row" should be visible
90+
And "user_accounts" "table_row" should be visible
91+
And "users_online" "table_row" should be visible

0 commit comments

Comments
 (0)