diff --git a/README.md b/README.md index e15eba612..a6bd3cd76 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ This App is installed in the Nautobot Community Sandbox found over at [demo.naut Full web-based HTML documentation for this app can be found over on the [Nautobot Docs](https://docs.nautobot.com/projects/golden-config/en/latest/) website: -- [User Guide](https://docs.nautobot.com/projects/golden-config/en/latest/user/app_overview/) - Overview, Using the App, Getting Started, Navigating compliance (cli, json, custom), backup, app usage, intended state creation. +- [User Guide](https://docs.nautobot.com/projects/golden-config/en/latest/user/app_overview/) - Overview, Using the App, Getting Started, Navigating compliance (cli, json, xml, custom, hierarchical), backup, app usage, intended state creation. - [Administrator Guide](https://docs.nautobot.com/projects/golden-config/en/latest/admin/install/) - How to Install, Configure, Upgrade, or Uninstall the App. - [Developer Guide](https://docs.nautobot.com/projects/golden-config/en/latest/dev/contributing/) - Extending the App, Code Reference, Contribution Guide. - [Release Notes / Changelog](https://docs.nautobot.com/projects/golden-config/en/latest/admin/release_notes/) diff --git a/changes/979.added b/changes/979.added new file mode 100644 index 000000000..ec3a776ed --- /dev/null +++ b/changes/979.added @@ -0,0 +1 @@ +Added hier config compatability to compliance rules for matching nested config using hier config tags \ No newline at end of file diff --git a/docs/user/app_feature_compliance.md b/docs/user/app_feature_compliance.md index 1cfe51eb1..cf8a34a75 100644 --- a/docs/user/app_feature_compliance.md +++ b/docs/user/app_feature_compliance.md @@ -61,6 +61,9 @@ For JSON based configs, the match is based on JSON's structure top level key nam !!! note "Config to Match" is mandatory for CLI configurations. If config to match is not defined for JSON, the complete JSON configuration will be compared. If the config to match is defined, comparison will take place only for defined keys. +!!! tip "Advanced CLI Compliance: Hierarchical Configuration" + For more sophisticated CLI compliance checking, you can use **Filtered Configuration Compliance** which leverages the `hier_config`. Start the "Config to Match" field with `# hier_config` followed by YAML rule blocks using the unified `match_rules` syntax. See [Filtered Configuration Compliance](./app_feature_compliancefiltered.md) for detailed documentation and examples. + !!! note If the data is accidentally "corrupted" with a bad tested match, simply delete the devices an re-run the compliance process. diff --git a/docs/user/app_feature_compliancecli.md b/docs/user/app_feature_compliancecli.md index 05b30fca0..6c8bc8a65 100644 --- a/docs/user/app_feature_compliancecli.md +++ b/docs/user/app_feature_compliancecli.md @@ -3,6 +3,9 @@ !!! note This document provides instructions for `CLI` configuration type based compliance. The other option is `JSON` based [structured data compliance](./app_feature_compliancejson.md). +!!! tip "Advanced CLI Compliance" + For more sophisticated CLI compliance checking with advanced filtering capabilities, see [Filtered Configuration Compliance](./app_feature_compliancefiltered.md) which provides advanced configuration matching using the `hier_config` library. + ## Configuration Compliance Parsing Engine Configuration compliance is different than a simple UNIX diff. While the UI provides both, the compliance metrics are not influenced by the UNIX diff diff --git a/docs/user/app_feature_compliancefiltered.md b/docs/user/app_feature_compliancefiltered.md new file mode 100644 index 000000000..bca477b37 --- /dev/null +++ b/docs/user/app_feature_compliancefiltered.md @@ -0,0 +1,215 @@ +# Filtered Configuration Compliance + +## Overview + +Filtered configuration compliance is a powerful extension to the standard CLI-based compliance checking that leverages the `hier_config` library to provide advanced configuration parsing and comparison capabilities. This feature allows you to define complex matching rules to focus compliance checks on specific configuration sections or patterns. + +Unlike standard CLI compliance which matches configuration sections based on simple line-starting patterns, Filtered configuration compliance identifies and compares configuration elements based on their hierarchical relationships and specific attributes, which allows for filtering based on nested config lines. + +## When to Use Filtered Configuraiton Compliance + +This feature is particularly useful for: + +- Configurations that are benign or non-consequential to the configurations. +- When you are building out your compliance journey, and not prepared to include all configurations based on simple line matching. + +!!! warning + Should not be used to provide full line matches or matches that are better served as data, as an example you should not do this `- startswith: description USER PORT` but should do this `- startswith: description`. + +## Requirements + +### Platform Support + +Filtered configuration compliance requires: + +1. **CLI Configuration Type**: Only `CLI` configuration types are supported for hierarchical compliance +2. **Platform Compatibility**: Your device platform must be supported by the `hier_config` library +3. **Network Driver Mapping**: The platform must have a valid `hier_config` mapping in its `network_driver_mappings` + +### Repository Settings + +The same repository settings required for standard compliance apply: + +- Backup repository for storing device configurations +- Intended configuration repository +- Proper `backup_path_template` and `intended_path_template` configuration + +## Configuring Filtered Compliance Rules + +### Rule Identification + +To create a filtered compliance rule, start the **Config to Match** field with the comment: + +`# hier_config` + +This comment must be the first line. The content that follows should be YAML consisting of one or more rule blocks. Each rule block uses the `match_rules` key containing an ordered list of predicates (e.g. `startswith`, `contains`, `re_search`, `endswith`). + +### Syntax + +Example with two rule blocks: + +```yaml +# hier_config +- match_rules: + - startswith: router bgp + - startswith: neighbor +- match_rules: + - startswith: interface + - contains: GigabitEthernet +``` + +### Example Configuration Rules + +#### Example 1: BGP Neighbor Configuration + +To check compliance for all BGP neighbor configurations: + +```yaml +# hier_config +- match_rules: + - startswith: router bgp + - startswith: neighbor +``` + +This rule will match configurations like: +``` +router bgp 65001 + neighbor 10.1.1.1 + remote-as 65002 + description PEER_ROUTER_A + neighbor 10.1.1.2 + remote-as 65003 + description PEER_ROUTER_B +``` + +#### Example 2: Interface MTU Settings + +To check compliance for MTU settings on all interfaces: + +```yaml +# hier_config +- match_rules: + - startswith: interface + - contains: mtu +``` + +This rule will match configurations like: +``` +interface GigabitEthernet0/1 + mtu 9000 +interface GigabitEthernet0/2 + mtu 1500 +``` + +#### Example 3: SNMP Configuration + +To check compliance for SNMP server configurations: + +```yaml +# hier_config +- match_rules: + - startswith: snmp-server +``` + +This rule will match configurations like: +``` +snmp-server community public RO +snmp-server community private RW +snmp-server host 192.168.1.100 version 2c public +``` + +#### Example 4: Access Control Lists + +To check compliance for specific access control list entries: + +```yaml +# hier_config +- match_rules: + - startswith: ip access-list + - contains: SECURITY +``` + +This rule will match configurations like: +``` +ip access-list extended SECURITY_IN + permit tcp any host 192.168.1.100 eq 443 + deny ip any any log +ip access-list extended SECURITY_OUT + permit ip 192.168.1.0 0.0.0.255 any + deny ip any any log +``` + +## Fallback Behavior for Empty Intended Configuration + +In some cases, the intended configuration may not contain any elements that match the compliance rules, resulting in an empty intended configuration text. To handle this scenario, the system includes a fallback mechanism: + +**Interface Fallback**: When the intended configuration text is empty, the system automatically looks for top-level interface configurations in the running configuration and uses them as the intended configuration baseline. + +This fallback behavior: + +1. **Triggers** when `intended_text` is empty after tag-based filtering +3. **Filters** for lines that start with "interface" +4. **Uses** these interface declarations as the intended configuration for comparison + +**Example Scenario**: +- Your hierarchical rule targets specific VLAN configurations +- The intended configuration template doesn't include those VLANs +- The running configuration has interface declarations +- The system uses the interface lines from running config as the baseline + +This ensures that remediation will not remove interfaces when intended configuration is empty for an interface. + +## Troubleshooting + +### Common Issues + +#### Platform Not Supported +**Error**: `Unsupported platform for hier_config v3: ` + +**Solution**: Ensure your device platform has a valid `hier_config` mapping in its `network_driver_mappings`. Check the platform configuration in Nautobot. + +#### Invalid YAML Syntax +**Error**: `Invalid YAML in match_config: ` + +**Solution**: Validate your YAML syntax in the **Config to Match** field. Ensure proper indentation and structure. + +#### Wrong Configuration Type +**Error**: `Hier config compliance rules are only supported for CLI config types` + +**Solution**: Change the **Config Type** to `CLI` for hierarchical compliance rules. + +### Debug Tips + +1. **Check Platform Support**: Verify your platform's `network_driver_mappings` include `hier_config` +2. **Review hier_config Documentation**: Consult the [hier_config documentation](https://hier-config.readthedocs.io/en/latest/tags/) for advanced tagging examples +3. **Start Simple**: Begin with basic matching rules and gradually add complexity +4. **Empty Intended Configuration**: If you notice interface lines appearing in compliance results when not expected, check if your intended configuration contains the elements targeted by your hierarchical rules. The system automatically falls back to using running config interface declarations when intended configuration is empty + +## Best Practices + +### Rule Design + +1. **Specific Matching**: Create focused rules that target specific configuration elements +2. **Logical Grouping**: Group related configuration elements in single rules when appropriate +3. **Clear Naming**: Use descriptive feature names that clearly indicate what the rule validates +4. **Complete Intended Configurations**: Ensure your intended configuration templates include all elements targeted by filtered compliance rules to avoid unexpected fallback behavior where interface lines from running config are used as intended baseline + +## Advanced Usage + +### Combining Multiple Rules + +You can create multiple hierarchical compliance rules for the same platform to check different aspects of the configuration independently. + +### Integration with Configuration Management + +Filtered configuration compliance works seamlessly with existing configuration management workflows: + +- Backup processes remain unchanged +- Intended configuration generation follows the same patterns +- Compliance reporting provides the same interface with enhanced precision + +## Related Documentation + +- [Standard CLI Compliance](./app_feature_compliancecli.md) +- [Configuration Compliance Overview](./app_feature_compliance.md) +- [hier_config Documentation](https://hier-config.readthedocs.io/) diff --git a/mkdocs.yml b/mkdocs.yml index 82516074e..92f290aa1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -114,6 +114,7 @@ nav: - Navigate Compliance JSON: "user/app_feature_compliancejson.md" - Navigate Compliance XML: "user/app_feature_compliancexml.md" - Navigate Compliance Custom: "user/app_feature_compliancecustom.md" + - Navigate Compliance Filtered: "user/app_feature_compliancefiltered.md" - Navigate Intended: "user/app_feature_intended.md" - Navigate SoT Agg: "user/app_feature_sotagg.md" - Navigate Configuration Post-Processing: "user/app_feature_config_postprocessing.md" diff --git a/nautobot_golden_config/nornir_plays/config_compliance.py b/nautobot_golden_config/nornir_plays/config_compliance.py index 5a2531efd..a3123b487 100644 --- a/nautobot_golden_config/nornir_plays/config_compliance.py +++ b/nautobot_golden_config/nornir_plays/config_compliance.py @@ -2,12 +2,17 @@ # pylint: disable=relative-beyond-top-level import difflib +import hashlib +import json import logging import os from collections import defaultdict from datetime import datetime +import hier_config +import yaml from django.utils.timezone import make_aware +from hier_config.utils import HCONFIG_PLATFORM_V2_TO_V3_MAPPING from lxml import etree from nautobot_plugin_nornir.constants import NORNIR_SETTINGS from nautobot_plugin_nornir.plugins.inventory.nautobot_orm import NautobotORMInventory @@ -16,6 +21,7 @@ from nornir.core.plugins.inventory import InventoryPluginRegister from nornir.core.task import Result, Task from nornir_nautobot.exceptions import NornirNautobotException +from pydantic import TypeAdapter from nautobot_golden_config.choices import ComplianceRuleConfigTypeChoice from nautobot_golden_config.exceptions import ComplianceFailure @@ -171,8 +177,13 @@ def run_compliance( # pylint: disable=too-many-arguments,too-many-locals intended_cfg = _open_file_config(intended_file) for rule in rules[obj.platform.network_driver]: - _actual = get_config_element(rule, backup_cfg, obj, logger) - _intended = get_config_element(rule, intended_cfg, obj, logger) + # Check for hier config compliance rule + section_type = rule["section"][0] + if "# hier_config" in section_type: + _actual, _intended = process_nested_compliance_rule_hier_config(rule, backup_cfg, intended_cfg, obj, logger) + else: + _actual = get_config_element(rule, backup_cfg, obj, logger) + _intended = get_config_element(rule, intended_cfg, obj, logger) # using update_or_create() method to conveniently update actual obj or create new one. ConfigCompliance.objects.update_or_create( @@ -194,6 +205,98 @@ def run_compliance( # pylint: disable=too-many-arguments,too-many-locals return Result(host=task.host) +def process_nested_compliance_rule_hier_config(rule, backup_cfg, intended_cfg, obj, logger): + """ + Processes nested compliance rules using hierarchical configuration comparison. + + This function compares backup (running) and intended configurations using hierarchical config parsing. + It applies tag-based filtering according to match criteria defined in the compliance rule's match_config. + + Args: + rule (dict): Contains compliance rule details, including match configuration. + backup_cfg (str): The backup (running) configuration as a string. + intended_cfg (str): The intended configuration as a string. + obj (object): Object containing platform and device information. + logger (logging.Logger): Logger instance for logging errors and debug information. + + Returns: + tuple[str, str]: Filtered running and intended configuration text for compliance comparison. + + Notes: + The match config must have # hier_config as a comment at the top of the YAML block. + + The match_config field in the rule should define hierarchical config tags and matching criteria in YAML format. + See: https://hier-config.readthedocs.io/en/latest/tags/ + """ + # Check if the compliance rule is of type CLI + if rule["obj"].config_type != ComplianceRuleConfigTypeChoice.TYPE_CLI: + error_msg = "Hier config compliance rules are only supported for CLI config types." + logger.error(error_msg, extra={"object": obj}) + raise NornirNautobotException(error_msg) + + # Convert v2 platform to v3 platform + v2_platform = obj.platform.network_driver_mappings.get("hier_config") + platform = HCONFIG_PLATFORM_V2_TO_V3_MAPPING.get(v2_platform) + + if not platform: + error_msg = f"Unsupported platform for hier_config v3: {v2_platform}" + logger.error(error_msg, extra={"object": obj}) + raise NornirNautobotException(error_msg) + + # Create config objects using v3 API + running_config = hier_config.get_hconfig(platform_or_driver=platform, config_raw=backup_cfg) + generated_config = hier_config.get_hconfig(platform_or_driver=platform, config_raw=intended_cfg) + + v3_tags = tuple() + tag_names = set() + + # Create hier config v3 tags + try: + match_config = yaml.safe_load(rule["obj"].match_config) + for tag_rule in match_config: + # Create a unique tag name for each match rule + tag_name = hashlib.sha1(json.dumps(tag_rule, sort_keys=True).encode()).hexdigest() # noqa: S324 + tag_rule["apply_tags"] = frozenset([tag_name]) + tag_names.add(tag_name) + v3_tags = TypeAdapter(tuple[hier_config.models.TagRule, ...]).validate_python(match_config) + except yaml.YAMLError as e: + error_msg = f"Invalid YAML in match_config: {str(e)}" + logger.error(error_msg, extra={"object": obj}) + raise NornirNautobotException(error_msg) + + # Apply tags to the running and generated configs + for tag_rule in v3_tags: + for child in running_config.get_children_deep(tag_rule.match_rules): + child.tags_add(tag_rule.apply_tags) + + for tag_rule in v3_tags: + for child in generated_config.get_children_deep(tag_rule.match_rules): + child.tags_add(tag_rule.apply_tags) + + # Filter configs by the applied tags + running_config_generator = running_config.all_children_sorted_by_tags(tag_names, set()) + running_text = "\n".join(c.cisco_style_text() for c in running_config_generator) + + intended_config_generator = generated_config.all_children_sorted_by_tags(tag_names, set()) + intended_text = "\n".join(c.cisco_style_text() for c in intended_config_generator) + + # If the intended text is None, inject the top level parent lines from running config into the intended config. + # This will prevent the removal of top-level interfaces or other sections that are not matched by the tags + # For example, if the intent is to remove 'flow exporter' from the config when remediating + # Instead of this: + # no interface GigabitEthernet0/0 + # we will have this: + # no flow exporter 192.0.0.1 + if not intended_text: + running_config_top_level_lines = [] + for child in running_config.all_children_sorted_by_tags(tag_names, set()): + if child.real_indent_level == 0 and child.cisco_style_text().startswith("interface"): + running_config_top_level_lines.append(child.cisco_style_text()) + intended_text = "\n".join(running_config_top_level_lines) + + return running_text, intended_text + + def config_compliance(job): # pylint: disable=unused-argument """ Nornir play to generate configurations. diff --git a/nautobot_golden_config/tests/test_nornir_plays/test_config_compliance.py b/nautobot_golden_config/tests/test_nornir_plays/test_config_compliance.py index 36c05a15b..05733d746 100644 --- a/nautobot_golden_config/tests/test_nornir_plays/test_config_compliance.py +++ b/nautobot_golden_config/tests/test_nornir_plays/test_config_compliance.py @@ -4,8 +4,15 @@ import unittest from unittest.mock import MagicMock, Mock, patch +import yaml +from nornir_nautobot.exceptions import NornirNautobotException + from nautobot_golden_config.choices import ComplianceRuleConfigTypeChoice -from nautobot_golden_config.nornir_plays.config_compliance import get_config_element, get_rules +from nautobot_golden_config.nornir_plays.config_compliance import ( + get_config_element, + get_rules, + process_nested_compliance_rule_hier_config, +) class ConfigComplianceTest(unittest.TestCase): @@ -48,3 +55,447 @@ def test_get_config_element_match_config_absent(self): mock_rule["obj"].config_type = ComplianceRuleConfigTypeChoice.TYPE_JSON return_config = json.dumps(get_config_element(mock_rule, mock_config, mock_obj, None)) self.assertEqual(return_config, mock_config) + + +class ProcessNestedComplianceRuleHierConfigTest(unittest.TestCase): + """Test Hierarchical Configuration Compliance Function.""" + + def setUp(self): + """Set up test fixtures.""" + self.mock_logger = Mock() + self.mock_obj = Mock() + self.mock_obj.platform.network_driver_mappings = {"hier_config": "cisco_ios"} + + # Sample configuration data + self.backup_cfg = """ +interface GigabitEthernet0/1 + description WAN_INTERFACE + ip address 192.168.1.1 255.255.255.0 + mtu 1500 + no shutdown +! +interface GigabitEthernet0/2 + description LAN_INTERFACE + ip address 10.0.0.1 255.255.255.0 + mtu 9000 + no shutdown +! +router bgp 65001 + neighbor 10.1.1.1 remote-as 65002 + neighbor 10.1.1.1 description PEER_A + neighbor 10.1.1.2 remote-as 65003 + neighbor 10.1.1.2 description PEER_B +! +""" + + self.intended_cfg = """ +interface GigabitEthernet0/1 + description WAN_INTERFACE + ip address 192.168.1.1 255.255.255.0 + mtu 1500 + no shutdown +! +interface GigabitEthernet0/2 + description LAN_INTERFACE + ip address 10.0.0.1 255.255.255.0 + mtu 1500 + no shutdown +! +router bgp 65001 + neighbor 10.1.1.1 remote-as 65002 + neighbor 10.1.1.1 description PEER_A + neighbor 10.1.1.2 remote-as 65003 + neighbor 10.1.1.2 description PEER_B +! +""" + + def test_non_cli_config_type_raises_exception(self): + """Test that non-CLI config types raise an exception.""" + mock_rule = {"obj": Mock(config_type=ComplianceRuleConfigTypeChoice.TYPE_JSON, match_config="test")} + + with self.assertRaises(NornirNautobotException) as context: + process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + self.assertIn("Hier config compliance rules are only supported for CLI config types", str(context.exception)) + self.mock_logger.error.assert_called_once() + + def test_unsupported_platform_raises_exception(self): + """Test that unsupported platforms raise an exception.""" + mock_rule = {"obj": Mock(config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, match_config="test")} + + # Mock an unsupported platform + self.mock_obj.platform.network_driver_mappings = {"hier_config": "unsupported_platform"} + + with patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING", {}): + with self.assertRaises(NornirNautobotException) as context: + process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + self.assertIn("Unsupported platform for hier_config v3", str(context.exception)) + self.mock_logger.error.assert_called_once() + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.TypeAdapter") + def test_hier_config_syntax_success(self, mock_type_adapter, mock_platform_mapping, mock_hier_config): + """Test successful processing with hier_config syntax (# hier_config).""" + mock_platform_mapping.get.return_value = "cisco_ios" + mock_running_config = Mock() + mock_generated_config = Mock() + mock_hier_config.get_hconfig.side_effect = [mock_running_config, mock_generated_config] + + # Mock tag rule + mock_tag_rule = Mock() + mock_tag_rule.match_rules = ["interface"] + mock_tag_rule.apply_tags = frozenset(["test_tag"]) + mock_type_adapter_instance = Mock() + mock_type_adapter_instance.validate_python.return_value = (mock_tag_rule,) + mock_type_adapter.return_value = mock_type_adapter_instance + + # Mock config children + mock_child = Mock() + mock_child.cisco_style_text.return_value = "interface GigabitEthernet0/1" + mock_running_config.get_children_deep.return_value = [mock_child] + mock_generated_config.get_children_deep.return_value = [mock_child] + mock_running_config.all_children_sorted_by_tags.return_value = [mock_child] + mock_generated_config.all_children_sorted_by_tags.return_value = [mock_child] + + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="# hier_config\n- match_rules:\n - startswith: interface", + ) + } + + running_text, intended_text = process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + # Verify results + self.assertEqual(running_text, "interface GigabitEthernet0/1") + self.assertEqual(intended_text, "interface GigabitEthernet0/1") + mock_hier_config.get_hconfig.assert_called() + mock_type_adapter_instance.validate_python.assert_called_once() + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.yaml.safe_load") + def test_invalid_yaml_raises_exception(self, mock_yaml_load, mock_platform_mapping, mock_hier_config): + """Test that invalid YAML in compliance rule raises an exception.""" + mock_platform_mapping.get.return_value = "cisco_ios" + + # Mock YAML parsing to raise an exception + mock_yaml_load.side_effect = yaml.YAMLError("Invalid YAML syntax") + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="# hier_config\ninvalid: yaml: content: [", + ) + } + with self.assertRaises(NornirNautobotException) as context: + process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + self.assertIn("Invalid YAML in match_config", str(context.exception)) + self.mock_logger.error.assert_called_once() + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.TypeAdapter") + def test_multiple_rules(self, mock_type_adapter, mock_platform_mapping, mock_hier_config): + """Test processing with multiple match configs in a single compliance rule.""" + mock_platform_mapping.get.return_value = "cisco_ios" + mock_running_config = Mock() + mock_generated_config = Mock() + mock_hier_config.get_hconfig.side_effect = [mock_running_config, mock_generated_config] + + # Mock tag rules + mock_tag_rule1 = Mock() + mock_tag_rule1.match_rules = ["interface"] + mock_tag_rule1.apply_tags = frozenset(["interface_tag"]) + mock_tag_rule2 = Mock() + mock_tag_rule2.match_rules = ["router bgp"] + mock_tag_rule2.apply_tags = frozenset(["bgp_tag"]) + mock_type_adapter_instance = Mock() + mock_type_adapter_instance.validate_python.return_value = (mock_tag_rule1, mock_tag_rule2) + mock_type_adapter.return_value = mock_type_adapter_instance + + # Mock config children + mock_interface_child = Mock() + mock_interface_child.cisco_style_text.return_value = "interface GigabitEthernet0/1" + mock_bgp_child = Mock() + mock_bgp_child.cisco_style_text.return_value = "router bgp 65001" + mock_running_config.get_children_deep.return_value = [mock_interface_child, mock_bgp_child] + mock_generated_config.get_children_deep.return_value = [mock_interface_child, mock_bgp_child] + mock_running_config.all_children_sorted_by_tags.return_value = [mock_interface_child, mock_bgp_child] + mock_generated_config.all_children_sorted_by_tags.return_value = [mock_interface_child, mock_bgp_child] + + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="""# hier_config +- match_rules: + - startswith: interface +- match_rules: + - startswith: router bgp""", + ) + } + + running_text, intended_text = process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + expected_text = "interface GigabitEthernet0/1\nrouter bgp 65001" + self.assertEqual(running_text, expected_text) + self.assertEqual(intended_text, expected_text) + mock_type_adapter_instance.validate_python.assert_called_once() + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.TypeAdapter") + def test_empty_filtered_config_with_interface_fallback( + self, mock_type_adapter, mock_platform_mapping, mock_hier_config + ): + """Test processing when intended config is empty but running config has interfaces (fallback behavior).""" + # Setup mocks + mock_platform_mapping.get.return_value = "cisco_ios" + mock_running_config = Mock() + mock_generated_config = Mock() + mock_hier_config.get_hconfig.side_effect = [mock_running_config, mock_generated_config] + # Mock tag rule via TypeAdapter + mock_tag_rule = Mock() + mock_tag_rule.match_rules = ["flow exporter"] + mock_tag_rule.apply_tags = frozenset(["test_tag"]) + mock_type_adapter_instance = Mock() + mock_type_adapter_instance.validate_python.return_value = (mock_tag_rule,) + mock_type_adapter.return_value = mock_type_adapter_instance + + # Mock running config with interface and flow exporter + mock_interface_child = Mock() + mock_interface_child.cisco_style_text.return_value = "interface GigabitEthernet0/1" + mock_interface_child.real_indent_level = 0 + + mock_flow_child = Mock() + mock_flow_child.cisco_style_text.return_value = "flow exporter 192.0.0.1" + mock_flow_child.real_indent_level = 0 + + # Only the flow exporter should match the tag rule + mock_running_config.get_children_deep.return_value = [mock_flow_child] + # Fallback second call coverage + mock_running_config.all_children_sorted_by_tags.side_effect = [ + [mock_flow_child], + [mock_interface_child, mock_flow_child], + ] + + # Generated config has no matching elements (empty intended config) + mock_generated_config.get_children_deep.return_value = [] + mock_generated_config.all_children_sorted_by_tags.return_value = [] + + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="# hier_config\n- match_rules:\n - startswith: flow exporter", + ) + } + + running_text, intended_text = process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + # Running text should have the flow exporter + self.assertEqual(running_text, "flow exporter 192.0.0.1") + # Intended text should fallback to interface lines from running config + self.assertEqual(intended_text, "interface GigabitEthernet0/1") + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.TypeAdapter") + def test_empty_filtered_config_no_interfaces(self, mock_type_adapter, mock_platform_mapping, mock_hier_config): + """Test processing when both configs are empty and no interfaces are available for fallback.""" + # Setup mocks + mock_platform_mapping.get.return_value = "cisco_ios" + mock_running_config = Mock() + mock_generated_config = Mock() + mock_hier_config.get_hconfig.side_effect = [mock_running_config, mock_generated_config] + + # Mock tag rule + mock_tag_rule = Mock() + mock_tag_rule.match_rules = ["nonexistent"] + mock_tag_rule.apply_tags = frozenset(["test_tag"]) + mock_type_adapter_instance = Mock() + mock_type_adapter_instance.validate_python.return_value = (mock_tag_rule,) + mock_type_adapter.return_value = mock_type_adapter_instance + + # Mock empty results for both configs + mock_running_config.get_children_deep.return_value = [] + mock_generated_config.get_children_deep.return_value = [] + mock_running_config.all_children_sorted_by_tags.return_value = [] + mock_generated_config.all_children_sorted_by_tags.return_value = [] + + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="# hier_config\n- match_rules:\n - startswith: nonexistent", + ) + } + + running_text, intended_text = process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + # Should return empty strings when no matches and no interfaces for fallback + self.assertEqual(running_text, "") + self.assertEqual(intended_text, "") + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.TypeAdapter") + def test_interface_fallback_only_top_level_interfaces( + self, mock_type_adapter, mock_platform_mapping, mock_hier_config + ): + """Test that fallback only includes top-level interface lines.""" + # Setup mocks + mock_platform_mapping.get.return_value = "cisco_ios" + mock_running_config = Mock() + mock_generated_config = Mock() + mock_hier_config.get_hconfig.side_effect = [mock_running_config, mock_generated_config] + + # Mock tag rule + mock_tag_rule = Mock() + mock_tag_rule.match_rules = ["router bgp"] + mock_tag_rule.apply_tags = frozenset(["test_tag"]) + mock_type_adapter_instance = Mock() + mock_type_adapter_instance.validate_python.return_value = (mock_tag_rule,) + mock_type_adapter.return_value = mock_type_adapter_instance + + # Mock children with different indent levels + mock_interface_child = Mock() + mock_interface_child.cisco_style_text.return_value = "interface GigabitEthernet0/1" + mock_interface_child.real_indent_level = 0 + mock_sub_interface_child = Mock() + mock_sub_interface_child.cisco_style_text.return_value = "interface GigabitEthernet0/1.100" + mock_sub_interface_child.real_indent_level = 1 + mock_other_child = Mock() + mock_other_child.cisco_style_text.return_value = "hostname router1" + mock_other_child.real_indent_level = 0 + mock_bgp_child = Mock() + mock_bgp_child.cisco_style_text.return_value = "router bgp 65001" + mock_running_config.get_children_deep.return_value = [mock_bgp_child] + mock_running_config.all_children_sorted_by_tags.side_effect = [ + [mock_bgp_child], + [mock_interface_child, mock_sub_interface_child, mock_other_child, mock_bgp_child], + ] + # Generated config is empty + mock_generated_config.get_children_deep.return_value = [] + mock_generated_config.all_children_sorted_by_tags.return_value = [] + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="# hier_config\n- match_rules:\n - startswith: router bgp", + ) + } + running_text, intended_text = process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + # Running text should have the BGP config + self.assertEqual(running_text, "router bgp 65001") + # Intended text should only have top-level interface (not sub-interface or hostname) + self.assertEqual(intended_text, "interface GigabitEthernet0/1") + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.TypeAdapter") + def test_multiple_interface_fallback(self, mock_type_adapter, mock_platform_mapping, mock_hier_config): + """Test fallback behavior with multiple top-level interfaces.""" + # Setup mocks + mock_platform_mapping.get.return_value = "cisco_ios" + mock_running_config = Mock() + mock_generated_config = Mock() + mock_hier_config.get_hconfig.side_effect = [mock_running_config, mock_generated_config] + + # Mock tag rule + mock_tag_rule = Mock() + mock_tag_rule.match_rules = ["snmp-server"] + mock_tag_rule.apply_tags = frozenset(["test_tag"]) + mock_type_adapter_instance = Mock() + mock_type_adapter_instance.validate_python.return_value = (mock_tag_rule,) + mock_type_adapter.return_value = mock_type_adapter_instance + mock_interface1 = Mock() + mock_interface1.cisco_style_text.return_value = "interface GigabitEthernet0/1" + mock_interface1.real_indent_level = 0 + mock_interface2 = Mock() + mock_interface2.cisco_style_text.return_value = "interface GigabitEthernet0/2" + mock_interface2.real_indent_level = 0 + mock_snmp_child = Mock() + mock_snmp_child.cisco_style_text.return_value = "snmp-server community public" + + # Running config has SNMP + mock_running_config.get_children_deep.return_value = [mock_snmp_child] + + # Mock the second call to all_children_sorted_by_tags for interface fallback + mock_running_config.all_children_sorted_by_tags.side_effect = [ + [mock_snmp_child], + [mock_interface1, mock_interface2, mock_snmp_child], + ] + + # Generated config is empty + mock_generated_config.get_children_deep.return_value = [] + mock_generated_config.all_children_sorted_by_tags.return_value = [] + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="# hier_config\n- match_rules:\n - startswith: snmp-server", + ) + } + running_text, intended_text = process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + # Running text should have the SNMP config + self.assertEqual(running_text, "snmp-server community public") + # Intended text should have both interfaces + self.assertEqual(intended_text, "interface GigabitEthernet0/1\ninterface GigabitEthernet0/2") + + @patch("nautobot_golden_config.nornir_plays.config_compliance.hier_config") + @patch("nautobot_golden_config.nornir_plays.config_compliance.HCONFIG_PLATFORM_V2_TO_V3_MAPPING") + @patch("nautobot_golden_config.nornir_plays.config_compliance.TypeAdapter") + def test_tag_application_to_configs(self, mock_type_adapter, mock_platform_mapping, mock_hier_config): + """Test that tags are properly applied to both running and generated configs in unified syntax.""" + mock_platform_mapping.get.return_value = "cisco_ios" + mock_running_config = Mock() + mock_generated_config = Mock() + mock_hier_config.get_hconfig.side_effect = [mock_running_config, mock_generated_config] + + # Mock tag rule + mock_tag_rule = Mock() + mock_tag_rule.match_rules = ["interface"] + mock_tag_rule.apply_tags = frozenset(["interface_tag"]) + mock_type_adapter_instance = Mock() + mock_type_adapter_instance.validate_python.return_value = (mock_tag_rule,) + mock_type_adapter.return_value = mock_type_adapter_instance + + # Mock config children + mock_child = Mock() + mock_child.cisco_style_text.return_value = "interface GigabitEthernet0/1" + mock_running_config.get_children_deep.return_value = [mock_child] + mock_generated_config.get_children_deep.return_value = [mock_child] + mock_running_config.all_children_sorted_by_tags.return_value = [mock_child] + mock_generated_config.all_children_sorted_by_tags.return_value = [mock_child] + + mock_rule = { + "obj": Mock( + config_type=ComplianceRuleConfigTypeChoice.TYPE_CLI, + match_config="# hier_config\n- match_rules:\n - startswith: interface", + ) + } + + process_nested_compliance_rule_hier_config( + mock_rule, self.backup_cfg, self.intended_cfg, self.mock_obj, self.mock_logger + ) + + # Verify tags were added to children from both configs + self.assertEqual(mock_child.tags_add.call_count, 2) + mock_child.tags_add.assert_called_with(frozenset(["interface_tag"]))