Hash Reports

changes/997.added

@@ -0,0 +1 @@
+Added configuration hash grouping feature for identifying and grouping devices with identical non-compliant configurations.

docs/user/app_feature_hash.md

@@ -0,0 +1,97 @@
+# Configuration Hash Grouping
+
+The **Configuration Hash Grouping** feature enables administrators to identify devices that have identical non-compliant configurations, making it easier to troubleshoot and fix configuration issues that affect multiple devices simultaneously. This feature groups devices by their configuration hash values, allowing you to see patterns in configuration drift and apply fixes to entire groups at once.
+
+## Overview
+
+When configuration compliance issues affect multiple devices with identical misconfigurations, the traditional approach of reviewing each device individually can be time-consuming and inefficient. The Configuration Hash Grouping feature solves this by:
+
+- Automatically grouping devices with identical configuration hashes
+- Providing a unified view of devices sharing the same configuration issues
+- Enabling bulk remediation operations on groups of devices
+- Simplifying troubleshooting of widespread configuration problems
+
+## How It Works
+
+The Configuration Hash Grouping feature uses a three-model architecture to efficiently organize and display configuration data:
+
+### Architecture Components
+
+1. **ConfigHashGrouping**: Groups devices with identical configuration hashes
+2. **ConfigComplianceHash**: Links individual devices to configuration hash groups
+3. **ConfigCompliance**: Provides the base compliance data (existing model, modified for integration)
+
+### Hash Generation Process
+
+When configuration compliance jobs run, the system:
+
+1. Computes SHA-256 hashes of device configuration content
+2. Creates or finds existing ConfigHashGrouping records for each unique hash
+3. Links devices to the appropriate hash groups via ConfigComplianceHash records
+4. Stores the actual configuration content once per unique hash for display purposes
+
+This approach eliminates duplicate storage while maintaining fast access to configuration data for analysis.
+
+## Accessing Configuration Hash Grouping
+
+To access the Configuration Hash Grouping feature:
+
+1. Navigate to **Golden Config** in the main navigation menu
+2. Under the **Manage** section, select **Hash Grouping Report**
+3. The main view displays groups of devices with identical configuration hashes
+
+!!! note
+    You must have the `view_configcompliance` permission to access this feature.
+
+## Configuration Hash Grouping Views
+
+### Main Grouping View
+
+The main Configuration Hash Grouping view (`/config-compliance/hash-grouping/`) displays:
+
+- **Feature Name**: The compliance rule feature being evaluated
+- **Device Count**: Number of devices sharing the same configuration hash (clickable to view devices)
+- **Configuration Preview**: Expandable view of the actual configuration content
+- **Action Buttons**: Quick access to remediation jobs and other operations
+
+### Device-Level Hash View
+
+The device-level view (`/config-compliance/config-hash/`) provides:
+
+- Individual device details linked to their hash groups
+- Device-specific configuration information
+- Direct navigation to device compliance details
+
+### Interactive Features
+
+The user interface includes several interactive elements:
+
+- **Expand/Collapse**: Toggle individual configuration displays
+- **Master Toggle**: Expand or collapse all configurations at once
+- **AJAX Loading**: Smooth loading of configuration content without page refreshes
+- **Fixed-Width Containers**: Consistent layout that prevents content shifting
+
+## API Access
+
+The Configuration Hash Grouping feature provides REST API access for programmatic integration:
+
+### Endpoints
+
+- **ConfigHashGrouping**: `/api/plugins/golden-config/config-hash-grouping/`
+- **ConfigComplianceHash**: `/api/plugins/golden-config/config-compliance-hash/`
+
+### Example API Usage
+
+```bash
+# Get all hash groups
+curl -H "Authorization: Token YOUR_TOKEN" \
+  http://nautobot/api/plugins/golden-config/config-hash-grouping/
+
+# Get devices in a specific hash group
+curl -H "Authorization: Token YOUR_TOKEN" \
+  http://nautobot/api/plugins/golden-config/config-compliance-hash/
+```
+
+## Summary
+
+The Configuration Hash Grouping feature represents a significant enhancement to Nautobot Golden Config's compliance capabilities. By grouping devices with identical configuration hashes, it provides network administrators with powerful tools for identifying, analyzing, and resolving configuration issues at scale. The feature's three-model architecture ensures excellent performance while maintaining data integrity, and its seamless integration with existing Golden Config functionality makes it immediately useful in any network automation workflow.

mkdocs.yml

@@ -122,6 +122,7 @@ nav:
       - Navigate Configuration Post-Processing: "user/app_feature_config_postprocessing.md"
       - Navigate Config Plans: "user/app_feature_config_plans.md"
       - Navigate Remediation: "user/app_feature_remediation.md"
+      - Navigate Config Hashes: "user/app_feature_hash.md"
       - Frequently Asked Questions: "user/faq.md"
       - External Interactions: "user/external_interactions.md"
       - Troubleshooting:

nautobot_golden_config/api/serializers.py

@@ -162,3 +162,23 @@ def get_remote_branches(self, obj):
     class Meta:  # noqa: D106  # undocumented-public-nested-class
         model = GitRepository
         fields = "__all__"
+
+
+class ConfigHashGroupingSerializer(NautobotModelSerializer, TaggedModelSerializerMixin):
+    """Serializer for ConfigHashGrouping object."""
+
+    class Meta:
+        """Set Meta Data for ConfigHashGrouping, will serialize all fields."""
+
+        model = models.ConfigHashGrouping
+        fields = "__all__"
+
+
+class ConfigComplianceHashSerializer(NautobotModelSerializer, TaggedModelSerializerMixin):
+    """Serializer for ConfigComplianceHash object."""
+
+    class Meta:
+        """Set Meta Data for ConfigComplianceHash, will serialize all fields."""
+
+        model = models.ConfigComplianceHash
+        fields = "__all__"

nautobot_golden_config/api/urls.py

@@ -16,6 +16,8 @@
 router.register("remediation-setting", views.RemediationSettingViewSet)
 router.register("config-postprocessing", views.ConfigToPushViewSet)
 router.register("config-plan", views.ConfigPlanViewSet)
+router.register("config-hash-grouping", views.ConfigHashGroupingViewSet)
+router.register("config-compliance-hash", views.ConfigComplianceHashViewSet)
 
 urlpatterns = [
     path(

nautobot_golden_config/api/views.py

@@ -210,6 +210,22 @@ def get_serializer_context(self):
         return context
 
 
+class ConfigHashGroupingViewSet(NautobotModelViewSet):  # pylint:disable=too-many-ancestors
+    """API viewset for interacting with ConfigHashGrouping objects."""
+
+    queryset = models.ConfigHashGrouping.objects.all()
+    serializer_class = serializers.ConfigHashGroupingSerializer
+    filterset_class = filters.ConfigHashGroupingFilterSet
+
+
+class ConfigComplianceHashViewSet(NautobotModelViewSet):  # pylint:disable=too-many-ancestors
+    """API viewset for interacting with ConfigComplianceHash objects."""
+
+    queryset = models.ConfigComplianceHash.objects.all()
+    serializer_class = serializers.ConfigComplianceHashSerializer
+    filterset_class = filters.ConfigComplianceHashFilterSet
+
+
 class GenerateIntendedConfigException(APIException):
     """Exception for when the intended config cannot be generated."""
 

nautobot_golden_config/filters.py

@@ -1,6 +1,7 @@
 """Filtering for nautobot_golden_config."""
 
 import django_filters
+from django.db.models import Count, Exists, F, OuterRef, Q
 from nautobot.apps.filters import (
     MultiValueDateTimeFilter,
     NaturalKeyOrPKMultipleChoiceFilter,
@@ -146,6 +147,56 @@
         to_field_name="slug",
         label="ComplianceFeature (slug)",
     )
+    compliance = django_filters.BooleanFilter(
+        field_name="compliance",
+        label="Compliance Status",
+    )
+    config_hash_group = django_filters.CharFilter(
+        method="filter_by_hash_group",
+        label="Config Hash Group",
+    )
+    config_hash = django_filters.CharFilter(
+        method="filter_by_config_hash",
+        label="Config Hash",
+    )
+
+    def filter_by_hash_group(self, queryset, _name, value):
+        """Filter ConfigCompliance records by config hash group ID."""
+        if not value:
+            return queryset
+
+        try:
+            # Get the hash group
+            hash_group = models.ConfigHashGrouping.objects.get(pk=value)
+
+            # Find all devices that are linked to this hash group via ConfigComplianceHash
+            devices_in_group = models.ConfigComplianceHash.objects.filter(
+                config_group=hash_group, config_type="actual"
+            ).values_list("device_id", flat=True)
+
+            # Filter ConfigCompliance records to show only these devices for this rule
+            return queryset.filter(device_id__in=devices_in_group, rule=hash_group.rule)
+
+        except models.ConfigHashGrouping.DoesNotExist:
+            # If hash group doesn't exist, return empty queryset
+            return queryset.none()
+
+    def filter_by_config_hash(self, queryset, _name, value):
+        """Filter ConfigCompliance by hash value, exact or prefix."""
+        if not value:
+            return queryset
+        matching_groups = models.ConfigHashGrouping.objects.filter(
+            config_hash__istartswith=value,
+        )
+        if not matching_groups.exists():
+            return queryset.none()
+        matching_hash_rows = models.ConfigComplianceHash.objects.filter(
+            device=OuterRef("device"),
+            rule=OuterRef("rule"),
+            config_type="actual",
+            config_group__in=matching_groups,
+        )
+        return queryset.filter(Exists(matching_hash_rows))
 
     class Meta:
         """Meta class attributes for ConfigComplianceFilter."""
@@ -154,6 +205,134 @@
         fields = "__all__"
 
 
+class ConfigHashGroupingFilterSet(NautobotFilterSet):
+    """Custom filter for configuration hash grouping that handles device filtering properly."""
+
+    feature = django_filters.ModelMultipleChoiceFilter(
+        field_name="rule__feature__name",
+        queryset=models.ComplianceFeature.objects.all(),
+        to_field_name="name",
+        label="Feature",
+    )
+
+    device = NaturalKeyOrPKMultipleChoiceFilter(
+        queryset=Device.objects.all(),
+        to_field_name="name",
+        label="Device (name or ID)",
+        method="filter_by_device",
+    )
+
+    class Meta:
+        """Meta class attributes for ConfigHashGroupingFilterSet."""
+
+        model = models.ConfigHashGrouping
+        fields = "__all__"
+
+    def filter_by_device(self, queryset, _name, value):
+        """Filter ConfigHashGrouping records by devices that are members of the groups."""
+        if not value:
+            return queryset
+
+        # Get device IDs from the filter value
+        device_ids = []
+        for device in value:
+            if hasattr(device, "id"):
+                device_ids.append(device.id)
+            else:
+                device_ids.append(device)
+
+        # Find all ConfigHashGrouping IDs where these devices have corresponding ConfigComplianceHash records
+        hash_group_ids = (
+            models.ConfigComplianceHash.objects.filter(device_id__in=device_ids, config_group__isnull=False)
+            .values_list("config_group_id", flat=True)
+            .distinct()
+        )
+
+        return queryset.filter(id__in=hash_group_ids)
+
+
+class ConfigComplianceHashFilterSet(GoldenConfigFilterSet):  # pylint: disable=too-many-ancestors
+    """Custom filter for mismatch grouping that handles device filtering properly."""
+
+    location = TreeNodeMultipleChoiceFilter(
+        queryset=Location.objects.all(),
+        field_name="device__location",
+        to_field_name="name",
+        label="Location (name)",
+    )
+    platform = NaturalKeyOrPKMultipleChoiceFilter(
+        field_name="device__platform",
+        queryset=Platform.objects.all(),
+        to_field_name="name",
+        label="Platform (name or ID)",
+    )
+
+    device = NaturalKeyOrPKMultipleChoiceFilter(
+        field_name="device",
+        queryset=Device.objects.all(),
+        to_field_name="name",
+        label="Device (name or ID)",
+    )
+
+    def filter_device(self, queryset, _name, value):
+        """Custom device filtering for grouped mismatch data."""
+        # Get the devices to filter by
+        device_ids = [device.id if hasattr(device, "id") else device for device in value]
+
+        # Find ConfigComplianceHash records for these devices that correspond to non-compliant actual configs
+        hash_records = (
+            models.ConfigComplianceHash.objects.filter(
+                device_id__in=device_ids,
+                config_type="actual",
+                device__configcompliance__rule=F("rule"),
+                device__configcompliance__compliance=False,
+            )
+            .values("rule", "config_hash")
+            .distinct()
+        )
+
+        # Build filters for rule+hash combinations
+        hash_filters = Q()
+        filter_count = 0
+        for record in hash_records:
+            if record["config_hash"]:
+                hash_filters |= Q(rule=record["rule"], config_hash=record["config_hash"])
+                filter_count += 1
+
+        if hash_filters:
+            # Filter the base ConfigComplianceHash records before they get grouped
+            base_qs = models.ConfigComplianceHash.objects.filter(
+                config_type="actual",
+                device__configcompliance__rule=F("rule"),
+                device__configcompliance__compliance=False,
+            ).filter(hash_filters)
+
+            # Apply grouping to the filtered base queryset
+            grouped_qs = (
+                base_qs.values(
+                    "rule__feature__id", "rule__feature__name", "rule__feature__slug", "config_hash", "config_content"
+                )
+                .annotate(
+                    device_count=Count("device", distinct=True),
+                    feature_id=F("rule__feature__id"),
+                    feature_name=F("rule__feature__name"),
+                    feature_slug=F("rule__feature__slug"),
+                )
+                .filter(device_count__gt=1)
+                .order_by("-device_count", "rule__feature__name")
+            )
+
+            return grouped_qs
+
+        return queryset.none()
+
+    class Meta:
+        """Boilerplate filter Meta data for Config Hash."""
+
+        model = models.ConfigComplianceHash
+        fields = "__all__"
+
+
 class ComplianceFeatureFilterSet(NautobotFilterSet):
     """Inherits Base Class NautobotFilterSet."""
 
@@ -257,7 +436,7 @@
         method="filter_device_id",
     )
 
-    def filter_device_id(self, queryset, name, value):  # pylint: disable=unused-argument
+    def filter_device_id(self, queryset, _name, value):
         """Filter by Device ID."""
         if not value:
             return queryset