Skip to content

fix: filter exact variable names from expression evaluation to avoid …#7396

Closed
X3r0Day wants to merge 1 commit into
projectdiscovery:devfrom
X3r0Day:dev
Closed

fix: filter exact variable names from expression evaluation to avoid …#7396
X3r0Day wants to merge 1 commit into
projectdiscovery:devfrom
X3r0Day:dev

Conversation

@X3r0Day
Copy link
Copy Markdown

@X3r0Day X3r0Day commented May 11, 2026

Closes #7395

Proposed changes

Filter out expressions that are exact keys in the variable map from govaluate evaluation. Hyphenated template variables like request-id are misparsed as subtraction (request - id) causing "No parameter 'request' found." on every request - these are simple variable references handled by replacer.Replace and should not be re-evaluated.

Proof

  • All 13 existing expression package tests pass
  • HTTP protocol package tests pass
  • Built binary tested against CVE-2025-55182 template: requests now appear as [VER] Sent HTTP request instead of [WRN] Could not execute request

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Summary by CodeRabbit

  • Bug Fixes
    • Fixed expression evaluation to prevent parsing errors and improve handling of variable references.

Review Change Stack

Copilot AI review requested due to automatic review settings May 11, 2026 20:14
@auto-assign auto-assign Bot requested a review from Mzack9999 May 11, 2026 20:14
@neo-by-projectdiscovery-dev
Copy link
Copy Markdown

neo-by-projectdiscovery-dev Bot commented May 11, 2026

Neo - PR Security Review

No security issues found

Comment @pdneo help for available commands. · Open in Neo

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 11, 2026

Walkthrough

The evaluate function in the expression handler now pre-filters discovered expressions to exclude exact matches for variable names already in the base map. This prevents govaluate from misparsing hyphenated identifiers (e.g., request-id as request - id subtraction) during compilation and evaluation.

Changes

Expression Filter Pre-processing

Layer / File(s) Summary
Expression Discovery Filter
pkg/protocols/common/expressions/expressions.go
evaluate filters FindExpressions results to exclude expression strings that exactly match keys in base, preventing govaluate from misparsing simple variable references with hyphens or special characters.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

A rabbit's hop through expressions so fine,
Filters the hyphens, makes variables align,
No more request - id confusion to parse,
Simple names pass through, complex ones spark! 🐰✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately describes the main fix: filtering exact variable names from expression evaluation to prevent hyphenated variables from being misparsed as arithmetic operations.
Linked Issues check ✅ Passed The PR directly implements the fix required by issue #7395: filtering out expressions matching exact variable keys to prevent govaluate misparse errors on hyphenated template variables like request-id.
Out of Scope Changes check ✅ Passed The change scope is limited to the expressions.go file with pre-filtering logic; all modifications directly address the root cause of issue #7395 without introducing unrelated changes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a regression in the template expression engine where hyphenated template variable names (e.g. request-id) were incorrectly treated as govaluate expressions and misparsed as subtraction, causing evaluation failures during request execution.

Changes:

  • Filter out expressions that exactly match keys in the variable map before govaluate evaluation.
  • Preserve normal placeholder substitution for simple variable references (handled via replacer.Replace) while avoiding mis-evaluation of hyphenated names.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +48 to +57
// Filter out expressions that are exact variable names in base
// these are simple variable references handled by replacer.Replace and
// evaluating them as govaluate expressions can misparse hyphens as subtraction
filtered := make([]string, 0, len(expressions))
for _, expr := range expressions {
if _, ok := base[expr]; !ok {
filtered = append(filtered, expr)
}
}
expressions = filtered
Comment on lines +48 to +50
// Filter out expressions that are exact variable names in base
// these are simple variable references handled by replacer.Replace and
// evaluating them as govaluate expressions can misparse hyphens as subtraction
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/protocols/common/expressions/expressions.go`:
- Around line 48-57: Add a unit test in the expressions package that ensures
hyphenated variable names are treated as simple variable references (not parsed
as subtraction) by the replacer/Evaluate flow: create a base map containing keys
like "request-id" and "nextjs-html" and assert that Evaluate (or the public
evaluation wrapper that uses the logic in expressions.go which filters exact
variable names from govaluate parsing) returns "abc123" for "{{request-id}}",
"ID: abc123" for "ID: {{request-id}}", and "abc123/content" for
"{{request-id}}/{{nextjs-html}}"; this will prevent regressions to the filtering
logic that skips entries present in base and relies on replacer.Replace rather
than govaluate parsing.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b23f7c59-1da8-4f1e-b392-8e13f7b16c80

📥 Commits

Reviewing files that changed from the base of the PR and between d65c795 and a308a8c.

📒 Files selected for processing (1)
  • pkg/protocols/common/expressions/expressions.go

Comment on lines +48 to +57
// Filter out expressions that are exact variable names in base
// these are simple variable references handled by replacer.Replace and
// evaluating them as govaluate expressions can misparse hyphens as subtraction
filtered := make([]string, 0, len(expressions))
for _, expr := range expressions {
if _, ok := base[expr]; !ok {
filtered = append(filtered, expr)
}
}
expressions = filtered
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win

Add test coverage for hyphenated variable names to prevent regression.

While the fix correctly addresses issue #7395, the PR checklist indicates no tests were added. Since this fixes a regression introduced between v3.7.1 and v3.8.0, a test case specifically covering hyphenated variable names (e.g., request-id, nextjs-html) would prevent future regressions.

🧪 Suggested test case structure

Consider adding a test in the expression package that verifies:

// Test that hyphenated variable names are treated as simple variables,
// not parsed as subtraction expressions
func TestEvaluateHyphenatedVariableNames(t *testing.T) {
    base := map[string]interface{}{
        "request-id": "abc123",
        "nextjs-html": "content",
    }
    
    // Test single hyphenated variable
    result, err := Evaluate("{{request-id}}", base)
    // expect: "abc123", no govaluate parse error
    
    // Test hyphenated variable in text
    result, err = Evaluate("ID: {{request-id}}", base)
    // expect: "ID: abc123"
    
    // Test multiple hyphenated variables
    result, err = Evaluate("{{request-id}}/{{nextjs-html}}", base)
    // expect: "abc123/content"
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/protocols/common/expressions/expressions.go` around lines 48 - 57, Add a
unit test in the expressions package that ensures hyphenated variable names are
treated as simple variable references (not parsed as subtraction) by the
replacer/Evaluate flow: create a base map containing keys like "request-id" and
"nextjs-html" and assert that Evaluate (or the public evaluation wrapper that
uses the logic in expressions.go which filters exact variable names from
govaluate parsing) returns "abc123" for "{{request-id}}", "ID: abc123" for "ID:
{{request-id}}", and "abc123/content" for "{{request-id}}/{{nextjs-html}}"; this
will prevent regressions to the filtering logic that skips entries present in
base and relies on replacer.Replace rather than govaluate parsing.

Comment on lines +51 to +57
filtered := make([]string, 0, len(expressions))
for _, expr := range expressions {
if _, ok := base[expr]; !ok {
filtered = append(filtered, expr)
}
}
expressions = filtered
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this as nothing more than a temporary workaround because it only addresses the callers of evaluate(). The problem is that any other direct caller of FindExpressions(..., base) is still going to run into issues where they see the exact placeholder keys being treated as expressions.

Also, this approach allocates.

@dwisiswant0
Copy link
Copy Markdown
Member

Superseded by #7397.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Template variables with hyphens (e.g. request-id) fail with "No parameter 'request' found."

3 participants