We are fetching Falsepositive

[gotspatel]

i think i found one major fault

the rule created

uri		PHISHTANK_0009282311_MATCH	/www\.google\.com\/url/
score		PHISHTANK_0009282311_MATCH	5.0
describe	PHISHTANK_0009282311_MATCH	Phishtank ID 9282311 (Other) 

the entry in json

    {
        "phish_id": 9282311,
        "url": "https://www.google.com/url?q=https%3A%2F%2Faccess-port-ggl0rw3zkononuwgjpgwgsh.vercel.app&sa=D&sntz=1&usg=AOvVaw1bg884qO0M5p6ZfkBpXDD5#a@b.com",
        "phish_detail_url": "http://www.phishtank.com/phish_detail.php?phish_id=9282311",
        "submission_time": "2025-12-03T06:55:37+00:00",
        "verified": "yes",
        "verification_time": "2025-12-15T05:42:05+00:00",
        "online": "yes",
        "details": [
            {
                "ip_address": "142.251.16.103",
                "cidr_block": "142.251.16.0/24",
                "announcing_network": "15169",
                "rir": "arin",
                "country": "US",
                "detail_time": "2025-12-03T07:03:48+00:00"
            }
        ],
        "target": "Other"
    }

we are using a false positive here https://www.google.com/url instead of https%3A%2F%2Faccess-port-ggl0rw3zkononuwgjpgwgsh.vercel.app&sa=D&sntz=1&usg=AOvVaw1bg884qO0M5p6ZfkBpXDD5#a@b.com

Am I correct in this interpretation?

[kawaiipantsu]

Yes, that would give some false positive hits as it would match any/all google URI's related to /url.
The fix would be to include more of the Phishtank url, I think back then I simply did it this way as it ended up being to hard/difficult to sanitize the urls correctly if you included the whole thing. It was just easier and more clean to cut of by the "?" and truncate the rest.

But the solution would be to include everything or at least more.
I have no timeline on this as I'm busy on the ipdigger/subdigger tools as we speak.

[gotspatel]

I have moved the complete processing to python now and resolved the same and others also, but for this fix i think you can test this function, when you have time

or suggest if maybe i am doing it wrong

// Function to build a SpamAssassin rule from phishtank array entry
function buildRule($entry) {

    $ptID = trim($entry["phish_id"]);
    $id   = sprintf("PHISHTANK_%010s", $ptID);
    cLog("  - Building rule: " . $id);

    $ptURL  = trim($entry["phish_detail_url"]);
    $ptDate = trim($entry["submission_time"]);

    /*
     * ------------------------------------------------------------------
     * FIX: Preserve full URL semantics (Python parity)
     * ------------------------------------------------------------------
     * - Decode HTML entities (fixes &)
     * - Parse URL normally
     * - Build match using host + path + query + fragment
     *
     * This prevents false positives like:
     *   www.google.com/url
     * by ensuring the rule matches the FULL redirect URL,
     * not just the redirect endpoint.
     */
    $rawURL = html_entity_decode(trim($entry["url"]), ENT_QUOTES | ENT_HTML5);
    $parsed = parse_url($rawURL);

    if (!isset($parsed['host'])) {
        cLog("    ! Invalid URL, skipping rule");
        return null;
    }

    // Host is mandatory
    $matchURL = $parsed['host'];

    // Path (default "/" if missing, matches Python behavior)
    if (isset($parsed['path'])) {
        $matchURL .= $parsed['path'];
    } else {
        $matchURL .= '/';
    }

    // Query (THIS IS THE CRITICAL FIX)
    if (isset($parsed['query']) && $parsed['query'] !== '') {
        $matchURL .= '?' . $parsed['query'];
    }

    // Fragment (rare, but Python keeps it)
    if (isset($parsed['fragment']) && $parsed['fragment'] !== '') {
        $matchURL .= '#' . $parsed['fragment'];
    }

    // Escape for SpamAssassin regex (PHP equivalent of re.escape)
    $matchURL_regex = preg_quote($matchURL, '/');

    /*
     * ------------------------------------------------------------------
     * Original flags and scoring (unchanged)
     * ------------------------------------------------------------------
     */
    $bValidated      = strtolower(trim($entry["verified"])) == "yes" ? "1" : "0";
    $bActive         = strtolower(trim($entry["online"])) == "yes" ? "1" : "0";
    $score           = strlen(SCORE_DEFAULT) >= 1 ? SCORE_DEFAULT : "10";
    $score_validated = strlen(SCORE_EXTRA_VALIDATED) >= 1 ? SCORE_EXTRA_VALIDATED : "10";
    $score_online    = strlen(SCORE_EXTRA_ACTIVE) >= 1 ? SCORE_EXTRA_ACTIVE : "10";
    $ptTarget        = "(" . trim($entry["target"]) . ")";

    /*
     * ------------------------------------------------------------------
     * Rule output (unchanged)
     * ------------------------------------------------------------------
     */
    $rule = <<<EOD
## Rule ID.......: ${id}
## PhishTank URL.: ${ptURL}
## Submitted.....: ${ptDate}
##
## DEFAULT MATCH RULE
uri\t\t${id}_MATCH\t/${matchURL_regex}/
score\t\t${id}_MATCH\t${score}
describe\t${id}_MATCH\tPhishtank ID ${ptID} ${ptTarget}

## VALIDATED RULE
meta\t\t__${id}_VALCHECK\t(1*${bValidated}) > 0
meta\t\t${id}_VALIDATED\t${id}_MATCH && __${id}_VALCHECK
score\t\t${id}_VALIDATED\t${score_validated}
describe\t${id}_VALIDATED\tValidated phishing url

## ACTIVE/ONLINE RULE
meta\t\t__${id}_ONCHECK\t(1*${bActive}) > 0
meta\t\t${id}_ONLINE\t${id}_MATCH && __${id}_ONCHECK
score\t\t${id}_ONLINE\t${score_online}
describe\t\t${id}_ONLINE\tPhishing URL still online
##
EOD;

    return $rule;
}

Kind Regards

[kawaiipantsu]

If you test and it's working, you can create a pull request etc.

[gotspatel]

Oh It works 100% and no complaint from Spamassassin also, but I really don't know how to make a PR

SAMPLE1:

   {
       "phish_id": 9324403,
       "url": "https://www.google.com/url?q=https://00o0o00o0o.skapi.com/ionos/1bgionos.html&sa=D&sntz=1&usg=AOvVaw0q_tryahwY5bNjiVWi7VUp#email",
       "phish_detail_url": "http://www.phishtank.com/phish_detail.php?phish_id=9324403",
       "submission_time": "2026-01-28T08:15:29+00:00",
       "verified": "yes",
       "verification_time": "2026-01-28T08:22:00+00:00",
       "online": "yes",
       "details": [
           {
               "ip_address": "142.251.163.103",
               "cidr_block": "142.251.163.0/24",
               "announcing_network": "15169",
               "rir": "arin",
               "country": "US",
               "detail_time": "2026-01-28T08:22:53+00:00"
           }
       ],
       "target": "Other"
   },

##  ____  _     _     _   _____           _      ____        _
## |  _ \| |__ (_)___| |_|_   _|_ _ _ __ | | __ |  _ \ _   _| | ___
## | |_) | '_ \| / __| '_ \| |/ _` | '_ \| |/ / | |_) | | | | |/ _ \
## |  __/| | | | \__ \ | | | | (_| | | | |   <  |  _ <| |_| | |  __/
## |_|   |_| |_|_|___/_| |_|_|\__,_|_| |_|_|\_\ |_| \_\\__,_|_|\___|
##
## Rule ID.......: PHISHTANK_0009324403
## PhishTank URL.: http://www.phishtank.com/phish_detail.php?phish_id=9324403
## Submitted.....: 2026-01-28T08:15:29+00:00
## 
## DEFAULT MATCH RULE
uri			PHISHTANK_0009324403_MATCH	/www\.google\.com\/url\?q\=https\:\/\/00o0o00o0o\.skapi\.com\/ionos\/1bgionos\.html&sa\=D&sntz\=1&usg\=AOvVaw0q_tryahwY5bNjiVWi7VUp\#email/
score		PHISHTANK_0009324403_MATCH	5.0
describe	PHISHTANK_0009324403_MATCH	PHISHTANK - ID 9324403 (Other) 

## VALIDATED RULE
meta		__PHISHTANK_0009324403_VALCHECK	(1*1) > 0
meta		PHISHTANK_0009324403_VALIDATED	PHISHTANK_0009324403_MATCH && __PHISHTANK_0009324403_VALCHECK
score		PHISHTANK_0009324403_VALIDATED	2.0
describe	PHISHTANK_0009324403_VALIDATED	PHISHTANK - Validated phishing url

## ACTIVE/ONLINE RULE
meta		__PHISHTANK_0009324403_ONCHECK	(1*1) > 0
meta		PHISHTANK_0009324403_ONLINE	PHISHTANK_0009324403_MATCH && __PHISHTANK_0009324403_ONCHECK
score		PHISHTANK_0009324403_ONLINE	2.0
describe	PHISHTANK_0009324403_ONLINE	PHISHTANK - Phishing URL still online

##

SAMPLE 2:

{
        "phish_id": 9312891,
        "url": "https://www.google.com/url?q=https%3A%2F%2Fgoogleidmen.dynv6.net%2Fgoogle.com&sa=D&sntz=1&usg=AOvVaw3CCbESUnW_5YK4bwvkytqI",
        "phish_detail_url": "http://www.phishtank.com/phish_detail.php?phish_id=9312891",
        "submission_time": "2026-01-15T14:11:17+00:00",
        "verified": "yes",
        "verification_time": "2026-01-15T14:21:51+00:00",
        "online": "yes",
        "details": [
            {
                "ip_address": "192.178.155.103",
                "cidr_block": "192.178.155.0/24",
                "announcing_network": "15169",
                "rir": "arin",
                "country": "US",
                "detail_time": "2026-01-15T14:13:05+00:00"
            }
        ],
        "target": "Internal Revenue Service"
    },

##  ____  _     _     _   _____           _      ____        _
## |  _ \| |__ (_)___| |_|_   _|_ _ _ __ | | __ |  _ \ _   _| | ___
## | |_) | '_ \| / __| '_ \| |/ _` | '_ \| |/ / | |_) | | | | |/ _ \
## |  __/| | | | \__ \ | | | | (_| | | | |   <  |  _ <| |_| | |  __/
## |_|   |_| |_|_|___/_| |_|_|\__,_|_| |_|_|\_\ |_| \_\\__,_|_|\___|
##
## Rule ID.......: PHISHTANK_0009312891
## PhishTank URL.: http://www.phishtank.com/phish_detail.php?phish_id=9312891
## Submitted.....: 2026-01-15T14:11:17+00:00
## 
## DEFAULT MATCH RULE
uri			PHISHTANK_0009312891_MATCH	/www\.google\.com\/url\?q\=https%3A%2F%2Fgoogleidmen\.dynv6\.net%2Fgoogle\.com&sa\=D&sntz\=1&usg\=AOvVaw3CCbESUnW_5YK4bwvkytqI/
score		PHISHTANK_0009312891_MATCH	5.0
describe	PHISHTANK_0009312891_MATCH	PHISHTANK - ID 9312891 (Internal Revenue Service) 

## VALIDATED RULE
meta		__PHISHTANK_0009312891_VALCHECK	(1*1) > 0
meta		PHISHTANK_0009312891_VALIDATED	PHISHTANK_0009312891_MATCH && __PHISHTANK_0009312891_VALCHECK
score		PHISHTANK_0009312891_VALIDATED	2.0
describe	PHISHTANK_0009312891_VALIDATED	PHISHTANK - Validated phishing url

## ACTIVE/ONLINE RULE
meta		__PHISHTANK_0009312891_ONCHECK	(1*1) > 0
meta		PHISHTANK_0009312891_ONLINE	PHISHTANK_0009312891_MATCH && __PHISHTANK_0009312891_ONCHECK
score		PHISHTANK_0009312891_ONLINE	2.0
describe	PHISHTANK_0009312891_ONLINE	PHISHTANK - Phishing URL still online

##

Kind Regards,

[kawaiipantsu]

You can make your changes here: https://github.com/kawaiipantsu/spamassassin-rules/tree/14-we-are-fetching-falsepositive