Improve parsing of JSON-LD to minimize parsing errors

[pkdash]

For user submitted records DSP scraps the JSON-LD data from landing page of of those records and converts it to a python dict using json.loads(). This conversion to dict will fail if the scrapped JSON-LD is not a valid JSON-Encoded string. Though it may not be possible to handle all cases of bad JSON-LD, we should try to preprocess the JSON-LD to remove control chars and extra quotes etc before using it in json.loads(). Here is an example of such a preprocessing function:

import re

def preprocess_json_string(raw_str):
    """
    Cleans a JSON string to reduce the chance of parsing errors.
    
    - Removes wrapping quotes (if present)
    - Removes control characters (except \t, \n, \r)
    - Escapes stray backslashes
    """
    if not isinstance(raw_str, str):
        return raw_str  # Skip preprocessing if not a string

    # Strip extra quotes around the whole string (common in user inputs or logs)
    if raw_str.startswith('"') and raw_str.endswith('"'):
        raw_str = raw_str[1:-1]

    # Unescape incorrectly escaped quotes (e.g., \" inside a quoted string)
    raw_str = raw_str.replace('\\"', '"')

    # Escape unescaped backslashes (avoid breaking valid escape sequences)
    raw_str = re.sub(r'(?<!\\)\\(?![\\/"bfnrtu])', r'\\\\', raw_str)

    # Remove control characters (except \n, \r, \t)
    raw_str = re.sub(r'[\x00-\x08\x0B\x0C\x0E-\x1F]', '', raw_str)

    return raw_str