Inconsistent tag/version parsing behavior of Version.parse() and Version.from_any_vcs()?

[martinpakosch]

Hi @mtkennerly,

it's me again. While I am playing around with the Dunamai API I noticed an unexpected inconsistency. I am wondering, if the difference between the Version.parse() and the Version.from_any_vcs() parsing behavior is intended or not. You describe the parse() method as an heuristic approach, but why not use the approach via _match_version_pattern, that is also used by the VCS functions to parse string versions? Wouldn't this be more reliable and more consistent? At lease that was what I expected. :) Wouldn't it be nicer to have a more consistent behavior? Or also adapt another API function to simulate VCS parsing behavior for version strings (just talking about the Tag-part of the existing parsing)?

Here is a summary what I have tested so far. Take a look at the if version_string is not None: blocks.

from dunamai import Pattern, Style, Vcs, Version, _match_version_pattern

VERSION_PATTERN: str | Pattern = Pattern.DefaultUnprefixed
"""RegEx pattern or preset used to match the version tag."""

VERSION_PATTERN_PREFIX: str | None = "poc-"
"""RegEx prefix used in the version pattern to match a version tag."""

This code using Version.parse()...

if version_string is not None:
    version = Version.parse(
        version_string, Pattern.parse(VERSION_PATTERN, VERSION_PATTERN_PREFIX)
    )
else:
    version = Version.from_any_vcs(
        pattern=VERSION_PATTERN,
        pattern_prefix=VERSION_PATTERN_PREFIX,
        strict=True,
    )

...behaves different (less reliable) while parsing version strings than this code (no part of the API)...

if version_string is not None:
    matched_pattern = _match_version_pattern(
        pattern=VERSION_PATTERN,
        pattern_prefix=VERSION_PATTERN_PREFIX,
        sources=[version_string],
        latest_source=True,
        strict=True,
    )
    if not matched_pattern:
        msg = f"failed to parse version string: '{version_string}'"
        raise ValueError(msg)
    _, base, stage, _, tagged_metadata, epoch = matched_pattern
    version = Version(
        base, stage=stage, tagged_metadata=tagged_metadata, epoch=epoch
    )
else:
    version = Version.from_any_vcs(
        pattern=VERSION_PATTERN,
        pattern_prefix=VERSION_PATTERN_PREFIX,
        strict=True,
    )

Let me explain the setup. With the code above, you should have everything you need for a comparison of VCS readout vs. both approaches. In my case it is part of a custom version.py as I am currently evaluating the API to also provide "manual" release-version management. :) However, the details do not matter. I use the following shell command to evaluate this. Just set the GITTAG at the beginning to experiment. Your Python file will need to read the CLI Args to get the version string, if you stick to this setup (could be also hard-coded, that's clear). :) Ignore the OUT lines as they use opinionated serialization. ;-)

This is with the upper approach using VCS compared to Version.parse()...

$ GITTAG=poc-2.9.4+local; git tag -a $GITTAG -m testing HEAD~2; python ./version.py; python ./version.py --version $GITTAG; git tag -d $GITTAG
IN: None
DEBUG: {'_matched_tag': 'poc-2.9.4+local',
 '_newer_unmatched_tags': [],
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': 'dunamai',
 'commit': '035652a',
 'concerns': set(),
 'dirty': True,
 'distance': 2,
 'epoch': None,
 'revision': None,
 'stage': None,
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': datetime.datetime(2025, 7, 15, 0, 3, 57, tzinfo=datetime.timezone.utc),
 'vcs': <Vcs.Git: 'git'>}
OUT: 2.9.4-post.2+local.g035652a.d20250715

IN: poc-2.9.4+local
DEBUG: {'_matched_tag': None,
 '_newer_unmatched_tags': None,
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': None,
 'commit': 'local',    <<<<< ???
 'concerns': set(),
 'dirty': None,
 'distance': 0,
 'epoch': None,
 'revision': None,
 'stage': None,
 'tagged_metadata': None,    <<<<< ???
 'timestamp': None,
 'vcs': <Vcs.Any: 'any'>}
OUT: 2.9.4
Deleted tag 'poc-2.9.4+local' (was c76bfa0)

------------------------------

$ GITTAG=poc-2.9.4-alpha.2+local; git tag -a $GITTAG -m testing HEAD~2; python ./version.py; python ./version.py --version $GITTAG; git tag -d $GITTAG
IN: None
DEBUG: {'_matched_tag': 'poc-2.9.4-alpha.2+local',
 '_newer_unmatched_tags': [],
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': 'dunamai',
 'commit': '035652a',
 'concerns': set(),
 'dirty': True,
 'distance': 2,
 'epoch': None,
 'revision': 2,    <<<<<< OK
 'stage': 'alpha',    <<<<<< OK
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': datetime.datetime(2025, 7, 15, 0, 3, 57, tzinfo=datetime.timezone.utc),
 'vcs': <Vcs.Git: 'git'>}
OUT: 2.9.4-alpha.2.post.2+local.g035652a.d20250715
IN: poc-2.9.4-alpha.2+local
DEBUG: {'_matched_tag': None,
 '_newer_unmatched_tags': None,
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': None,
 'commit': 'local',    <<<<< ???
 'concerns': set(),
 'dirty': None,
 'distance': 0,
 'epoch': None,
 'revision': 2,    <<<<<< OK
 'stage': 'alpha',    <<<<<< OK
 'tagged_metadata': None,    <<<<< ???
 'timestamp': None,
 'vcs': <Vcs.Any: 'any'>}
OUT: 2.9.4-alpha.2
Deleted tag 'poc-2.9.4-alpha.2+local' (was db114e4)

------------------------------

$ GITTAG=poc-2.9.4-dev.6+local; git tag -a $GITTAG -m testing HEAD~2; python ./version.py; python ./version.py --version $GITTAG; git tag -d $GITTAG
IN: None
DEBUG: {'_matched_tag': 'poc-2.9.4-dev.6+local',
 '_newer_unmatched_tags': [],
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': 'dunamai',
 'commit': '035652a',
 'concerns': set(),
 'dirty': True,
 'distance': 2,
 'epoch': None,
 'revision': 6,    <<<<<< OK
 'stage': 'dev',    <<<<<< OK
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': datetime.datetime(2025, 7, 15, 0, 3, 57, tzinfo=datetime.timezone.utc),
 'vcs': <Vcs.Git: 'git'>}
OUT: 2.9.4-dev.6.post.2+local.g035652a.d20250715
IN: poc-2.9.4-dev.6+local
DEBUG: {'_matched_tag': None,
 '_newer_unmatched_tags': None,
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': None,
 'commit': 'local',    <<<<< ???
 'concerns': set(),
 'dirty': None,
 'distance': 6,    <<<<< ???
 'epoch': None,
 'revision': None,    <<<<< ???
 'stage': None,    <<<<< ???
 'tagged_metadata': None,    <<<<< ???
 'timestamp': None,
 'vcs': <Vcs.Any: 'any'>}
OUT: 2.9.4-post.6+local
Deleted tag 'poc-2.9.4-dev.6+local' (was 07d8f4d)

This is with the lower approach using VCS compared to _match_version_pattern()... which is not part of the public API...

$ GITTAG=poc-2.9.4+local; git tag -a $GITTAG -m testing HEAD~2; python ./version.py; python ./version.py --version $GITTAG; git tag -d $GITTAG
IN: None
DEBUG: {'_matched_tag': 'poc-2.9.4+local',
 '_newer_unmatched_tags': [],
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': 'dunamai',
 'commit': '035652a',
 'concerns': set(),
 'dirty': True,
 'distance': 2,
 'epoch': None,
 'revision': None,
 'stage': None,
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': datetime.datetime(2025, 7, 15, 0, 3, 57, tzinfo=datetime.timezone.utc),
 'vcs': <Vcs.Git: 'git'>}
OUT: 2.9.4-post.2+local.g035652a.d20250715
IN: poc-2.9.4+local
DEBUG: {'_matched_tag': None,
 '_newer_unmatched_tags': None,
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': None,
 'commit': None,
 'concerns': set(),
 'dirty': None,
 'distance': 0,
 'epoch': None,
 'revision': None,
 'stage': None,
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': None,
 'vcs': <Vcs.Any: 'any'>}    <<<<<< OK, but maybe I should set it to None for my purpose. :D
OUT: 2.9.4+local
Deleted tag 'poc-2.9.4+local' (was baab322)

------------------------------

$ GITTAG=poc-2.9.4-alpha.2+local; git tag -a $GITTAG -m testing HEAD~2; python ./version.py; python ./version.py --version $GITTAG; git tag -d $GITTAG
IN: None
DEBUG: {'_matched_tag': 'poc-2.9.4-alpha.2+local',
 '_newer_unmatched_tags': [],
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': 'dunamai',
 'commit': '035652a',
 'concerns': set(),
 'dirty': True,
 'distance': 2,
 'epoch': None,
 'revision': 2,    <<<<<< OK
 'stage': 'alpha',    <<<<<< OK
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': datetime.datetime(2025, 7, 15, 0, 3, 57, tzinfo=datetime.timezone.utc),
 'vcs': <Vcs.Git: 'git'>}
OUT: 2.9.4-alpha.2.post.2+local.g035652a.d20250715
IN: poc-2.9.4-alpha.2+local
DEBUG: {'_matched_tag': None,
 '_newer_unmatched_tags': None,
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': None,
 'commit': None,
 'concerns': set(),
 'dirty': None,
 'distance': 0,
 'epoch': None,
 'revision': 2,    <<<<<< OK
 'stage': 'alpha',    <<<<<< OK
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': None,
 'vcs': <Vcs.Any: 'any'>}
OUT: 2.9.4-alpha.2+local
Deleted tag 'poc-2.9.4-alpha.2+local' (was 10c6801)

------------------------------

$ GITTAG=poc-2.9.4-dev.6+local; git tag -a $GITTAG -m testing HEAD~2; python ./version.py; python ./version.py --version $GITTAG; git tag -d $GITTAG
IN: None
DEBUG: {'_matched_tag': 'poc-2.9.4-dev.6+local',
 '_newer_unmatched_tags': [],
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': 'dunamai',
 'commit': '035652a',
 'concerns': set(),
 'dirty': True,
 'distance': 2,
 'epoch': None,
 'revision': 6,    <<<<<< OK
 'stage': 'dev',    <<<<<< OK
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': datetime.datetime(2025, 7, 15, 0, 3, 57, tzinfo=datetime.timezone.utc),
 'vcs': <Vcs.Git: 'git'>}
OUT: 2.9.4-dev.6.post.2+local.g035652a.d20250715
IN: poc-2.9.4-dev.6+local
DEBUG: {'_matched_tag': None,
 '_newer_unmatched_tags': None,
 '_smart_bumped': False,
 'base': '2.9.4',
 'branch': None,
 'commit': None,
 'concerns': set(),
 'dirty': None,
 'distance': 0,
 'epoch': None,
 'revision': 6,    <<<<<< OK
 'stage': 'dev',    <<<<<< OK
 'tagged_metadata': 'local',    <<<<<< OK
 'timestamp': None,
 'vcs': <Vcs.Any: 'any'>}
OUT: 2.9.4-dev.6+local
Deleted tag 'poc-2.9.4-dev.6+local' (was 7086789)

Hope you get the point of my concerns. :D

I know, especially when taking dev or even post tags into account, this gets complicated as it partially might colide with the behavior how Version.serialize() uses them to express the commit distance with/without bumping. And maybe the revision of dev/post tags should always match the distance, but I do not aggree with that.

From a release perspective a post or dev revision should not be bound to the distance... I should be free to start with dev.1 or post.1 on a distance of e.g. 6 if I really want to make such a release (to have a clean version numbering) - generally speaking. Because the distance is something that depends on how I handle my repo commit history and how often I commit to e.g. main. Though, I do not like the idea of having a mismatch of a dev/post revision and the corresponding commit distance either as this would mislead to the wrong commit this release came from... theoretically, as we have tags, right? ;)

You know what I mean? So from this point of view, the VCS parsing works as expected and well (at least tested with Git). The Version.parse() does not in my opinion - to much heuristic.

This is just a notification about what I noticed during my POC. It's up to you, if you pick this up for a change or if you say "works as designed". :D

Cheers, Martin

[mtkennerly]

Thanks for the detailed info!

Version.parse does use _match_version_pattern internally, but also then tries to extract some information from the tagged_metadata and tries to interpret how Dunamai's default format uses dev/post. This is just because Version.parse doesn't have access to the VCS history, so it tries to make up for the missing information, but that also makes it inexact.

• -dev.6 being parsed as distance=6 is expected, since Version.parse is geared toward parsing Dunamai's own default format. However, it may be worth adding an argument to opt out of this. Also, I think it only works right now for exactly dev, but not a compound like alpha.1.dev.2, which it probably should support.
• +local being parsed as commit="local" is because the function currently expects an order like +{distance}.{commit}.{other}, so it's over-eagerly trying to parse anything it can as the commit first. Parsing the distance and commit out of the tagged metadata is probably too prone to false positives like you're seeing, so it may be better to just remove that handling or make it opt-in.

I'll play around with some potential alternatives/additions here.

If you'll always have access to the VCS history in your script, though, Version.from_any_vcs and similar are certainly more robust options.

[martinpakosch]

Hi,

Version.parse does use _match_version_pattern internally

Oh, must have missed this detail in the code. 😇

I think I get the point, what you've designed the parse() method for now. And it's absolute valid to have a method, that focuses on parsing the default serialization format to at least try to get back the Version object the string was created from. I only came up with this string parsing because I love flexibility... But I also know how hard it is to parse "the unknown" with regex. 😁 The question is more likely, what level of detail do you expect to be parsed correctly, right? Is it common to expect a string for parsing with commit hashes etc.? I have no experience here and it is up to each user, isn't it?

However, as you wrote about your thoughts regarding possible opt-ins, why not redesign your pattern approach to cover this? I suddenly think of your default version source pattern:

VERSION_SOURCE_PATTERN = r"""
    (?x)                                                        (?# ignore whitespace)
    ^v((?P<epoch>\d+)!)?(?P<base>\d+(\.\d+)*)                   (?# v1.2.3 or v1!2000.1.2)
    ([-._]?((?P<stage>[a-zA-Z]+)[-._]?(?P<revision>\d+)?))?     (?# b0)
    (\+(?P<tagged_metadata>.+))?$                               (?# +linux)
""".strip()

I guess with this default pattern you are able to support the least common denominator of the PEP440, SemVer and PVP formats and that's good for compatibility! But you seem just to focus on one stage, ignoring the compound variants you mention, or detailed metadata formats. I am not saying you should redesign the default pattern, but maybe extend the capabilities for users and the parse method using such pattern? Why not introduce additional capture groups like compound (e.g. catching additional dev or post), distance (expecting to match a number), tagged_commit, tagged_dirty (handled internally as boolean)? Then tagged_metadata could be used as before for local version specs.

Then I could provide a pattern like this if I need to parse such detailed versions and both, VCS and string parsing could benefit from it - explicit is better than implicit 😉:

VERSION_SOURCE_PATTERN = r"""
    (?x)                                                        (?# ignore whitespace)
    ^v((?P<epoch>\d+)!)?(?P<base>\d+(\.\d+)*)        (?# v1.2.3 or v1!2000.1.2)
    ([-._]?((?P<stage>[a-zA-Z]+)[-._]?(?P<revision>\d+)?))?       (?# b0)
    ([-._]?((?P<compound>dev|post)[-._]?(?P<distance>\d+)?))?      (?# dev7)
    (\+(?P<tagged_metadata>[a-zA-Z0-9]+)([.]?(?P<tagged_commit>[a-zA-Z0-9]+))?([.]?(?P<tagged_dirty>\.?dirty))?)?$      (?# +linux.gabcdef1.dirty)
""".strip()

This is just a quick idea, not tested or thought about in depth. But maybe another approach to consider, so the user can provide a more explicit pattern if his versioning scheme needs that.