Support .gitattributes export-ignore in distribution archives by jberdine · Pull Request #515 · tarides/dune-release

jberdine · 2026-01-24T22:50:38Z

Parse .gitattributes files and exclude paths marked with the export-ignore
attribute from distribution tarballs. This is the mechanism used by
git-archive to exclude files, allowing projects to exclude dev-only files
like dune-workspace from releases.

Supported patterns:

Exact matches: dune-workspace
Directory patterns: .github/**
Glob patterns: .log, test_, file?.txt
Double star: /build, src//test.ml

Paths are normalized before matching (handles ./ and ../).

Not supported:

Escaped patterns (\! for literal !)
Quoted patterns ("a b" for patterns with spaces)
Case insensitivity (core.ignorecase)
Negation patterns (!pattern)
Subdirectory .gitattributes files

Testing:

Unit tests comparing behavior against git check-attr
Tests cover pattern matching and parsing edge cases
.gitattributes content is generated from test cases to ensure sync
Archive integration test verifies end-to-end exclusion without requiring git

Signed-off-by: Josh Berdine josh@berdine.net

Parse .gitattributes files and exclude paths marked with the export-ignore attribute from distribution tarballs. This is the mechanism used by git-archive to exclude files, allowing projects to exclude dev-only files like dune-workspace from releases. Supported patterns: - Exact matches: dune-workspace - Directory patterns: .github/** - Glob patterns: *.log, test_*, file?.txt - Double star: **/build, src/**/test.ml Paths are normalized before matching (handles ./ and ../). Not supported: - Escaped patterns (\\! for literal \!) - Quoted patterns ("a b" for patterns with spaces) - Case insensitivity (core.ignorecase) - Negation patterns (\!pattern) - Subdirectory .gitattributes files Testing: - Unit tests comparing behavior against git check-attr - Tests cover pattern matching and parsing edge cases - .gitattributes content is generated from test cases to ensure sync - Archive integration test verifies end-to-end exclusion without requiring git Signed-off-by: Josh Berdine <josh@berdine.net>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Leonidas-from-XIV

Very nice PR, the extensive test coverage is definitely a highlight and good to see as the glob to regex code is somewhat hairy.

I have some suggestions for improvements here.

Leonidas-from-XIV · 2026-03-26T10:39:27Z

-let path_set_of_dir dir ~exclude_paths =
-  let not_excluded p = Ok (not (Fpath.Set.mem (Fpath.base p) exclude_paths)) in
+let path_set_of_dir dir ~exclude_paths ~export_ignore =
+  let not_excluded p =


To avoid the negation, maybe rename it to included. It's kind of hard to wrap one's head around the double negated name and then the negation in the code.

Renamed. My thinking was that conceptually what is being done is "excluding" and so kept that base term, but fair point about double-negation.

Leonidas-from-XIV · 2026-03-26T10:41:25Z

+        (* Skip directories - they will be created implicitly when their
+           contents are added. This ensures that directories whose contents
+           are excluded via export-ignore patterns don't appear as empty
+           directories in the archive. *)


But isn't this what one would expect to do? If I exclude a directory, I would expect the directory to show up, but not its contents.

Git does this but its more to do with the fact that Git doesn't track directories at all, so an empty directory is unrepresentable. But dune-release doesn't have this issue.

My thinking is that the context here is reading a .gitattributes file, and so the expectation is to interpret it the way that git archive does (as much as possible). As a user, if some workflows used git archive and some dune release, and I had to adjust the .gitattributes file because they interpreted it differently, I would assume that dune was wrong since the name .gitattributes indicates that git "owns" the format.

Leonidas-from-XIV · 2026-03-26T10:43:45Z

-    is preserved.
+(** [tar dir ~exclude_paths ~export_ignore ~root ~mtime] is a (us)tar archive
+    that contains the file hierarchy [dir] except:
+    - relative hierarchies present in [exclude_paths] (basename matching)


I wonder if it wouldn't make more sense to just translate exclude_paths to a Gitattributes.pattern and use the more generic mechanism. That way there's only one way to ignore paths.

That could make sense. My worry is that exclude_paths excludes based on basename, so at any depth, while Gitattributes.patterns are path-anchored. Translating exclude_paths could have a semantic change if not done exactly right. So I think such a change ought to be in a separate PR, and I would want to have a much better idea for a good way to test it than I have at the moment.

Leonidas-from-XIV · 2026-03-26T10:45:11Z

+    (any chars except /), [?] (single char except /), and [**] (any path
+    segments, but only when adjacent to /). *)
+let glob_to_re pattern =
+  let buf = Buffer.create (String.length pattern * 2) in


Why is the initial size pattern * 2?

Dropped the * 2. It is a guess, based on the observation that each char of the pattern might expand into 2 regex chars, but the buffer grows fine so it is simpler to just skip the * 2.

Leonidas-from-XIV · 2026-03-26T10:46:22Z

+let glob_to_re pattern =
+  let buf = Buffer.create (String.length pattern * 2) in
+  Buffer.add_char buf '^';
+  let len = String.length pattern in


Nitpick but the length of the pattern is determined twice, so could be pulled up in the function an reused.

Leonidas-from-XIV · 2026-03-26T10:54:07Z

+  (* Strip UTF-8 BOM if present at start of file *)
+  let content =
+    if String.is_prefix ~affix:"\xef\xbb\xbf" content then
+      String.Sub.to_string (String.sub ~start:3 content)


And use String.length utf8_bom here.

Leonidas-from-XIV · 2026-03-26T10:56:41Z

+
+(** {1 Patterns} *)
+
+type pattern


Suggested change

type pattern

type t

Leonidas-from-XIV · 2026-03-26T10:57:53Z

+    let files =
+      [
+        ("CHANGES.md", "changes");
+        ("foo.opam", "opam");


Suggested change

("foo.opam", "opam");

("foo.opam", {|opam-version: "2.0"|});

Leonidas-from-XIV · 2026-03-26T10:59:24Z

+    in
+    List.fold_left
+      (fun acc file -> acc >>= fun () -> create_file file)
+      (Ok ()) files


I think List.iter followed by some kind of Result.List.all_unit (or the like) would be a bit easier to understand but it's of minor importance.

Perhaps. I like the way the current formulation uses >>=/Result.bind to chain the results just like the rest of the code.

Leonidas-from-XIV · 2026-03-26T11:03:04Z

+- Double star in path: `**/build`, `src/**/test.ml`
+- Path normalization: handles `./` and `../` in paths
+
+**Not supported:**


I think it would be good to state what "not supported" means in practice. Will it error? Will it ignore the pattern?

Good point. I have pushed a commit that detects the unsupported pattern syntax, and updated the README to distinguish such cases (that are skipped with a warning) and other limitations (that are silent).

Avoid the double negation in the predicate name used by OS.Path.fold's `Sat filter. Signed-off-by: Josh Berdine <josh@berdine.net>

The Buffer grows on demand; the 2x factor was a guess with no measurement behind it. Signed-off-by: Josh Berdine <josh@berdine.net>

Compute the pattern length once and reuse for both the Buffer size and the loop bound. Signed-off-by: Josh Berdine <josh@berdine.net>

Bind `utf8_bom` so the byte sequence appears once and the skip length is derived from it rather than a magic 3. Signed-off-by: Josh Berdine <josh@berdine.net>

Follow the standard OCaml convention of naming the primary type of a module `t`. Signed-off-by: Josh Berdine <josh@berdine.net>

Signed-off-by: Josh Berdine <josh@berdine.net>

Detect negation (`\!pattern`), escape (`\\`), quoting (`"`), and character class (`[...]`) syntax in parse_pattern and skip the pattern with a Logs.warn rather than compiling to a regex that silently matches no files. Change parse_pattern to return a t option to reflect this. Split the README "not supported" list: syntactic cases are warned-and- skipped uniformly, while case-insensitivity and subdirectory .gitattributes are documented as separate limitations. Signed-off-by: Josh Berdine <josh@berdine.net>

github-actions Bot reviewed Jan 24, 2026

View reviewed changes

Comment thread CHANGES.md Outdated

Update CHANGES.md

892f343

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Leonidas-from-XIV reviewed Mar 26, 2026

View reviewed changes

jberdine added 7 commits April 22, 2026 22:33

Rename not_excluded predicate to included

7864bf5

Avoid the double negation in the predicate name used by OS.Path.fold's `Sat filter. Signed-off-by: Josh Berdine <josh@berdine.net>

Drop speculative 2x buffer size in glob_to_re

1342036

The Buffer grows on demand; the 2x factor was a guess with no measurement behind it. Signed-off-by: Josh Berdine <josh@berdine.net>

Hoist String.length in glob_to_re

0988a73

Compute the pattern length once and reuse for both the Buffer size and the loop bound. Signed-off-by: Josh Berdine <josh@berdine.net>

Name the UTF-8 BOM constant

f5f00f6

Bind `utf8_bom` so the byte sequence appears once and the skip length is derived from it rather than a magic 3. Signed-off-by: Josh Berdine <josh@berdine.net>

Rename Gitattributes.pattern to Gitattributes.t

1a002b0

Follow the standard OCaml convention of naming the primary type of a module `t`. Signed-off-by: Josh Berdine <josh@berdine.net>

Use plausible opam content in archive test fixture

5dad686

Signed-off-by: Josh Berdine <josh@berdine.net>

Conversation

jberdine commented Jan 24, 2026

Uh oh!

Uh oh!

Leonidas-from-XIV left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants