Avoid multiple progress bars

[manuelgloeckler]

🚀 Multiple progress bars can lead to confusing printouts.

In e.g. #1473 progress bars confusingly reported different number of samples, which indicated that the wrong number of samples were generated which however was not the case (see analysis on last comment).

In this specific case, because a IID score method was used, three possible origins exist:

• Initial hyperparameter estimation run, calls sample here but show_progressbar=False so this should be suppressed.
• accept_reject_sample has rejection sampling progressbar
• Diffuser.run has diffusion per-step progressbar.

One can think about handling "hierarchy" of progressbars better (or removing/surpressing lower level bars). I have no concrete idea on resolving this for now.

[Jocho-Smith]

Hi @manuelgloeckler,

I looked into this a bit and here is what I can say so far:

• Generally speaking I found that from time to time people get confused about defaults in progress bars or have custom needs in their use case.
• In libraries like PyTorch Lightning, maintainers explicitly opt for a strong default + extensibility, instead of trying to satisfy every preference. The default progress bar (tqdm-based) is opinionated, but users can override it entirely via callbacks if needed.
• More generally, there is a guide by tqdm in progress bar design.
• I noticed in sbi/inference/posteriors/mcmc_posterior.py line 888 show_progress_bars is passed to pymc.sample() in the backend, effectively "delegating progress bar policy" to the backend of PyMC, rather than defining a strict convention at the sbi level. This could potentially cause inconsistencies.

Given all this, it seems the real question is: How should sbi define and enforce its own progress bar convention for strong defaults?

What are your thoughts about this? What do you think is the underlying main question here?

I looked into the other sbi tutorial notebooks (with and without IID score method), and there I didn't see multiple progress bars occurring. I would expect this to happen more often though.

[janfb]

Hey @Jocho-Smith, thanks for looking into this in detail!

I had a closer look as well and here's what I found:

The nested bar problem (from #1473)

I think what is happening is due to nested pbars: When VectorFieldPosterior.sample() runs with reject_outside_prior=True (the default), the call chain is:

accept_reject_sample (outer bar: "Drawing N samples")
  └─ [retry loop, may run multiple times]
       └─ _sample_via_diffusion
            └─ Diffuser.run (inner bar: "Generating N samples in M steps")

The single show_progress_bars flag is forwarded to both the outer rejection bar and every inner Diffuser.run() call. Each retry iteration spawns fresh inner bars that stack in the terminal. There's currently no way to show just the outer bar.

I suggest we fix this by changing vector_field_posterior.py: when calling accept_reject_sample, set "show_progress_bars": False inside proposal_sampling_kwargs regardless of the outer flag. The outer rejection bar already tracks overall progress toward num_samples; the inner diffusion bars are redundant noise. Would you agree? Please check locally with different combos (reject_outside_prior=True / False, show_progress_bars=True / False).

Broader inconsistencies (for context, not necessarily for this PR)

I also noticed a few other things across the codebase:

1. Naming split: show_progress_bars (plural) is used in all posteriors/trainers/samplers, but show_progress_bar (singular) is used in VIPosterior.train(), simulate_for_sbi(), and all diagnostics (sbc, tarp, lc2st). This forces manual renaming. Standardizing on plural (the majority convention) would be a clean separate PR.

2. utils/metrics.py:640 — Distance.__call__() uses tqdm(range(num_batches)) with no disable= parameter, so this bar always shows regardless of user preference. This function is not used a lot I guess, so it hasn't surfaced yet.

3. No propagation from trainer to posterior: The trainer stores self._show_progress_bars but build_posterior() never passes it to the posterior object. Users must re-specify it on every .sample() call. We should forward it from the trainer; that's actually something that has been annoying me for a while :)

I think these additional points are orthogonal to the nested bar fix, we don't want to make this a major refactoring but rather keep the PRs focused. Thus, I'd suggest scoping this issue to the nested bar problem and tracking the rest separately if anyone wants to pick them up.

What do you think? Would you like to take on the nested bar fix?

[janfb]

I was looking into this more carefully when thinking about the problems mentioned in #1403, and I noticed that my suggestion above was not ideal: simply hardcoding show_progress_bars: False in proposal_sampling_kwargs doesn't work as intended: the reject_outside_prior=False bypass path reuses the same kwargs dict, so the diffusion bar would get suppressed there too, but in that case there's no outer bar and the user would see nothing at all.

So the core challenge is that "inner bars" (e.g. the Diffuser.run) should be suppressed when called from inside an outer sampler (e.g. accept_reject_sample), but should show normally when called standalone. To avoid passed something like a is_nested kwarg around, one approach would be a thread-local flag that the outer bar sets, so inner bars can check whether they're nested. But there might be other ways to solve this too.

Generally, I think we need a separate file to handle this, e.g., sbi/utils/pbar.py would be a good. Note that accept_reject_sample also wraps proposals in DirectPosterior, so the solution should work generally, not just for the VectorField case.

This issue is still orthogonal to #1403 which would only handle how we display the pbars.

[Jocho-Smith]

Hi @janfb, thanks for your detailed reply!

so the diffusion bar would get suppressed there too, but in that case there's no outer bar and the user would see nothing at all.

Exactly. That was the point that made me surrender in #1473.

I'll experiment with both the is_nested kwarg and the thread-local flag and will write some tests with the suggested combos "(reject_outside_prior=True / False, show_progress_bars=True / False)"

I'm not quite sure yet how to assert on progress bars in pytest, but I'll think about it.

Also can you pls specify what exactly the content of sbi/utils/pbar.py should be? Are you thinking about moving the status bar logic to this place, or write functions that specifically only print status bars in a standalone manner?