Fix savings_yesterday_predbat fluctuating throughout the day on dynamic tariffs

[Copilot]

savings_yesterday_predbat was non-deterministic across its hourly recalculations — the value would swing wildly (spikes of 150+ p, then collapse to ~0 or negative) within a single completed day.

Root cause

calculate_yesterday() builds past_rates via history_to_future_rates(..., end_record + self.minutes_now), which grows as the day progresses to include today's rates beyond minute 1440. The baseline charge-window threshold was then:

rate_low = min(past_rates.values())  # included today's rates

On Agile (or any tariff with a cheap/negative slot today), once that slot entered past_rates, rate_low dropped below yesterday's minimum — e.g. from 5p → 0p. rate_scan_window then found no charge windows in yesterday's data (all of yesterday's rates > 0p), so the baseline ran with no scheduled charging, producing a higher metric_baseline and thus inflated apparent savings. The next recalculation (before that cheap slot was included) would find charge windows again and give a lower value — hence the oscillation.

The same issue affected the flat-tariff variability guard (min != max): today's variable rates could trigger charge-window detection even when yesterday had a flat tariff.

Fix — output.py

Extracted a new compute_rate_low_for_yesterday(past_rates, end_record) helper method on the Output class. It restricts both the rate_low threshold and the variability check to yesterday's rate range only (k < end_record), so they are unaffected by today's rates being progressively appended:

def compute_rate_low_for_yesterday(self, past_rates, end_record):
    yesterday_values = [v for k, v in past_rates.items() if k < end_record]
    if yesterday_values:
        return min(yesterday_values)
    if past_rates:
        return min(past_rates.values())
    return 0.0

calculate_yesterday() now calls self.compute_rate_low_for_yesterday(past_rates, end_record) instead of the inline min(past_rates.values()). The variability guard for past_rates_no_io is similarly scoped to k < end_record.

The past_rates dict still covers end_record + minutes_now entries so that overnight charge windows spanning midnight are detected correctly — only the threshold derivation is scoped to yesterday.

Test — tests/test_savings_stability.py

New test calls the production methods directly to exercise the fix:

• history_to_future_rates() builds past_rates exactly as calculate_yesterday() does for three time-of-day scenarios (midnight, mid-morning, noon)
• compute_rate_low_for_yesterday() is asserted to return the stable 5p yesterday minimum at all three points, regardless of today's 0p Agile slot entering the dict
• rate_scan_window() confirms the correct yesterday charge window is found with the fixed threshold and nothing is found with the old broken 0p threshold
• All shared my_predbat state (combine_charge_slots, minutes_now, forecast_minutes) is saved and restored via try/finally with deterministic values set before scanning

[Copilot]

Pull request overview

Fixes non-deterministic savings_yesterday_predbat recalculation on dynamic tariffs by ensuring the baseline “lowest rate” threshold and flat-tariff variability detection are derived only from yesterday’s rate range, preventing today’s progressively-added rates from skewing the baseline.

Changes:

• Scope rate_low derivation in calculate_yesterday() to k < end_record (yesterday-only), with safe fallbacks.
• Scope the “variable tariff” check (min != max) to yesterday-only rates to avoid false positives from today’s rates.
• Add a new unit test and register it in the unit test runner.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
apps/predbat/output.py	Restricts baseline threshold + variability checks to yesterday’s rate range to stabilize savings calculations.
apps/predbat/tests/test_savings_stability.py	Adds a regression-style test scenario for the threshold stability / window finding behavior.
apps/predbat/unit_test.py	Registers the new savings_stability test in the unit test harness.

[springfall2008]

@copilot apply changes based on this feedback

[Copilot]

Done in da44d39. Extracted the rate_low selection into a compute_rate_low_for_yesterday() method on the Output class, which calculate_yesterday() now calls. The test was rewritten to call the production methods directly: history_to_future_rates() to build past_rates exactly as production does, compute_rate_low_for_yesterday() to assert stability across three minutes_now values, and rate_scan_window() to confirm correct window detection. All shared state (combine_charge_slots, minutes_now, forecast_minutes) is now saved and restored in a try/finally block with deterministic values set before rate_scan_window is called.

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/repos/springfall2008/batpred/contents/apps/predbat
  • Triggering command: /home/REDACTED/work/batpred/batpred/coverage/venv/bin/python3 python3 ../apps/predbat/unit_test.py --test savings_stability (http block)
  • Triggering command: /home/REDACTED/work/batpred/batpred/coverage/venv/bin/python3 python3 ../apps/predbat/unit_test.py --test savings_stability --test find_charge_window --test basic_rates --test compare (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)