Skip to content

Update CMORizer CLARA-AVHRR (A3)#4463

Open
axel-lauer wants to merge 16 commits into
mainfrom
update_clara-avhrr
Open

Update CMORizer CLARA-AVHRR (A3)#4463
axel-lauer wants to merge 16 commits into
mainfrom
update_clara-avhrr

Conversation

@axel-lauer

@axel-lauer axel-lauer commented May 19, 2026

Copy link
Copy Markdown
Contributor

Description

This PR replaces the old NCL CMORizer for CLARA-AVHRR data with a new Python version. The processed CLARA-AVHRR data (version A3) is obtained from the Copernicus Climate Data Store (CDS) and replaces the old version (A2).

Please note: I will not fix the remaining Codacy issues.

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

New or updated data reformatting script

@axel-lauer axel-lauer changed the title Update CMORizer CLARA-AVHRR Update CMORizer CLARA-AVHRR (A3) May 19, 2026
@axel-lauer axel-lauer requested a review from hb326 May 19, 2026 11:45
@axel-lauer axel-lauer marked this pull request as ready for review May 19, 2026 11:45
@axel-lauer axel-lauer requested a review from a team as a code owner May 19, 2026 11:45
Comment thread esmvaltool/cmorizers/data/downloaders/datasets/clara_avhrr.py

@hb326 hb326 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works like a charm. Thanks, @axel-lauer!
I tried the downloader and formatter both.

One little thing: I commented in one of the files that it might be good to add a comment about why the data download end-date is set to 2020. This will most likely be helpful in a few month when we try to update the data until 2025...

@hb326 hb326 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the notes, @axel-lauer!
Looks good from my side now (I did not look at any Codacy issues...).

@schlunma schlunma left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 comments regarding Codacy problems, others can be ignored.

Comment on lines +177 to +178
with gzip.open(file_path, "rb") as f_in:
with Path(outdir / file_path.stem).open("rb") as f_out:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
with gzip.open(file_path, "rb") as f_in:
with Path(outdir / file_path.stem).open("rb") as f_out:
with (
gzip.open(file_path, "rb") as f_in,
Path(outdir / file_path.stem).open("rb") as f_out,
):

Comment on lines +180 to +181
except Exception as ex:
logger.info("%s: no data downloaded for %s", type(ex), var_name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will more or less silently skip missing variables. Is this desirable? Can the dataset be used if files are missing? If not, I would remove this try/except block here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Manu - perhaps do a check on the required variables, and if at least one is missing, then raise an exception than mask via try/except

@valeriupredoi

Copy link
Copy Markdown
Contributor

@schlunma are you doing a technical review? If not, I can do Axel a solid 🍻

@hb326 hb326 added this to the v2.15.0 milestone May 26, 2026
@schlunma

Copy link
Copy Markdown
Contributor

@schlunma are you doing a technical review? If not, I can do Axel a solid 🍻

No, please go for it 🍻

@valeriupredoi

Copy link
Copy Markdown
Contributor

@schlunma are you doing a technical review? If not, I can do Axel a solid 🍻

No, please go for it 🍻

cheers, buds, on it now then 🍻

@valeriupredoi valeriupredoi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good and many thanks for yet another retired NCL script @axel-lauer 🍻 Two very small observations from me, pun intended

Comment on lines +180 to +181
except Exception as ex:
logger.info("%s: no data downloaded for %s", type(ex), var_name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Manu - perhaps do a check on the required variables, and if at least one is missing, then raise an exception than mask via try/except

Comment thread esmvaltool/cmorizers/data/formatters/datasets/clara_avhrr.py Outdated
Co-authored-by: Valeriu Predoi <valeriu.predoi@gmail.com>
@rbeucher

Copy link
Copy Markdown
Contributor

Hi all — we’re getting ready for the next ESMValTool release, version 2.15, and are planning to freeze the code on 26 June 2026.

Could you please let us know whether you think you’ll be able to finalise the work in this PR by then? If not, no problem — it would just help us plan what can realistically make it into the release.

@flicj191 tagging Felicity for visibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update/add cmorizer for CLARA-A3

5 participants