Skip to content

Generalise table preprocessing #2791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kosack merged 36 commits into from
Oct 10, 2025
Merged

Generalise table preprocessing #2791

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
6605589
test
Jun 26, 2025
1a68509
first working version, ruff
Jun 26, 2025
0791398
adding/fixing tests, ruff
Jun 28, 2025
473c0a1
ruff
Jun 30, 2025
298dc02
updating docs
Jun 30, 2025
9aaba99
fix the docs, ruff
Jun 30, 2025
8042312
addressing sonarqube, ruff
Jul 1, 2025
5d7b24b
addressing review, ruff
Jul 1, 2025
c07cb52
modifying renaming functionality
Jul 4, 2025
44a66a9
Improving changelog
Jul 7, 2025
2ca592e
Improving changelog
Jul 7, 2025
9042d95
Removing disable_column_renaming
Jul 8, 2025
4661647
addressing review
Jul 9, 2025
dd4902a
simpifying the table construction
Jul 15, 2025
42f58d2
addressing review
Jul 15, 2025
36f3eeb
relocating preprocessing to io
Jul 16, 2025
d035775
fixing docs and circular imports
Jul 16, 2025
40d9f50
fixing tests
Jul 16, 2025
15396e3
fixing import that strangely wasn't caught by the tests
Jul 16, 2025
ccd427b
trying to fix the minimal dependencies issue
Jul 17, 2025
8c924fa
Revert "trying to fix the minimal dependencies issue"
mexanick Jul 17, 2025
fb5f8ea
moving all tests that implicitly use pyirf to irf
Jul 17, 2025
7f3a16c
removing duplicated lines
Jul 17, 2025
5d07e5a
Make pyirf dependency optional, fix dl2_tables_preprocessing exposure
mexanick Jul 17, 2025
802319d
resolving conflicts
Jul 17, 2025
9af99dd
Fix min tests for preprocessing
mexanick Jul 18, 2025
4a234a6
Remove "u.dimensionless_unscaled" units from gh_score and weights.
mexanick Jul 22, 2025
7d4960d
fix test fixture (new order of columns)
mexanick Jul 22, 2025
e9a17ca
improving the help of a config parameter
Jul 28, 2025
1fa7012
supported pointing frames
Aug 4, 2025
6bfc371
Variable names, warning, imports
Sep 21, 2025
fed7c7f
Variable names, warning, imports
Sep 21, 2025
4061c9d
Put annotation when needed by the new importing scheme
Sep 21, 2025
11d98e2
Reverting to correct exception
Sep 22, 2025
e94ac42
Merge branch 'main' into generalise_table_preprocessing
maxnoe Oct 9, 2025
d685e25
Merge remote-tracking branch 'origin/main' into generalise_table_prep…
maxnoe Oct 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/changes/2791.optimization.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Currently, for telescope cross calibration we have implemented our own table filtering. However similar functionality has been implemented in IRF tools. Our idea is to reuse the existing functionality from IRF tools for cross calibration. The added benefits is increased maintainability. Moreover, cross calibration memory efficiency will be improved by reading data by chunks. Finally, this can potentially benefit other usecases that load DL2 tables.
Comment thread
maxnoe marked this conversation as resolved.
Outdated

In order to do that, we will propose some modifications in irf/preprocessing. However we should ensure that the default behaviour stays the same.
3 changes: 3 additions & 0 deletions docs/changes/2791.refactoring.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Currently, for telescope cross calibration we have implemented our own table filtering. However similar functionality has been implemented in IRF tools. Our idea is to reuse the existing functionality from IRF tools for cross calibration. The added benefits is increased maintainability. Moreover, cross calibration memory efficiency will be improved by reading data by chunks. Finally, this can potentially benefit other usecases that load DL2 tables.

In order to do that, we will propose some modifications in irf/preprocessing. However we should ensure that the default behaviour stays the same.
Loading