Skip to content

Extending HDF5 Support #5391

@mcmehrtens

Description

@mcmehrtens

While trying to integrate AMReX's HDF5 capabilities into my research group's in-house code, Alamo, we ran into some rough edges we thought might be better solved upstream.

Configuring AMReX for HDF5

  1. AMReX doesn't have an --enable-hdf5 flag like it does for bittree and some of the other Extern sources. This flag would simply write USE_HDF5 = TRUE or USE_HDF5 = FALSE to the GNUmakefile.
  2. AMReX doesn't have an entry in GNUmakefile.in for including the Extern/HDF5 sources (see the existing GNUmakefile.in for an example). The absence of this Pdirs line caused our Alamo build to fail (I'd have to do more digging to figure out exactly why; we have a tight coupling with AMReX).

Configuring HDF5

  1. AMReX doesn't expose the ability to alter either the HDF5 alignment size or the HDF5 metadata block size. The defaults are reasonable for an HPC environment with a distributed file-system, but caused abysmal storage overhead on our macOS machines. The ability to configure these using environment variables (or otherwise) would be very helpful since we run Alamo + AMReX on a wide variety of platforms (MacBooks, workstations, HPC clusters, etc.).

Proposed solution

I have a draft PR I'll open shortly after opening this that implements fixes for the above items. We've been using this branch to test HDF5 support for Alamo/AMReX.

I appreciate your support and your time!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions