Experiment YAML Structure

Designing the experiment.yaml File

To fully realize the vision of a dynamic, Python-driven run system (as detailed in Refactoring gcm_run.j to Python), all configuration data currently scattered across gcm_setup arguments, templated @ variables in gcm_run.j, AGCM.rc, and CAP.rc needs to be consolidated into a single source of truth: experiment.yaml.

The overarching script (submit_run.py) will read this YAML file to dynamically calculate nodes, construct the MPI execution command, configure the model, and submit the job.

Below is an analysis of what gcm_run.j expects, categorized into logical sections for an experiment.yaml.

---

Proposed experiment.yaml Schema

1. Job & Resource Management

This section replaces the hardcoded #SBATCH / #PBS directives and the rigid $NX / $NY math.

job:
  account: s1234
  queue: compute
  walltime: "12:00:00"
  qos: "default"
  scheduler: "SLURM"  # SLURM or PBS
  
resources:
  # The Python script will automatically compute total cores and nodes 
  # based on NX, NY, and the machine's hardware profile.
  layout:
    NX: 4
    NY: 24
  ioserver:
    enabled: true
    nodes: 1
    backend_pes: 12
  # Optional override for standard calculations
  cpus_per_node: 48

2. Time Management

This replaces the CAP.rc parsing (END_DATE, JOB_SGMT, NUM_SGMT) and manual cap_restart tracking. It uses standard ISO 8601 strings instead of the old "YYYYMMDD hhmmss" packtime format.

time:
  start_time: "2000-04-14T21:00:00"
  end_time: "2000-05-15T21:00:00"
  segment_length: "P1M"     # ISO 8601 duration (1 month)
  heartbeat_dt: 450         # Replaces CAP.rc HEARTBEAT_DT and propagates to MOM/CICE

3. Model Configuration & Resolution

This replaces many of the settings read from AGCM.rc and templated via gcm_setup.

model:
  atmosphere:
    IM: 144
    JM: 91
    LM: 72
  ocean:
    type: "MOM6"      # e.g., MOM5, MOM6, DATAOCEAN, MIT
    IM: 360
    JM: 200
    LM: 75
    coupled: true
  waves:
    enabled: false
    model: "WW3"
  chemistry:
    emissions: "OPS_EMISSIONS" # or AMIP_EMISSIONS
    use_extdata2g: true

4. Replay Configuration

Consolidates the REPLAY_MODE flags from AGCM.rc.

replay:
  mode: "Regular" # Exact, Regular, or None
  ana_expid: "my_analysis_run"
  ana_location: "/path/to/analysis"
  replay_file: "my_analysis_run.inst6_3d_ana_Nv.%y4%m2%d2_%h2z.nc4"
  replay_file09: "my_analysis_run.inst6_3d_ana_Nv.%y4%m2%d2_%h2z.nc4"

5. Infrastructure & Environment

Replaces the templated variables (@GEOSDIR, @SITE) inserted by gcm_setup.

infrastructure:
  site: "NCCS"
  geos_install_dir: "/path/to/GEOSgcm/install"

6. Restarts & Directories

Defines where the run lives and where to source its initial state.

experiment:
  id: "c180_MOM6"
  directory: "/path/to/experiment/dir"
  
restarts:
  initial_dir: "/path/to/initial/restarts"
  read_by_face: false
  write_by_face: false
  num_readers: 6    # Enforced multiple of 6 for face writing/reading
  num_writers: 6

---

Transitioning gcm_setup

If we adopt this experiment.yaml approach, the role of gcm_setup fundamentally changes:

1. Old Role: gcm_setup queries the user for options, performs node math, generates .rc files, and creates templated C-shell scripts (gcm_run.j).
2. New Role: gcm_setup queries the user for options, generates .rc files, copies initial data, and generates the experiment.yaml file.

Then, the user simply runs:

python submit_run.py experiment.yaml

Which reads the YAML, parses the machine architecture, creates a temporary scheduler script, and submits the job.

---

Appendix: Singularity Support

Support for Singularity runs can be added back into the configuration as needed:

infrastructure:
  container:
    enabled: true
    sandbox: "/path/to/ubuntu_sandbox"
    bind_paths:
      - "/discover/nobackup"
      - "/gpfsm"