Update diag_table #157

minghangli-uni · 2025-01-17T01:04:37Z

See COSIMA/access-om3#257

minghangli-uni · 2025-01-17T01:07:35Z

Hi @aekiss , can you please review it when you are available?

anton-seaice · 2025-01-17T03:32:29Z

!test repro

github-actions · 2025-01-17T03:36:38Z

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

dev-025deg_jra55do_ryf_iss257 (checksums created using commit 4a663e3), against
dev-025deg_jra55do_ryf (checksums in commit f37396e)

Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/4a663e368c8b2a720fb3197d21ff0743b2cabf50, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/35754730407.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/12822161716/artifacts/2444836580.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/f37396e0047d59ddafefb435c08073a0256cfcef/testing/checksum

anton-seaice · 2025-01-17T03:43:10Z

Hi @minghangli-uni - test run failed with:

FATAL from PE     0: diag_axis_mod::diag_axis_init: num_axis_sets (26) exceeds max_num_axis_sets (25).  Increase max_num_axis_sets via diag_manager_nml.

Looks like it happened on most PEs

minghangli-uni · 2025-01-17T04:03:36Z

!test repro

github-actions · 2025-01-17T04:14:38Z

✅ The Bitwise Reproducibility Check Succeeded ✅

When comparing:

dev-025deg_jra55do_ryf_iss257 (checksums created using commit ccc44fb), against
dev-025deg_jra55do_ryf (checksums in commit f37396e)

Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/ccc44fb4ac8bce98dcc997ef86ed8e2d302ec227, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/35755655798.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/12822441249/artifacts/2444929021.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/f37396e0047d59ddafefb435c08073a0256cfcef/testing/checksum

anton-seaice · 2025-01-23T03:24:30Z

Re-running the CI after #163 means the tests now pass

aekiss · 2025-01-28T04:14:37Z

Thanks @minghangli-uni, I think the diag_table_source.yaml that was used should also be part of this PR for provenance and ease of updates.

minghangli-uni · 2025-01-28T04:27:39Z

Thanks @aekiss . The diag_table_source.yaml configuration file has been attached.

anton-seaice · 2025-01-31T05:06:05Z

We could set 'prsn', 'prlq', 'vprec', 'evs', 'fsitherm','friver', 'ficeberg' to save daily output whilst we are confirming the freshwater conservation. Is it ok to do in this PR ?

based on this code snippet

  ! adjust the NET fresh-water flux to zero, if flagged
  if (CS%adjust_net_fresh_water_to_zero) then
    sign_for_net_FW_bug = 1.
    if (CS%use_net_FW_adjustment_sign_bug) sign_for_net_FW_bug = -1.
    do j=js,je ; do i=is,ie
      net_FW(i,j) = US%RZ_T_to_kg_m2s * &
                    (((fluxes%lprec(i,j)   + fluxes%fprec(i,j) + fluxes%seaice_melt(i,j)) + &
                      (fluxes%lrunoff(i,j) + fluxes%frunoff(i,j))) + &
                      (fluxes%evap(i,j)    + fluxes%vprec(i,j)) ) * US%L_to_m**2*G%areaT(i,j)
      net_FW2(i,j) = net_FW(i,j) / (US%L_to_m**2*G%areaT(i,j))
    enddo ; enddo

minghangli-uni · 2025-01-31T05:21:26Z

We could set 'prsn', 'prlq', 'vprec', 'evs', 'fsitherm','friver', 'ficeberg' to save daily output whilst we are confirming the freshwater conservation. Is it ok to do in this PR ?

I am not sure. They are mainly for testing freshwater conservation, it may not be necessary for everyone and might impact performance for others using the same configuration. To confirm freshwater conservation, I’d suggest adding these parameters in daily format for your test rather than including them here for general use?

Any thoughts @aekiss ?

dougiesquire · 2025-02-20T03:32:59Z

To confirm freshwater conservation, I’d suggest adding these parameters in daily format for your test rather than including them here for general use?

Yeah I think I agree. We can't cater for everyone and I think the list here, which I understand was mostly taken from OM2, was arrived at after iteration with users. It probably provides a pretty good starting point.

@minghangli-uni, could you please rebase this onto the latest dev-025deg_jra55do_ryf?

… (comment)

aekiss · 2025-02-21T05:13:55Z

diag_table_source.yaml

+            uhGM: # Time Mean Diffusive Zonal Thickness Flux
+            vhGM: # Time Mean Diffusive Meridional Thickness Flux


only necessary in configs that have GM

aekiss · 2025-02-21T05:25:17Z

diag_table_source.yaml

+            heat_content_cond: # Heat content (relative to 0degC) of water condensing into ocean
+            heat_content_evap: # Heat content (relative to 0degC) of water evaporating from ocean
+            heat_content_fprec: # Heat content (relative to 0degC) of frozen prec entering ocean
+            heat_content_frunoff: # Heat content (relative to 0C) of solid runoff into ocean
+            heat_content_lrunoff: # Heat content (relative to 0C) of liquid runoff into ocean
+            heat_content_lprec: # Heat content (relative to 0degC) of liquid precip entering ocean
+            heat_content_vprec: # Heat content (relative to 0degC) of virtual precip entering ocean
+            hfrunoffds: # Heat content (relative to 0C) of liquid+solid runoff into ocean
+            hfrainds: # Heat content (relative to 0degC) of liquid+frozen precip entering ocean
+            hfevapds: # Heat content (relative to 0degC) of net mass leaving ocean ocean via evap and ice form heat_content_massout
+            Heat_PmE: # Heat flux into ocean from mass flux into ocean


some of the above may be zero? if so, delete

aekiss · 2025-02-21T05:26:43Z

diag_table_source.yaml

+            hfsso: # Sensible heat flux into ocean sensible
+            prlq: # Liquid precipitation into ocean lprec
+            prsn: # Frozen precipitation into ocean fprec
+            vprec: # Virtual liquid precip into ocean due to SSS restoring


aren't we doing SSS restoring via salt flux, rather than water?

aekiss · 2025-02-21T05:29:54Z

diag_table_source.yaml

+            file_name_dimension: 2d  # descriptor for filename, e.g. 3d, 2d, scalar
+            reduction_method: max  # mean, snap, rms, pow##, min, max, or diurnal##
+        fields:
+            mlotst: # Mixed layer depth (delta rho = 0.03) MLD_003


Suggested change

mlotst: # Mixed layer depth (delta rho = 0.03) MLD_003

mlotst: # Mixed layer depth (delta rho = 0.03) MLD_003

sos: # Sea Surface Salinity

tos: # Sea Surface Temperature

SSH: # Sea Surface Height

min and max sos, tos, SSH may be helpful for spotting locations of instability

aekiss · 2025-02-21T05:30:16Z

diag_table_source.yaml

+            file_name_dimension: 2d  # descriptor for filename, e.g. 3d, 2d, scalar
+            reduction_method: min  # mean, snap, rms, pow##, min, max, or diurnal##
+        fields:
+            tos: # Sea Surface Temperature


Suggested change

tos: # Sea Surface Temperature

mlotst: # Mixed layer depth (delta rho = 0.03) MLD_003

sos: # Sea Surface Salinity

tos: # Sea Surface Temperature

SSH: # Sea Surface Height

aekiss · 2025-02-21T05:35:07Z

diag_table_source.yaml

+                    - output_freq_units
+                - file_name_date
+            output_freq_units: days  # time units for output: years, months, days, hours, minutes, or seconds
+            reduction_method: mean  # mean, snap, rms, pow##, min, max, or diurnal##


Suggested change

reduction_method: mean # mean, snap, rms, pow##, min, max, or diurnal##

reduction_method: snap # mean, snap, rms, pow##, min, max, or diurnal##

snapshots of global integrals will probably give better parallel performance than averages, as they don't require collectives and synchronisation of all MPI ranks every timestep

aekiss

LGTM, thanks @minghangli-uni! Just a few suggested changes to make before merging.

aekiss · 2025-02-21T06:34:43Z

diag_table_source.yaml

+            # A separator is not used prior to the first item.
+                - file_name_prefix
+                - file_name_dimension
+            reduction_method: snap  # mean, snap, rms, pow##, min, max, or diurnal##


Suggested change

reduction_method: snap # mean, snap, rms, pow##, min, max, or diurnal##

packing: 1 # double precision

reduction_method: snap # mean, snap, rms, pow##, min, max, or diurnal##

probably a good idea to have double precision grid data

aekiss · 2025-02-21T06:41:56Z

diag_table_source.yaml

+                - output_freq
+                - '':
+                    - output_freq_units
+                - file_name_date


Suggested change

- file_name_date

- reduction_method

- file_name_date

minghangli-uni requested a review from aekiss January 17, 2025 01:06

anton-seaice mentioned this pull request Jan 22, 2025

Turn-off config CI checks until configs are further developed #163

Merged

minghangli-uni mentioned this pull request Jan 29, 2025

Diagnostic table files management COSIMA/access-om3#259

Open

minghangli-uni added 3 commits February 20, 2025 16:23

Update diag_table based on the agreed scheme in COSIMA/access-om3#190…

c261490

… (comment)

Update diag_manager_nml

f5a5d03

Add diag_table_source.yaml for provenance

60bc213

minghangli-uni force-pushed the dev-025deg_jra55do_ryf_iss257 branch from 31497d2 to 60bc213 Compare February 20, 2025 05:24

aekiss reviewed Feb 21, 2025

View reviewed changes

aekiss approved these changes Feb 21, 2025

View reviewed changes

aekiss reviewed Feb 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update diag_table #157

Update diag_table #157

minghangli-uni commented Jan 17, 2025

minghangli-uni commented Jan 17, 2025

anton-seaice commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

anton-seaice commented Jan 17, 2025 •

edited

Loading

minghangli-uni commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

anton-seaice commented Jan 23, 2025

aekiss commented Jan 28, 2025

minghangli-uni commented Jan 28, 2025

anton-seaice commented Jan 31, 2025 •

edited

Loading

minghangli-uni commented Jan 31, 2025

dougiesquire commented Feb 20, 2025

aekiss Feb 21, 2025

aekiss Feb 21, 2025 •

edited

Loading

aekiss Feb 21, 2025

aekiss Feb 21, 2025 •

edited

Loading

aekiss Feb 21, 2025 •

edited

Loading

aekiss Feb 21, 2025

aekiss left a comment •

edited

Loading

aekiss Feb 21, 2025

aekiss Feb 21, 2025

		uhGM: # Time Mean Diffusive Zonal Thickness Flux
		vhGM: # Time Mean Diffusive Meridional Thickness Flux

	reduction_method: mean # mean, snap, rms, pow##, min, max, or diurnal##
	reduction_method: snap # mean, snap, rms, pow##, min, max, or diurnal##

	reduction_method: snap # mean, snap, rms, pow##, min, max, or diurnal##
	packing: 1 # double precision
	reduction_method: snap # mean, snap, rms, pow##, min, max, or diurnal##

Update diag_table #157

Are you sure you want to change the base?

Update diag_table #157

Conversation

minghangli-uni commented Jan 17, 2025

minghangli-uni commented Jan 17, 2025

anton-seaice commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

anton-seaice commented Jan 17, 2025 • edited Loading

minghangli-uni commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

anton-seaice commented Jan 23, 2025

aekiss commented Jan 28, 2025

minghangli-uni commented Jan 28, 2025

anton-seaice commented Jan 31, 2025 • edited Loading

minghangli-uni commented Jan 31, 2025

dougiesquire commented Feb 20, 2025

aekiss Feb 21, 2025

Choose a reason for hiding this comment

aekiss Feb 21, 2025 • edited Loading

Choose a reason for hiding this comment

aekiss Feb 21, 2025

Choose a reason for hiding this comment

aekiss Feb 21, 2025 • edited Loading

Choose a reason for hiding this comment

aekiss Feb 21, 2025 • edited Loading

Choose a reason for hiding this comment

aekiss Feb 21, 2025

Choose a reason for hiding this comment

aekiss left a comment • edited Loading

Choose a reason for hiding this comment

aekiss Feb 21, 2025

Choose a reason for hiding this comment

aekiss Feb 21, 2025

Choose a reason for hiding this comment

anton-seaice commented Jan 17, 2025 •

edited

Loading

anton-seaice commented Jan 31, 2025 •

edited

Loading

aekiss Feb 21, 2025 •

edited

Loading

aekiss Feb 21, 2025 •

edited

Loading

aekiss Feb 21, 2025 •

edited

Loading

aekiss left a comment •

edited

Loading