Calibrating CMOR3 & 4 forward development plans #730

Activity

1. matthew-mizielinski commented on Feb 26, 2024

   

   matthew-mizielinski

   on Feb 26, 2024

   More actions

   It isn't immediately obvious to me that the changes for the mip-cmor-tables use will break CMOR for use of other sets of MIP tables, but I can see that maintaining the tests using the tables may be a little more tricky, and the tests in this repository would benefit from some reorganisation/attention.

   Regarding the different forthcoming releases, I'd recommend creating and maintaining a "release branch" for version 3.7 while development proceeds on main for v3.8. A fix to 3.7 can then be applied to the release branch and released, and then it can be cherry-picked into main while v3.8 is being prepared. Once v3.8 is ready create a v3.8_release branch and any bug fixes to that version can proceed while developing v3.9.

   The important thing with release branches is to ensure that a fix applied to the v3.7_release branch is propagated to main and other release branches if necessary. It is moderately straightforward to cherry-pick changes (providing changes to release branches are "squash and merge"'d) from one branch to another as long as the code has not diverged too far from the branch on which it is developed. It is also key that care is taken to try to keep a linear history within the repository as merges can interleave commits from one branch into another on merge
2. durack1 commented on Feb 26, 2024

   

   durack1

   on Feb 26, 2024

   CollaboratorAuthor

   More actions

   It isn't immediately obvious to me that the changes for the mip-cmor-tables use will break CMOR for use of other sets of MIP tables, but I can see that maintaining the tests using the tables may be a little more tricky, and the tests in this repository would benefit from some reorganisation/attention.

   It's the formatting change strings to lists for dimensions and modeling_realm (e.g. CMIP6 Amon tas and MIP APmon tas) that breaks backward compatibility.

   Of course, another option would be to reformat the cmip6-cmor-tables to align with CMOR3.8, but that would need to be thought about.
3. matthew-mizielinski commented on Feb 27, 2024

   

   matthew-mizielinski

   on Feb 27, 2024

   More actions

   It isn't immediately obvious to me that the changes for the mip-cmor-tables use will break CMOR for use of other sets of MIP tables, but I can see that maintaining the tests using the tables may be a little more tricky, and the tests in this repository would benefit from some reorganisation/attention.

   It's the formatting change strings to lists for dimensions and modeling_realm (e.g. CMIP6 Amon tas and MIP APmon tas) that breaks backward compatibility.

   I would have thought it possible to run the equivalent of the following python code for the dimensions and modeling_realm

   if isinstance(variable_entry['dimensions'], list):
       dimensions = variable_entry['dimensions']
   else:
       dimensions = variable_entry['dimensions'].split(' ')

   Of course, another option would be to reformat the cmip6-cmor-tables to align with CMOR3.8, but that would need to be thought about.

   I wouldn't even consider this while CMIP6 is still operational
4. mauzey1 commented on Feb 27, 2024

   

   mauzey1

   on Feb 27, 2024

   Collaborator

   More actions

   If the code that reads variable values from the tales encounters an array, then it will convert that array into a string.
   

   cmor/Src/cmor_tables.c

   Lines 215 to 233 in 10bb94a

   	/* -------------------------------------------------------------------- */
   	/* Attribute values that are arrays will have their array elements */
   	/* combined into space-separated lists. */
   	/* -------------------------------------------------------------------- */
   	if(json_object_is_type(value, json_type_array)) {
   	jsonArray = json_object_get_array(value);
   	arrayLen = array_list_length(jsonArray);
   	for (k = 0; k < arrayLen; k++) {
   	jsonItem = (json_object *) array_list_get_idx(jsonArray, k);
   	if (k == 0) {
   	strcpy(szValue, json_object_get_string(jsonItem));
   	} else {
   	strcat(szValue, " ");
   	strcat(szValue, json_object_get_string(jsonItem));
   	}
   	}
   	} else {
   	strcpy(szValue, json_object_get_string(value));
   	}

   
   Values for dimensions and modeling_realm will get treated as a space-separated list in a string regardless of their format in the tables.
5. durack1 commented on Feb 27, 2024

   

   durack1

   on Feb 27, 2024

   CollaboratorAuthor

   More actions

   @mauzey1 excellent, if I am reading (and interpreting this correctly) in then means that we can continue to use CMOR3.8 with the cmip6-cmor-tables, right?
6. mauzey1 commented on Feb 27, 2024

   

   mauzey1

   on Feb 27, 2024

   Collaborator

   More actions

   Yes, CMOR 3.8 should still work with cmip6-cmor-tables.
7. durack1 commented on Mar 10, 2024

   

   durack1

   on Mar 10, 2024

   CollaboratorAuthor

   More actions

   @mauzey1 just circling around on this, and correcting the erroneous assumption that CMOR3.8/.9 was going to break backward (cmip6-cmor-tables) support, great.

   Just trying to calibrate if anything is holding up the CMOR3.8 release, and if so, it'd be great to get any remaining issues identified so these can be resolved and CMOR3.8 can be released
8. durack1

   added

   documentation

   on Apr 7, 2024
9. durack1

   added this to the 4.0/Future milestone on Apr 7, 2024