Handling unmerged data in `_refln.` blocks in SF mmCIF

[paulsbond]

There are some SF mmCIF files (e.g. 1VLY and 3E0Z) that have unmerged data in _refln. blocks instead of _diffrn_refln. blocks. Should Gemmi handle those or do they need to be remediated by the PDB?

[CV-GPhL]

If one can truly make sense out of it - yes. 1VLY shows e.g.

loop_
_audit.revision_id 
_audit.creation_date 
_audit.update_record 
1   2004-10-05 
;The data loops contain the following:
structure factors used in refinement.
unmerged original index intensities of data set used for refinement, this data set is also used for phasing.
unmerged original index intensities of data set used for phasing.
unmerged original index intensities of data set used for phasing.
unmerged original index intensities of data set used for phasing.
experimental phases.
density modified experimental phases.
;

and

 Block  I/F max(redundancy)  _diffrn.details
 -------------------------------------------------
   1     F     1
   2     I     7           unmerged original index intensities of data set used for refinement, which is also used for phasing.
   3     I     6           unmerged original index intensities of data set used for phasing.
   4     I     2           unmerged original index intensities of data set used for phasing.
   5     -     1           experimental phases.
   6     -     1           density modified experimental phases.

So one could probably get e.g. the inverse-variance weighted mean intensity for those unmerged datablocks - which would be nice, yes. Maybe a first step is some analysis function that could go through a generic list of reflections and see if they indices occupy only one asymmetric unit or if they occupy several, how many (relative to the number given by symmetry). If one sees only one set if I+ and I- for example, then those might be hte merged intensities of Friedel mates kept apart or such.

Do we know if the PDB upon deposition is refusing unmerged data in _refln caegory? And if it does: is there a cut-off date of PDB entries before which there might be such situations? It would be god to know if that feature would be for remediation or something generically useful in future too.

Just my 2 cents ...