Which enhancements act at which processing-chain stage

Generated from arrayprobe_enhancements.registry (each module's declared stage) and .pipeline (which are wired to the automatic driver); never hand-typed. Rebuilt by the documentation build.

The chain (09-conventions.md): decode (raw binary to per-record images) -> framing, imgProc_sm (images to a proc file) -> size distribution, sizeDist (a proc file to an sd file) -> combine (a flight's sd files to one product) -> an optional post-combine export. Two workflows hang off this spine rather than sitting on it, and are the model for "when does this become relevant":

See also Enhancements · Which enhancements apply to which engine · User guide for the commands themselves · All-on sets, per probe and engine.

UIOOPS enhancements (arrayprobe-uiops-enhancements), by stage

42 modules, each declaring one stage in arrayprobe_enhancements.registry. arrayprobe run --enhancements (enhancements_uiops/src/arrayprobe_enhancements/pipeline.py) applies 13 of 42 automatically once its [enhancements] table resolves them ON; the rest need the reason column below read before they do anything.

How to read the tables. Each row carries at most two tags:

Each Finding is the title of the defect the module corrects, as recorded in the review of the UIOOPS or SODA-2 source; — means it corrects no recorded defect.

The decode stage (raw binary to per-record images)

Replaces the decode itself. Cannot be bolted on after the decode stage has already written its file – there is nothing post hoc to correct, because the particles it would recover were never decoded. A user reaches these by running the corrected reader directly, or via a raw-format reader override; arrayprobe run --enhancements records them as skipped with a reason naming the correction, never silently drops them.

EnhancementFindingTagsWhy manualConsumesWhat to check
last_record_fixthe dropped final raw recordmanual stated limit—rawenhancements_uiops/docs/CORRECTIONS.md (its own last_record_fix.py section)
pms_c6_routing_fixthe misfiled F2DC scalar fieldsmanual stated limit—rawenhancements_uiops/docs/CORRECTIONS.md (its own pms_c6_routing_fix.py section)

The framing stage (imgProc_sm, per-record images to a proc file)

Acts on a proc file (or the per-particle image array the framing stage holds in memory while building one). Relevant once the framing stage has run at least once – the diode workflow's first pass is exactly this file.

EnhancementFindingTagsWhy manualConsumesWhat to check
diode_range_fixthe image-widening diode numbermanual needs workno driver adapter yetimagesenhancements_uiops/docs/CORRECTIONS.md (its own diode_range_fix.py section)
dmax_sensitivity—manual stated limitno driver adapter: reports a spread across definitions, writes no corrected fileimage_diam_minR, image_length, image_widthenhancements_uiops/docs/CORRECTIONS.md (its own dmax_sensitivity.py section)
effective_density—manual stated limitno driver adapter: needs area_ratio, not a stored proc/sd columnholroyd_habit, image_area, image_diam_minRmodule docstring – “A published, habit-dependent ice effective-density relation, from a table.”
hole_edge_swapthe swapped hole variablesauto—edge_at_max_hole, max_hole_diameterenhancements_uiops/docs/CORRECTIONS.md (its own hole_edge_swap.py section)
int_arr_rollback_fixthe missing probetype-1 rollback correctionauto—Time_in_secondsenhancements_uiops/docs/CORRECTIONS.md (its own int_arr_rollback_fix.py section)
mask_presence_fixthe silent missing mask filemanual needs workno driver adapter yet—enhancements_uiops/docs/CORRECTIONS.md (its own mask_presence_fix.py section)
ml_accept_reject—manual stated limitno driver adapter: an analysis (CNN suggestion), writes no corrected fileimagesmodule docstring – “Per-particle accept/reject classification beyond UIOOPS's own rule-based ladder.”
outoffocusthe unapplied Korolev size correctionauto—image_diam_minR, imagesenhancements_uiops/docs/CORRECTIONS.md (its own outoffocus.py section)
partial_image_habit—manual stated limitno driver adapter: a population analysis that writes no corrected file, and no retrieval matrix is held for any probe this driver runs – Korolev & Sussman (2000)'s are the OAP-2DC's and do not transfer (p. 1057)imagesenhancements_uiops/docs/CORRECTIONS.md (its own partial_image_habit.py section)
particle_clustering—manual stated limitno driver adapter: an analysis, writes a labels sidecar rather than a corrected proc/sd file; whether the driver should run analyses at all is not yet decidedarea_hole_ratio, edge_at_max_hole, image_area, image_auto_reject, image_diam_circle_fit, image_diam_minR, image_length, image_perimeter, image_width, max_hole_diameter, percent_shadow_areaenhancements_uiops/docs/CORRECTIONS.md (its own particle_clustering.py section)
particle_number_all_fixthe sometimes-written Particle_number_allauto—Particle_number_allenhancements_uiops/docs/CORRECTIONS.md (its own particle_number_all_fix.py section)
particle_time_glitch_fixthe first-midnight-only rolloverauto—Time_in_seconds, particle_timeenhancements_uiops/docs/CORRECTIONS.md (its own particle_time_glitch_fix.py section)
particle_time_int8_fixthe int8 particle-time collapseauto—Time, Time_in_seconds, parent_rec_num, particle_microsec, particle_millisecenhancements_uiops/docs/CORRECTIONS.md (its own particle_time_int8_fix.py section)
spec_artefact_filter—manual needs workno entry_pointimagesmodule docstring – “SPEC's own 2D-S artefact-removal algorithm, as a selectable second rejection scheme.”
spot_edge_fixthe missing decimal in spot_edgemanual stated limitno driver adapter: needs area_ratio, not a stored proc/sd columnedge_at_max_hole, max_hole_diameter, part_zenhancements_uiops/docs/CORRECTIONS.md (its own spot_edge_fix.py section)
wide_hollow_fixthe diode-32 hollow testmanual stated limitno driver adapter: needs a reconstructed particle image, not a proc/sd columnimagesenhancements_uiops/docs/CORRECTIONS.md (its own wide_hollow_fix.py section)
zero_image_recoverythe abandoned zero-image recordmanual needs workno driver adapter yetimagesenhancements_uiops/docs/CORRECTIONS.md (its own zero_image_recovery.py section)

The size-distribution stage and the combine step (sizeDist, a proc file to an sd file, then a flight's sd files to one combined product)

Acts on an sd file, or on a value the size-distribution stage (sizeDist) itself reads or computes – including a flight's already-combined product, which EnhancementSpec.stage's own docstring folds into this same value. Relevant once the size-distribution stage has run, or, for the inter-arrival modules, once a threshold file exists to feed the size-distribution stage's optional twelfth argument.

EnhancementFindingTagsWhy manualConsumesWhat to check
analysis—manual needs workno entry_pointbin_dD, bin_mid, conc_minR, count, mass, massBLmodule docstring – “A post-sizeDist analysis layer: PSD moments, mass-weighted mean diameter, and leg-scale aggregation with propagated Poisson uncertainty.”
conc_arear_fixthe double-scaled area-ratio concentrationmanual needs workno driver adapter yetimage_diam_AreaRenhancements_uiops/docs/CORRECTIONS.md (its own conc_arear_fix.py section)
distance_rejection—manual stated limitno driver adapter: needs tas and a threshold_distance, neither a proc/sd columnTime_in_secondsmodule docstring – “Inter-particle distance rejection, as an alternative to a fixed time threshold.”
field2006_shatter—auto—Time_in_secondsenhancements_uiops/docs/CORRECTIONS.md (its own field2006_shatter.py section)
int16_double_fixthe int16 diameter saturationmanual needs workno entry_pointimage_length, image_widthenhancements_uiops/docs/CORRECTIONS.md (its own int16_double_fix.py section)
intarr_policy—manual stated limitno driver adapter: the rule acts on the threshold file sizeDist reads (write_threshold_file) and the neighbour count inside sizeDist's per-second flagging, both before any sd file existsint_arrenhancements_uiops/docs/CORRECTIONS.md (its own intarr_policy.py section)
intarr_threshold_fixthe global-minimum inter-arrival thresholdmanual needs workno driver adapter yetint_arrenhancements_uiops/docs/CORRECTIONS.md (its own intarr_threshold_fix.py section)
overload_accounting_fixthe no-accept second overwriteauto—DMT_DOF_SPEC_OVERLOAD, Time_in_seconds, sample_vol, sum_IntArrenhancements_uiops/docs/CORRECTIONS.md (its own overload_accounting_fix.py section)
poisson_restore—manual stated limitno driver adapter: needs false_rejection_fraction from a separate inter-arrival analysisconc_minRmodule docstring – “Korolev & Field (2015) Eq. 2 – statistically restore intact particles the inter-arrival-time shattering rejection removes as false positives.”
poisson_uncertainty—manual needs workno entry_pointcountmodule docstring – “Per-bin and total Poisson counting uncertainty on sizeDist output.”
pressure_units_fixthe unchecked pressure unitauto——enhancements_uiops/docs/CORRECTIONS.md (its own pressure_units_fix.py section)
probe_crossover—manual needs workno entry_pointbin_dD, bin_max, bin_min, conc_minR, count, timemodule docstring – “The 2D-S/HVPS crossover diagnostic: do two probes agree where their size ranges overlap?”
published_dof—auto—bin_dD, bin_mid, conc_minR, nenhancements_uiops/docs/CORRECTIONS.md (its own published_dof.py section)
reaccept_every_event_fixthe first-event-only reacceptance scanmanual stated limitno driver adapter: re-running sizeDist needs the run's own call arguments (pressure, threshold file), not recoverable from the .call stamp – same gap as pressure_units_fix; use reaccept_every_event_fix.run_sizedist(..., reaccept_every_event=True) from PythonTime_in_seconds, image_auto_reject, image_diam_minRenhancements_uiops/docs/CORRECTIONS.md (its own reaccept_every_event_fix.py section)
reshape_fixthe missing mean-ratio transposemanual needs workno driver adapter yetmean_area_ratioenhancements_uiops/docs/CORRECTIONS.md (its own reshape_fix.py section)
rollover_fixthe first-midnight-only rollovermanual stated limitno driver adapter: in-memory time-array correction, no proc/sd file shape to copy-then-correcttimehhmmssenhancements_uiops/docs/CORRECTIONS.md (its own rollover_fix.py section)
tas_grid_fixthe duplicate TAS-second particle lossmanual stated limitno chain adapter: run_flight.py applies it inside S4 itself (DRIVER_PROTECTIVE; regular grid with a TAS chain, collapsed probe grid without), stamped tas_grid= in the sd .call—enhancements_uiops/docs/CORRECTIONS.md (its own tas_grid_fix.py section)

post-combine (an export or copy of an already-written port output file)

Acts on a finished port output file, any stage's. Relevant last, once there is a file to export or restructure.

EnhancementFindingTagsWhy manualConsumesWhat to check
arco_export—auto—*module docstring – “Analysis-ready export of a port output file — lossless by default.”
arm_export—auto—timemodule docstring – “A CF-1.9 + ARM Data File Standards structural copy of a port output file.”
cf_export—auto—*module docstring – “An opt-in CF-1.10 copy of a port sd (sizeDist) output file.”

no file stage (a reference table or pure function)

No proc/sd/product file to act on – a lookup table or a function that takes call arguments the run configuration supplies rather than anything read from a file.

EnhancementFindingTagsWhy manualConsumesWhat to check
brown_francis_1995—manual stated limitno file stage—module docstring – “Brown & Francis (1995) mass-diameter power law – closing the brown_francis_1995 gap in arrayprobe_enhancements.md_relations.”
md_relations—manual stated limitno file stage—module docstring – “A registry of mass-diameter relations – the mechanism, and only what has actually been read.”
probe_uncertainty—manual stated limitno file stage—module docstring – “A published per-instrument-class uncertainty table, shipped with the outputs.”

SODA-2 corrections (arrayprobe-soda2-enhancements) – no per-stage split

19 modules. SODA-2 is a single processing lineage with one source tree, not staged decode/imgProc/sizeDist/combine like UIOOPS – arrayprobe_soda2_enhancements.registry's own docstring gives the reason there is no engine matrix, and the same fact means there is no stage matrix either: a correction here applies or it does not, judged by reachability (live/latent/flagged, the SODA-2 review's own classification of how often the defect fires on real data), not by where in a chain it sits.

How to read this table. The same tags as the UIOOPS tables, read from arrayprobe_soda2_enhancements._adapter's ROUTES and NOT_ROUTED:

Reachability is the SODA-2 review's own classification of how often the defect fires on real data. Each Finding is the title of the defect, as that review records it.

CorrectionFindingTagsWhy manualReachabilityEntry pointWhat to check
aspr_nan_fixaspr < 1.0 is MIN, and it maps every non-finite result onto exactly 1.0auto—livecorrected_asprenhancements_soda2/docs/CORRECTIONS.md (its own aspr_nan_fix.py section)
blank_roi_dof_fixA blank ROI is accepted as within depth of fieldauto—flaggedcorrected_binary_dofenhancements_soda2/docs/CORRECTIONS.md (its own blank_roi_dof_fix.py section)
den_clamp_fixThe den clamp guards a 0/0 that cannot happen, and bites one that canauto—livecorrected_asprenhancements_soda2/docs/CORRECTIONS.md (its own den_clamp_fix.py section)
dilation_clamp_fixThe dilation kernel clamps against the roi's own slice count, not the padded array it dilatesauto—livecorrected_label_blobs_dilationenhancements_soda2/docs/CORRECTIONS.md (its own dilation_clamp_fix.py section)
double_pi_fix!pi is single precision, and it silently changes min_circle_fast's corner countauto—livecorrected_aspect_ratioenhancements_soda2/docs/CORRECTIONS.md (its own double_pi_fix.py section)
empty_roi_diodegaps_fixThe empty-ROI diodegaps value is 1, not 0, through the same no-match where() sentinel S4-t and S5-f both reach by different routesmanual stated limitinline arithmetic inside process_buffer_tail (processbuffer_tail.py:283); no function boundary to substituteflaggedcorrected_diodegapsenhancements_soda2/docs/CORRECTIONS.md (its own empty_roi_diodegaps_fix.py section)
fillholes_int_labelsfillholes wraps its hole labels modulo 256, so region 256 is never filled and region 257 shares region 1's fateauto—latentcorrected_fillholesenhancements_soda2/docs/CORRECTIONS.md (its own fillholes_int_labels.py section)
hk_probeid_columnsspec_process_hk's voltage/count columns default to the horizontal-channel set for every probeid but 'V'auto—flaggedcorrected_process_hkenhancements_soda2/docs/CORRECTIONS.md (its own hk_probeid_columns.py section)
label_blobs_int_countlabel_blobs's default byte return wraps at 256 blobsauto—latentcorrected_label_blobsenhancements_soda2/docs/CORRECTIONS.md (its own label_blobs_int_count.py section)
last_buffer_index_fixThe last buffer of every file is never indexed, and its own header is never range-checkedauto—livecorrected_spec_indexenhancements_soda2/docs/CORRECTIONS.md (its own last_buffer_index_fix.py section)
no_stuck_diode_fixThe no-stuck-diode case is a silent scalar-index bug on wide probes, and halts only on a narrow oneauto—livecorrected_fixstuckbitsenhancements_soda2/docs/CORRECTIONS.md (its own no_stuck_diode_fix.py section)
pbp_inttime_consistent.pbp.csv and .pbp.nc write two different values for the same inttime columnmanual stated limitits writer needs interarrival, which the driver's write_pbp_csv_buffer call (process_2d.py:389) does not passlivecorrected_write_pbp_csv_bufferenhancements_soda2/docs/CORRECTIONS.md (its own pbp_inttime_consistent.py section)
pbp_rejectionflag_ubyte.pbp.nc's rejectionflag is a signed byteauto—livecorrected_open_pbp_ncdfenhancements_soda2/docs/CORRECTIONS.md (its own pbp_rejectionflag_ubyte.py section)
poisson_fill_guardA zero or shrinking fill area drives the Poisson correction through a floating-point error, then silently down the "no spot" pathauto—liveguarded_poisson_spot_correctenhancements_soda2/docs/CORRECTIONS.md (its own poisson_fill_guard.py section)
rejection_tally_fixBit 128 (depth-of-field) is never counted in soda2_particlesort's per-reason tallymanual stated limitreturns a separate corrected tally beside the port's own; call it on the run's arrays, it substitutes no driver calllivecorrected_count_rejectedenhancements_soda2/docs/CORRECTIONS.md (its own rejection_tally_fix.py section)
rollover_modulus_fixThe 2D-S rollover constant is 2³²−1, one tick short of the true modulusmanual stated limitinline arithmetic inside process_spec_buffer (processbuffer_spec.py:305); no function boundary to substitutelivecorrected_inttimeenhancements_soda2/docs/CORRECTIONS.md (its own rollover_modulus_fix.py section)
run_diagnosticsA ≥1000-slice frame or a >10 000-line buffer is dropped with no count keptmanual stated limita diagnostic: reports on a run and changes no outputflaggedspec_buffer_diagnosticsenhancements_soda2/docs/CORRECTIONS.md (its own run_diagnostics.py section)
same_time_buffer_fixA same-time buffer stamps its first particle with the previous buffer's last timeauto—livecorrected_process_spec_bufferenhancements_soda2/docs/CORRECTIONS.md (its own same_time_buffer_fix.py section)
zero_area_return_fixThe zero-area return hands back defaults that read as measurementsauto—flaggedcorrected_soda2_findsizeenhancements_soda2/docs/CORRECTIONS.md (its own zero_area_return_fix.py section)