fix(inversion): guard data_subtracted_dict on data=None, harden sparse noise check and data-term dtype

[Jammy2211]

Overview

Corrective follow-up to phase 1 of https://github.com/orgs/PyAutoLabs/discussions/13 (PyAutoArray#589, PyAutoGalaxy#637, PyAutoLens#757). An independent Codex review reproduced three defects on the sparse data=None path (record: PyAutoLabs/PyAutoLens#757 (comment)): AbstractInversion.data_subtracted_dict raises TypeError / yields {mapper: None} when the inversion was built without data, which breaks subplot_of_mapper / subplot_mappings and the two workspace interferometer/features/pixelization/fit.py scripts; check_noise_map_real_imag_equal uses an absolute tolerance of 1e-8; and apply_sparse_operator squares complex64 data in float32 so the cached data_term disagrees with the chunked builder.

Plan

• Raise a clear InversionException from data_subtracted_dict when the inversion has no data, pointing at fit.inversion_with_data.
• Catch that exception in the two mapper subplots so the data-subtracted panel is skipped cleanly.
• Set atol=0.0 on the real/imag noise equality check.
• Promote to complex128 before computing the cached data term so one-shot equals chunked for any input dtype.
• Add tests for all four; then migrate the two workspace pixelization fit.py scripts to fit.inversion_with_data (library first, workspace follows).

Detailed implementation plan

Affected Repositories

• PyAutoArray (primary)
• autogalaxy_workspace, autolens_workspace (workspace follow-up, /ship_workspace)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoArray	main	clean
./autogalaxy_workspace	main	clean
./autolens_workspace	main	clean

Worktree guard clear on all three.

Suggested branch: feature/sparse-data-none-guard

Implementation Steps

1. autoarray/inversion/inversion/abstract.py data_subtracted_dict (~L868): if self.data is None: raise exc.InversionException(...) naming fit.inversion_with_data / fit.data.
2. autoarray/inversion/plot/inversion_plots.py: the except (AttributeError, KeyError) guards at ~L87 (subplot_of_mapper) and ~L397 (subplot_mappings) also catch exc.InversionException; do not widen to bare Exception.
3. autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py check_noise_map_real_imag_equal (~L58, L61): atol=0.0 on np.allclose and np.isclose, default rtol kept.
4. autoarray/dataset/interferometer/dataset.py apply_sparse_operator (~L391-394): compute data_term from complex128-promoted local copies of data and noise; dirty image expression unchanged.
5. Tests: DatasetInterface(data=None) inversion → pytest.raises(exc.InversionException, match="inversion_with_data"); both subplots run when data_subtracted_dict raises; sigma 1e-9+2e-9j → exc.DatasetException, equal 1e-9 sigmas pass; complex64 apply_sparse_operator data_term == complex128 value == sparse_terms_from_chunks(...).data_term.
6. Workspace (after the library PR): scripts/interferometer/features/pixelization/fit.py in autogalaxy_workspace (~L319) and autolens_workspace (~L291): inversion = fit.inversion_with_data plus one sentence of prose; regenerate notebooks per each workspace's convention; smoke both scripts.

Key Files

• autoarray/inversion/inversion/abstract.py — guard
• autoarray/inversion/plot/inversion_plots.py — handlers
• autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py — noise check
• autoarray/dataset/interferometer/dataset.py — data term dtype
• autogalaxy_workspace / autolens_workspace scripts/interferometer/features/pixelization/fit.py — accessor migration

Original Prompt

Click to expand starting prompt

Sparse data=None path: guard data_subtracted_dict, harden noise check and data-term dtype (Discussion #13 review fixes)

Type: bug
Target: PyAutoArray
Repos:

• PyAutoArray
• autogalaxy_workspace
• autolens_workspace
  Themes:
• interferometer
• sparse-operator
• visualization
  Difficulty: small
  Autonomy: supervised
  Priority: high
  Status: draft
  Consequence: glance
  Witness: on a sparse-operator dataset with the precomputed-data-term gate on (fit.inversion.dataset.data is None), fit.inversion.data_subtracted_dict raises InversionException naming inversion_with_data (no TypeError, no {mapper: None}), and InversionPlotter.subplot_of_mapper / subplot_mappings on that inversion skip cleanly instead of raising; check_noise_map_real_imag_equal rejects sigma 1e-9 vs 2e-9; apply_sparse_operator on complex64 data gives data_term equal to the complex128 value and to sparse_terms_from_chunks; the two workspace interferometer/features/pixelization/fit.py scripts run under the smoke profile using fit.inversion_with_data.
  Review-minutes: 5
  Unattended: ready
  Parent: complete/2026/09/interferometer-streaming-visibilities.md

Source: Codex gpt-6-astra review of PyAutoArray#589 / PyAutoGalaxy#637 / PyAutoLens#757
(record: PyAutoLabs/PyAutoLens#757 (comment)), all four
claims reproduced 2026-09-30.

Why

Phase 1 of https://github.com/orgs/PyAutoLabs/discussions/13 builds the sparse inversion with
data=None when no galaxy has a non-linear light profile. Three review findings follow from
the merged code:

• B (introduced, user-facing). AbstractInversion.data_subtracted_dict
  (autoarray/inversion/inversion/abstract.py ~L868-885) does copy.copy(self.data) then
  -=; with data=None a multi-object inversion raises
  TypeError: unsupported operand type(s) for -: 'NoneType' and 'complex' and a single-mapper
  inversion yields {mapper: None}, which makes subplot_of_mapper fail with a ValueError
  in plot/array.py. The handlers in autoarray/plot/.../inversion_plots.py (~L87, ~L397)
  catch only (AttributeError, KeyError). The library visualizers already pass
  fit.inversion_with_data, but autogalaxy_workspace/scripts/interferometer/features/pixelization/fit.py
  (~L321) passes fit.inversion and now fails on PyAutoGalaxy main; the autolens_workspace copy
  (~L293) fails once PyAutoLens#757 lands.
• D (pre-existing, hardening). check_noise_map_real_imag_equal
  (inversion_interferometer_util.py ~L58, L61) uses np.allclose/np.isclose at default
  atol=1e-8, so sigma 1e-9 vs 2e-9 passes and the sparse curvature is 1.5x wrong. Unreachable
  with Jy-unit data but a scale-dependent check is wrong on principle.
• A (edge, pre-existing path). Interferometer.apply_sparse_operator
  (autoarray/dataset/interferometer/dataset.py ~L391-394) squares data.real/data.imag in
  the data's own dtype; complex64 data + complex128 noise gives data_term 700140000.0 vs exact
  700140007.0 (log_evidence +3.5), and disagrees with sparse_terms_from_chunks, which promotes
  to complex128. Loaders always yield complex128, so only hand-built complex64 arrays hit it.

Finding C (noise map mutated after apply_sparse_operator stales the operator) predates phase 1
and is out of scope; document only if convenient.

What

1. AbstractInversion.data_subtracted_dict: when self.data is None, raise
   exc.InversionException with a message pointing at fit.inversion_with_data (and fit.data).
2. inversion_plots.py handlers around data_subtracted_dict reads: also catch
   exc.InversionException (and TypeError), so the mapper subplots skip that panel cleanly.
3. check_noise_map_real_imag_equal: atol=0.0 on both calls (keep the default rtol).
4. apply_sparse_operator: cast data (and noise) to float64 components before squaring so
   one-shot == chunked for any input dtype.
5. Tests in test_autoarray: (1) raises with the named message; (2) plotter skips; (3) the
   1e-9/2e-9 rejection; (4) complex64 data_term == complex128 == chunked.
6. Workspace: both interferometer/features/pixelization/fit.py scripts → fit.inversion_with_data
   in the inversion plotter call, with a one-line prose note on why; regenerate notebooks per
   each workspace's convention. Library first (/ship_library), workspace follows
   (/ship_workspace).

🤖 Generated with Claude Code

[Jammy2211]

Library PR created

#591

• test_autoarray/: 1765 passed
• Workspace follow-up (autogalaxy_workspace, autolens_workspace pixelization fit.py → fit.inversion_with_data) ships next via /ship_workspace
• Next: /prm once CI is green (library first)

🤖 Generated with Claude Code

[Jammy2211]

Shipped

• PyAutoArray: fix(inversion): guard data_subtracted_dict on data=None, harden sparse noise check and data-term dtype #591 (merge d429844) — data_subtracted_dict guard + plotter handlers, atol=0.0 noise check, complex128 data term. Labelled pending-release.
• autogalaxy_workspace: interferometer/pixelization: use fit.inversion_with_data for the mapper diagnostics (Discussion #13) autogalaxy_workspace#253 (merge 826b7f2b); autolens_workspace: interferometer/pixelization: use fit.inversion_with_data for the mapper diagnostics (Discussion #13) autolens_workspace#581 (merge b1cbf848) — pixelization fit.py → fit.inversion_with_data.

Merged library-first after PyAutoLens#757. Review finding C (noise map mutated after apply_sparse_operator) is pre-existing and left as is.

🤖 Generated with Claude Code