feat(interferometer): pass data=None on the sparse precomputed-data-term path (Discussion #13 lens parity)

[Jammy2211]

Summary

PyAutoLens parity for phase 1 of GitHub Discussion https://github.com/orgs/PyAutoLabs/discussions/13 ("Streaming visibilities for memory efficiency", HRSAstro), mirroring PyAutoLabs/PyAutoGalaxy#637 on top of PyAutoLabs/PyAutoArray#589. Closes #756.

On the sparse interferometer path a lens fit with no non-linear light profile still allocated two N_vis arrays per likelihood evaluation: profile_visibilities (Visibilities.zeros) and profile_subtracted_visibilities (data - zeros), because FitInterferometer.tracer_to_inversion passed the subtracted data unconditionally. This PR:

• gates tracer_to_inversion on autogalaxy's uses_precomputed_data_term_from(dataset, galaxies, data, noise_map) (sparse operator carries data_term, the fitted data/noise map are the dataset's own, no galaxy has a non-linear light profile) and passes data=None when it holds, so fast_chi_squared reads the cached scalar and neither visibility property is evaluated on the likelihood path; the checks are identity/type checks, so the branch is fixed at trace time and jit-safe;
• adds FitInterferometer.inversion_with_data: the inversion, or a shallow copy carrying fit.data when the inversion ran with data=None, for output paths (data_subtracted_dict → subplot_of_mapper); the interferometer visualizer uses it;
• makes profile_visibilities / profile_subtracted_visibilities cached properties (as profile_image already was and as autogalaxy now has them), so "never evaluated" is observable.

Dense-path and light-profile fits are bit-identical to before.

Witness (1e5-vis NUFFT, 616-pixel mask, Isothermal lens + 15x15 rectangular source, InversionInterferometerSparseNumba): figure_of_merit gated vs gate forced off differs by exactly 0.0; tracemalloc peak per figure_of_merit 3.20 MB gated at both 1e5 and 4e5 visibilities (flat in N_vis, below 16·N_vis bytes) vs 7.59 MB / 21.99 MB ungated; 16.2 ms vs 18.0 / 26.6 ms per evaluation.

API Changes

Additive. New public name FitInterferometer.inversion_with_data. Behaviour change: on the sparse path with no non-linear light profile, fit.inversion.dataset.data is now None (use fit.inversion_with_data or fit.data for outputs). profile_visibilities and profile_subtracted_visibilities are now cached per fit instance. Requires PyAutoArray ≥ the #589 merge and PyAutoGalaxy ≥ the #637 merge (both on main, pending release).
See full details below.

Test Plan

• pytest test_autolens/interferometer — 29 passed (new: gate true + inversion.dataset.data is None; sparse pixelization-only vs dense at rel 1e-8; inversion_with_data shares the reconstruction and carries fit.data; dense fit returns inversion_with_data is inversion; spies on visibilities_from / Visibilities.zeros record no calls and neither profile property is cached after figure_of_merit; light-profile fit gate false and bit-equal; jax.jit sparse fit matches numpy; gate forced off → figure_of_merit bit-equal)
• pytest test_autolens — 766 passed, 1 xfailed
• Red check: with the source edits reverted, 5 of the 11 tests in the file fail (all new/extended)
• Witness script (numbers above)
• CI green on unittest 3.12 / 3.13 / nojax

Full API Changes (for automation & release notes)

Added

• autolens.interferometer.fit_interferometer.FitInterferometer.inversion_with_data -> Optional[aa.AbstractInversion] — the inversion carrying the fitted visibilities as its dataset data, for output/plot paths
• FitInterferometer._uses_precomputed_data_term (private property) — uses_precomputed_data_term_from(dataset, tracer.galaxies, data, noise_map)

Changed Behaviour

• FitInterferometer.tracer_to_inversion passes data=None to aa.DatasetInterface when _uses_precomputed_data_term holds (sparse operator with data_term, dataset's own data/noise map, no non-linear light profile); profile_visibilities / profile_subtracted_visibilities are then never evaluated on the likelihood path
• FitInterferometer.profile_visibilities and FitInterferometer.profile_subtracted_visibilities are cached_property (were plain properties); values unchanged
• autolens/interferometer/model/visualizer.py passes fit.inversion_with_data to the inversion plotter

Migration

• Code reading fit.inversion.dataset.data / fit.inversion.data_subtracted_dict on a sparse pixelization-only lens fit should use fit.inversion_with_data (or fit.data).

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

[Jammy2211]

Independent review (Codex gpt-6-astra) — this PR + phase 1 (PyAutoArray#589, PyAutoGalaxy#637)

Three read-only Codex reviews (one per repo) returned 6 findings = 4 distinct claims. Every claim was reproduced with scripts before being reported here; nothing has been edited in response yet.

#	Claim	Verdict	Where	Real exposure
B	fit.inversion.data_subtracted_dict breaks when the inversion ran with data=None (gate on): multi-object → TypeError: unsupported operand type(s) for -: 'NoneType' and 'complex' at autoarray/inversion/inversion/abstract.py:885; single mapper → {mapper: None} and subplot_of_mapper raises ValueError at plot/array.py:206. The inversion_plots.py:87/:397 handlers catch only (AttributeError, KeyError).	Reproduced — introduced by phase 1 / this PR	autoarray + plotters	Yes. autogalaxy_workspace scripts/interferometer/features/pixelization/fit.py:321-322 passes fit.inversion on a sparse dataset (live on PyAutoGalaxy main now); autolens_workspace same path :293 will hit it once this PR lands. Library visualizers are already on inversion_with_data.
A	apply_sparse_operator squares data.real/data.imag in the data's own dtype: complex64 data + complex128 noise gives data_term 700140000.0 vs exact 700140007.0; log_evidence +3.5; chunked path (sparse_terms_from_chunks, promotes to complex128) disagrees with one-shot.	Reproduced — pre-existing precision path, exposed by the cache	autoarray/dataset/interferometer/dataset.py:391-394	Low. Visibilities.from_fits and the float (N,2) constructor cast to complex128; only a caller passing a complex64 ndarray directly reaches it. complex128 data: all paths bit-equal.
C	Replacing/mutating dataset.noise_map after apply_sparse_operator leaves noise_normalization (and data_term) stale: 7.35 vs 12.90 for σ 1→2.	Reproduced — but pre-existing: W~, dirty image and the sparse curvature (4× off) were already stale in that scenario before phase 1; phase 1 adds two more stale scalars.	autoarray/fit/fit_interferometer.py:142; noise_map is a plain attribute	Low. No library or workspace path reassigns noise_map on a sparse dataset (all dataset.noise_map = sites are imaging, pre-operator).
D	check_noise_map_real_imag_equal uses np.allclose default atol=1e-8, so σ_real=1e-9 vs σ_imag=2e-9 passes; sparse curvature then 1.5× dense, log_evidence −2.77.	Reproduced — check predates phase 1 (1ae2f2f3, 2026-08-28); phase 1 moved it into the shared helper	inversion_interferometer_util.py:58,61	Not reachable with Jy-unit data (sdp81 ALMA σ min 6e-5 → tolerance ≈1.7e-4 relative; factor-2 mismatch accepted only below ~1e-8).

Recommended follow-up (not done here — scope decision for the human):

1. B (fix): in AbstractInversion.data_subtracted_dict raise a clear InversionException when self.data is None; add it (and TypeError) to the inversion_plots.py handlers; switch the two workspace scripts to fit.inversion_with_data. PyAutoArray corrective PR + two workspace PRs.
2. D (cheap hardening, same PR as 1): atol=0.0 on both allclose/isclose calls.
3. A (cheap, same PR): cast to complex128 before squaring in apply_sparse_operator, so one-shot == chunked for any input dtype.
4. C: document as unsupported (mutating a dataset after apply_sparse_operator was already wrong) or make noise_map/data setters drop sparse_operator. Judgment call.

None of the four changes the parity or memory claims in this PR's body for complex128 data. Reproduction scripts: session scratchpad repro_{A,B,C,D}.py.

🤖 Generated with Claude Code