feat(plot): interferometer plots on array-free datasets — natural-weighted dirty panels, inversion subplot fix (streaming phase 3)

[Jammy2211]

Summary

Streaming phase 3 (Mind epic streaming-visibilities; https://github.com/orgs/PyAutoLabs/discussions/13). Closes #596. Siblings: PyAutoLabs/PyAutoGalaxy#640, PyAutoLabs/PyAutoLens#761.

The go/no-go benchmark on the epic ledger settled why this continues: the in-memory sparse path runs out of memory at ~1e6 visibilities on a 16 GB laptop (its one-shot NUFFT precision build allocates ~3 kB per visibility), while from_stream stays at 1.5–1.8 GB to 5e7 and reaches 2e8 in ~40 min at chunk 65536. A streamed fit was usable only with visualization off; this PR makes the interferometer plots work on an array-free dataset, branching in place so filenames and call sites are unchanged and in-memory output is untouched.

• aa.fit.fit_interferometer.dirty_model_image_natural_from(dataset, image) — the natural-weighted dirty image of a model image, Re(Fᴴ W F m)/Σw as one W̃ multiply on the real-space grid via sparse_operator.operated_matrix_slim_from; Σw from sparse_terms.sum_weights or the noise map. On an in-memory dataset it matches the transformer path at rel 3e-14 (witness) / 1e-10 (test).
• Dataset plots: subplot_interferometer_dataset / subplot_interferometer_dirty_images draw a 1×2 "Dirty Image (Natural)" / "Dirty Beam (Natural)" on array-free datasets (same filenames); fits_interferometer guards the data reads and writes DIRTY_IMAGE_NATURAL / DIRTY_BEAM extensions on array-free.
• Fit plots: subplot_fit_interferometer / subplot_fit_interferometer_dirty_images take an optional model_image and draw the natural dirty image / model / residual on array-free datasets; visibility-space panels are skipped, not approximated.
• Inversion plots (live bug, in-memory too when a transformer is absent): _recon_array read mapped_reconstructed_operated_data_dict first, which raises InversionException on an array-free interferometer inversion, and the handlers caught only (AttributeError, KeyError), so subplot_of_mapper / subplot_mappings escaped. Array-free interferometer inversions (_is_array_free_interferometer: the class has a transformer attribute and it is None — imaging inversions are untouched) now read mapped_reconstructed_data_dict; exc.InversionException is caught in both handlers.

Witness (1e5 streamed NUFFT visibilities, 40×40 mask, pixelization-only, af.m.MockSearch with save_for_aggregator, visualization ON): ag writes 7 files (dataset.fits, dataset.png, fit.png, fit_dirty_images.fits, galaxies.png, galaxy_images.fits, inversion_0_0.png) on the array-free dataset and the same 7 in memory (natural vs classic EXTNAMEs); al writes 10 each. In-memory fit_dirty_images.fits arrays equal the fit's dirty_* exactly.

API Changes

Additive. New autoarray.fit.fit_interferometer.dirty_model_image_natural_from; subplot_fit_interferometer / subplot_fit_interferometer_dirty_images gain model_image=None; array-free datasets get a 1×2 natural dataset subplot and natural FITS extensions under the same filenames; subplot_of_mapper / subplot_mappings no longer raise on an array-free interferometer inversion. In-memory datasets and imaging inversions are byte-unchanged.
See full details below.

Test Plan

• pytest test_autoarray — 1898 passed
• New: array-free dataset subplots + fits EXTNAMEs; array-free fit subplots with/without model_image; subplot_of_mapper / subplot_mappings on an array-free inversion (titles recorded; red with _is_array_free_interferometer forced False); dirty_model_image_natural_from vs transformer path at rel 1e-10
• Downstream suites on the same-named branches: test_autogalaxy 1293 passed, test_autolens 774 passed + 1 xfailed
• CI green on unittest 3.12 / 3.13 / nojax

Heart RED override (development only)

Heart verdict at ship: RED (2026-09-30T18:32Z): release validation FAILED (stage integrate) (unrelated release-integrate leg) plus two canonical-checkout drift reasons from another session (PyAutoLens: on branch feature/point-solver-padding-backend (not main), PyAutoLens: 2 uncommitted source change(s)) — none caused by these branches. Live human authorization in-session 2026-09-30 for issue #596 and the three feature/streaming-p3-visualizer branches ("Authorize override for #596"): commit, push and the pending-release PRs only. Branch gates: test_autoarray 1898, test_autogalaxy 1293, test_autolens 776 + 1 xfailed; red-checks on the inversion-plot fix and the array-free tests; Codex (gpt-6-astra) review: FINDINGS (1): model-image composition for mixed galaxies, fixed in-branch + red-checked. Merge needs its own explicit human command with every check green.

Full API Changes (for automation & release notes)

Added

• autoarray.fit.fit_interferometer.dirty_model_image_natural_from(dataset, image) -> Array2D — natural-weighted dirty model image W̃·m/Σw (needs a sparse_operator)

Changed Signature

• autoarray.dataset.plot.interferometer_plots unchanged signatures; autoarray.fit.plot.fit_interferometer_plots.subplot_fit_interferometer(..., model_image=None) and subplot_fit_interferometer_dirty_images(..., model_image=None)

Changed Behaviour

• subplot_interferometer_dataset / subplot_interferometer_dirty_images — on dataset.is_array_free, a 1×2 natural dirty image / beam under the same filename
• fits_interferometer — guards data; writes DIRTY_IMAGE_NATURAL / DIRTY_BEAM on array-free
• subplot_fit_interferometer / subplot_fit_interferometer_dirty_images — natural panels on array-free (model/residual when model_image given), visibility panels skipped
• inversion_plots._recon_array — reads mapped_reconstructed_data_dict for array-free interferometer inversions; subplot_of_mapper / subplot_mappings catch exc.InversionException

Migration

• None.

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JZZksyZ8LTA4LLxoZjQMNF

[Jammy2211]

Independent review (Codex gpt-6-astra) of the three branches before PR-open

• P2 — Natural model/residual images omit ordinary light in mixed-component galaxies. autogalaxy/interferometer/fit_interferometer.py:438, autolens/interferometer/fit_interferometer.py:356.

  On an in-memory dataset after apply_sparse_operator(), put an ordinary Sersic and a linear Gaussian—or pixelization—in the same galaxy. galaxy_image_dict replaces that galaxy’s ordinary image with its linear reconstruction. Summing its values therefore omits the ordinary component: dirty_model_image_natural is too small, and dirty_residual_map_natural contains the omitted component’s dirty image.

  Classification: the dictionary overwrite is pre-existing; using it as the complete model introduces this defect into the new properties. Assemble the model from profile_image plus inversion.mapped_reconstructed_data.

  The added tests miss this: they use pixelization-only models and construct their Fourier reference from model_image_natural itself, rather than independently from fit.model_data.

FINDINGS (1)

Disposition: FINDINGS (1) — model_image_natural summed galaxy_image_dict, whose entries are overwritten by a galaxy's linear reconstruction, so a galaxy with both ordinary and linear light lost its ordinary component (reproduced: rel error 1.0 in ag, 1.14/1.31 in al); fixed in-branch as profile_image + inversion.mapped_reconstructed_data, with mixed-galaxy tests against an independent transformer reference from fit.model_data (rel ≤ 2e-14), red-checked. Test names: test__natural_dirty_images__galaxy_with_ordinary_and_linear_light__matches_model_data[linear_light_profile|pixelization] in both repos.

🤖 Generated with Claude Code