FlyHack: Stress-Testing Mechanistic Claims in a Connectome-Constrained Visual Model

Aashish Dhawan

Connectome-constrained neural networks make biological components directly accessible to intervention, but their mechanistic conclusions can still depend on the comparison model, intervention, circuit representation and trained solution. We stress-test these dependencies in flyvis, a \emph{Drosophila} motion-vision model, using its 50-model ensemble, task-matched causal CNN controls and replicated interventions. At the level of model behaviour, three-level reverse-phi reversal in flyvis is strongly period- and velocity-dependent and also appears in task-matched CNNs; neither family reproduces the fly's stationary-sawtooth response. Adversarial direction reversal requires a perturbation bound of 0.21--0.27 times stimulus contrast in flyvis versus 0.13--0.25 in CNNs, but the gap over a parameter-matched CNN largely closes for larger CNNs. At the level of mechanism, matching attribution to the intervention matters: for held-out-mean replacement, first-order edge attribution patching reaches pooled AUROC 0.970 and Spearman 0.872, versus 0.933 and 0.126 when its zero-aligned score is evaluated against the same ground truth. Circuit conclusions also vary with representation and…