Archive MaleCNS research sources, provenance, bibliography and immutable data inventory

This commit is contained in:
Digital Drosophila Archive
2026-09-29 03:38:32 +08:00
commit 3981628ff8
682 changed files with 161475 additions and 0 deletions
@@ -0,0 +1,5 @@
31309546251666ba48de58185a0a738d7e70680c04934e4152c572110c874ee6 phase4z-protocol.md
0a9136c5543531ae5ce76eeef9a512f09b94ed2b96f8b14d2e8f7e6782b7e40d phase4z.py
f03b20f03d4f0eb56d58986c84aebbee689b3c9e6b606d4758b51ad8bd745c03 phase4z.yaml
d2007b312b587cc105f13b3ef68c64fb145e7b30a6addfe789a0558c365a0b36 phase4z_describe.py
988ec5d2316b1eea21676fe4b6641eb7a43d93901e69cda97a857c470b824393 manifest.json
@@ -0,0 +1,24 @@
{
"files": [
{
"bytes": 3169,
"path": "phase4z-protocol.md",
"sha256": "31309546251666ba48de58185a0a738d7e70680c04934e4152c572110c874ee6"
},
{
"bytes": 22251,
"path": "phase4z.py",
"sha256": "0a9136c5543531ae5ce76eeef9a512f09b94ed2b96f8b14d2e8f7e6782b7e40d"
},
{
"bytes": 1591,
"path": "phase4z.yaml",
"sha256": "f03b20f03d4f0eb56d58986c84aebbee689b3c9e6b606d4758b51ad8bd745c03"
},
{
"bytes": 15121,
"path": "phase4z_describe.py",
"sha256": "d2007b312b587cc105f13b3ef68c64fb145e7b30a6addfe789a0558c365a0b36"
}
]
}
@@ -0,0 +1,23 @@
# Phase 4Z: event-resolved PN feedback and input acceptance
This diagnostic stage follows user approval to investigate upstream feedback and delivered-input differences. All historical inputs and model files remain read-only. No learning, gain adjustment, changed thresholds, novel transmitter sign, mask expansion or game interface is permitted.
## Registered scope
Replay six old calibration odors with seed 131101 under the two existing Phase 4Y conditions: frozen reference and diagnostic DPM fast-current-off. This is reproduction of a sealed intervention, not promotion of it. Only that previous in-memory effective-current overlay is allowed. Save outputs in new Phase 4Z directories. Preserve current delay/refractory and request acceptance semantics. Do not force a matched delivered spike train by overriding refractory state.
Observer targets are KC 82312, every actual annotation-ALPN input to it, and ALLN nodes 1197/1381/1511 found on a prior Phase 4W time-supported ancestry. Selection is exploratory and explicitly based on prior outcomes. Do not call it blind validation or unique-cause identification.
Use exact incoming-edge event reconstruction and CPU single-cell replay to verify queue, V/g, natural spikes, output spikes and accepted requests. Conditional omissions retain other measured arrivals and APL; they are not closed-loop whole-brain ablations. Distinguish anatomical recurrence, event-supported recurrence and experimentally demonstrated necessity.
Reference reproduction must match Phase 4Y counts, final V/g, trace and KC events. Frozen-reference whole-brain events must additionally match Phase 4W. Repeat both first-odor conditions and rerun five frozen deterministic reference trials. Abort result generation on any mismatch; do not repair frozen data. Hash history and all new final artifacts.
## Literature context, not parameter authority
- Eckstein et al., Cell 2024, doi:10.1016/j.cell.2024.03.016: DPM dopamine predictions in FlyWire/hemibrain identified as mispredictions. This is not direct ground truth for the two MaleCNS bodies. https://pmc.ncbi.nlm.nih.gov/articles/PMC11106717/
- Haynes et al., eLife 2015, doi:10.7554/eLife.03868: DPM GABA/serotonin-related physiology; does not provide a universal MaleCNS edge-specific current model. https://pmc.ncbi.nlm.nih.gov/articles/PMC4305081/
- Hu et al., PNAS 2010, doi:10.1073/pnas.0914912107: functional mushroom-body-to-antennal-lobe feedback. Existence of such feedback does not prove the current residual loop traverses KC/MBON. https://pubmed.ncbi.nlm.nih.gov/20479249/
- Localized inhibition in the Drosophila mushroom body, 2020: spatially localized APL inhibition. No changes to frozen APL parameters are authorized here. https://pmc.ncbi.nlm.nih.gov/articles/PMC7541083/
- Shiu et al., Nature 2024, doi:10.1038/s41586-024-07763-9: sensorimotor LIF model, not universal validation of associative-memory physiology. https://pmc.ncbi.nlm.nih.gov/articles/PMC11446845/
No slow receptor time constants, biological sign correction or parameter tuning is derived from these citations. Source code SHA-256 serves as this audit's reproducible version identifier.
@@ -0,0 +1,283 @@
"""Event-resolved read-only diagnosis of frozen trials and existing 4Y hypotheses."""
from pathlib import Path
import argparse,json,time
import numpy as np
import pandas as pd
import torch,yaml
import phase4p,phase4p_finalize,phase4s,phase4v,phase4w,phase4y
from phase4p_revalidate import DeterministicAPLSim,door_inputs,make_drive
from phase4r import save,digest,arrhash
from phase4h_audit import verify_checksum
from transmitter_roles_v1 import roles
def checks(cfg):
old=yaml.safe_load((Path(cfg['prior_model'])/'phase4y.yaml').read_text())
out=phase4y.checks(old)
for key in ('prior_model','prior_root','event_experiment'):
p=Path(cfg[key]);out['Z_input_'+key]=verify_checksum(p,p/'checksums.sha256')
assert all(x['passed'] for x in out.values());return out
class Observer(phase4w.Observer):
def __init__(self,rt,watch,requests,targets):
super().__init__(rt,watch)
self.requests=requests;self.targets=np.asarray(targets,np.int32);self.request_rows=[]
def force_step(self,*a):
step=self.rt.step_index;ii=np.flatnonzero(self.requests[step])
if len(ii):
tg=a[4][torch.as_tensor(ii,device=self.rt.device)]
x=torch.stack((a[0][tg],self.rt.g[tg],a[1][tg].float(),a[2][tg].float())).cpu().numpy()
self.ext.force_step(*a)
if len(ii):
after=a[2][tg].cpu().numpy()
for j,k in enumerate(ii):
node=int(self.targets[k]);status='preexisting' if x[3,j] else ('blocked' if x[2,j] else 'accepted')
self.request_rows.append({'step':step,'neuron_idx':node,'status':status,
'pre_force_v':float(x[0,j]),'pre_force_g':float(x[1,j]),'pre_force_ref':int(x[2,j]),
'preexisting_spike':bool(x[3,j]),'actual_output_spike':bool(after[j])})
assert bool(after[j])==(status!='blocked')
if status=='accepted':self.forced.append((step,node))
def role_codes(meta):
x=roles(meta);out=x.role_group.to_numpy(object).copy()
for cls,code in [('ALPN','ALPN'),('ALLN','ALLN'),('Kenyon_Cell','KC'),('MBON','MBON'),('DAN','DAN')]:
out[meta.annotation_class.eq(cls).to_numpy()]=code
out[meta.annotation_type.eq('DPM').to_numpy()]='DPM'
out[meta.annotation_type.eq('APL').to_numpy()]='APL'
return out
def replay(arrival,overlay,requests,rt):
f=np.float32;v=f(rt.rest);g=f(0);ref=0;events=[];states=[];accepted=[]
for step,a in enumerate(arrival):
g=f(f(g*rt.gd)+a);ref=max(ref-1,0)
v=f(v+f(rt.alpha*f(g-f(v-rt.rest)))) if ref==0 else f(rt.resetv)
fire=ref==0 and v>=rt.threshold
if fire:v=f(rt.resetv);ref=rt.ref_steps
states.append((v,g,float(fire)))
if requests[step] and not fire and ref==0:
v=f(rt.resetv);ref=rt.ref_steps;fire=True;accepted.append(step)
if fire:events.append(step)
g=f(np.float64(g)+overlay[step])
return np.asarray(events,np.int32),np.asarray(states,np.float32),v,g,np.asarray(accepted,np.int32)
def summarize_drive(cfg,exp,name,sim,obs,result,edges,codes,request_frame):
rt=sim.rt;state=np.asarray(obs.states);T=len(state);ev=obs.events()
assert np.array_equal(np.bincount(ev.neuron_idx,minlength=sim.n),result['counts'])
arrivals=ev.merge(edges,left_on='neuron_idx',right_on='source_neuron_idx').drop(columns='neuron_idx')
arrivals=arrivals.rename(columns={'step':'emission_step'})
arrivals['arrival_step']=arrivals.emission_step+rt.delay_steps
arrivals=arrivals[arrivals.arrival_step<T].sort_values(['target_neuron_idx','arrival_step','edge_id'],kind='stable')
# Compact event-derived edge accounting retains exact edge IDs; whole events permit full reconstruction.
summaries=[];replays=[];reg=[]
for wi,node in enumerate(obs.watch):
a=arrivals[arrivals.target_neuron_idx==node]
grouped={int(t):g for t,g in a.groupby('arrival_step',sort=False)}
ar=np.zeros(T,np.float32)
for t,g in grouped.items():ar[t]=np.cumsum(g.effective_weight.to_numpy(np.float32),dtype=np.float32)[-1]
assert np.array_equal(ar,state[:,3,wi]),f'arrival mismatch {node}'
after=np.r_[state[1:,1,wi],result['final_g'][node]]
overlay=after.astype(np.float64)-state[:,5,wi].astype(np.float64)
if codes[node]!='KC':assert not np.any(overlay),'unexpected non-KC overlay'
req=np.zeros(T,bool);q=request_frame[request_frame.neuron_idx==node];req[q.step.to_numpy(np.int32)]=True
base=replay(ar,overlay,req,rt)
expected=ev[ev.neuron_idx==node].step.to_numpy(np.int32)
eq={'queue_exact':True,'events_exact':np.array_equal(base[0],expected),
'post_state_exact':np.array_equal(base[1],state[:,4:7,wi]),
'final_v_exact':base[2]==result['final_v'][node],'final_g_exact':base[3]==result['final_g'][node],
'accepted_exact':np.array_equal(base[4],q[q.status=='accepted'].step.to_numpy(np.int32))}
assert all(eq.values()),(name,int(node),eq)
reg.append({'trial':name,'neuron_idx':int(node),**{k:bool(v) for k,v in eq.items()}})
# Predeclared population omissions plus each actual incoming ALPN edge to the target KC.
modes=[('base',None)]+[('omit_'+c,a.source_role.eq(c).to_numpy()) for c in cfg['isolated_omissions']]
if node==cfg['target_KC']:
modes += [('omit_edge_'+str(e),a.edge_id.eq(e).to_numpy()) for e in edges.loc[(edges.target_neuron_idx==node)&(edges.source_role=='ALPN'),'edge_id']]
for mode,omit in modes:
aa=ar
if omit is not None:
aa=np.zeros(T,np.float32)
for t,g in a.loc[~omit].groupby('arrival_step',sort=False):
aa[int(t)]=np.cumsum(g.effective_weight.to_numpy(np.float32),dtype=np.float32)[-1]
rr=base if omit is None else replay(aa,overlay,req,rt)
replays.append({'trial':name,'neuron_idx':int(node),'mode':mode,'spikes':len(rr[0]),
'late_spikes':int(np.count_nonzero(rr[0]>=round(cfg['late_start_ms']/rt.dt))),
'accepted_requests':len(rr[4]),'max_abs_v_diff':float(np.abs(rr[1][:,0].astype(float)-base[1][:,0]).max())})
for period,lo,hi in [('input_on',0,1200),('post_input',1200,T),('late',5000,T)]:
aa=a[(a.arrival_step>=lo)&(a.arrival_step<hi)]
for eid,g in aa.groupby('edge_id',sort=False):
e=g.iloc[0];weight=float(e.effective_weight)
summaries.append({'trial':name,'period':period,'target_neuron_idx':int(node),'edge_id':int(eid),
'source_neuron_idx':int(e.source_neuron_idx),'source_role':str(e.source_role),
'raw_synapse_count':int(e.raw_synapse_count),'effective_weight':weight,'edge_event_count':len(g),
'signed_arrival_sum':len(g)*weight,'positive_arrival_sum':len(g)*max(0.,weight),
'negative_arrival_sum':len(g)*min(0.,weight)})
return ev,reg,replays,summaries
def request_comparison(exp,oi,seed,a,b):
z=a.merge(b,on=['step','neuron_idx'],suffixes=('_reference','_off'),validate='one_to_one',how='outer',indicator=True)
assert z['_merge'].eq('both').all();z=z.drop(columns='_merge')
z['status_changed']=z.status_reference!=z.status_off
z['state_changed']=(z.pre_force_v_reference!=z.pre_force_v_off)|(z.pre_force_g_reference!=z.pre_force_g_off)|(z.pre_force_ref_reference!=z.pre_force_ref_off)
z.to_parquet(exp/'input-audit'/f'{oi}-{seed}-paired-requests.parquet',index=False)
diff=z[z.status_changed];st=z[z.state_changed]
row={'odor_index':oi,'seed':seed,'scheduled_requests':len(z),'status_changed_requests':len(diff),
'first_status_difference_ms':float(diff.step.min()/10) if len(diff) else None,
'first_requested_state_difference_ms':float(st.step.min()/10) if len(st) else None,
'delivered_output_differences':int((z.actual_output_spike_reference!=z.actual_output_spike_off).sum()),
'neurons_with_status_difference':int(diff.neuron_idx.nunique()),'requests_exact':True}
for cond,q in [('reference',a),('off',b)]:
for status in ('accepted','preexisting','blocked'):row[cond+'_'+status]=int(q.status.eq(status).sum())
return row
def run(cfg,exp,model):
for d in ('protocol','audit','events','states','drive','replay','ancestry','input-audit','regression','performance'):(exp/d).mkdir(parents=True,exist_ok=True)
assert not (exp/'protocol/preregistered.json').exists(),'Refusing overwrite'
save(exp/'audit/preflight.json',checks(cfg))
vc=yaml.safe_load((Path(cfg['input_model'])/'phase4v.yaml').read_text())
assert vc['short_duration_ms']==120 and vc['observation_duration_ms']==1000
save(exp/'protocol/preregistered.json',{'config':cfg,'source_hashes':{p.name:digest(p) for p in model.iterdir() if p.is_file()},
'before_simulation':True,'selection_is_exploratory':True,'input_request_clamp_not_attempted':True})
pc=yaml.safe_load((Path(cfg['runtime_root'])/'phase4p.yaml').read_text())
rt=phase4p.CanonicalRuntime(pc,Path(cfg['runtime_root']),exp);sim=DeterministicAPLSim(rt);sim.install(None)
assert sim.n==166700 and len(rt.raw_np)==25582938 and rt.dt==0.1 and rt.delay_steps==18
codes=role_codes(sim.meta);original_install=sim.install;basehash=arrhash(sim.base.cpu().numpy());weights0=rt.weights.cpu().numpy().copy()
target=cfg['target_KC'];incoming=np.flatnonzero((rt.indices_np==target)&(codes[sim.sources]=='ALPN'))
pn=np.unique(sim.sources[incoming]);watch=np.unique(np.r_[target,pn,cfg['local_loop_nodes']]).astype(np.int32)
assert codes[target]=='KC' and all(codes[cfg['local_loop_nodes']]=='ALLN')
sim.meta.iloc[watch].to_parquet(exp/'audit/watched-metadata.parquet',index=False)
save(exp/'protocol/watched.json',{'target_KC':target,'all_incoming_ALPNs':pn.tolist(),'ALLN_from_prior_event_path':cfg['local_loop_nodes'],'all_watched':watch.tolist()})
dpm=np.flatnonzero(sim.meta.annotation_type.eq('DPM'));de=np.flatnonzero(np.isin(sim.sources,dpm));dg=torch.as_tensor(de,device=sim.device)
assert len(dpm)==2 and len(de)==4934
factor=1.
def install(E):
assert E is None;original_install(None);rt.weights[dg]=sim.base[dg]*float(factor)
sim.install=install
ids=np.flatnonzero(np.isin(rt.indices_np,watch))
edge_base=pd.DataFrame({'edge_id':ids,'source_neuron_idx':sim.sources[ids],'target_neuron_idx':rt.indices_np[ids],
'raw_synapse_count':rt.raw_np[ids],'source_role':codes[sim.sources[ids]]})
edge_base.assign(frozen_effective_weight=weights0[ids]).to_parquet(exp/'audit/incoming-edges.parquet',index=False)
sim.meta.iloc[np.unique(sim.sources[ids])].to_parquet(exp/'audit/incoming-source-metadata.parquet',index=False)
csc=(np.argsort(rt.indices_np,kind='stable'),np.r_[0,np.cumsum(np.bincount(rt.indices_np,minlength=sim.n))])
oldcode=phase4w.classify(sim.meta)
oc,panel,vectors,selection,mapping=door_inputs();panel=panel[panel.split=='calibration'].reset_index(drop=True)
targetkc=np.unique(sim.identity.loc[sim.identity.compartment.eq(vc['target_compartment']),'source_neuron_idx'].to_numpy(np.int32))
regs=[];replays=[];drives=[];paths=[];pathsum=[];requests=[];trials=[];inputhash=[]
start=time.perf_counter();torch.cuda.reset_peak_memory_stats()
for oi in cfg['odor_indices']:
seed=cfg['seed'];odor=panel.iloc[oi];targets,rates=make_drive(oc,panel,vectors,selection,mapping,odor.odor_id,'physiological')
forced=phase4v.forcing(sim,targets,rates,vc['seed_namespace']+oi*vc['odor_seed_offset']+seed,vc,'offset')
ff=forced.cpu().numpy();inputhash.append({'odor_index':oi,'forcing':arrhash(ff),'targets_rates':arrhash(targets,rates)})
reqpair={}
for spec in cfg['conditions']:
factor=spec['factor'];condition=spec['name'];name=f'{oi}-{seed}-{condition}'
obs=Observer(rt,watch,ff,targets);obs.attach()
try:r=phase4v.observed(sim,targets,forced,targetkc,vc)
finally:obs.detach()
old=np.load(Path(cfg['prior_root'])/'states'/f'{name}.npz')
eq={k:bool(np.array_equal(old[k],r[k])) for k in ('counts','final_v','final_g','trace')}
oldev=pd.read_parquet(Path(cfg['prior_root'])/'events'/f'{name}.parquet')
eq['KC_events']=arrhash(oldev.step.to_numpy(),oldev.neuron_idx.to_numpy())==arrhash(r['kc_event_step'],r['kc_event_node'])
eq['finite']=r['finite'];assert all(eq.values()),(name,eq)
regs.append({'trial':name,'prior_Y_exact':eq})
rq=pd.DataFrame(obs.request_rows);rq.to_parquet(exp/'input-audit'/f'{name}-requests.parquet',index=False);reqpair[condition]=rq
last=r['samples'][-1]
assert len(rq)==int(ff.sum())==last['PN_requested_cumulative']
for status,field in [('accepted','PN_accepted_cumulative'),('preexisting','PN_preexisting_cumulative'),('blocked','PN_blocked_cumulative')]:
assert int(rq.status.eq(status).sum())==last[field]
weights=rt.weights.cpu().numpy().copy();expected=weights0.copy();expected[de]*=np.float32(factor);assert np.array_equal(weights,expected)
edges=edge_base.assign(effective_weight=weights[ids])
ev,reg,rep,drive=summarize_drive(cfg,exp,name,sim,obs,r,edges,codes,rq)
regs[-1]['CPU_shadow_replay']=reg;replays+=rep;drives+=drive
ev.to_parquet(exp/'events'/f'{name}-whole-brain.parquet',index=False)
np.savez_compressed(exp/'states'/f'{name}-watched.npz',state=np.asarray(obs.states),watch=watch)
if condition=='frozen_reference':
oldfull=pd.read_parquet(Path(cfg['event_experiment'])/'events'/f'{oi}-{seed}-offset-whole-brain.parquet')
regs[-1]['W_full_events_exact']=bool(ev.equals(oldfull));assert regs[-1]['W_full_events_exact']
ancestry_cfg={'watch_neuron_idx':[target],'late_start_ms':cfg['late_start_ms'],'ancestry':cfg['ancestry']}
pr,ps=phase4w.ancestry(ancestry_cfg,sim,ev,obs.forced,csc,weights,oldcode,name)
# Correct orthogonal role labels; historical W category field never used as biological DAN identity.
for row in pr:
row['source_category']=str(codes[row['source_neuron_idx']]);row['target_category']=str(codes[row['target_neuron_idx']])
for item in ps:
selected=[p for p in pr if p['endpoint']==item['endpoint']]
item['categories']=sorted({p['source_category'] for p in selected})
item['repeated_neuron_ids']=len({p['source_neuron_idx'] for p in selected})<len(selected)
paths+=pr;pathsum+=ps
trials.append({'trial':name,'odor_index':oi,'seed':seed,'condition':condition,'whole_spikes':int(r['counts'].astype(np.int64).sum()),
'KC82312_late_spikes':int(((ev.neuron_idx==target)&(ev.step>=5000)).sum()),'finite':r['finite'],'wall_seconds':r['wall_seconds']})
if oi==cfg['repeat_odor_index']:
again=Observer(rt,watch,ff,targets);again.attach()
try:rr=phase4v.observed(sim,targets,forced,targetkc,vc)
finally:again.detach()
repeat={'events':ev.equals(again.events()),'states':np.array_equal(obs.states,again.states),'requests':rq.equals(pd.DataFrame(again.request_rows))}
assert all(repeat.values());regs[-1]['repeat_exact']=repeat
save(exp/'regression/trials.json',regs)
print(json.dumps({'trial_complete':name,'watched_neurons':len(watch),'incoming_edges':len(ids),'CPU_checks':len(reg)}),flush=True)
requests.append(request_comparison(exp,oi,seed,reqpair['frozen_reference'],reqpair['DPM_fast_current_off']))
sim.install=original_install;original_install(None)
assert arrhash(sim.base.cpu().numpy())==basehash and np.array_equal(rt.weights.cpu().numpy(),weights0)
save(exp/'regression/restore.json',{'base_unchanged':True,'APL_overlay_restored':True,'new_weight_hypotheses':False})
assert phase4s.evidence_regression(cfg,exp,rt,sim)
for folder,name,rows in [('audit','trials',trials),('replay','results',replays),('drive','edge-accounting',drives),('ancestry','event-supported-edges',paths),('input-audit','paired-summary',requests)]:
pd.DataFrame(rows).to_parquet(exp/folder/f'{name}.parquet',index=False)
save(exp/'ancestry/summary.json',pathsum);save(exp/'protocol/input-hashes.json',inputhash)
save(exp/'audit/results.json',{'phase':'4Z','status':'COMPLETE-EVENT-RESOLVED-PN-AUDIT','trials':len(trials),'exact_repeat_trials':2,
'Y_reproductions':len(trials),'W_full_event_reproductions':6,'CPU_shadow_replay_checks':sum(len(x['CPU_shadow_replay']) for x in regs),
'reference_pass':True,'all_finite':True,'input_pairs':len(requests),'pairs_with_status_difference':sum(x['status_changed_requests']>0 for x in requests),
'full_brain_loop_necessity_proven':False,'biological_sign_proven':False,'weight_training':False,'candidate':False,'frozen_baseline_changed':False})
save(exp/'performance/summary.json',{'wall_seconds_excluding_build':time.perf_counter()-start,'peak_VRAM_bytes':torch.cuda.max_memory_allocated(),
'GPU':torch.cuda.get_device_name(0),'diagnostic_overhead_included':True})
def describe(cfg,exp):
d=pd.read_parquet(exp/'drive/edge-accounting.parquet');rep=pd.read_parquet(exp/'replay/results.parquet')
q=pd.read_parquet(exp/'input-audit/paired-summary.parquet');meta=pd.read_parquet(exp/'audit/incoming-source-metadata.parquet')
late=d[d.period=='late'].copy();den=late.groupby(['trial','target_neuron_idx']).positive_arrival_sum.transform('sum')
late['positive_fraction']=np.where(den>0,late.positive_arrival_sum/den,np.nan)
late=late.merge(meta[['neuron_idx','body_id','annotation_type','annotation_class','neurotransmitter','nt_ground_truth']],left_on='source_neuron_idx',right_on='neuron_idx',how='left',validate='many_to_one').drop(columns='neuron_idx')
late.to_parquet(exp/'drive/late-input-fractions.parquet',index=False)
lines=['# Phase 4Z — PN feedback and input acceptance audit','',
'Diagnostic only. No training, no new sign hypothesis, no candidate. The two branches reproduce sealed Phase4Y reference and DPM-off conditions.',
'Target and three ALLNs were selected from prior data; this is mechanism exploration, not heldout validation.',
'All source fractions are positive arrival-weight accounting, not causal percentages or voltage attribution.','',
'## Late positive inputs to KC 82312','', '| condition | source idx | body_id | type | mean fraction |','|---|---:|---:|---|---:|']
z=late[late.target_neuron_idx==cfg['target_KC']].copy();z['condition']=z.trial.str.split('-',n=2).str[-1]
for (cond,idx,body,typ),g in z.groupby(['condition','source_neuron_idx','body_id','annotation_type'],dropna=False):
if g.positive_arrival_sum.sum()>0:lines.append(f'| {cond} | {idx} | {body} | {typ} | {g.positive_fraction.mean():.6f} |')
lines+=['','## Conditional single-cell replay','',
'Other recorded arrivals and measured APL remain fixed; original force requests and refractory logic retained. This does not test whole-brain loop necessity.',
'| neuron_idx | condition | mode | late spike range |','|---:|---|---|---:|']
rep['condition']=rep.trial.str.split('-',n=2).str[-1]
for (node,cond,mode),g in rep.groupby(['neuron_idx','condition','mode']):
if mode=='base' or node==cfg['target_KC'] or (node in cfg['local_loop_nodes'] and mode=='omit_ALLN'):
lines.append(f'| {node} | {cond} | {mode} | {g.late_spikes.min()}–{g.late_spikes.max()} |')
lines+=['','## Input acceptance','', '| odor | requests | accepted reference | accepted off | status differences | first difference ms |','|---:|---:|---:|---:|---:|---:|']
for r in q.itertuples():lines.append(f'| {r.odor_index} | {r.scheduled_requests} | {r.reference_accepted} | {r.off_accepted} | {r.status_changed_requests} | {r.first_status_difference_ms} |')
lines+=['','Identical requests do not establish matched delivered inputs. Refusal and preexisting spikes depend on endogenous state. No forcing clamp or altered refractory rule was introduced.',
'Ancestry edges require actual emission +18 steps <= downstream spike; greedy ranking is not a unique or sufficient causal path. Repeated neuron IDs indicate a time-supported recurrent motif, not proof it is the only sustaining loop.',
'Frozen LIF/APL, raw graph/counts, eligibility and masks unchanged. All detailed state/event checks are in regression/.']
(exp/'REPORT.md').write_text('\n'.join(lines)+'\n',encoding='utf-8')
def seal(cfg,exp,model):
assert not (exp/'checksums.sha256').exists(),'Already sealed'
describe(cfg,exp)
now=checks(cfg);hist=phase4p_finalize.integrity();pre=json.loads((exp/'protocol/preregistered.json').read_text())
ok=hist['all_static_checks_passed'] and now==json.loads((exp/'audit/preflight.json').read_text()) and all(digest(model/n)==h for n,h in pre['source_hashes'].items())
save(exp/'audit/frozen-integrity.json',{'pass':ok,'history':hist,'inputs':now});assert ok
r=json.loads((exp/'audit/results.json').read_text());r['integrity_pass']=True;save(exp/'audit/results.json',r)
for root in (model,exp):
rows=[{'path':p.relative_to(root).as_posix(),'sha256':digest(p),'bytes':p.stat().st_size} for p in sorted(root.rglob('*')) if p.is_file() and p.name not in ('manifest.json','checksums.sha256')]
save(root/'manifest.json',{'files':rows});rows.append({'path':'manifest.json','sha256':digest(root/'manifest.json')})
(root/'checksums.sha256').write_text(''.join(x['sha256']+' '+x['path']+'\n' for x in rows))
check=verify_checksum(root,root/'checksums.sha256');assert check['passed'];print(json.dumps(check),flush=True)
if __name__=='__main__':
p=argparse.ArgumentParser();p.add_argument('task',choices=['run','seal']);p.add_argument('--config',required=True);a=p.parse_args()
cfg=yaml.safe_load(Path(a.config).read_text());(run if a.task=='run' else seal)(cfg,Path(cfg['experiment_root']),Path(cfg['model_root']))
@@ -0,0 +1,32 @@
phase: 4Z
model_root: /data/digital-drosophila/models/male-cns-lif/pn-feedback-audit-v1
experiment_root: /data/digital-drosophila/experiments/phase4z-pn-feedback-audit
prior_model: /data/digital-drosophila/models/male-cns-lif/dpm-hypothesis-v1
prior_root: /data/digital-drosophila/experiments/phase4y-dpm-hypothesis
runtime_root: /data/digital-drosophila/models/male-cns-lif/deterministic-runtime-v2
phase4p_root: /data/digital-drosophila/experiments/phase4p-deterministic-runtime
input_model: /data/digital-drosophila/models/male-cns-lif/csus-audit-v1
input_experiment: /data/digital-drosophila/experiments/phase4v-csus-semantics
event_experiment: /data/digital-drosophila/experiments/phase4w-late-kc-feedback
odor_indices: [0, 1, 2, 3, 4, 5]
seed: 131101
target_KC: 82312
local_loop_nodes: [1197, 1381, 1511]
selection: target is remaining Phase4Y persistent KC; all seven actual incoming ALPNs plus three ALLN nodes in sealed Phase4W first-late ancestry; exploratory diagnostic, not heldout evidence
conditions:
- {name: frozen_reference, factor: 1.0}
- {name: DPM_fast_current_off, factor: 0.0}
late_start_ms: 500
repeat_odor_index: 0
ancestry: {max_hops: 512}
isolated_omissions: [ALPN, ALLN, KC, DPM, DAN]
scope:
new_weight_hypotheses: false
previous_4Y_conditions_only: true
frozen_dynamics_changed: false
new_inputs: false
training: false
candidate: false
game_or_RL: false
conditional_replay: all other recorded arrivals and measured APL fixed; original request schedule and force semantics retained; not full-brain intervention
no_claim_of_unique_cause: true
@@ -0,0 +1,150 @@
"""Post-run descriptive audit only; no selection, simulation or model mutation."""
from pathlib import Path
import json,hashlib,importlib.util
import numpy as np
import pandas as pd
ROOT=Path('/data/digital-drosophila')
exp=ROOT/'experiments/phase4z-pn-feedback-audit'
assert not (exp/'checksums.sha256').exists(),'Sealed output is immutable'
(exp/'audit/descriptive-analysis-provenance.json').write_text(json.dumps({
'source':str(Path(__file__).resolve()),'sha256':hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
'post_run_descriptive_analysis':True,'not_part_of_simulation_preregistration':True,
'new_simulations_or_selection':False},indent=2)+'\n')
meta=pd.read_parquet(exp/'audit/incoming-source-metadata.parquet').set_index('neuron_idx')
watch=pd.read_parquet(exp/'audit/watched-metadata.parquet').set_index('neuron_idx')
d=pd.read_parquet(exp/'drive/edge-accounting.parquet');d=d[d.period=='late'].copy()
d['condition']=d.trial.str.split('-',n=2).str[-1]
rep=pd.read_parquet(exp/'replay/results.parquet');rep['condition']=rep.trial.str.split('-',n=2).str[-1]
path=pd.read_parquet(exp/'ancestry/event-supported-edges.parquet')
q=pd.read_parquet(exp/'input-audit/paired-summary.parquet')
current=json.loads((exp/'protocol/input-hashes.json').read_text())
prior=json.loads((ROOT/'experiments/phase4y-dpm-hypothesis/protocol/input-hashes.json').read_text())
old={x['odor']:x for x in prior if x['seed']==131101}
input_checks=[{'odor_index':x['odor_index'],
'forcing_hash_exact':x['forcing']==old[x['odor_index']]['forcing_hash'],
'targets_rates_hash_exact':x['targets_rates']==old[x['odor_index']]['targets_rates_hash']} for x in current]
assert all(x['forcing_hash_exact'] and x['targets_rates_hash_exact'] for x in input_checks)
(exp/'regression/input-hashes-vs-Y.json').write_text(json.dumps({'pass':True,'checks':input_checks},indent=2)+'\n')
rows=[]
for (trial,node),g in d.groupby(['trial','target_neuron_idx']):
den=g.positive_arrival_sum.sum()
for role,z in g.groupby('source_role'):
rows.append({'trial':trial,'target_neuron_idx':node,'source_role':role,
'positive_fraction':float(z.positive_arrival_sum.sum()/den) if den else None,
'positive_arrival_sum':float(z.positive_arrival_sum.sum()),'negative_arrival_sum':float(z.negative_arrival_sum.sum())})
pd.DataFrame(rows).to_parquet(exp/'drive/late-population-accounting.parquet',index=False)
top=[];timing=[];transitions=[]
for trial,g in d[d.target_neuron_idx==82312].groupby('trial'):
a=g[g.source_role=='ALPN'].sort_values(['positive_arrival_sum','edge_id'],ascending=[False,True]).iloc[0]
top.append({'trial':trial,'source_neuron_idx':int(a.source_neuron_idx),'body_id':int(meta.loc[a.source_neuron_idx,'body_id']),
'type':str(meta.loc[a.source_neuron_idx,'annotation_type']),'edge_id':int(a.edge_id),
'positive_fraction':float(a.positive_arrival_sum/g.positive_arrival_sum.sum()),'edge_event_count':int(a.edge_event_count)})
pd.DataFrame(top).to_parquet(exp/'drive/dominant-late-PN.parquet',index=False)
motifs=[]
for trial,g in path[path.endpoint=='first_late'].groupby('trial'):
g=g.sort_values('backward_hop')
pairs=g.groupby(['source_neuron_idx','target_neuron_idx']).size()
for (src,dst),n in pairs.items():
if (dst,src) in pairs.index:
motifs.append({'trial':trial,'source_neuron_idx':int(src),'target_neuron_idx':int(dst),
'forward_occurrences_on_ancestry':int(n),'reverse_occurrences_on_ancestry':int(pairs.loc[(dst,src)]),
'claim':'actual time-supported reciprocal motif on ranked path, not unique necessity'})
pd.DataFrame(motifs).to_parquet(exp/'ancestry/reciprocal-event-motifs.parquet',index=False)
loop=path[(path.endpoint=='first_late')&path.source_neuron_idx.isin([1381,1511])&path.target_neuron_idx.isin([1381,1511])]
for trial,g in loop.groupby('trial'):
timing.append({'trial':trial,'recurrent_edge_events_on_path':len(g),
'first_emission_ms':float(g.source_step.min()/10),'last_arrival_ms':float(g.arrival_step.max()/10),
'all_delays_exact':bool(((g.arrival_step-g.source_step)==18).all()),
'all_arrivals_precede_target':bool((g.arrival_step<=g.target_step).all())})
pd.DataFrame(timing).to_parquet(exp/'ancestry/loop-timing.parquet',index=False)
for r in q.itertuples():
z=pd.read_parquet(exp/'input-audit'/f'{r.odor_index}-{r.seed}-paired-requests.parquet')
diff=z[z.status_changed]
for (before,after),g in diff.groupby(['status_reference','status_off']):
transitions.append({'odor_index':r.odor_index,'reference_status':before,'off_status':after,'count':len(g)})
pd.DataFrame(transitions).to_parquet(exp/'input-audit/status-transitions.parquet',index=False)
isirows=[]
for f in sorted((exp/'states').glob('*-watched.npz')):
trial=f.name.removesuffix('-watched.npz');z=np.load(f);st=z['state'];ids=z['watch']
for j,node in enumerate(ids):
steps=np.flatnonzero(st[5000:,6,j])+5000;isi=np.diff(steps)
isirows.append({'trial':trial,'neuron_idx':int(node),'late_natural_spikes':len(steps),
'late_rate_hz':2*len(steps),'ISI_min_ms':float(isi.min()/10) if len(isi) else None,
'ISI_mean_ms':float(isi.mean()/10) if len(isi) else None,'ISI_max_ms':float(isi.max()/10) if len(isi) else None,
'ISI_equal_refractory_fraction':float(np.mean(isi==22)) if len(isi) else None,
'late_post_g_mean':float(st[5000:,5,j].mean(dtype=np.float64)),
'late_post_g_min':float(st[5000:,5,j].min()),'late_post_g_max':float(st[5000:,5,j].max())})
isidf=pd.DataFrame(isirows);isidf.to_parquet(exp/'drive/late-ISI-and-g.parquet',index=False)
# Read-only coverage audit of the frozen graded APL replacement, not an intervention.
canonical=ROOT/'datasets/male-cns-v1.0/processed/neuron-only'
fullmeta=pd.read_parquet(canonical/'neuron-index/unified-metadata.parquet').sort_values('neuron_idx').reset_index(drop=True)
assert np.array_equal(fullmeta.neuron_idx.to_numpy(),np.arange(166700))
ptr=np.load(canonical/'gpu/csr-indptr.int64.npy',mmap_mode='r')
dst=np.load(canonical/'gpu/csr-indices.int32.npy',mmap_mode='r')
raw=np.load(canonical/'gpu/csr-weights.int64.npy',mmap_mode='r')
aplids=np.flatnonzero(fullmeta.annotation_type.eq('APL'));assert len(aplids)==2
aplrows=[]
for src in aplids:
for eid in range(int(ptr[src]),int(ptr[src+1])):
node=int(dst[eid]);m=fullmeta.iloc[node]
aplrows.append({'edge_id':eid,'source_neuron_idx':int(src),'source_body_id':int(fullmeta.iloc[src].body_id),
'target_neuron_idx':node,'target_body_id':int(m.body_id),'target_class':m.annotation_class,
'target_type':m.annotation_type,'raw_synapse_count':int(raw[eid]),
'frozen_APL_fast_edge_zeroed':True,'graded_replacement_target':bool(m.annotation_class=='Kenyon_Cell')})
apldf=pd.DataFrame(aplrows);apldf.to_parquet(exp/'audit/APL-target-coverage.parquet',index=False)
omitted=apldf[~apldf.graded_replacement_target]
focus=omitted[omitted.target_neuron_idx.isin(watch[watch.annotation_class=='ALPN'].index)]
spec=importlib.util.find_spec('phase4p_revalidate');assert spec and spec.origin
srcfile=Path(spec.origin);assert srcfile.exists()
aplsummary={'scope':'static coverage of frozen DeterministicAPLSim.install and graded output matrix; not proof of functional effect',
'APL_neurons':len(aplids),'all_outgoing_edges':len(apldf),'all_outgoing_synapse_count':int(apldf.raw_synapse_count.sum()),
'non_KC_edges_without_graded_replacement':len(omitted),'non_KC_synapse_count_without_graded_replacement':int(omitted.raw_synapse_count.sum()),
'non_KC_target_neurons':int(omitted.target_neuron_idx.nunique()),'watched_PN_edges':len(focus),
'watched_PN_synapse_count':int(focus.raw_synapse_count.sum()),'source':str(srcfile),
'source_sha256':hashlib.sha256(srcfile.read_bytes()).hexdigest(),'baseline_not_modified':True,
'cannot_replace_terminal_inhibition_with_whole_PN_inhibition_without_new_hypothesis':True}
(exp/'audit/APL-target-coverage.json').write_text(json.dumps(aplsummary,indent=2)+'\n')
lines=['# Phase 4Z — mechanism interpretation','',
'Post-run descriptive analysis, not a new candidate gate. No tuning, learning, new stimulus or sign hypothesis.',
'The earlier generic label PN feedback is refined below using actual ALLN cell identities. Do not assume the residual loop is KC→MBON→PN feedback merely because that pathway exists in the literature.','',
'## Watched identities','', '| neuron_idx | body_id | class | type | metadata transmitter | raw ground_truth |','|---:|---:|---|---|---|---|']
for idx,r in watch.iterrows():lines.append(f'| {idx} | {r.body_id} | {r.annotation_class} | {r.annotation_type} | {r.neurotransmitter} | {r.nt_ground_truth} |')
lines+=['','## Representative first-late ancestry','',
'Odor 0 / seed 131101 / frozen reference. Ordered source spike → edge arrival → target spike (ms). These are ranked event-supported contributions; no unique predecessor assertion.',
'| source idx/type | target idx/type | source time | arrival time | target time | contacts | effective weight |','|---|---|---:|---:|---:|---:|---:|']
example=path[(path.trial=='0-131101-frozen_reference')&(path.endpoint=='first_late')].sort_values('backward_hop').head(5).iloc[::-1]
for r in example.itertuples():lines.append(f'| {r.source_neuron_idx}/{r.source_type} | {r.target_neuron_idx}/{r.target_type} | {r.source_step/10:.1f} | {r.arrival_step/10:.1f} | {r.target_step/10:.1f} | {r.raw_synapse_count} | {r.effective_weight:.6f} |')
lines+=['','## Dominant PN arrival contribution','', '| trial | PN idx/type | positive arrival fraction |','|---|---|---:|']
for r in top:lines.append(f'| {r["trial"]} | {r["source_neuron_idx"]}/{r["type"]} | {100*r["positive_fraction"]:.3f}% |')
lines+=['','Fractions are sums of positive synaptic arrivals, not percentages of causal responsibility. Negative input, recorded APL and membrane dynamics are retained in exact CPU replay.',
f'The preidentified 1381↔1511 event motif occurs in {len(timing)}/12 first-late ancestry paths. Full temporal edges and alternative inputs are retained.',
'Conditional omission results are in replay/results.parquet and REPORT.md. They retain other source events; loss of spikes there is not evidence that a whole-brain ablation would produce the same result.',
'Critically, omitting ALLN arrivals to a watched ALLN does not silence it in conditional replay: recorded PN and other inputs can still sustain high firing. Thus the 1381↔1511 motif cannot be called the sole cause or an isolated autonomous oscillator.',
'Removing the existing VA6_adPN→KC82312 edge in conditional replay eliminates target late firing; removing ALLN input to VA6_adPN also eliminates its late firing. These are two conditional links, not a demonstrated whole-brain necessity chain.','',
'## Late firing of preidentified ALLNs','', '| neuron_idx | late rate range Hz | min/max ISI ms | fraction ISI = refractory, range |','|---:|---:|---:|---:|']
for node,g in isidf[isidf.neuron_idx.isin([1197,1381,1511])].groupby('neuron_idx'):
lines.append(f'| {node} | {g.late_rate_hz.min()}–{g.late_rate_hz.max()} | {g.ISI_min_ms.min():.2f}/{g.ISI_max_ms.max():.2f} | {g.ISI_equal_refractory_fraction.min():.5f}–{g.ISI_equal_refractory_fraction.max():.5f} |')
lines+=['','This is a measured property of the mathematical model, not a validated biological firing rate. Window-boundary counting can put spikes/window slightly above 1/refractory. No new biological runaway threshold is inferred.','',
'## Frozen APL replacement target coverage','',
f'The frozen implementation zeroes all {len(apldf)} APL fast outgoing edges and reintroduces graded inhibition only on KC targets. {len(omitted)} non-KC edges ({int(omitted.raw_synapse_count.sum())} anatomical contacts) have no direct graded replacement in this implementation.',
f'This includes {len(focus)} edges / {int(focus.raw_synapse_count.sum())} anatomical contacts onto the seven watched PNs. Raw edges and counts remain intact. These edges were not removed by this audit.',
'This is an implementation coverage gap relative to the anatomical graph, not evidence that restoring a whole-neuron negative current would be biologically correct. APL inhibition of PN boutons is spatial/terminal-specific and a point-neuron approximation may not capture it.',
'Primary study: Prisco et al., eLife 10:e74172 (2021), https://pubmed.ncbi.nlm.nih.gov/34964714/ . It reports reciprocal APL inhibition targeting PN boutons and KC dendrites; it does not provide these MaleCNS edge-specific effective parameters.',
'No frozen APL parameter, edge, sign or output target was modified. Functional necessity of this coverage gap remains untested.','',
'## What input matching means','',
f'All six paired request schedules match. Status differs for {int(q.status_changed_requests.sum())} requests in total; actual output at requested times differs for {int(q.delivered_output_differences.sum())} requests.',
'An accepted→preexisting transition changes the accepted-new count without removing the output spike. A blocked transition can change actual delivery. Thus accepted-new totals alone are insufficient to describe input matching.',
'The first differing request is state dependent; this is an endogenous response to an existing intervention, not evidence of different external RNG. No clamp, injected substitute events or refractory override was introduced.',
'The earlier closed-loop intervention remains interpretable as a total system effect under matched requests. It is not an estimate of a direct DPM effect holding actual PN output trains fixed.','',
'## Literature boundary','',
'Excitatory local antennal-lobe circuits and reciprocal local-neuron/PN excitation have experimental support. They must not automatically be relabeled inhibitory because the model has persistent activity.',
'Shang et al., Cell 2007: https://pubmed.ncbi.nlm.nih.gov/17289577/',
'Huang et al., Neuron 2010, functional connectivity of excitatory local interneurons: https://doi.org/10.1016/j.neuron.2010.08.025',
'These experiments do not validate uniform current weights, this MaleCNS individual, or its simulated persistent firing. No receptor, adaptation or electrical-synapse parameters are inferred or inserted.',
'Next priority is independent assessment of local AL-loop dynamics and input-interface causal controls, not changing dopamine learning rate. No official model replacement is justified.']
(exp/'audit/MECHANISM.md').write_text('\n'.join(lines)+'\n',encoding='utf-8')
print(json.dumps({'dominant_PNs':sorted({r['source_neuron_idx'] for r in top}),
'dominant_PN_positive_fraction_range':[min(r['positive_fraction'] for r in top),max(r['positive_fraction'] for r in top)],
'trials_with_preidentified_reciprocal_motif':len(timing),
'status_changed_requests':int(q.status_changed_requests.sum()),'delivered_output_differences':int(q.delivered_output_differences.sum())}),flush=True)