Seal Phase 4AH conditional KC drive audit and source accounting
This commit is contained in:
Digital Drosophila Archive
@@ -0,0 +1,71 @@
|
||||
# Phase 4AH — sealed-trajectory KC drive accounting
|
||||
|
||||
This diagnostic was specified after Phase 4AG's fixed CAL failed. It is not a
|
||||
new hypothesis trial, a preregistered discovery, a new brain simulation, a
|
||||
parameter search, a held-out test, or training. All observations are conditional
|
||||
on the existing logged trajectories. A priori here means before this accounting
|
||||
is computed, not before the original CAL outcomes were known.
|
||||
|
||||
## Scope and fixed inputs
|
||||
|
||||
Read all eight Phase 4AG-r2 fixture r0 records and all 60 CAL r0 plus 12 first-seed
|
||||
r1 records. No outcome-based subset choice. Read actual static canonical metadata,
|
||||
all real incoming edges of the relevant KC, the exact frozen coefficients and SI
|
||||
weights, requested and actual events, recorded float32 APL inhibition, and final
|
||||
states. Keep all previous artifacts read-only. Do not import a GPU runtime or
|
||||
advance a new whole-brain state. Use at most four CPU threads. No external data
|
||||
download is required.
|
||||
|
||||
## Exact target-state reconstruction
|
||||
|
||||
Compute KC spikes independently, never force the logged KC spike/reset sequence.
|
||||
Upstream arrivals use the logged whole-brain source events at exactly t-18.
|
||||
At each tick decrement refractory, update available V/g in the exact separate
|
||||
float64 operation order, apply accepted request override, strict threshold,
|
||||
reset and ref22, then sequential canonical-order incoming edges only if post-reset
|
||||
ref==0. Finally widen stored float32 h to float64, multiply by float64 .001 and
|
||||
subtract unconditionally. In particular preserve the stored reset coefficient
|
||||
bits; do not substitute hand-typed -0.052 or recompute exponentials.
|
||||
|
||||
Verify all target spikes/counts and final V/g/ref byte-for-byte. For fixtures also
|
||||
verify saved pre-APL and post-APL dense target histories at every tick. Full CAL
|
||||
has whole-brain hashes only: these cannot verify each-tick KC subset states.
|
||||
Never claim all-tick independently observed CAL state equivalence. On any failure,
|
||||
save diagnostic failure and stop without adjusting tolerances or publishing a
|
||||
mechanistic PASS.
|
||||
|
||||
## Accounting (not counterfactual causality)
|
||||
|
||||
Per KC and source class report actual due/accepted/rejected edges, original
|
||||
contact counts, nominal signed SI increments (positive and negative separately),
|
||||
and zeroed-APL arrivals. Keep actual anatomical population and dopamine annotation
|
||||
distinct; unavailable metadata is unknown, not a guessed identity.
|
||||
|
||||
At each tick ledger the actual g changes due to available-state decay, reset,
|
||||
sequential arrivals and graded APL write. Sum these differences in extended
|
||||
precision only as a non-feedback accounting device; report reconstruction residual
|
||||
and an explicit roundoff bound. The actual replay always stays float64.
|
||||
Nominal per-source increments do not exactly sum to applied floating increments;
|
||||
report that rounding residual rather than altering the source labels or dynamics.
|
||||
|
||||
Record APL writes during ref versus available state, pre-first-spike APL writes,
|
||||
negative-g exposure, threshold margin, PN input acceptance, and active versus
|
||||
silent KC summaries. KC-active/silent strata are descriptive and outcome-defined,
|
||||
not an experimental comparison. Cumulative increments and state-time integrals
|
||||
must retain distinct units and must not be called instantaneous conductance or
|
||||
a causal percentage of spike production.
|
||||
|
||||
## Gate and interpretation
|
||||
|
||||
Gate: all 80 prescribed records reconstruct exactly under the stated observable
|
||||
scope, inputs/checksums stay unchanged, all accounting is finite, and no missing
|
||||
records are silently skipped. Repeated-record metrics must agree excluding wall
|
||||
time. Pure adversarial tests are required before registration.
|
||||
|
||||
Passing is PASS-CONDITIONAL-KC-DRIVE-AUDIT, not a learning/candidate gate. Do not
|
||||
infer that removing APL would restore sparse/separable coding, that refractory
|
||||
APL accumulation is the cause of CAL failure, or that a larger drive is justified.
|
||||
An actual counterfactual would require a separately declared new-model hypothesis.
|
||||
No gain/threshold/eta changes, new edges, mask expansion, held-out tuning, rewards,
|
||||
games, decoder, visual input or Web. Preserve failed records. New output budget
|
||||
2 GiB; source and experiment get SHA-256 manifests at completion.
|
||||
@@ -0,0 +1,197 @@
|
||||
"""Non-feedback drive ledger. None of these summaries affect the replay."""
|
||||
import numpy as np
|
||||
|
||||
|
||||
def bit_equal(a, b):
|
||||
a, b = np.asarray(a), np.asarray(b)
|
||||
return a.dtype == b.dtype and a.shape == b.shape and np.array_equal(
|
||||
np.ascontiguousarray(a).view(np.uint8), np.ascontiguousarray(b).view(np.uint8))
|
||||
|
||||
|
||||
def fraction(numerator, denominator):
|
||||
return None if denominator == 0 else float(numerator / denominator)
|
||||
|
||||
|
||||
class DriveAccounting:
|
||||
def __init__(self, target_globals, incoming, raw_contacts, source_groups, group_names,
|
||||
expected_events, coeff, expected_counts, histories=None):
|
||||
self.targets = target_globals
|
||||
self.incoming = incoming
|
||||
self.raw = raw_contacts
|
||||
self.groups = source_groups
|
||||
self.names = group_names
|
||||
self.coeff = coeff
|
||||
self.expected_counts = expected_counts
|
||||
self.active = expected_counts > 0
|
||||
self.histories = histories or {}
|
||||
self.expected = {}
|
||||
for tick in np.unique(expected_events['step']):
|
||||
self.expected[int(tick)] = expected_events['neuron_idx'][expected_events['step'] == tick]
|
||||
n = len(target_globals)
|
||||
self.n = n
|
||||
self.first = np.full(n, -1, np.int32)
|
||||
self.tick_rows = []
|
||||
self.group_stats = {name: {key: np.zeros(n, np.float64 if 'SI' in key else np.int64)
|
||||
for key in ('due_edges', 'accepted_edges', 'rejected_edges', 'due_contacts',
|
||||
'accepted_contacts', 'rejected_contacts', 'accepted_exc_SI',
|
||||
'accepted_inh_abs_SI', 'rejected_exc_SI', 'rejected_inh_abs_SI')}
|
||||
for name in group_names}
|
||||
self.metrics = {key: np.zeros(n, np.float64) for key in (
|
||||
'APL_write_SI', 'APL_write_ref_SI', 'APL_write_available_SI', 'APL_pre_first_spike_SI',
|
||||
'APL_same_tick_first_spike_SI', 'negative_g_ticks', 'refractory_ticks',
|
||||
'g_end_time_integral_V_ms', 'V_end_time_integral_V_ms', 'accepted_exc_before_first_SI',
|
||||
'accepted_inh_before_first_abs_SI', 'APL_zeroed_due_edges', 'APL_zeroed_due_contacts')}
|
||||
self.min_g = np.full(n, np.inf)
|
||||
self.max_v = np.full(n, -np.inf)
|
||||
self.ledger = {key: np.zeros(n, np.longdouble) for key in ('decay', 'reset', 'arrivals', 'APL')}
|
||||
self.ledger_abs = np.zeros(n, np.longdouble)
|
||||
self.arrival_rounding = np.zeros(n, np.longdouble)
|
||||
# Source/sign lanes are a conditional accounting, never fed to V/g.
|
||||
self.shadow_names = [name + '_' + sign for name in group_names for sign in ('exc', 'inh')] + ['graded_APL']
|
||||
self.shadow = np.zeros((len(self.shadow_names), n), np.float64)
|
||||
self.shadow_integral = np.zeros_like(self.shadow)
|
||||
self.initial_shadow_integral = np.zeros(n, np.float64)
|
||||
self.shadow_max_residual = 0.0
|
||||
self.callback_count = 0
|
||||
self.initial_g = None
|
||||
|
||||
def __call__(self, tick, s):
|
||||
fired_nodes = self.targets[s['fired']]
|
||||
expected = self.expected.get(tick, np.empty(0, np.int32))
|
||||
if not bit_equal(fired_nodes, expected):
|
||||
raise ValueError(f'KC spike divergence at tick {tick}: computed={fired_nodes.tolist()} expected={expected.tolist()}')
|
||||
if self.initial_g is None:
|
||||
self.initial_g = s['old_g'].copy()
|
||||
# Fixture initial g is a separate lane, not attributed to any input class.
|
||||
self.initial_shadow = s['old_g'].copy()
|
||||
for name, history in self.histories.items():
|
||||
key = {'pre_v': 'v_pre_apl', 'pre_g': 'g_pre_apl', 'pre_ref': 'ref_pre_apl',
|
||||
'v': 'v', 'g': 'g', 'ref': 'ref'}[name]
|
||||
if not bit_equal(history[tick], s[key]):
|
||||
raise ValueError(f'Fixture dense {name} divergence at tick {tick}')
|
||||
before_first = self.first < 0
|
||||
first_now = before_first & s['fired']
|
||||
self.first[first_now] = tick
|
||||
self.max_v = np.maximum(self.max_v, s['v_threshold'])
|
||||
self.min_g = np.minimum(self.min_g, s['g'])
|
||||
ref_mask = s['ref'] > 0
|
||||
delta = s['delta_si']
|
||||
self.metrics['APL_write_SI'] += delta
|
||||
self.metrics['APL_write_ref_SI'] += delta * ref_mask
|
||||
self.metrics['APL_write_available_SI'] += delta * ~ref_mask
|
||||
# Strictly prior ticks: arrival/APL at first-spike tick cannot cause that spike.
|
||||
self.metrics['APL_pre_first_spike_SI'] += delta * (before_first & ~s['fired'])
|
||||
self.metrics['APL_same_tick_first_spike_SI'] += delta * first_now
|
||||
self.metrics['negative_g_ticks'] += s['g'] < 0
|
||||
self.metrics['refractory_ticks'] += ref_mask
|
||||
self.metrics['g_end_time_integral_V_ms'] += s['g'] * .1
|
||||
self.metrics['V_end_time_integral_V_ms'] += s['v'] * .1
|
||||
stages = {'decay': ('g_decayed', 'old_g'), 'reset': ('g_after_reset', 'g_decayed'),
|
||||
'arrivals': ('g_pre_apl', 'g_after_reset'), 'APL': ('g', 'g_pre_apl')}
|
||||
for label, (after, before) in stages.items():
|
||||
diff = s[after].astype(np.longdouble) - s[before].astype(np.longdouble)
|
||||
self.ledger[label] += diff
|
||||
self.ledger_abs += np.abs(diff)
|
||||
due = s['due_edge_indices']
|
||||
dst = self.incoming.dst_local[due]
|
||||
weights = self.incoming.weights[due]
|
||||
groups = self.groups[due]
|
||||
accepted = s['arrival_accepted']
|
||||
positive, negative = weights > 0, weights < 0
|
||||
nominal = np.zeros(self.n, np.longdouble)
|
||||
np.add.at(nominal, dst[accepted], weights[accepted].astype(np.longdouble))
|
||||
actual = s['g_pre_apl'].astype(np.longdouble) - s['g_after_reset'].astype(np.longdouble)
|
||||
self.arrival_rounding += actual - nominal
|
||||
self.shadow[:, s['available']] *= self.coeff['gd']
|
||||
self.initial_shadow[s['available']] *= self.coeff['gd']
|
||||
self.shadow[:, s['fired']] = 0.0
|
||||
self.initial_shadow[s['fired']] = 0.0
|
||||
row = dict(tick=tick, time_ms=tick * .1, KC_spikes=int(s['fired'].sum()),
|
||||
KC_negative_g=int(np.count_nonzero(s['g'] < 0)), KC_refractory=int(ref_mask.sum()),
|
||||
APL_write_SI_sum=float(delta.sum()), APL_ref_write_SI_sum=float(delta[ref_mask].sum()))
|
||||
for gi, name in enumerate(self.names):
|
||||
mask = groups == gi
|
||||
acc, rej = mask & accepted, mask & ~accepted
|
||||
stat = self.group_stats[name]
|
||||
for stem, selected in (('due', mask), ('accepted', acc), ('rejected', rej)):
|
||||
np.add.at(stat[stem + '_edges'], dst[selected], 1)
|
||||
np.add.at(stat[stem + '_contacts'], dst[selected], self.raw[due[selected]])
|
||||
for stem, selected in (('accepted', acc), ('rejected', rej)):
|
||||
for suffix, signmask, sign in (('exc_SI', positive, 1), ('inh_abs_SI', negative, -1)):
|
||||
pick = selected & signmask
|
||||
np.add.at(stat[stem + '_' + suffix], dst[pick], sign * weights[pick])
|
||||
row[name + '_' + stem + '_' + suffix] = float((sign * weights[pick]).sum())
|
||||
for sign_index, signmask in ((0, positive), (1, negative)):
|
||||
selected = acc & signmask
|
||||
np.add.at(self.shadow[2 * gi + sign_index], dst[selected], weights[selected])
|
||||
self.shadow[-1] -= delta
|
||||
self.shadow_integral += self.shadow * .1
|
||||
self.initial_shadow_integral += self.initial_shadow * .1
|
||||
self.shadow_max_residual = max(self.shadow_max_residual,
|
||||
float(np.max(np.abs(self.shadow.sum(axis=0) + self.initial_shadow - s['g']))))
|
||||
no_first_yet = before_first[dst] & ~s['fired'][dst]
|
||||
for metric, selected, sign in (
|
||||
('accepted_exc_before_first_SI', accepted & positive & no_first_yet, 1),
|
||||
('accepted_inh_before_first_abs_SI', accepted & negative & no_first_yet, -1)):
|
||||
np.add.at(self.metrics[metric], dst[selected], sign * weights[selected])
|
||||
apl_id = self.names.index('APL')
|
||||
zeroed = (groups == apl_id) & (weights == 0)
|
||||
np.add.at(self.metrics['APL_zeroed_due_edges'], dst[zeroed], 1)
|
||||
np.add.at(self.metrics['APL_zeroed_due_contacts'], dst[zeroed], self.raw[due[zeroed]])
|
||||
for stratum, mask in (('ever_active', self.active), ('never_active', ~self.active)):
|
||||
row[stratum + '_mean_g_V'] = None if not mask.any() else float(s['g'][mask].mean())
|
||||
row[stratum + '_mean_V_V'] = None if not mask.any() else float(s['v'][mask].mean())
|
||||
self.tick_rows.append(row)
|
||||
self.callback_count += 1
|
||||
|
||||
def finish(self, final_g):
|
||||
reconstructed = self.initial_g.astype(np.longdouble) + sum(self.ledger.values())
|
||||
residual = reconstructed - final_g.astype(np.longdouble)
|
||||
bound = (64 * np.finfo(np.longdouble).eps * (self.callback_count + 1) *
|
||||
(self.ledger_abs + np.abs(self.initial_g.astype(np.longdouble)) + np.abs(final_g)))
|
||||
assert np.all(np.abs(residual) <= bound), 'g ledger residual exceeds stated arithmetic bound'
|
||||
data = dict(neuron_idx=self.targets, spike_count=self.expected_counts, first_spike_tick=self.first,
|
||||
minimum_end_g_V=self.min_g, maximum_prethreshold_V=self.max_v,
|
||||
best_threshold_margin_mV=(self.coeff['threshold'] - self.max_v) * 1000,
|
||||
g_ledger_residual_V=residual.astype(np.float64),
|
||||
g_ledger_bound_V=bound.astype(np.float64), arrival_rounding_V=self.arrival_rounding.astype(np.float64),
|
||||
initial_g_state_area_V_ms=self.initial_shadow_integral,
|
||||
source_g_area_residual_V_ms=(self.shadow_integral.sum(axis=0) + self.initial_shadow_integral -
|
||||
self.metrics['g_end_time_integral_V_ms']),
|
||||
**self.metrics)
|
||||
for name, values in self.group_stats.items():
|
||||
for metric, values2 in values.items():
|
||||
data[name + '_' + metric] = values2
|
||||
for name, value in self.ledger.items():
|
||||
data['actual_g_delta_' + name + '_V'] = value.astype(np.float64)
|
||||
for index, name in enumerate(self.shadow_names):
|
||||
data['source_g_area_' + name + '_V_ms'] = self.shadow_integral[index]
|
||||
for key, value in data.items():
|
||||
assert np.isfinite(value).all(), 'Nonfinite accounting: ' + key
|
||||
aggregates = {}
|
||||
for label, mask in (('all', np.ones(self.n, bool)), ('ever_active', self.active), ('never_active', ~self.active)):
|
||||
groups = {name: {key: float(value[mask].sum()) if 'SI' in key else int(value[mask].sum())
|
||||
for key, value in metrics.items()} for name, metrics in self.group_stats.items()}
|
||||
exc = sum((self.group_stats[n]['accepted_exc_SI'] for n in self.names), np.zeros(self.n))
|
||||
alp = self.group_stats['ALPN']['accepted_exc_SI']
|
||||
aggregates[label] = dict(neurons=int(mask.sum()), source_groups=groups,
|
||||
no_accepted_exc_neurons=int(np.count_nonzero(mask & (exc == 0))),
|
||||
no_accepted_ALPN_exc_neurons=int(np.count_nonzero(mask & (alp == 0))),
|
||||
has_exc_but_never_active=int(np.count_nonzero(mask & (exc > 0) & ~self.active)),
|
||||
metrics_sum={k: float(v[mask].sum()) for k, v in self.metrics.items()},
|
||||
source_g_area_V_ms={name: float(self.shadow_integral[i, mask].sum()) for i, name in enumerate(self.shadow_names)},
|
||||
initial_g_state_area_V_ms=float(self.initial_shadow_integral[mask].sum()),
|
||||
source_g_area_residual_V_ms=float(data['source_g_area_residual_V_ms'][mask].sum()),
|
||||
maximum_prethreshold_V=None if not mask.any() else float(self.max_v[mask].max()),
|
||||
minimum_end_g_V=None if not mask.any() else float(self.min_g[mask].min()))
|
||||
return data, dict(strata=aggregates, ledger_max_abs_error_V=float(np.max(np.abs(residual))),
|
||||
ledger_max_bound_V=float(bound.max()), ledger_roundoff_bound_pass=True,
|
||||
ledger_accumulator_mantissa_bits=int(np.finfo(np.longdouble).nmant),
|
||||
ledger_bound_formula='64 * eps(longdouble) * (ticks+1) * (sum_abs_actual_stage_deltas + abs(initial_g) + abs(final_g)) per KC',
|
||||
source_shadow_max_residual_V=self.shadow_max_residual,
|
||||
source_area_max_residual_V_ms=float(np.max(np.abs(data['source_g_area_residual_V_ms']))),
|
||||
source_shadow_meaning='Signed source/sign g bookkeeping under observed availability/reset only; non-feedback, not counterfactual',
|
||||
cumulative_increment_units='SI volts; sums of individual increments, not instantaneous conductance',
|
||||
source_state_area_units='V*ms; post-APL source-state left samples times dt .1ms',
|
||||
activity_strata_outcome_defined=True, APL_cause_of_CAL_failure_proven=False,
|
||||
full_CAL_all_tick_subset_state_independently_verified=False)
|
||||
@@ -0,0 +1,12 @@
|
||||
fc254f996eab81c8ad76c4eb3dad274823f0e23502e59c3da6b5cbbd68444e8e PROTOCOL.md
|
||||
c4703d36587bd40a2a231d5098b8d9991407ba80f6f2b488ffad068c7a74fde6 accounting.py
|
||||
c7eb06a19d9f1f28ecf71c5596d10aa11bb405d09b1167546da347eb58d8da9a context.py
|
||||
ce0d0bfa2679c18573acab673b28d167ed300ff7669486c2e93fdc1a6097fc05 finalize.py
|
||||
8504b38219bac952b07a67e7dc37a520f0a68238cb0b0dd56f4b4b7948c5233b populations.py
|
||||
fecf8324b5fca4b65d8f24f3ed80d20bdb1fc5654294958f34d58dd36625c59b prepare.py
|
||||
aa3187cb23712c09ab603e27e60424294db92de8086d60a95cf7a629d0521fdc replay_core.py
|
||||
724c4f25341dc0883951ef5abdf9fc7dc72fba146a000b9cf9bca759e9c7623a run_audit.py
|
||||
a10f67c42162f8b07f0d760108d5ebf575f6dddc588752acc4499b43bac3ec94 test_accounting.py
|
||||
65285a6679e140f9f7cc16cb806e7bddf0f0d288c6a7d3fa2879a7cd745dacc1 test_populations.py
|
||||
17b94e6c549e60c5adb9eebf0a5dc0679a95670265b3a332a49a2e6610a976c2 test_replay_core.py
|
||||
fa89cb0a6e2dc77462057b672ffbcfabbe74b893d8f70e4b41d8b5202de15530 manifest.json
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Phase 4AH: read-only accounting of sealed Phase 4AG trajectories."""
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path('/data/digital-drosophila')
|
||||
MODEL = Path(__file__).resolve().parent
|
||||
EXP = ROOT / 'experiments/phase4ah-kc-drive-audit'
|
||||
AG_MODEL = ROOT / 'models/male-cns-lif/author-apl-lab-v2'
|
||||
AG = ROOT / 'experiments/phase4ag-author-apl-r2'
|
||||
META = ROOT / 'datasets/male-cns-v1.0/processed/neuron-only/neuron-index/unified-metadata.parquet'
|
||||
COEFF = ROOT / 'experiments/phase4af-author-fullgraph/oracle/no_input/coefficients.json'
|
||||
|
||||
|
||||
def digest(path):
|
||||
h = hashlib.sha256()
|
||||
with Path(path).open('rb') as f:
|
||||
for b in iter(lambda: f.read(8 << 20), b''):
|
||||
h.update(b)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def save(path, value):
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open('x', encoding='utf-8') as f:
|
||||
json.dump(value, f, indent=2, ensure_ascii=False, allow_nan=False)
|
||||
f.write('\n')
|
||||
|
||||
|
||||
def sources():
|
||||
return {p.name: digest(p) for p in sorted(MODEL.iterdir()) if p.is_file()
|
||||
and p.name not in ('manifest.json', 'checksums.sha256')}
|
||||
|
||||
|
||||
def read(path):
|
||||
return json.loads(Path(path).read_text(encoding='utf-8'))
|
||||
|
||||
|
||||
def integrity():
|
||||
spec = importlib.util.spec_from_file_location('sealed_ag_context', AG_MODEL / 'context.py')
|
||||
previous = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(previous)
|
||||
result = previous.integrity()
|
||||
import sys
|
||||
sys.path.insert(0, str(ROOT / 'models/male-cns-lif/terminal-scope-diagnostic-v1'))
|
||||
from diagnostic import manifest_check
|
||||
for p in (AG_MODEL, AG):
|
||||
result[str(p.relative_to(ROOT))] = manifest_check(p, digest)
|
||||
assert all(r['pass'] for r in result.values())
|
||||
return result
|
||||
|
||||
|
||||
def check():
|
||||
registered = read(EXP / 'protocol/preregistered.json')
|
||||
assert registered['sources'] == sources(), 'Registered audit source changed'
|
||||
for row in registered['inputs']:
|
||||
assert digest(ROOT / row['path']) == row['sha256'], row['path']
|
||||
if (EXP / 'protocol/prepared-hashes.json').exists():
|
||||
for row in read(EXP / 'protocol/prepared-hashes.json'):
|
||||
assert digest(EXP / row['path']) == row['sha256'], row['path']
|
||||
return registered
|
||||
|
||||
|
||||
def seal(root):
|
||||
rows = [dict(path=str(p.relative_to(root)), bytes=p.stat().st_size, sha256=digest(p))
|
||||
for p in sorted(root.rglob('*')) if p.is_file() and '__pycache__' not in p.parts]
|
||||
save(root / 'manifest.json', dict(root=str(root), files=rows))
|
||||
with (root / 'checksums.sha256').open('x') as f:
|
||||
for row in rows:
|
||||
f.write(row['sha256'] + ' ' + row['path'] + '\n')
|
||||
f.write(digest(root / 'manifest.json') + ' manifest.json\n')
|
||||
@@ -0,0 +1,154 @@
|
||||
"""Aggregate and seal descriptive accounting without learning promotion."""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from context import ROOT, MODEL, EXP, AG, AG_MODEL, read, save, digest, check, integrity, seal
|
||||
from accounting import bit_equal
|
||||
|
||||
|
||||
def summarize():
|
||||
inventory = read(EXP / 'protocol/inventory.json')
|
||||
expected_inventory = [(name, 0, 'fixture') for name in read(AG_MODEL / 'config.json')['fixtures']]
|
||||
for row in read(AG / 'protocol/calibration-cases.json'):
|
||||
expected_inventory.append((row['name'], 0, 'calibration'))
|
||||
if row['seed'] == 51101:
|
||||
expected_inventory.append((row['name'], 1, 'calibration'))
|
||||
identities = [(r['case'], r['repeat'], r['kind']) for r in inventory]
|
||||
assert len(identities) == len(set(identities)) == 80 and identities == expected_inventory
|
||||
reports = []
|
||||
modes = {}
|
||||
for item in inventory:
|
||||
folder = EXP / 'trials' / (item['case'] + '-r' + str(item['repeat']))
|
||||
result = read(folder / 'summary.json')
|
||||
assert result['pass_all'] and all(result['exact'].values())
|
||||
assert result['ledger_roundoff_bound_pass'] and result['steps'] > 0
|
||||
for name, sha in result['input_sha256'].items():
|
||||
assert digest(ROOT / item['source'] / name) == sha
|
||||
# Re-read saved numeric evidence; a successful receipt alone is not proof.
|
||||
spec = read(AG / 'cases' / item['case'] / 'spec.json')
|
||||
assert (result['case'], result['repeat'], result['kind']) == (item['case'], item['repeat'], item['kind'])
|
||||
assert result['steps'] == spec['steps'] and result['duration_ms'] == spec['duration_ms']
|
||||
assert result['mode'] == spec.get('mode') and result['seed'] == spec.get('seed')
|
||||
assert result['odor_id'] == spec.get('odor_id')
|
||||
nodes = np.load(spec['graph_dir'] + '/nodes.npy', mmap_mode='r')
|
||||
kc = np.load(AG / 'apl-bundle/kc_globals.npy', mmap_mode='r')
|
||||
targets, positions, _ = np.intersect1d(nodes, kc, assume_unique=True, return_indices=True)
|
||||
table = pd.read_parquet(folder / 'per-kc.parquet')
|
||||
with np.load(folder / 'replay-final.npz') as actual, np.load(ROOT / item['source'] / 'state.npz') as expected:
|
||||
assert bit_equal(actual['target_globals'], targets)
|
||||
for key in ('v', 'g', 'ref', 'counts'):
|
||||
assert bit_equal(actual[key], expected[key][positions]), key
|
||||
event_mask = np.isin(expected['neuron_idx'], targets)
|
||||
for key in ('step', 'neuron_idx'):
|
||||
assert bit_equal(actual[key], expected[key][event_mask]), key
|
||||
assert bit_equal(table.neuron_idx.to_numpy(), targets)
|
||||
assert bit_equal(table.spike_count.to_numpy(), actual['counts'])
|
||||
assert bit_equal(table.first_spike_tick.to_numpy(), actual['first_spike'])
|
||||
assert np.isfinite(table.select_dtypes(include='number').to_numpy()).all()
|
||||
assert (table.g_ledger_residual_V.abs() <= table.g_ledger_bound_V).all()
|
||||
reports.append(result)
|
||||
if item['kind'] != 'calibration' or item['repeat'] != 0:
|
||||
continue
|
||||
modes.setdefault(result['mode'], []).append(result)
|
||||
aggregate = {}
|
||||
for mode, rows in modes.items():
|
||||
assert len(rows) == 30
|
||||
group_totals = {}
|
||||
for group in rows[0]['strata']['all']['source_groups']:
|
||||
group_totals[group] = {metric: sum(r['strata']['all']['source_groups'][group][metric] for r in rows)
|
||||
for metric in rows[0]['strata']['all']['source_groups'][group]}
|
||||
active = np.array([r['KC_active'] for r in rows], dtype=np.int64)
|
||||
strata = {}
|
||||
for stratum in ('all', 'ever_active', 'never_active'):
|
||||
strata[stratum] = {metric: sum(r['strata'][stratum]['metrics_sum'][metric] for r in rows)
|
||||
for metric in rows[0]['strata'][stratum]['metrics_sum']}
|
||||
strata[stratum].update(neuron_trial_count=sum(r['strata'][stratum]['neurons'] for r in rows),
|
||||
no_accepted_exc_neuron_trials=sum(r['strata'][stratum]['no_accepted_exc_neurons'] for r in rows),
|
||||
no_accepted_ALPN_exc_neuron_trials=sum(r['strata'][stratum]['no_accepted_ALPN_exc_neurons'] for r in rows),
|
||||
has_exc_but_never_active_neuron_trials=sum(r['strata'][stratum]['has_exc_but_never_active'] for r in rows))
|
||||
aggregate[mode] = dict(trials=30, KC_active_mean=float(active.mean()),
|
||||
KC_active_min=int(active.min()), KC_active_max=int(active.max()),
|
||||
KC_active_fraction_mean=float(active.mean() / 4064), source_groups=group_totals,
|
||||
strata=strata,
|
||||
input_totals={key: sum(r['input'][key] for r in rows) for key in
|
||||
('requests', 'accepted', 'rejected', 'design_target_PN_spikes', 'non_design_ALPN_spikes')},
|
||||
min_max_design_targets=[min(r['input']['odor_design_target_count'] for r in rows),
|
||||
max(r['input']['odor_design_target_count'] for r in rows)],
|
||||
neuron_trial_not_unique_neuron_counts=True)
|
||||
flat = []
|
||||
for row in reports:
|
||||
if row['kind'] == 'calibration' and row['repeat'] == 0:
|
||||
flat.append(dict(case=row['case'], mode=row['mode'], seed=row['seed'], odor_id=row['odor_id'],
|
||||
KC_active=row['KC_active'], KC_spikes=row['KC_spikes'],
|
||||
requests=row['input']['requests'], accepted_requests=row['input']['accepted'],
|
||||
ALPN_actual_spikes=row['input']['actual_spikes_by_source_group']['ALPN'],
|
||||
no_exc_neurons=row['strata']['all']['no_accepted_exc_neurons'],
|
||||
no_ALPN_exc_neurons=row['strata']['all']['no_accepted_ALPN_exc_neurons'],
|
||||
APL_sum_SI=row['strata']['all']['metrics_sum']['APL_write_SI'],
|
||||
APL_ref_sum_SI=row['strata']['all']['metrics_sum']['APL_write_ref_SI'],
|
||||
maximum_ledger_error_V=row['ledger_max_abs_error_V'],
|
||||
source_shadow_max_residual_V=row['source_shadow_max_residual_V']))
|
||||
assert len(reports) == 80 and len(flat) == 60 and len(aggregate) == 2
|
||||
return reports, aggregate, flat
|
||||
|
||||
|
||||
def main():
|
||||
check()
|
||||
assert not (EXP / 'audit/failure.json').exists(), 'Preserved failure cannot be promoted'
|
||||
completed = read(EXP / 'audit/completed.json')
|
||||
assert completed['pass_all'] and len(completed['trials']) == 80 and len(completed['repeat_checks']) == 12
|
||||
inventory = read(EXP / 'protocol/inventory.json')
|
||||
expected_ids = {(r['case'], r['repeat']) for r in inventory}
|
||||
actual_ids = [(r['case'], r['repeat']) for r in completed['trials']]
|
||||
assert len(actual_ids) == len(set(actual_ids)) == 80 and set(actual_ids) == expected_ids
|
||||
assert all(r['pass_all'] for r in completed['trials'])
|
||||
repeat_ids = [r['case'] for r in completed['repeat_checks']]
|
||||
assert len(repeat_ids) == len(set(repeat_ids)) == 12
|
||||
assert set(repeat_ids) == {r['case'] for r in inventory if r['repeat'] == 1}
|
||||
assert all(r['pass_all'] and all(r['checks'].values()) for r in completed['repeat_checks'])
|
||||
reports, aggregate, flat = summarize()
|
||||
pd.DataFrame(flat).to_csv(EXP / 'audit/calibration-drive-summary.csv', index=False)
|
||||
after = integrity()
|
||||
save(EXP / 'regression/old-integrity-after.json', after)
|
||||
prior_five = read(AG / 'regression/frozen-five.json')
|
||||
assert prior_five['pass_all']
|
||||
result = dict(status='PASS-CONDITIONAL-KC-DRIVE-AUDIT', reconstructed_trials=80,
|
||||
fixture_full_target_history_trials=8, CAL_final_and_event_trials=72, exact_accounting_repeats=12,
|
||||
modes=aggregate, max_ledger_residual_V=max(r['ledger_max_abs_error_V'] for r in reports),
|
||||
max_source_shadow_residual_V=max(r['source_shadow_max_residual_V'] for r in reports),
|
||||
max_source_area_residual_V_ms=max(r['source_area_max_residual_V_ms'] for r in reports),
|
||||
CPU_accounting_wall_sum_seconds=sum(r['CPU_audit_wall_seconds'] for r in reports),
|
||||
old_integrity_pass=all(r['pass'] for r in after.values()),
|
||||
frozen_five_reexecuted_this_audit=False, previous_sealed_five_reference_pass=True,
|
||||
previous_five_reference_sha256=digest(AG / 'regression/frozen-five.json'),
|
||||
previous_CAL_failure_not_changed=True, new_brain_simulation=False, GPU_used=False,
|
||||
training=False, heldout_access=False, candidate_generated=False, learning_achieved=False,
|
||||
APL_causal_failure_claim=False, counterfactual_tested=False,
|
||||
limitations=['Logged upstream and APL are conditioning inputs, not independent components',
|
||||
'CAL has no independently saved per-tick KC state histories; only events and final states checked',
|
||||
'Source state lanes are non-feedback linear bookkeeping under observed reset/availability',
|
||||
'First-spike APL/ref mechanisms cannot establish whole-network counterfactual causality',
|
||||
'Counts pooled across trials are neuron-trial counts, not distinct neuron identities'],
|
||||
payload_bytes_before_manifest=sum(p.stat().st_size for p in EXP.rglob('*') if p.is_file()))
|
||||
assert result['payload_bytes_before_manifest'] < 2 * 1024 ** 3
|
||||
save(EXP / 'audit/final-status.json', result)
|
||||
lines = ['# Phase 4AH — conditional KC drive audit', '', 'Status: `'+result['status']+'`.', '',
|
||||
'80 existing records reconstructed; 8 fixture pre/post histories and 72 CAL event/final states exact.',
|
||||
'12 repeated CAL records produced identical accounting. No new brain simulation, GPU use, tuning or learning.',
|
||||
'Full prior frozen integrity passed. Five reference simulations were not rerun; the sealed prior PASS is referenced by SHA.',
|
||||
'Previous CAL failure remains unchanged. Conditional replay is not an APL-removal counterfactual.', '',
|
||||
'|Mode|Mean active KC fraction|Requests|Accepted requests|ALPN accepted excitatory increment sum (V)|APL write sum (V)|',
|
||||
'|---|---:|---:|---:|---:|---:|']
|
||||
for mode, row in aggregate.items():
|
||||
lines.append(f"|{mode}|{row['KC_active_fraction_mean']:.8f}|{row['input_totals']['requests']}|{row['input_totals']['accepted']}|{row['source_groups']['ALPN']['accepted_exc_SI']:.9g}|{row['strata']['all']['APL_write_SI']:.9g}|")
|
||||
lines += ['', 'Increment sums span 30 trials × 4064 KC × time; not instantaneous conductance or causal percentages.',
|
||||
'See audit/final-status.json, audit/calibration-drive-summary.csv and trials/*/per-kc.parquet for detailed evidence.']
|
||||
with (EXP / 'REPORT.md').open('x') as f:
|
||||
f.write('\n'.join(lines) + '\n')
|
||||
check()
|
||||
seal(MODEL)
|
||||
seal(EXP)
|
||||
print(result['status'], flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"root": "/data/digital-drosophila/models/male-cns-lif/kc-drive-audit-v1",
|
||||
"files": [
|
||||
{
|
||||
"path": "PROTOCOL.md",
|
||||
"bytes": 4208,
|
||||
"sha256": "fc254f996eab81c8ad76c4eb3dad274823f0e23502e59c3da6b5cbbd68444e8e"
|
||||
},
|
||||
{
|
||||
"path": "accounting.py",
|
||||
"bytes": 12819,
|
||||
"sha256": "c4703d36587bd40a2a231d5098b8d9991407ba80f6f2b488ffad068c7a74fde6"
|
||||
},
|
||||
{
|
||||
"path": "context.py",
|
||||
"bytes": 2757,
|
||||
"sha256": "c7eb06a19d9f1f28ecf71c5596d10aa11bb405d09b1167546da347eb58d8da9a"
|
||||
},
|
||||
{
|
||||
"path": "finalize.py",
|
||||
"bytes": 10619,
|
||||
"sha256": "ce0d0bfa2679c18573acab673b28d167ed300ff7669486c2e93fdc1a6097fc05"
|
||||
},
|
||||
{
|
||||
"path": "populations.py",
|
||||
"bytes": 9356,
|
||||
"sha256": "8504b38219bac952b07a67e7dc37a520f0a68238cb0b0dd56f4b4b7948c5233b"
|
||||
},
|
||||
{
|
||||
"path": "prepare.py",
|
||||
"bytes": 5160,
|
||||
"sha256": "fecf8324b5fca4b65d8f24f3ed80d20bdb1fc5654294958f34d58dd36625c59b"
|
||||
},
|
||||
{
|
||||
"path": "replay_core.py",
|
||||
"bytes": 14287,
|
||||
"sha256": "aa3187cb23712c09ab603e27e60424294db92de8086d60a95cf7a629d0521fdc"
|
||||
},
|
||||
{
|
||||
"path": "run_audit.py",
|
||||
"bytes": 12406,
|
||||
"sha256": "724c4f25341dc0883951ef5abdf9fc7dc72fba146a000b9cf9bca759e9c7623a"
|
||||
},
|
||||
{
|
||||
"path": "test_accounting.py",
|
||||
"bytes": 13018,
|
||||
"sha256": "a10f67c42162f8b07f0d760108d5ebf575f6dddc588752acc4499b43bac3ec94"
|
||||
},
|
||||
{
|
||||
"path": "test_populations.py",
|
||||
"bytes": 5472,
|
||||
"sha256": "65285a6679e140f9f7cc16cb806e7bddf0f0d288c6a7d3fa2879a7cd745dacc1"
|
||||
},
|
||||
{
|
||||
"path": "test_replay_core.py",
|
||||
"bytes": 12276,
|
||||
"sha256": "17b94e6c549e60c5adb9eebf0a5dc0679a95670265b3a332a49a2e6610a976c2"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
"""Pure, disjoint source labels from frozen anatomical metadata evidence.
|
||||
|
||||
No files, trajectories, Torch, simulation, or parameters are touched. The
|
||||
caller must verify context.META against its registered SHA-256 before loading.
|
||||
Class membership follows Phase 4AG/4X; transmitter metadata is an orthogonal
|
||||
diagnostic, never anatomical DAN identity or a reward/valence assertion.
|
||||
"""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
|
||||
N_NEURONS = 166700
|
||||
REQUIRED_COLUMNS = ("neuron_idx", "body_id", "annotation_class", "annotation_type",
|
||||
"neurotransmitter", "nt_ground_truth")
|
||||
GROUPS = ("KC", "APL", "ALPN", "MBON", "anatomical_DAN",
|
||||
"dopamine_annotation_only", "other", "unclassified")
|
||||
METADATA_SHA256 = "922af16846a065c3dbfca7c11146b83ca217fcbd7392edc745f444a04eae8c63"
|
||||
FROZEN_4X_ROLES_SHA256 = "76ba0cae6c5db417ec030456797f7935216c6877bd339d5e435133a0061e3930"
|
||||
FROZEN_4X_SIDECAR_SHA256 = "8b51d12d84d66cdca66aeb7d052a24e520b9c28f462bd6bf87f37dda85c33284"
|
||||
|
||||
|
||||
def require(condition, message):
|
||||
if not condition:
|
||||
raise ValueError(message)
|
||||
|
||||
|
||||
def _text_column(meta, name):
|
||||
column = meta[name]
|
||||
# Do not silently turn corrupt numeric metadata into a plausible label.
|
||||
require(bool(column.map(lambda value: pd.isna(value) or isinstance(value, str)).all()),
|
||||
"Expected nullable string metadata: " + name)
|
||||
return column.fillna("").astype(str).to_numpy()
|
||||
|
||||
|
||||
def classify(meta):
|
||||
"""Return (Unicode labels[N], JSON-serializable report), indexed by global ID.
|
||||
|
||||
The full metadata must already be in exact neuron_idx order. Conflicting
|
||||
exact identities and absent/explicitly unknown identities are retained as
|
||||
unclassified instead of silently resolving them by assignment precedence.
|
||||
DPM is reported explicitly but belongs to other (unless its annotations
|
||||
conflict). A PAM/PPL1/DAN name prefix alone never creates a DAN identity.
|
||||
"""
|
||||
require(isinstance(meta, pd.DataFrame), "classify expects a pandas DataFrame")
|
||||
missing = set(REQUIRED_COLUMNS) - set(meta.columns)
|
||||
require(not missing, "Missing actual metadata schema fields: " + ", ".join(sorted(missing)))
|
||||
require(meta.columns.is_unique, "Duplicate metadata columns")
|
||||
require(len(meta) == N_NEURONS, "Expected the complete 166700-neuron metadata")
|
||||
ids = meta["neuron_idx"].to_numpy()
|
||||
bodies = meta["body_id"].to_numpy()
|
||||
require(ids.dtype == np.int32 and np.array_equal(ids, np.arange(N_NEURONS, dtype=np.int32)),
|
||||
"neuron_idx must be int32, complete, unique and in canonical row order")
|
||||
require(bodies.dtype == np.uint64 and len(np.unique(bodies)) == N_NEURONS,
|
||||
"body_id must be uint64 and unique")
|
||||
cls, typ = _text_column(meta, "annotation_class"), _text_column(meta, "annotation_type")
|
||||
nt_raw = _text_column(meta, "neurotransmitter")
|
||||
ground_truth = _text_column(meta, "nt_ground_truth")
|
||||
# Match frozen 4X normalization for the transmitter axis only. Anatomical
|
||||
# class/type membership retains exact, case-sensitive original values.
|
||||
nt = np.char.lower(np.char.strip(nt_raw.astype(str)))
|
||||
dopamine = nt == "dopamine"
|
||||
dpm = typ == "DPM"
|
||||
masks = {"KC": cls == "Kenyon_Cell", "APL": typ == "APL", "ALPN": cls == "ALPN",
|
||||
"MBON": cls == "MBON", "anatomical_DAN": cls == "DAN"}
|
||||
hits = np.stack(list(masks.values())).sum(axis=0)
|
||||
# Preserve the frozen 4X DAN/DPM conflict explicitly, and do not silently
|
||||
# prefer an exact APL type over an incompatible exact anatomical class.
|
||||
conflict = (hits > 1) | (dpm & (hits > 0))
|
||||
labels = np.full(N_NEURONS, "other", dtype="<U32")
|
||||
for name, mask in masks.items():
|
||||
labels[mask & ~conflict] = name
|
||||
unassigned = hits == 0
|
||||
diagnostic_dopamine = dopamine & unassigned & ~dpm & ~conflict
|
||||
labels[diagnostic_dopamine] = "dopamine_annotation_only"
|
||||
# These are explicit missingness/uncertainty markers, not inferred types.
|
||||
unknown_tokens = ("", "unknown", "unclear", "unclassified")
|
||||
class_unknown = np.isin(np.char.lower(np.char.strip(cls.astype(str))), unknown_tokens)
|
||||
type_unknown = np.isin(np.char.lower(np.char.strip(typ.astype(str))), unknown_tokens)
|
||||
no_anatomical_annotation = class_unknown & type_unknown
|
||||
dan_name_hint = np.char.startswith(typ.astype(str), ("PAM")) | \
|
||||
np.char.startswith(typ.astype(str), ("PPL1")) | \
|
||||
np.char.startswith(typ.astype(str), ("DAN"))
|
||||
uncertain_dan_prefix = dan_name_hint & ~masks["anatomical_DAN"]
|
||||
unresolved = unassigned & ~dpm & ~diagnostic_dopamine & (
|
||||
no_anatomical_annotation | (class_unknown & uncertain_dan_prefix))
|
||||
labels[unresolved | conflict] = "unclassified"
|
||||
# Orthogonal flags remain auditable even when the primary label is disjoint.
|
||||
flag_masks = {"annotation_DPM": dpm, "metadata_dopamine": dopamine,
|
||||
"annotation_DAN": masks["anatomical_DAN"], "identity_conflict": conflict,
|
||||
"multiple_exact_group_hits": hits > 1,
|
||||
"no_anatomical_annotation": no_anatomical_annotation,
|
||||
"DAN_name_prefix_without_DAN_class": uncertain_dan_prefix,
|
||||
"metadata_dopamine_not_annotation_DAN": dopamine & ~masks["anatomical_DAN"],
|
||||
"annotation_DAN_not_metadata_dopamine": masks["anatomical_DAN"] & ~dopamine,
|
||||
"has_raw_ground_truth": np.char.strip(ground_truth.astype(str)) != ""}
|
||||
counts = {name: int(np.count_nonzero(labels == name)) for name in GROUPS}
|
||||
require(sum(counts.values()) == N_NEURONS and set(np.unique(labels)).issubset(GROUPS),
|
||||
"Output groups do not partition the complete graph")
|
||||
require(not np.any((labels == "anatomical_DAN") & dpm), "DPM must never be assigned anatomical_DAN")
|
||||
report = {"neurons": N_NEURONS, "groups": counts, "group_order": list(GROUPS),
|
||||
"all_neurons_assigned_once": True, "labels_index": "global neuron_idx, identical to canonical row index",
|
||||
"identity_conflicts_present": bool(np.any(conflict)),
|
||||
"pass_all": bool(not np.any(conflict)),
|
||||
"pass_all_scope": "schema, complete disjoint accounting and no conflicting exact identities; not biological validation",
|
||||
"flags": {name: int(mask.sum()) for name, mask in flag_masks.items()},
|
||||
"flag_counts_by_group": {name: {group: int(np.count_nonzero(mask & (labels == group)))
|
||||
for group in GROUPS} for name, mask in flag_masks.items()},
|
||||
"DPM_neuron_idx": ids[dpm].tolist(), "identity_conflict_neuron_idx": ids[conflict].tolist(),
|
||||
"uncertain_DAN_prefix_neuron_idx": ids[uncertain_dan_prefix].tolist(),
|
||||
"definitions": {
|
||||
"KC": "annotation_class exactly Kenyon_Cell",
|
||||
"APL": "annotation_type exactly APL",
|
||||
"ALPN": "annotation_class exactly ALPN; annotation_class olfactory is NOT included",
|
||||
"MBON": "annotation_class exactly MBON",
|
||||
"anatomical_DAN": "annotation_class exactly DAN, no exact identity conflict; not inferred from NT or PAM/PPL1 prefix",
|
||||
"dopamine_annotation_only": "normalized neurotransmitter=dopamine, no named exact group, no DPM; transmitter-evidence diagnostic, NOT DAN",
|
||||
"other": "remaining annotated neurons, explicitly including non-conflicting annotation_type DPM",
|
||||
"unclassified": "conflicting exact identity, no anatomical class/type evidence, or only uncertain DAN-name hint without class/NT evidence"},
|
||||
"uncertainty_policy": {
|
||||
"explicit_unknown_tokens_casefolded": list(unknown_tokens),
|
||||
"PAM_PPL1_DAN_prefix_never_assigns_identity": True,
|
||||
"transmitter_role_and_anatomical_identity_are_separate": True,
|
||||
"DPM_retained_as_other_with_explicit_flag": True,
|
||||
"missing_ground_truth_is_not_filled_or_guessed": True,
|
||||
"other_does_not_mean_known_function": True},
|
||||
"provenance": {
|
||||
"metadata_path_contract": "context.META; caller must verify its preregistered SHA-256 before loading",
|
||||
"metadata_expected_sha256_from_AG_r2": METADATA_SHA256,
|
||||
"frozen_rules_path": "/data/digital-drosophila/models/male-cns-lif/transmitter-audit-v1/transmitter_roles_v1.py",
|
||||
"frozen_rules_sha256": FROZEN_4X_ROLES_SHA256,
|
||||
"frozen_sidecar_path": "/data/digital-drosophila/experiments/phase4x-transmitter-provenance/roles/neuron-evidence-axes.parquet",
|
||||
"frozen_sidecar_sha256": FROZEN_4X_SIDECAR_SHA256,
|
||||
"frozen_MB_DAN_mapping_path": "/data/digital-drosophila/experiments/phase4h-dopamine-plasticity/mappings/dan-to-mbon-confirmed.parquet",
|
||||
"frozen_MB_DAN_mapping_used_to_assign_identity": False,
|
||||
"file_hash_verified_by_this_pure_function": False,
|
||||
"differences_from_4X_reporting": "ALPN separated from broader olfactory class; DPM grouped other with flag; absent/conflicting identity explicitly unclassified"},
|
||||
"source_graph_changed": False, "simulation_executed": False,
|
||||
"heldout_dynamic_access": False, "reward_or_valence_inferred": False}
|
||||
return labels, report
|
||||
@@ -0,0 +1,75 @@
|
||||
"""Register conditional, CPU-only accounting; no new brain trajectories."""
|
||||
from datetime import datetime, timezone
|
||||
import platform
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from context import ROOT, MODEL, EXP, AG, AG_MODEL, META, COEFF, read, save, digest, sources, integrity
|
||||
from populations import classify, METADATA_SHA256, FROZEN_4X_ROLES_SHA256
|
||||
|
||||
|
||||
def main():
|
||||
assert not EXP.exists() or not any(EXP.iterdir()), 'Never reuse a populated audit directory'
|
||||
prior = read(AG / 'audit/final-status.json')
|
||||
assert prior['status'] == 'FAIL-FIXED-CAL-REPRESENTATION' and prior['numerical_gate_pass']
|
||||
for name in ('protocol', 'regression', 'trials', 'audit', 'logs'):
|
||||
(EXP / name).mkdir(parents=True, exist_ok=False)
|
||||
save(EXP / 'audit/preflight-directory.json', dict(
|
||||
first_attempt='Parent experiments directory was not writable; failed before creating any audit directory/file or registration',
|
||||
resolution='Administrator creates only the exact new empty audit directory owned by qishizhe; parent/frozen permissions unchanged',
|
||||
empty_scaffold_allowed=True, scientific_accounting_started=False))
|
||||
command = [sys.executable, '-B', '-m', 'unittest', '-v',
|
||||
'test_replay_core', 'test_populations', 'test_accounting']
|
||||
completed = subprocess.run(command, cwd=MODEL, capture_output=True, text=True)
|
||||
with (EXP / 'regression/unit-tests.log').open('x') as f:
|
||||
f.write(completed.stdout + completed.stderr)
|
||||
save(EXP / 'regression/unit-tests.json', dict(command=command, exit_code=completed.returncode,
|
||||
pass_all=completed.returncode == 0, sha256=digest(EXP / 'regression/unit-tests.log')))
|
||||
assert completed.returncode == 0, 'Pure audit tests failed; preserve output and stop'
|
||||
paths = [AG / 'manifest.json', AG / 'checksums.sha256', AG_MODEL / 'manifest.json',
|
||||
AG_MODEL / 'checksums.sha256', META, COEFF,
|
||||
ROOT / 'models/male-cns-lif/author-runtime-lab-v1/author_ext.cu',
|
||||
ROOT / 'models/male-cns-lif/transmitter-audit-v1/transmitter_roles_v1.py',
|
||||
ROOT / 'datasets/male-cns-v1.0/processed/neuron-only/gpu/csr-weights.int64.npy']
|
||||
assert digest(META) == METADATA_SHA256
|
||||
assert digest(ROOT / 'models/male-cns-lif/transmitter-audit-v1/transmitter_roles_v1.py') == FROZEN_4X_ROLES_SHA256
|
||||
save(EXP / 'protocol/preregistered.json', dict(timestamp_utc=datetime.now(timezone.utc).isoformat(),
|
||||
sources=sources(), inputs=[dict(path=str(p.relative_to(ROOT)), sha256=digest(p)) for p in paths],
|
||||
parent_commit=subprocess.check_output(['git', '-C', str(ROOT), 'rev-parse', 'HEAD'], text=True).strip(),
|
||||
scope='Post-outcome conditional shadow accounting; no new brain simulation, no heldout, no tuning/training',
|
||||
before_accounting=True, prior_CAL_result_known=True, output_budget_bytes=2 * 1024 ** 3,
|
||||
CPU_only=True, max_threads=4, python=sys.version, numpy=np.__version__, pandas=pd.__version__,
|
||||
machine=platform.machine(), longdouble_mantissa_bits=int(np.finfo(np.longdouble).nmant)))
|
||||
save(EXP / 'regression/old-integrity-before.json', integrity())
|
||||
meta = pd.read_parquet(META).sort_values('neuron_idx').reset_index(drop=True)
|
||||
assert len(meta) == 166700 and np.array_equal(meta.neuron_idx, np.arange(166700))
|
||||
labels, report = classify(meta)
|
||||
assert report['pass_all'], 'Ambiguous/conflicting source identities: audit required'
|
||||
for label, file in (('KC', 'kc_globals.npy'), ('APL', 'apl_globals.npy'), ('ALPN', 'pn_globals.npy')):
|
||||
assert np.array_equal(np.flatnonzero(labels == label), np.load(AG / 'apl-bundle' / file))
|
||||
save(EXP / 'protocol/population-audit.json', report)
|
||||
save(EXP / 'protocol/metadata-schema.json', {c: dict(dtype=str(meta[c].dtype),
|
||||
nulls=int(meta[c].isna().sum())) for c in meta.columns})
|
||||
meta[['neuron_idx', 'body_id', 'annotation_class', 'annotation_type', 'neurotransmitter']].assign(
|
||||
audit_source_group=labels).to_parquet(EXP / 'protocol/neuron-source-groups.parquet', index=False)
|
||||
inventory = [dict(case=name, repeat=0, kind='fixture') for name in read(AG_MODEL / 'config.json')['fixtures']]
|
||||
for row in read(AG / 'protocol/calibration-cases.json'):
|
||||
inventory.append(dict(case=row['name'], repeat=0, kind='calibration'))
|
||||
if row['seed'] == 51101:
|
||||
inventory.append(dict(case=row['name'], repeat=1, kind='calibration'))
|
||||
assert len(inventory) == 80 and len({(r['case'], r['repeat']) for r in inventory}) == 80
|
||||
for row in inventory:
|
||||
source = AG / 'gpu' / (row['case'] + '-r' + str(row['repeat']))
|
||||
assert read(source / 'summary.json')['complete']
|
||||
row['source'] = str(source.relative_to(ROOT))
|
||||
save(EXP / 'protocol/inventory.json', inventory)
|
||||
save(EXP / 'protocol/prepared-hashes.json', [dict(path=str(p.relative_to(EXP)), sha256=digest(p))
|
||||
for p in sorted((EXP / 'protocol').iterdir()) if p.is_file() and p.name != 'prepared-hashes.json'])
|
||||
print('REGISTERED 8 fixtures + 60 CAL + 12 repeats; CPU-only existing-trajectory accounting', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,253 @@
|
||||
"""CPU-only, target-local shadow of the frozen author LIF + legacy APL schedule.
|
||||
|
||||
Targets generate their own threshold events. Recorded whole-graph events are
|
||||
used ONLY as upstream, fixed-delay inputs; recorded target spikes are never
|
||||
forced into the target state. APL h is an observed external component trace.
|
||||
This is a conditional shadow reconstruction, not independent whole-brain or
|
||||
independent APL validation. Nothing in this module imports Brian or Torch,
|
||||
runs a scientific trial, reads a file, or writes a file.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
N_GLOBAL = 166700
|
||||
DELAY_STEPS = 18
|
||||
REF_STEPS = 22
|
||||
COEFFICIENT_KEYS = ("gd", "c0", "cg", "cv", "threshold", "reset", "seed_volt")
|
||||
|
||||
|
||||
def _require(condition, message):
|
||||
if not condition:
|
||||
raise ValueError(message)
|
||||
|
||||
|
||||
def _vector(value, dtype, name, size=None, finite=False):
|
||||
value = np.asarray(value)
|
||||
_require(value.dtype == np.dtype(dtype) and value.ndim == 1, f"{name} must be a one-dimensional {dtype} array")
|
||||
_require(size is None or len(value) == size, f"{name} has the wrong length")
|
||||
if finite:
|
||||
_require(np.isfinite(value).all(), f"{name} contains a nonfinite value")
|
||||
return value
|
||||
|
||||
|
||||
def _coefficients(coeff):
|
||||
_require(isinstance(coeff, Mapping), "coeff must be a mapping")
|
||||
result = {}
|
||||
for name in COEFFICIENT_KEYS:
|
||||
_require(name in coeff, f"Missing coefficient: {name}")
|
||||
value = coeff[name]
|
||||
_require(not isinstance(value, (bool, np.bool_)) and isinstance(value, (int, float, np.integer, np.float64)),
|
||||
f"{name} must be a double-precision scalar, not a float32 or nonnumeric value")
|
||||
result[name] = np.float64(value)
|
||||
_require(np.isfinite(result[name]), f"Nonfinite coefficient: {name}")
|
||||
_require(result["seed_volt"] > result["threshold"], "Common-request seed voltage must exceed the strict threshold")
|
||||
return result
|
||||
|
||||
|
||||
class IncomingIndex:
|
||||
"""Reusable immutable target-filtered anatomical edges in original order.
|
||||
|
||||
src_global:int32, dst_local:int32, edge_id:int64, weights:float64 (SI volts).
|
||||
All vectors have the same length. Sources and original global edge IDs
|
||||
must retain canonical increasing source/edge order. Edges are NOT removed
|
||||
because their weight is zero, nor reordered by target. The caller must
|
||||
supply the already-audited effective weights (all fast APL source edges
|
||||
zeroed in a separate copy); this object never edits its input arrays.
|
||||
"""
|
||||
|
||||
def __init__(self, src_global, dst_local, edge_id, weights, target_count):
|
||||
_require(isinstance(target_count, (int, np.integer)) and not isinstance(target_count, (bool, np.bool_)) and
|
||||
0 < target_count <= N_GLOBAL, "target_count must be a positive in-range integer")
|
||||
self.target_count = int(target_count)
|
||||
source = _vector(src_global, "int32", "src_global")
|
||||
count = len(source)
|
||||
destination = _vector(dst_local, "int32", "dst_local", count)
|
||||
ids = _vector(edge_id, "int64", "edge_id", count)
|
||||
weight = _vector(weights, "float64", "weights", count, finite=True)
|
||||
_require(np.all((source >= 0) & (source < N_GLOBAL)), "An incoming source is outside the global neuron range")
|
||||
_require(np.all((destination >= 0) & (destination < target_count)), "Incoming destination is outside the target-local range")
|
||||
_require(np.all(ids >= 0) and np.all(ids[1:] > ids[:-1]), "Original edge IDs must be unique and canonically increasing")
|
||||
_require(np.all(source[1:] >= source[:-1]), "Incoming sources must remain in original canonical order")
|
||||
self.src_global = np.array(source, dtype=np.int32, copy=True, order="C")
|
||||
self.dst_local = np.array(destination, dtype=np.int32, copy=True, order="C")
|
||||
self.edge_id = np.array(ids, dtype=np.int64, copy=True, order="C")
|
||||
self.weights = np.array(weight, dtype=np.float64, copy=True, order="C")
|
||||
self.sources, starts = np.unique(self.src_global, return_index=True)
|
||||
self.starts = np.asarray(starts, dtype=np.int64)
|
||||
self.stops = np.concatenate((self.starts[1:], np.array([count], np.int64))) if count else np.empty(0, np.int64)
|
||||
for value in (self.src_global, self.dst_local, self.edge_id, self.weights, self.sources, self.starts, self.stops):
|
||||
value.setflags(write=False)
|
||||
|
||||
def due(self, global_spikes):
|
||||
"""Return local edge rows in original queue order; no target sort/sum."""
|
||||
fired = _vector(global_spikes, "int32", "global_spikes")
|
||||
_require(np.all((fired >= 0) & (fired < N_GLOBAL)) and np.all(fired[1:] > fired[:-1]),
|
||||
"Presynaptic spikes must be sorted, unique, in-range globals")
|
||||
if not len(fired) or not len(self.sources):
|
||||
return np.empty(0, np.int64)
|
||||
positions = np.searchsorted(self.sources, fired)
|
||||
present = positions < len(self.sources)
|
||||
present[present] &= self.sources[positions[present]] == fired[present]
|
||||
positions = positions[present]
|
||||
if not len(positions):
|
||||
return np.empty(0, np.int64)
|
||||
# Each presynaptic group occupies a contiguous source-ordered block in
|
||||
# this filtered graph. Concatenation preserves every target's original
|
||||
# per-edge accumulation order, including zero-weight events.
|
||||
return np.concatenate([np.arange(self.starts[pos], self.stops[pos], dtype=np.int64) for pos in positions])
|
||||
|
||||
|
||||
class _EventIndex:
|
||||
def __init__(self, record, steps, name):
|
||||
_require(isinstance(record, Mapping), f"{name} must be a mapping")
|
||||
time_keys = [key for key in ("tick", "step") if key in record]
|
||||
_require(len(time_keys) == 1 and "neuron_idx" in record, f"{name} needs exactly one tick/step vector and neuron_idx")
|
||||
ticks = _vector(record[time_keys[0]], "int32", name + ".tick")
|
||||
nodes = _vector(record["neuron_idx"], "int32", name + ".neuron_idx", len(ticks))
|
||||
_require(np.all((ticks >= 0) & (ticks < steps)), f"{name} has an event outside the trial")
|
||||
_require(np.all((nodes >= 0) & (nodes < N_GLOBAL)), f"{name} contains an out-of-range global neuron")
|
||||
_require(np.all((ticks[1:] > ticks[:-1]) | ((ticks[1:] == ticks[:-1]) & (nodes[1:] > nodes[:-1]))),
|
||||
f"{name} must be sorted and unique by tick/global neuron")
|
||||
self.nodes = np.array(nodes, copy=True, order="C")
|
||||
counts = np.bincount(ticks, minlength=steps)
|
||||
self.pointer = np.concatenate((np.zeros(1, np.int64), np.cumsum(counts, dtype=np.int64)))
|
||||
self.nodes.setflags(write=False)
|
||||
self.pointer.setflags(write=False)
|
||||
|
||||
def at(self, tick):
|
||||
if tick < 0:
|
||||
return np.empty(0, np.int32)
|
||||
return self.nodes[self.pointer[tick]:self.pointer[tick + 1]]
|
||||
|
||||
|
||||
def _state_step(v, g, ref, request, coeff, capture):
|
||||
"""Separate NumPy array operations correspond to explicit CUDA RN ops."""
|
||||
next_ref = np.maximum(ref - np.int32(1), np.int32(0))
|
||||
available = next_ref == 0
|
||||
next_v, next_g = v.copy(), g.copy()
|
||||
oldg, oldv = g[available], v[available]
|
||||
next_g[available] = np.multiply(coeff["gd"], oldg)
|
||||
driven = np.multiply(coeff["cg"], oldg)
|
||||
leaked = np.multiply(coeff["cv"], oldv)
|
||||
combined = np.add(driven, leaked)
|
||||
next_v[available] = np.add(coeff["c0"], combined)
|
||||
result = {}
|
||||
if capture:
|
||||
result.update(old_v=v.copy(), old_g=g.copy(), ref_before=ref.copy(),
|
||||
available=available.copy(), request=request.copy(),
|
||||
request_accepted=(request & available), request_rejected=(request & ~available),
|
||||
v_integrated=next_v.copy(), g_decayed=next_g.copy())
|
||||
next_v[request & available] = coeff["seed_volt"]
|
||||
if capture:
|
||||
result["v_threshold"] = next_v.copy()
|
||||
fired = available & (next_v > coeff["threshold"])
|
||||
next_v[fired] = coeff["reset"]
|
||||
next_g[fired] = np.float64(0.0)
|
||||
next_ref[fired] = np.int32(REF_STEPS)
|
||||
_require(np.isfinite(next_v).all() and np.isfinite(next_g).all(), "State step produced a nonfinite value")
|
||||
result.update(v=next_v, g=next_g, ref=next_ref, fired=fired)
|
||||
if capture:
|
||||
result.update(v_after_reset=next_v.copy(), g_after_reset=next_g.copy(), ref_after_reset=next_ref.copy())
|
||||
return result
|
||||
|
||||
|
||||
def state_step(v, g, ref, request, coeff, *, capture=True):
|
||||
"""One group/request/threshold/reset step; no arrivals or APL yet.
|
||||
|
||||
Inputs are never modified. v/g are float64 SI, ref is int32 in [0,22],
|
||||
request is a bool vector. With capture=True this returns the corresponding
|
||||
pre-threshold/intermediate callback fields as well as v/g/ref/fired.
|
||||
"""
|
||||
voltage = _vector(v, "float64", "v", finite=True)
|
||||
n = len(voltage)
|
||||
conductance = _vector(g, "float64", "g", n, finite=True)
|
||||
refractory = _vector(ref, "int32", "ref", n)
|
||||
requested = _vector(request, "bool", "request", n)
|
||||
_require(n > 0 and np.all((refractory >= 0) & (refractory <= REF_STEPS)), "Invalid target count/refractory state")
|
||||
_require(isinstance(capture, bool), "capture must be Boolean")
|
||||
return _state_step(voltage, conductance, refractory, requested, _coefficients(coeff), capture)
|
||||
|
||||
|
||||
def replay(initial_v, initial_g, coeff, steps, target_globals, requests, events,
|
||||
incoming: IncomingIndex, h32, callback=None):
|
||||
"""Reconstruct targets with bounded working state and sparse output events.
|
||||
|
||||
initial_v/initial_g: float64 SI vectors, initially nonrefractory.
|
||||
target_globals: sorted unique int32 target IDs.
|
||||
requests/events: mappings with tick (or step) and neuron_idx int32 vectors;
|
||||
requests outside the target subset are ignored, not reassigned.
|
||||
h32: float32 (steps, target_count), ALREADY mapped to target_globals, in mV.
|
||||
Full 4064-KC input is valid when the target list is those same 4064 KCs.
|
||||
callback(tick, stage): optional copies of old states, available/request
|
||||
masks, integrated/threshold voltages, reset state, canonical due edges,
|
||||
accepted arrivals, pre-APL states, h32/delta_si, and final v/g/ref. It may
|
||||
raise to stop at the first caller-detected mismatch. Modifying callback
|
||||
arrays cannot affect this replay or its input arrays.
|
||||
|
||||
No dense history is stored. Memory is O(incoming edges + logged events +
|
||||
steps + targets + current due edges + predicted sparse events), excluding
|
||||
caller-owned h32 and anything the callback elects to retain.
|
||||
"""
|
||||
_require(isinstance(steps, (int, np.integer)) and not isinstance(steps, (bool, np.bool_)) and
|
||||
0 < steps <= np.iinfo(np.int32).max, "steps must be a positive int32-range integer")
|
||||
steps = int(steps)
|
||||
targets = _vector(target_globals, "int32", "target_globals")
|
||||
n = len(targets)
|
||||
_require(0 < n <= N_GLOBAL and np.all((targets >= 0) & (targets < N_GLOBAL)) and np.all(targets[1:] > targets[:-1]),
|
||||
"Targets must be sorted unique in-range globals")
|
||||
targets = targets.copy()
|
||||
v = _vector(initial_v, "float64", "initial_v", n, finite=True).copy()
|
||||
g = _vector(initial_g, "float64", "initial_g", n, finite=True).copy()
|
||||
constants = _coefficients(coeff)
|
||||
_require(isinstance(incoming, IncomingIndex) and incoming.target_count == n, "Incoming index target count mismatch")
|
||||
_require(callback is None or callable(callback), "callback must be callable or None")
|
||||
inhibition = np.asarray(h32)
|
||||
_require(inhibition.dtype == np.dtype("float32") and inhibition.shape == (steps, n), "h32 must be float32 (steps, targets), already target-mapped")
|
||||
for tick in range(steps):
|
||||
_require(np.isfinite(inhibition[tick]).all() and np.all(inhibition[tick] >= 0), "APL inhibition must be finite and nonnegative")
|
||||
request_index = _EventIndex(requests, steps, "requests")
|
||||
event_index = _EventIndex(events, steps, "events")
|
||||
ref = np.zeros(n, np.int32)
|
||||
counts = np.zeros(n, np.int32)
|
||||
first_spike = np.full(n, -1, np.int32)
|
||||
predicted_steps, predicted_nodes = [], []
|
||||
capture = callback is not None
|
||||
for tick in range(steps):
|
||||
requested_globals = request_index.at(tick)
|
||||
local = np.searchsorted(targets, requested_globals)
|
||||
present = local < n
|
||||
present[present] &= targets[local[present]] == requested_globals[present]
|
||||
request = np.zeros(n, dtype=bool)
|
||||
request[local[present]] = True
|
||||
stage = _state_step(v, g, ref, request, constants, capture)
|
||||
v, g, ref, fired = stage["v"], stage["g"], stage["ref"], stage["fired"]
|
||||
counts += fired.astype(np.int32)
|
||||
first_spike[fired & (first_spike < 0)] = np.int32(tick)
|
||||
fired_local = np.flatnonzero(fired)
|
||||
if len(fired_local):
|
||||
predicted_steps.append(np.full(len(fired_local), tick, np.int32))
|
||||
predicted_nodes.append(targets[fired_local].copy())
|
||||
due = incoming.due(event_index.at(tick - DELAY_STEPS))
|
||||
accepted = ref[incoming.dst_local[due]] == 0
|
||||
if len(due):
|
||||
accepted_edges = due[accepted]
|
||||
# add.at is deliberately unbuffered: start at existing g and add
|
||||
# each accepted edge in original order, NEVER sum weights first.
|
||||
np.add.at(g, incoming.dst_local[accepted_edges], incoming.weights[accepted_edges])
|
||||
if capture:
|
||||
stage.update(due_edge_indices=due.copy(), due_edge_id=incoming.edge_id[due].copy(),
|
||||
arrival_accepted=accepted.copy(), v_pre_apl=v.copy(), g_pre_apl=g.copy(), ref_pre_apl=ref.copy())
|
||||
delta_si = inhibition[tick].astype(np.float64) * np.float64(.001)
|
||||
g = np.subtract(g, delta_si)
|
||||
_require(np.isfinite(v).all() and np.isfinite(g).all(), "Arrival/APL stage produced a nonfinite value")
|
||||
if capture:
|
||||
stage.update(h32=inhibition[tick].copy(), delta_si=delta_si.copy(), v=v.copy(), g=g.copy(), ref=ref.copy(), fired=fired.copy())
|
||||
callback(tick, stage)
|
||||
return {"v": v.copy(), "g": g.copy(), "ref": ref.copy(), "counts": counts,
|
||||
"first_spike": first_spike, "target_globals": targets,
|
||||
"step": np.concatenate(predicted_steps) if predicted_steps else np.empty(0, np.int32),
|
||||
"neuron_idx": np.concatenate(predicted_nodes) if predicted_nodes else np.empty(0, np.int32)}
|
||||
@@ -0,0 +1,205 @@
|
||||
"""Reconstruct existing KC targets; never launch a brain simulation or CUDA."""
|
||||
from functools import lru_cache
|
||||
import time
|
||||
import traceback
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from context import ROOT, EXP, AG, META, COEFF, read, save, digest, check
|
||||
from replay_core import IncomingIndex, replay
|
||||
from accounting import DriveAccounting, bit_equal
|
||||
from populations import GROUPS
|
||||
|
||||
|
||||
@lru_cache(maxsize=2)
|
||||
def graph_view(graph_name):
|
||||
folder = AG / 'graphs' / graph_name
|
||||
nodes = np.load(folder / 'nodes.npy', mmap_mode='r')
|
||||
all_kc = np.load(AG / 'apl-bundle/kc_globals.npy', mmap_mode='r')
|
||||
apl = np.load(AG / 'apl-bundle/apl_globals.npy', mmap_mode='r')
|
||||
targets, node_positions, apl_columns = np.intersect1d(nodes, all_kc, assume_unique=True, return_indices=True)
|
||||
lookup = np.full(len(nodes), -1, np.int32)
|
||||
lookup[node_positions] = np.arange(len(targets), dtype=np.int32)
|
||||
destination = np.load(folder / 'dst.npy', mmap_mode='r')
|
||||
selected = np.flatnonzero(lookup[destination] >= 0)
|
||||
src_local = np.load(folder / 'src.npy', mmap_mode='r')[selected]
|
||||
src = nodes[src_local]
|
||||
dst = lookup[destination[selected]]
|
||||
ids = np.load(folder / 'edge_id.npy', mmap_mode='r')[selected]
|
||||
weights = np.load(folder / 'weights.npy', mmap_mode='r')[selected].copy()
|
||||
weights[np.isin(src, apl)] = 0.0
|
||||
original = np.load(ROOT / 'datasets/male-cns-v1.0/processed/neuron-only/gpu/csr-weights.int64.npy', mmap_mode='r')
|
||||
raw = original[ids]
|
||||
assert raw.dtype == np.int64 and np.all(raw > 0)
|
||||
incoming = IncomingIndex(src, dst, ids, weights, len(targets))
|
||||
group_frame = pd.read_parquet(EXP / 'protocol/neuron-source-groups.parquet')
|
||||
labels = group_frame.audit_source_group.to_numpy()
|
||||
group_codes = np.array([GROUPS.index(x) for x in labels], np.int32)[src]
|
||||
audit = dict(graph_name=graph_name, graph_nodes=len(nodes), graph_edges=len(destination),
|
||||
KC_targets=len(targets), incoming_edges=len(ids), incoming_contacts=int(raw.sum()),
|
||||
zeroed_APL_incoming_edges=int(np.isin(src, apl).sum()),
|
||||
groups={name: dict(edges=int(np.count_nonzero(group_codes == gi)),
|
||||
contacts=int(raw[group_codes == gi].sum())) for gi, name in enumerate(GROUPS)},
|
||||
source_graph=str(folder), conditional_target_filter_only=True, source_graph_modified=False)
|
||||
return incoming, raw, group_codes, targets, node_positions, apl_columns, audit
|
||||
|
||||
|
||||
def input_audit(case, source, spec, state, population):
|
||||
with np.load(case / 'requests.npz', allow_pickle=False) as data:
|
||||
req = {k: data[k].copy() for k in data.files}
|
||||
table = pd.read_csv(source / 'requests.csv')
|
||||
assert len(table) == len(req['tick'])
|
||||
assert np.array_equal(table.tick.to_numpy(), req['tick'])
|
||||
assert np.array_equal(table.neuron_idx.to_numpy(), req['neuron_idx'])
|
||||
assert (table.attempted == 1).all() and table.accepted.isin([0, 1]).all()
|
||||
assert np.array_equal(table.rejected.to_numpy(), 1 - table.accepted.to_numpy())
|
||||
request_keys = req['tick'].astype(np.int64) * 166700 + req['neuron_idx']
|
||||
event_keys = state['step'].astype(np.int64) * 166700 + state['neuron_idx']
|
||||
accepted = np.isin(request_keys, event_keys)
|
||||
assert np.array_equal(accepted, table.accepted.to_numpy().astype(bool))
|
||||
labels = population.audit_source_group.to_numpy()
|
||||
all_pn = np.flatnonzero(labels == 'ALPN').astype(np.int32)
|
||||
observed_count = np.bincount(state['neuron_idx'], minlength=166700)
|
||||
requested_count = np.bincount(req['neuron_idx'], minlength=166700)
|
||||
accepted_count = np.bincount(req['neuron_idx'][accepted], minlength=166700)
|
||||
targets = np.empty(0, np.int32)
|
||||
rates = np.empty(0, np.float32)
|
||||
if spec['kind'] == 'calibration':
|
||||
with np.load(case / 'pn-drive.npz', allow_pickle=False) as data:
|
||||
targets, rates = data['targets'].copy(), data['rates_hz'].copy()
|
||||
assert targets.dtype == np.int32 and rates.dtype == np.float32
|
||||
assert targets.shape == rates.shape == (spec['target_count'],)
|
||||
assert len(np.unique(targets)) == len(targets) and np.isfinite(rates).all() and (rates > 0).all()
|
||||
assert np.isin(targets, all_pn).all() and np.isin(req['neuron_idx'], targets).all()
|
||||
rate_map = dict(zip(targets.tolist(), rates.tolist()))
|
||||
pn_table = population.iloc[all_pn][['neuron_idx', 'body_id', 'annotation_type']].copy()
|
||||
pn_table['design_target'] = np.isin(all_pn, targets)
|
||||
pn_table['design_rate_hz'] = [rate_map.get(int(i), 0.0) for i in all_pn]
|
||||
pn_table['requests'] = requested_count[all_pn]
|
||||
pn_table['accepted_requests'] = accepted_count[all_pn]
|
||||
pn_table['rejected_requests'] = requested_count[all_pn] - accepted_count[all_pn]
|
||||
pn_table['actual_spikes'] = observed_count[all_pn]
|
||||
pn_table['spikes_not_coincident_with_accepted_request'] = observed_count[all_pn] - accepted_count[all_pn]
|
||||
assert (pn_table.spikes_not_coincident_with_accepted_request >= 0).all()
|
||||
info = dict(requests=len(request_keys), accepted=int(accepted.sum()), rejected=int((~accepted).sum()),
|
||||
actual_spikes_by_source_group={name: int(observed_count[labels == name].sum()) for name in GROUPS},
|
||||
all_ALPN=len(all_pn), odor_design_target_count=len(targets), PN_ever_requested=int(np.count_nonzero(requested_count[all_pn])),
|
||||
PN_ever_accepted_request=int(np.count_nonzero(accepted_count[all_pn])),
|
||||
PN_actual_active=int(np.count_nonzero(observed_count[all_pn])),
|
||||
design_target_PN_spikes=int(observed_count[targets].sum()),
|
||||
non_design_ALPN_spikes=int(observed_count[np.setdiff1d(all_pn, targets)].sum()),
|
||||
accepted_request_is_not_proof_spike_was_caused_exclusively_by_request=True,
|
||||
actual_requested_events_not_forced_to_equal_across_odors=True)
|
||||
return req, info, pn_table
|
||||
|
||||
|
||||
def audit_trial(row, population):
|
||||
start = time.perf_counter()
|
||||
name = row['case'] + '-r' + str(row['repeat'])
|
||||
out = EXP / 'trials' / name
|
||||
out.mkdir(exist_ok=False)
|
||||
source = ROOT / row['source']
|
||||
case = AG / 'cases' / row['case']
|
||||
spec = read(case / 'spec.json')
|
||||
graph_name = 'fixtures' if row['kind'] == 'fixture' else 'full'
|
||||
incoming, raw, groups, targets, positions, apl_columns, graph_audit = graph_view(graph_name)
|
||||
with np.load(source / 'state.npz', allow_pickle=False) as data:
|
||||
state = {k: data[k].copy() for k in data.files}
|
||||
assert set(state) == {'step', 'neuron_idx', 'counts', 'v', 'g', 'ref'}
|
||||
assert all(np.isfinite(v).all() for v in state.values())
|
||||
req, input_report, pn_table = input_audit(case, source, spec, state, population)
|
||||
event_mask = np.isin(state['neuron_idx'], targets)
|
||||
expected = {key: state[key][event_mask] for key in ('step', 'neuron_idx')}
|
||||
initial_v = np.full(len(targets), spec['model']['resting_potential_mv'], np.float64) * np.float64(.001)
|
||||
initial_g = np.zeros(len(targets), np.float64)
|
||||
if (case / 'arrays.npz').exists():
|
||||
with np.load(case / 'arrays.npz', allow_pickle=False) as data:
|
||||
initial_v = data['initial_v_mv'][positions] * np.float64(.001)
|
||||
initial_g = data['initial_g_mv'][positions] * np.float64(.001)
|
||||
histories = {}
|
||||
if row['kind'] == 'fixture':
|
||||
for key, suffix in (('v', 'float64'), ('g', 'float64'), ('ref', 'int32')):
|
||||
histories[key] = np.load(source / f'{key}-history.{suffix}.npy', mmap_mode='r')[:, positions]
|
||||
histories['pre_' + key] = np.load(source / f'preAPL-{key}.{suffix}.npy', mmap_mode='r')[:, positions]
|
||||
h = np.load(source / 'apl-inhibition.float32.npy', mmap_mode='r')
|
||||
assert h.shape == (spec['steps'], 4064)
|
||||
if not np.array_equal(apl_columns, np.arange(4064)):
|
||||
h = h[:, apl_columns]
|
||||
coeff = read(COEFF)
|
||||
accounting = DriveAccounting(targets, incoming, raw, groups, GROUPS, expected,
|
||||
coeff, state['counts'][positions], histories)
|
||||
result = replay(initial_v, initial_g, coeff, spec['steps'], targets, req,
|
||||
dict(step=state['step'], neuron_idx=state['neuron_idx']), incoming, h, accounting)
|
||||
exact = {key: bit_equal(result[key], state[key][positions]) for key in ('v', 'g', 'ref', 'counts')}
|
||||
exact.update({key: bit_equal(result[key], expected[key]) for key in ('step', 'neuron_idx')})
|
||||
assert all(exact.values()), 'Reconstruction diverged: ' + str(exact)
|
||||
assert accounting.callback_count == spec['steps']
|
||||
data, report = accounting.finish(result['g'])
|
||||
assert bit_equal(data['first_spike_tick'], result['first_spike'])
|
||||
per_kc = pd.DataFrame(data)
|
||||
per_kc.insert(1, 'body_id', population.iloc[targets].body_id.to_numpy())
|
||||
per_kc.to_parquet(out / 'per-kc.parquet', index=False)
|
||||
pn_table.to_parquet(out / 'per-pn.parquet', index=False)
|
||||
pd.DataFrame(accounting.tick_rows).to_parquet(out / 'per-tick.parquet', index=False)
|
||||
with (out / 'replay-final.npz').open('xb') as f:
|
||||
np.savez_compressed(f, **result)
|
||||
late_counts = np.bincount(state['neuron_idx'][state['step'] >= spec['steps'] - 18], minlength=166700)
|
||||
pending = late_counts[incoming.src_global]
|
||||
report.update(case=row['case'], repeat=row['repeat'], kind=row['kind'], pass_all=True,
|
||||
exact=exact, full_KC_sparse_events_and_final_state_exact=True,
|
||||
every_tick_fixture_pre_post_state_exact=bool(histories),
|
||||
graph=graph_audit, input=input_report, steps=spec['steps'], duration_ms=spec['duration_ms'],
|
||||
KC_active=int(np.count_nonzero(result['counts'])), KC_spikes=int(result['counts'].sum()),
|
||||
pending_after_record_window=dict(edge_events=int(pending.sum()), contacts=int(np.dot(pending, raw)),
|
||||
nominal_abs_SI=float(np.dot(pending, np.abs(incoming.weights)))),
|
||||
mode=spec.get('mode'), odor_id=spec.get('odor_id'), seed=spec.get('seed'),
|
||||
CPU_audit_wall_seconds=time.perf_counter() - start,
|
||||
original_brain_time_and_wall_not_rebenchmarked=True, training=False, new_brain_simulation=False,
|
||||
candidate_generated=False, input_sha256={filename: digest(source / filename)
|
||||
for filename in ('state.npz', 'apl-inhibition.float32.npy', 'requests.csv', 'summary.json')})
|
||||
save(out / 'summary.json', report)
|
||||
return report
|
||||
|
||||
|
||||
def main():
|
||||
check()
|
||||
prepared = read(EXP / 'protocol/prepared-hashes.json')
|
||||
for row in prepared:
|
||||
assert digest(EXP / row['path']) == row['sha256']
|
||||
population = pd.read_parquet(EXP / 'protocol/neuron-source-groups.parquet')
|
||||
inventory = read(EXP / 'protocol/inventory.json')
|
||||
reports = []
|
||||
for row in inventory:
|
||||
try:
|
||||
report = audit_trial(row, population)
|
||||
reports.append(dict(case=row['case'], repeat=row['repeat'], pass_all=report['pass_all']))
|
||||
assert sum(p.stat().st_size for p in EXP.rglob('*') if p.is_file()) < 2 * 1024 ** 3
|
||||
print(row['case'], row['repeat'], 'EXACT', report['KC_active'],
|
||||
'KC active; CPU audit seconds', round(report['CPU_audit_wall_seconds'], 3), flush=True)
|
||||
except Exception as error:
|
||||
save(EXP / 'audit/failure.json', dict(case=row, error=type(error).__name__, message=str(error),
|
||||
traceback=traceback.format_exc(), completed=reports, new_brain_simulation=False))
|
||||
raise
|
||||
repeat_checks = []
|
||||
for row in inventory:
|
||||
if row['repeat'] != 1:
|
||||
continue
|
||||
first = EXP / 'trials' / (row['case'] + '-r0')
|
||||
repeat = EXP / 'trials' / (row['case'] + '-r1')
|
||||
checks = {}
|
||||
for filename in ('per-kc.parquet', 'per-pn.parquet', 'per-tick.parquet'):
|
||||
a, b = pd.read_parquet(first / filename), pd.read_parquet(repeat / filename)
|
||||
checks[filename] = a.equals(b)
|
||||
with np.load(first / 'replay-final.npz') as a, np.load(repeat / 'replay-final.npz') as b:
|
||||
checks['replay-final'] = set(a.files) == set(b.files) and all(bit_equal(a[k], b[k]) for k in a.files)
|
||||
assert all(checks.values()), 'Repeat accounting differs'
|
||||
repeat_checks.append(dict(case=row['case'], pass_all=True, checks=checks))
|
||||
assert len(reports) == 80 and len(repeat_checks) == 12
|
||||
save(EXP / 'audit/completed.json', dict(pass_all=True, trials=reports, repeat_checks=repeat_checks,
|
||||
meaning='Exact conditional KC reconstruction and reproducible accounting only', candidate_generated=False))
|
||||
print('PASS 80/80 conditional reconstructions; 12/12 accounting repeats', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,246 @@
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
import numpy as np
|
||||
from accounting import DriveAccounting, bit_equal, fraction
|
||||
|
||||
|
||||
GROUPS = ('KC', 'APL', 'ALPN', 'MBON', 'anatomical_DAN',
|
||||
'dopamine_annotation_only', 'other', 'unclassified')
|
||||
|
||||
|
||||
def _events(ticks=(), nodes=()):
|
||||
return {'step': np.asarray(ticks, np.int32),
|
||||
'neuron_idx': np.asarray(nodes, np.int32)}
|
||||
|
||||
|
||||
def _account(targets, destinations=(), weights=(), raw=(), groups=(),
|
||||
expected=None, counts=None, gd=.5, histories=None):
|
||||
targets = np.asarray(targets, np.int32)
|
||||
incoming = SimpleNamespace(dst_local=np.asarray(destinations, np.int32),
|
||||
weights=np.asarray(weights, np.float64))
|
||||
coeff = {'gd': np.float64(gd), 'threshold': np.float64(-.045)}
|
||||
return DriveAccounting(targets, incoming, np.asarray(raw, np.int64),
|
||||
np.asarray(groups, np.int32), GROUPS,
|
||||
_events() if expected is None else expected, coeff,
|
||||
np.zeros(len(targets), np.int32) if counts is None else np.asarray(counts, np.int32),
|
||||
histories=histories)
|
||||
|
||||
|
||||
def _stage(old_g, g_decayed, g_after_reset, g_pre_apl, delta, *,
|
||||
available, fired, ref, due=(), accepted=(), v_threshold=None):
|
||||
"""Manually specified accounting callback; no runtime or replay dependency."""
|
||||
old_g = np.asarray(old_g, np.float64)
|
||||
n = len(old_g)
|
||||
delta = np.asarray(delta, np.float64)
|
||||
pre = np.asarray(g_pre_apl, np.float64)
|
||||
ref = np.asarray(ref, np.int32)
|
||||
threshold_v = np.full(n, -.05, np.float64) if v_threshold is None else np.asarray(v_threshold, np.float64)
|
||||
return {'old_g': old_g,
|
||||
'g_decayed': np.asarray(g_decayed, np.float64),
|
||||
'g_after_reset': np.asarray(g_after_reset, np.float64),
|
||||
'g_pre_apl': pre,
|
||||
'delta_si': delta,
|
||||
'g': np.subtract(pre, delta),
|
||||
'available': np.asarray(available, bool),
|
||||
'fired': np.asarray(fired, bool),
|
||||
'ref': ref,
|
||||
'ref_pre_apl': ref.copy(),
|
||||
'v': np.full(n, -.052, np.float64),
|
||||
'v_pre_apl': np.full(n, -.052, np.float64),
|
||||
'v_threshold': threshold_v,
|
||||
'due_edge_indices': np.asarray(due, np.int64),
|
||||
'arrival_accepted': np.asarray(accepted, bool)}
|
||||
|
||||
|
||||
def _two_tick_fixture():
|
||||
# KC 10 fires immediately, KC 20 first fires one tick later, KC 30 never
|
||||
# fires. All contacts are positive anatomical counts; signed effective
|
||||
# SI increments are an independent vector, including a zeroed APL edge.
|
||||
account = _account([10, 20, 30], [0, 1, 2, 2, 2, 0],
|
||||
[1., 2., 3., -1., 0., -2.], [4, 8, 12, 4, 16, 8],
|
||||
[2, 2, 0, 3, 1, 3], _events([0, 1], [10, 20]), [1, 1, 0])
|
||||
first = _stage([2., 4., 8.], [1., 2., 4.], [0., 2., 4.],
|
||||
[0., 4., 6.], [.25, .5, 1.], available=[1, 1, 1],
|
||||
fired=[1, 0, 0], ref=[22, 0, 0], due=np.arange(6),
|
||||
accepted=[0, 1, 1, 1, 1, 0], v_threshold=[-.044, -.049, -.05])
|
||||
second = _stage([-.25, 3.5, 5.], [-.25, 1.75, 2.5],
|
||||
[-.25, 0., 2.5], [-.25, 0., 2.5], [.25, .5, 1.],
|
||||
available=[0, 1, 1], fired=[0, 1, 0], ref=[21, 22, 0],
|
||||
v_threshold=[-.052, -.044, -.05])
|
||||
return account, first, second
|
||||
|
||||
|
||||
class AccountingTests(unittest.TestCase):
|
||||
def test_bitwise_signed_zero(self):
|
||||
self.assertFalse(bit_equal(np.array([0.0]), np.array([-0.0])))
|
||||
|
||||
def test_dtype_is_part_of_evidence(self):
|
||||
self.assertFalse(bit_equal(np.array([1], np.int32), np.array([1], np.int64)))
|
||||
|
||||
def test_shape_is_part_of_evidence(self):
|
||||
self.assertFalse(bit_equal(np.array([1.0]), np.array([[1.0]])))
|
||||
|
||||
def test_equal(self):
|
||||
self.assertTrue(bit_equal(np.array([1.0, -2.0]), np.array([1.0, -2.0])))
|
||||
|
||||
def test_zero_denominator_is_unknown(self):
|
||||
self.assertIsNone(fraction(1, 0))
|
||||
self.assertEqual(fraction(1, 2), .5)
|
||||
|
||||
def test_actual_stage_ledger_telescopes_without_source_feedback(self):
|
||||
account, first, second = _two_tick_fixture()
|
||||
account(0, first)
|
||||
account(1, second)
|
||||
data, report = account.finish(second['g'])
|
||||
for name, expected in {
|
||||
'decay': [-1., -3.75, -6.5], 'reset': [-1., -1.75, 0.],
|
||||
'arrivals': [0., 2., 2.], 'APL': [-.5, -1., -2.]}.items():
|
||||
np.testing.assert_array_equal(data['actual_g_delta_' + name + '_V'], expected)
|
||||
np.testing.assert_array_equal(data['g_ledger_residual_V'], np.zeros(3))
|
||||
np.testing.assert_array_equal(data['initial_g_state_area_V_ms'], [0., .2, .4 + .2])
|
||||
self.assertTrue(report['ledger_roundoff_bound_pass'])
|
||||
self.assertFalse(report['APL_cause_of_CAL_failure_proven'])
|
||||
self.assertFalse(report['full_CAL_all_tick_subset_state_independently_verified'])
|
||||
|
||||
def test_ref_write_includes_new_spike_and_already_refractory(self):
|
||||
account, first, second = _two_tick_fixture()
|
||||
account(0, first)
|
||||
account(1, second)
|
||||
data, _ = account.finish(second['g'])
|
||||
np.testing.assert_array_equal(data['APL_write_ref_SI'], [.5, .5, 0.])
|
||||
np.testing.assert_array_equal(data['APL_write_available_SI'], [0., .5, 2.])
|
||||
np.testing.assert_array_equal(data['APL_write_SI'],
|
||||
data['APL_write_ref_SI'] + data['APL_write_available_SI'])
|
||||
np.testing.assert_array_equal(data['refractory_ticks'], [2., 1., 0.])
|
||||
|
||||
def test_first_spike_tick_arrival_and_apl_are_not_prior_causes(self):
|
||||
account, first, second = _two_tick_fixture()
|
||||
account(0, first)
|
||||
account(1, second)
|
||||
data, _ = account.finish(second['g'])
|
||||
np.testing.assert_array_equal(data['first_spike_tick'], [0, 1, -1])
|
||||
np.testing.assert_array_equal(data['APL_pre_first_spike_SI'], [0., .5, 2.])
|
||||
np.testing.assert_array_equal(data['APL_same_tick_first_spike_SI'], [.25, .5, 0.])
|
||||
np.testing.assert_array_equal(data['accepted_exc_before_first_SI'], [0., 2., 3.])
|
||||
np.testing.assert_array_equal(data['accepted_inh_before_first_abs_SI'], [0., 0., 1.])
|
||||
|
||||
def test_group_contact_sign_and_rejected_edge_accounting(self):
|
||||
account, first, second = _two_tick_fixture()
|
||||
account(0, first)
|
||||
account(1, second)
|
||||
data, report = account.finish(second['g'])
|
||||
for name in GROUPS:
|
||||
np.testing.assert_array_equal(data[name + '_due_edges'],
|
||||
data[name + '_accepted_edges'] + data[name + '_rejected_edges'])
|
||||
np.testing.assert_array_equal(data[name + '_due_contacts'],
|
||||
data[name + '_accepted_contacts'] + data[name + '_rejected_contacts'])
|
||||
np.testing.assert_array_equal(data['ALPN_accepted_exc_SI'], [0., 2., 0.])
|
||||
np.testing.assert_array_equal(data['ALPN_rejected_exc_SI'], [1., 0., 0.])
|
||||
np.testing.assert_array_equal(data['MBON_accepted_inh_abs_SI'], [0., 0., 1.])
|
||||
np.testing.assert_array_equal(data['MBON_rejected_inh_abs_SI'], [2., 0., 0.])
|
||||
self.assertEqual(report['strata']['never_active']['neurons'], 1)
|
||||
self.assertEqual(report['strata']['never_active']['no_accepted_ALPN_exc_neurons'], 1)
|
||||
|
||||
def test_zeroed_apl_keeps_anatomical_edge_and_contact_counts(self):
|
||||
account, first, second = _two_tick_fixture()
|
||||
account(0, first)
|
||||
account(1, second)
|
||||
data, _ = account.finish(second['g'])
|
||||
np.testing.assert_array_equal(data['APL_zeroed_due_edges'], [0., 0., 1.])
|
||||
np.testing.assert_array_equal(data['APL_zeroed_due_contacts'], [0., 0., 16.])
|
||||
np.testing.assert_array_equal(data['APL_accepted_edges'], [0, 0, 1])
|
||||
np.testing.assert_array_equal(data['APL_accepted_exc_SI'], np.zeros(3))
|
||||
np.testing.assert_array_equal(data['APL_accepted_inh_abs_SI'], np.zeros(3))
|
||||
|
||||
def test_observer_never_modifies_supplied_callback_arrays(self):
|
||||
account, first, _ = _two_tick_fixture()
|
||||
original = {name: value.copy() for name, value in first.items()}
|
||||
account(0, first)
|
||||
for name, value in original.items():
|
||||
self.assertTrue(bit_equal(first[name], value), name)
|
||||
|
||||
def test_independent_spike_disagreement_stops_before_accounting(self):
|
||||
account, first, _ = _two_tick_fixture()
|
||||
first['fired'][0] = False
|
||||
with self.assertRaisesRegex(ValueError, 'KC spike divergence at tick 0'):
|
||||
account(0, first)
|
||||
self.assertEqual(account.callback_count, 0)
|
||||
|
||||
def test_dense_fixture_history_disagreement_is_not_tolerated(self):
|
||||
account, first, _ = _two_tick_fixture()
|
||||
account.histories = {'pre_g': first['g_pre_apl'][None, :].copy()}
|
||||
account.histories['pre_g'][0, 1] = np.nextafter(account.histories['pre_g'][0, 1], np.inf)
|
||||
with self.assertRaisesRegex(ValueError, 'Fixture dense pre_g divergence'):
|
||||
account(0, first)
|
||||
|
||||
def test_cancellation_reports_roundoff_instead_of_forcing_source_sum(self):
|
||||
account = _account([10], [0, 0], [2. ** 53, -(2. ** 53)],
|
||||
[1, 1], [2, 3], gd=1.)
|
||||
stage = _stage([1.], [1.], [1.], [0.], [0.],
|
||||
available=[1], fired=[0], ref=[0], due=[0, 1], accepted=[1, 1])
|
||||
account(0, stage)
|
||||
data, report = account.finish(stage['g'])
|
||||
np.testing.assert_array_equal(data['arrival_rounding_V'], [-1.])
|
||||
self.assertEqual(report['source_shadow_max_residual_V'], 1.)
|
||||
np.testing.assert_array_equal(data['initial_g_state_area_V_ms'], [.1])
|
||||
np.testing.assert_array_equal(data['source_g_area_residual_V_ms'], [.1])
|
||||
self.assertEqual(report['source_area_max_residual_V_ms'], .1)
|
||||
np.testing.assert_array_equal(data['g_ledger_residual_V'], [0.])
|
||||
|
||||
def test_initial_g_is_not_misattributed_to_a_source_group(self):
|
||||
account = _account([10])
|
||||
stage = _stage([2.], [1.], [1.], [1.], [0.],
|
||||
available=[1], fired=[0], ref=[0])
|
||||
account(0, stage)
|
||||
data, report = account.finish(stage['g'])
|
||||
for key, value in data.items():
|
||||
if key.startswith('source_g_area_'):
|
||||
np.testing.assert_array_equal(value, [0.])
|
||||
self.assertEqual(report['source_shadow_max_residual_V'], 0.)
|
||||
np.testing.assert_array_equal(data['actual_g_delta_decay_V'], [-1.])
|
||||
np.testing.assert_array_equal(data['initial_g_state_area_V_ms'], [.1])
|
||||
np.testing.assert_array_equal(data['source_g_area_residual_V_ms'], [0.])
|
||||
for stratum in ('all', 'never_active'):
|
||||
self.assertEqual(report['strata'][stratum]['initial_g_state_area_V_ms'], .1)
|
||||
self.assertEqual(report['strata'][stratum]['source_g_area_residual_V_ms'], 0.)
|
||||
self.assertEqual(report['strata']['ever_active']['initial_g_state_area_V_ms'], 0.)
|
||||
self.assertEqual(report['ledger_accumulator_mantissa_bits'], np.finfo(np.longdouble).nmant)
|
||||
self.assertIn('eps(longdouble)', report['ledger_bound_formula'])
|
||||
|
||||
def test_wrong_final_g_fails_ledger_closure(self):
|
||||
account, first, second = _two_tick_fixture()
|
||||
account(0, first)
|
||||
account(1, second)
|
||||
with self.assertRaisesRegex(AssertionError, 'g ledger residual'):
|
||||
account.finish(second['g'] + 1.)
|
||||
|
||||
def test_synthetic_replay_accounting_integration_keeps_delay_and_stage_order(self):
|
||||
from replay_core import IncomingIndex, replay
|
||||
targets = np.asarray([10, 20], np.int32)
|
||||
expected = _events([0, 19], [10, 20])
|
||||
coeff = {'gd': .5, 'c0': 0., 'cg': 1., 'cv': 1.,
|
||||
'threshold': 1., 'reset': 0., 'seed_volt': 2.}
|
||||
incoming = IncomingIndex(np.asarray([10], np.int32), np.asarray([1], np.int32),
|
||||
np.asarray([123], np.int64), np.asarray([2.], np.float64), 2)
|
||||
account = DriveAccounting(targets, incoming, np.asarray([5], np.int64),
|
||||
np.asarray([0], np.int32), GROUPS, expected, coeff, np.asarray([1, 1], np.int32))
|
||||
result = replay(np.zeros(2, np.float64), np.zeros(2, np.float64), coeff, 24,
|
||||
targets, _events([0], [10]), expected, incoming,
|
||||
np.zeros((24, 2), np.float32), callback=account)
|
||||
data, report = account.finish(result['g'])
|
||||
np.testing.assert_array_equal(result['step'], [0, 19])
|
||||
np.testing.assert_array_equal(result['neuron_idx'], targets)
|
||||
np.testing.assert_array_equal(result['counts'], [1, 1])
|
||||
np.testing.assert_array_equal(result['ref'], [0, 18])
|
||||
np.testing.assert_array_equal(data['first_spike_tick'], [0, 19])
|
||||
np.testing.assert_array_equal(data['KC_accepted_contacts'], [0, 5])
|
||||
np.testing.assert_array_equal(data['actual_g_delta_arrivals_V'], [0., 2.])
|
||||
np.testing.assert_array_equal(data['actual_g_delta_decay_V'], [0., -1.])
|
||||
np.testing.assert_array_equal(data['actual_g_delta_reset_V'], [0., -1.])
|
||||
np.testing.assert_array_equal(data['source_g_area_KC_exc_V_ms'], [0., .2])
|
||||
self.assertEqual(account.callback_count, 24)
|
||||
self.assertTrue(report['ledger_roundoff_bound_pass'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,105 @@
|
||||
"""Synthetic CPU-only classification tests; no scientific files are read."""
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from populations import N_NEURONS, GROUPS, classify
|
||||
|
||||
|
||||
def synthetic_metadata():
|
||||
return pd.DataFrame({"neuron_idx": np.arange(N_NEURONS, dtype=np.int32),
|
||||
"body_id": np.arange(1, N_NEURONS + 1, dtype=np.uint64),
|
||||
"annotation_class": np.full(N_NEURONS, "ALLN", dtype=object),
|
||||
"annotation_type": np.full(N_NEURONS, "unresolved_function", dtype=object),
|
||||
"neurotransmitter": np.full(N_NEURONS, "acetylcholine", dtype=object),
|
||||
"nt_ground_truth": np.full(N_NEURONS, None, dtype=object)})
|
||||
|
||||
|
||||
class PopulationTests(unittest.TestCase):
|
||||
def test_exact_groups_are_disjoint_and_full(self):
|
||||
meta = synthetic_metadata()
|
||||
classes = ["Kenyon_Cell", None, "ALPN", "MBON", "DAN", None, "olfactory", None]
|
||||
types = ["KCg", "APL", "VA6_adPN", "MBON01", "PAM15", None, "ORN", None]
|
||||
meta.loc[:7, "annotation_class"] = classes
|
||||
meta.loc[:7, "annotation_type"] = types
|
||||
meta.loc[5, "neurotransmitter"] = " DOPAMINE "
|
||||
before = meta.copy(deep=True)
|
||||
labels, report = classify(meta)
|
||||
self.assertEqual(labels[:8].tolist(), list(GROUPS))
|
||||
self.assertEqual(labels.dtype.kind, "U")
|
||||
self.assertEqual(sum(report["groups"].values()), N_NEURONS)
|
||||
self.assertTrue(report["all_neurons_assigned_once"])
|
||||
self.assertTrue(report["pass_all"])
|
||||
pd.testing.assert_frame_equal(meta, before)
|
||||
|
||||
def test_dopamine_does_not_prove_DAN_and_DPM_is_other(self):
|
||||
meta = synthetic_metadata()
|
||||
meta.loc[:3, "annotation_class"] = [None, None, "DAN", "ALPN"]
|
||||
meta.loc[:3, "annotation_type"] = ["DPM", "PAM-not-proof", "PAM15", "VA6_adPN"]
|
||||
meta.loc[:3, "neurotransmitter"] = ["dopamine", "dopamine", "gaba", "dopamine"]
|
||||
labels, report = classify(meta)
|
||||
self.assertEqual(labels[:4].tolist(), ["other", "dopamine_annotation_only", "anatomical_DAN", "ALPN"])
|
||||
self.assertEqual(report["DPM_neuron_idx"], [0])
|
||||
self.assertEqual(report["flags"]["annotation_DAN_not_metadata_dopamine"], 1)
|
||||
self.assertEqual(report["uncertain_DAN_prefix_neuron_idx"], [1])
|
||||
self.assertFalse(report["reward_or_valence_inferred"])
|
||||
|
||||
def test_DPM_and_exact_identity_conflicts_are_unclassified(self):
|
||||
meta = synthetic_metadata()
|
||||
meta.loc[:2, "annotation_class"] = ["DAN", "Kenyon_Cell", "MBON"]
|
||||
meta.loc[:2, "annotation_type"] = ["DPM", "APL", "DPM"]
|
||||
labels, report = classify(meta)
|
||||
self.assertEqual(labels[:3].tolist(), ["unclassified"] * 3)
|
||||
self.assertEqual(report["identity_conflict_neuron_idx"], [0, 1, 2])
|
||||
self.assertFalse(report["pass_all"])
|
||||
self.assertTrue(report["all_neurons_assigned_once"])
|
||||
|
||||
def test_prefix_and_missing_evidence_are_not_guessed(self):
|
||||
meta = synthetic_metadata()
|
||||
meta.loc[:6, "annotation_class"] = [None, "", "unknown", None, None, None, "olfactory"]
|
||||
meta.loc[:6, "annotation_type"] = ["PAM15", "PPL1-01", "DAN-like", None, "APL", "DPM", "ALPN-like"]
|
||||
labels, report = classify(meta)
|
||||
self.assertEqual(labels[:7].tolist(), ["unclassified"] * 4 + ["APL", "other", "other"])
|
||||
self.assertEqual(report["uncertain_DAN_prefix_neuron_idx"], [0, 1, 2])
|
||||
self.assertEqual(report["groups"]["ALPN"], 0)
|
||||
|
||||
def test_case_sensitive_anatomical_values_not_normalized_into_identity(self):
|
||||
meta = synthetic_metadata()
|
||||
meta.loc[:2, "annotation_class"] = ["dan", "ALPN ", "kenyon_cell"]
|
||||
meta.loc[:2, "annotation_type"] = ["type", "type", "type"]
|
||||
labels, _ = classify(meta)
|
||||
self.assertEqual(labels[:3].tolist(), ["other"] * 3)
|
||||
|
||||
def test_missing_schema_and_numeric_annotations_fail_closed(self):
|
||||
meta = synthetic_metadata()
|
||||
with self.assertRaises(ValueError):
|
||||
classify(meta.drop(columns="nt_ground_truth"))
|
||||
meta["annotation_type"] = meta["annotation_type"].astype(object)
|
||||
meta.loc[0, "annotation_type"] = 123
|
||||
with self.assertRaises(ValueError):
|
||||
classify(meta)
|
||||
|
||||
def test_index_dtype_order_and_duplicate_body_fail_closed(self):
|
||||
for failure in ("short", "reordered", "index_dtype", "duplicate_body", "body_dtype"):
|
||||
meta = synthetic_metadata()
|
||||
if failure == "short": meta = meta.iloc[:-1]
|
||||
if failure == "reordered": meta = meta.iloc[::-1]
|
||||
if failure == "index_dtype": meta["neuron_idx"] = meta.neuron_idx.astype(np.int64)
|
||||
if failure == "duplicate_body": meta.loc[1, "body_id"] = meta.loc[0, "body_id"]
|
||||
if failure == "body_dtype": meta["body_id"] = meta.body_id.astype(np.int64)
|
||||
with self.subTest(failure=failure), self.assertRaises(ValueError):
|
||||
classify(meta)
|
||||
|
||||
def test_raw_ground_truth_is_only_an_evidence_flag(self):
|
||||
meta = synthetic_metadata()
|
||||
meta.loc[0, "nt_ground_truth"] = "dopamine"
|
||||
labels, report = classify(meta)
|
||||
self.assertEqual(labels[0], "other")
|
||||
self.assertEqual(report["flags"]["has_raw_ground_truth"], 1)
|
||||
self.assertEqual(report["groups"]["anatomical_DAN"], 0)
|
||||
self.assertFalse(report["provenance"]["file_hash_verified_by_this_pure_function"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,237 @@
|
||||
"""Synthetic CPU-only contract tests; no real trajectories, GPU, or file writes."""
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
|
||||
from replay_core import IncomingIndex, replay, state_step
|
||||
|
||||
|
||||
def coefficients(**changes):
|
||||
result = dict(gd=1., c0=0., cg=0., cv=1., threshold=1., reset=0., seed_volt=2.)
|
||||
result.update(changes)
|
||||
return result
|
||||
|
||||
|
||||
def record(rows=(), key="tick"):
|
||||
values = np.array(rows, np.int32).reshape(-1, 2)
|
||||
return {key: values[:, 0], "neuron_idx": values[:, 1]}
|
||||
|
||||
|
||||
def empty_index(count=1):
|
||||
return IncomingIndex(np.empty(0, np.int32), np.empty(0, np.int32),
|
||||
np.empty(0, np.int64), np.empty(0, np.float64), count)
|
||||
|
||||
|
||||
def one_edge(weight=2.):
|
||||
return IncomingIndex(np.array([10], np.int32), np.array([0], np.int32),
|
||||
np.array([81], np.int64), np.array([weight], np.float64), 1)
|
||||
|
||||
|
||||
def shadow(steps=25, *, initial_v=None, initial_g=None, coeff=None, requests=None,
|
||||
events=None, incoming=None, h32=None, callback=None, targets=None):
|
||||
targets = np.array([100], np.int32) if targets is None else targets
|
||||
n = len(targets)
|
||||
return replay(np.zeros(n, np.float64) if initial_v is None else initial_v,
|
||||
np.zeros(n, np.float64) if initial_g is None else initial_g,
|
||||
coefficients() if coeff is None else coeff, steps, targets,
|
||||
record() if requests is None else requests,
|
||||
record() if events is None else events,
|
||||
empty_index(n) if incoming is None else incoming,
|
||||
np.zeros((steps, n), np.float32) if h32 is None else h32, callback)
|
||||
|
||||
|
||||
class StepTests(unittest.TestCase):
|
||||
def test_strict_threshold_and_positive_zero_reset(self):
|
||||
v = np.array([1., np.nextafter(1., np.inf)], np.float64)
|
||||
result = state_step(v, np.array([3., -3.]), np.zeros(2, np.int32), np.zeros(2, bool), coefficients())
|
||||
np.testing.assert_array_equal(result["fired"], [False, True])
|
||||
np.testing.assert_array_equal(result["ref"], [0, 22])
|
||||
self.assertEqual(result["g"].view(np.uint64)[1], 0)
|
||||
self.assertEqual(result["v_threshold"][1], v[1])
|
||||
|
||||
def test_voltage_uses_old_g_and_separate_expression_order(self):
|
||||
c = coefficients(gd=.4, cg=.2, cv=.3, c0=.1, threshold=100., seed_volt=101.)
|
||||
result = state_step(np.array([3.]), np.array([2.]), np.array([0], np.int32), np.array([False]), c)
|
||||
expected = np.float64(c["c0"]) + ((np.float64(c["cg"]) * np.float64(2.)) +
|
||||
(np.float64(c["cv"]) * np.float64(3.)))
|
||||
self.assertEqual(result["v"][0].hex(), expected.hex())
|
||||
self.assertEqual(result["g"][0].hex(), (np.float64(.4) * np.float64(2.)).hex())
|
||||
self.assertNotEqual(result["v"][0], .1 + (.2 * .8 + .3 * 3.))
|
||||
|
||||
def test_negative_g_and_voltage_freeze_when_refractory(self):
|
||||
result = state_step(np.array([-.052]), np.array([-.123]), np.array([22], np.int32),
|
||||
np.array([True]), coefficients(gd=.5, cg=1., c0=.3))
|
||||
self.assertEqual(result["v"][0], -.052)
|
||||
self.assertEqual(result["g"][0], -.123)
|
||||
self.assertEqual(result["ref"][0], 21)
|
||||
self.assertFalse(result["fired"][0])
|
||||
self.assertTrue(result["request_rejected"][0])
|
||||
|
||||
def test_ref_one_releases_before_request_and_reset(self):
|
||||
result = state_step(np.array([-.052]), np.array([-.123]), np.array([1], np.int32),
|
||||
np.array([True]), coefficients())
|
||||
self.assertTrue(result["available"][0])
|
||||
self.assertTrue(result["request_accepted"][0])
|
||||
self.assertEqual(result["v_threshold"][0], 2.)
|
||||
self.assertTrue(result["fired"][0])
|
||||
self.assertEqual(result["ref"][0], 22)
|
||||
self.assertEqual(result["g"][0], 0.)
|
||||
|
||||
def test_state_inputs_unchanged(self):
|
||||
v, g, ref, request = np.array([2.]), np.array([5.]), np.array([0], np.int32), np.array([False])
|
||||
inputs = [x.copy() for x in (v, g, ref, request)]
|
||||
state_step(v, g, ref, request, coefficients())
|
||||
for actual, expected in zip((v, g, ref, request), inputs):
|
||||
np.testing.assert_array_equal(actual, expected)
|
||||
|
||||
|
||||
class ReplayTests(unittest.TestCase):
|
||||
def test_logged_target_spikes_are_not_forced(self):
|
||||
result = shadow(events=record([(0, 100), (22, 100)]))
|
||||
self.assertEqual(len(result["step"]), 0)
|
||||
np.testing.assert_array_equal(result["first_spike"], [-1])
|
||||
|
||||
def test_predicted_events_exist_without_logged_target_spikes(self):
|
||||
result = shadow(requests=record([(0, 100), (1, 100), (21, 100), (22, 100), (23, 100)]))
|
||||
np.testing.assert_array_equal(result["step"], [0, 22])
|
||||
np.testing.assert_array_equal(result["neuron_idx"], [100, 100])
|
||||
np.testing.assert_array_equal(result["counts"], [2])
|
||||
np.testing.assert_array_equal(result["first_spike"], [0])
|
||||
|
||||
def test_anatomical_arrival_is_tick18_then_voltage_tick19(self):
|
||||
history = []
|
||||
shadow(steps=20, coeff=coefficients(cg=1., threshold=100., seed_volt=101.),
|
||||
events=record([(0, 10)]), incoming=one_edge(), callback=lambda t, s: history.append(s))
|
||||
self.assertEqual(history[17]["g"][0], 0.)
|
||||
self.assertEqual(history[18]["g"][0], 2.)
|
||||
self.assertEqual(history[18]["v"][0], 0.)
|
||||
self.assertEqual(history[19]["v"][0], 2.)
|
||||
np.testing.assert_array_equal(history[18]["due_edge_id"], [81])
|
||||
|
||||
def test_arrival_rejected_on_same_tick_target_fire(self):
|
||||
history = []
|
||||
result = shadow(steps=19, events=record([(0, 10)]), incoming=one_edge(),
|
||||
requests=record([(18, 100)]), callback=lambda t, s: history.append(s))
|
||||
np.testing.assert_array_equal(result["step"], [18])
|
||||
self.assertFalse(history[18]["arrival_accepted"][0])
|
||||
self.assertEqual(history[18]["g_pre_apl"][0], 0.)
|
||||
|
||||
def test_refractory_rejects18_and_release22_accepts(self):
|
||||
history = []
|
||||
shadow(steps=23, events=record([(0, 10), (4, 10)]), incoming=one_edge(),
|
||||
requests=record([(0, 100)]), callback=lambda t, s: history.append(s))
|
||||
self.assertFalse(history[18]["arrival_accepted"][0])
|
||||
self.assertTrue(history[22]["arrival_accepted"][0])
|
||||
self.assertEqual(history[22]["ref"][0], 0)
|
||||
self.assertEqual(history[22]["g_pre_apl"][0], 2.)
|
||||
|
||||
def test_apl_unconditional_during_freeze_and_release_decay(self):
|
||||
history = []
|
||||
h = np.ones((23, 1), np.float32)
|
||||
shadow(steps=23, coeff=coefficients(gd=.5), requests=record([(0, 100)]), h32=h,
|
||||
callback=lambda t, s: history.append(s))
|
||||
self.assertEqual(history[0]["g_after_reset"][0], 0.)
|
||||
self.assertEqual(history[0]["g"][0], -.001)
|
||||
for tick in range(1, 22):
|
||||
self.assertEqual(history[tick]["g_decayed"][0].hex(), history[tick]["old_g"][0].hex())
|
||||
self.assertEqual(history[tick]["g"][0].hex(), (history[tick]["old_g"][0] - np.float64(.001)).hex())
|
||||
self.assertEqual(history[22]["g_decayed"][0].hex(), (.5 * history[22]["old_g"][0]).hex())
|
||||
|
||||
def test_si_promotion_order_is_not_early_float32_scaling(self):
|
||||
h = np.array([[.01]], np.float32)
|
||||
result = shadow(steps=1, h32=h)
|
||||
expected = -(np.float64(h[0, 0]) * np.float64(.001))
|
||||
wrong = -np.float64(h[0, 0] * np.float32(.001))
|
||||
self.assertEqual(result["g"][0].hex(), expected.hex())
|
||||
self.assertNotEqual(result["g"][0].hex(), wrong.hex())
|
||||
|
||||
def test_arrivals_start_at_existing_g_in_original_edge_order(self):
|
||||
incoming = IncomingIndex(np.array([10, 11], np.int32), np.array([0, 0], np.int32),
|
||||
np.array([2, 81], np.int64), np.array([-1e16, 1.], np.float64), 1)
|
||||
result = shadow(steps=19, initial_g=np.array([1e16]), incoming=incoming, events=record([(0, 10), (0, 11)]))
|
||||
self.assertEqual(result["g"][0], 1.)
|
||||
self.assertNotEqual(result["g"][0], np.float64(1e16) + np.sum(incoming.weights))
|
||||
|
||||
def test_zero_weight_edges_still_present_in_due_trace(self):
|
||||
history = []
|
||||
shadow(steps=19, events=record([(0, 10)]), incoming=one_edge(0.), callback=lambda t, s: history.append(s))
|
||||
np.testing.assert_array_equal(history[18]["due_edge_id"], [81])
|
||||
np.testing.assert_array_equal(history[18]["arrival_accepted"], [True])
|
||||
|
||||
def test_non_target_requests_ignored_and_step_alias_supported(self):
|
||||
result = shadow(requests=record([(0, 10), (0, 100)], key="step"), events=record(key="step"))
|
||||
np.testing.assert_array_equal(result["step"], [0])
|
||||
np.testing.assert_array_equal(result["neuron_idx"], [100])
|
||||
|
||||
def test_callback_mutation_cannot_change_replay(self):
|
||||
args = dict(steps=23, requests=record([(0, 100)]), h32=np.ones((23, 1), np.float32))
|
||||
expected = shadow(**args)
|
||||
|
||||
def mutate(tick, stage):
|
||||
for value in stage.values():
|
||||
value[...] = 0
|
||||
|
||||
actual = shadow(**args, callback=mutate)
|
||||
for name in expected:
|
||||
np.testing.assert_array_equal(actual[name].view(np.uint8), expected[name].view(np.uint8))
|
||||
|
||||
def test_callback_can_fail_fast_without_later_ticks(self):
|
||||
visited = []
|
||||
|
||||
def stop(tick, stage):
|
||||
visited.append(tick)
|
||||
if tick == 2:
|
||||
raise RuntimeError("intentional mismatch")
|
||||
|
||||
with self.assertRaisesRegex(RuntimeError, "intentional mismatch"):
|
||||
shadow(callback=stop)
|
||||
self.assertEqual(visited, [0, 1, 2])
|
||||
|
||||
def test_no_dense_history_and_exact_output_dtypes(self):
|
||||
result = shadow()
|
||||
self.assertFalse(any(value.ndim > 1 for value in result.values()))
|
||||
for name in ("v", "g"):
|
||||
self.assertEqual(result[name].dtype, np.float64)
|
||||
for name in ("ref", "counts", "step", "neuron_idx", "first_spike", "target_globals"):
|
||||
self.assertEqual(result[name].dtype, np.int32)
|
||||
|
||||
|
||||
class ValidationTests(unittest.TestCase):
|
||||
def test_index_copies_inputs_and_reuses_source_lookup(self):
|
||||
src, dst, ids, w = np.array([3, 3, 8], np.int32), np.array([1, 0, 1], np.int32), np.array([8, 20, 99], np.int64), np.array([1., 0., -2.])
|
||||
index = IncomingIndex(src, dst, ids, w, 2)
|
||||
w[:] = 99
|
||||
np.testing.assert_array_equal(index.weights, [1., 0., -2.])
|
||||
np.testing.assert_array_equal(index.due(np.array([3, 7, 8], np.int32)), [0, 1, 2])
|
||||
self.assertFalse(index.weights.flags.writeable)
|
||||
|
||||
def test_rejects_bad_index_dtype_range_order_and_nonfinite(self):
|
||||
base = dict(src_global=np.array([10, 11], np.int32), dst_local=np.array([0, 0], np.int32),
|
||||
edge_id=np.array([3, 8], np.int64), weights=np.array([1., 2.]), target_count=1)
|
||||
changes = [dict(src_global=np.array([10, 11], np.int64)), dict(src_global=np.array([11, 10], np.int32)),
|
||||
dict(src_global=np.array([10, 166700], np.int32)), dict(dst_local=np.array([0, 1], np.int32)),
|
||||
dict(edge_id=np.array([3, 3], np.int64)), dict(weights=np.array([1., np.nan])),
|
||||
dict(weights=np.array([1., 2.], np.float32))]
|
||||
for change in changes:
|
||||
with self.subTest(change=change), self.assertRaises(ValueError):
|
||||
IncomingIndex(**(base | change))
|
||||
|
||||
def test_rejects_invalid_replay_arrays(self):
|
||||
variants = [dict(initial_v=np.array([0.], np.float32)), dict(initial_g=np.array([np.inf])),
|
||||
dict(targets=np.array([100], np.int64)), dict(targets=np.array([100, 100], np.int32)),
|
||||
dict(h32=np.zeros((25, 1), np.float64)), dict(h32=np.full((25, 1), np.nan, np.float32)),
|
||||
dict(h32=np.full((25, 1), -1, np.float32)), dict(coeff=coefficients(gd=np.float32(.9))),
|
||||
dict(requests=record([(0, 100), (0, 100)])), dict(events=record([(25, 10)])),
|
||||
dict(events=record([(0, 166700)])), dict(steps=0)]
|
||||
for change in variants:
|
||||
with self.subTest(change=change), self.assertRaises(ValueError):
|
||||
shadow(**change)
|
||||
|
||||
def test_no_scientific_runtime_import(self):
|
||||
self.assertNotIn("torch", sys.modules)
|
||||
self.assertNotIn("brian2", sys.modules)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
Reference in New Issue
Block a user