qishizhe/digital-drosophila
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Seal Phase 4AJ fixed APL timing hypothesis audit

Browse Source
This commit is contained in:
Digital Drosophila Archive
2026-09-30 12:08:25 +08:00
parent 8d6b063aaf
commit cb6937666f
1992 changed files with 258416 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
models/male-cns-lif/apl-timing-lab-v1/PROTOCOL.md
+72
View File
@@ -0,0 +1,72 @@
# Phase 4AJ: a single graded-APL timing hypothesis
Independent branch authorized by the human. All earlier models and data remain
read-only. The learning goal remains unmet; neither an audit nor fixed-CAL PASS
constitutes usable learning.
## Fixed intervention
Use the exact sealed AG author-style float64 LIF and float32 legacy graded APL.
Compare h applied at its generating tick with the identical generated operator
applied 18 steps later. No other change: dt=0.1 ms, anatomical delay=18 steps,
refractory=22 steps, tau_m=20 ms, tau_syn=5 ms, W_syn=0.275 mV, original signs,
threshold/rest/reset, all canonical IDs/edges, legacy APL tau=20 ms and gain=0.01.
The unchanged fast-APL zero mask includes 1,212 non-KC edges not replaced by the
graded operator. The branch does not silently fix this limitation.
This is engineering scheduling parity, not evidence of a measured 1.8 ms graded
APL delay. Whole h is delayed, including KC feedback; it is not a PN-only
intervention. APL still subtracts unconditionally at end-of-step, including
refractory KC, whereas ordinary synaptic arrivals are guarded by refractory.
Equal arrival time is not equivalent biological action.
## Stages and gates
1. Exact conditional reconstruction of all 420 metadata-defined ALLN in all
60 sealed CAL r0 records. Audit every simultaneous earliest non-design PN,
before first KC, and duration-matched never-active PN controls. PN→ALLN→PN
witnesses must be actual accepted timed edges; distinguish signed weights,
reset windows and zeroed edges. No shortest paths, unique-cause or necessity
claims. Unknown antecedents are retained and do not falsify numerical audit.
2. Fixed stored h trace shift audit: no loss, duplication, amplitude arithmetic
or brain simulation. Initial +0 padding and chronological unconsumed tail
explicitly retained. No extra brain/APL tail advance or future-decay claim.
3. Both arms: eight fixed real-edge fixtures, three fresh GPU repeats, independent
Brian LIF/own spikes/own FIFO, legacy APL component audit. Delay0 must reproduce
sealed AG scientific outputs bitwise.
4. Both arms: the same two fullgraph first-seed CAL probes, independent Brian
comparison and APL component audit. Numerical failure stops later execution.
5. Both arms: every original six-odor × two-mode × five-seed CAL, plus first-seed
repeat for each odor/mode. Exact source input files, not regenerated random
inputs. All results retained. Known baseline representation failure does not
skip treatment; no seed selection, threshold/gain search or gate relaxation.
6. Recompute comparisons from raw evidence, old checksums and five frozen
reference regressions. Seal truthful outcome with SHA-256 and source version.
## Interpretation limits
Generated h depends on changed closed-loop spikes, so branch inhibition integrals
need not match. Fixed-trace conservation is a transport check, not proof of
equal closed-loop dose. Accepted request counts can also change with refractory
state, despite byte-identical requested input schedules.
Brian state integration is independent but APL arithmetic is intentionally a
shared frozen component. Original operator AST replay and rounding audit remain
separate; no universal CPU/GPU equivalence is claimed from sampled fixtures.
CAL gate retains original 3–9% mean KC fraction, all-run limits, within/between
Jaccard and Euclidean gates, and minimum odor-pair count in both input modes.
Passing fixed CAL is not heldout generalization or associative memory. Further
progress requires independent seeds/novel odors, credit specificity, isolated and
whole-brain functional readouts, then conditioning with proper controls.
No training, eta/threshold/sign tuning, plastic mask expansion, heldout dynamic
access, visual input, decoder, RL, game, Web or new candidate in this experiment.
## Literature scope
Shiu et al. 2024, DOI 10.1038/s41586-024-07763-9, supplies the spiking-LIF
scheduling reference, not a measured MaleCNS graded-APL delay. Amin et al. 2020,
DOI 10.7554/eLife.56954, and Prisco et al. 2021 (PMC8741211) motivate local graded
APL/calyx interaction but do not determine this millisecond delay value. Prior
sealed source/literature audits retain versions and provenance.
models/male-cns-lif/apl-timing-lab-v1/al_origin_audit.py
+513
View File
@@ -0,0 +1,513 @@
"""Conditional audit of the earliest non-design ALPN cohort in 60 sealed CALs.
CPU only. No new network trajectory, learning, parameter change, heldout input,
or shortest-path attribution. Full ALLN inputs and compact outgoing witnesses
are retained separately so a potentially large temporal join is not expanded.
"""
from __future__ import annotations
from collections import defaultdict
from pathlib import Path
import traceback
import numpy as np
import pandas as pd
N = 166700
E = 25582938
DELAY = 18
BUDGET = 2 * 1024**3
MODES = ('physiological', 'matched_energy')
SEEDS = tuple(range(51101, 51106))
def require(value, message):
if not value:
raise ValueError(message)
def bit_equal(a, b):
return a.dtype == b.dtype and a.shape == b.shape and a.tobytes() == b.tobytes()
def first_ticks(ticks, nodes, targets, n=N):
first = np.full(n, np.iinfo(np.int32).max, np.int32)
np.minimum.at(first, nodes, ticks)
first[first == np.iinfo(np.int32).max] = -1
return first[targets]
def select_cohort(pn, first_pn, design, first_kc, steps):
"""Static-design complement, ALL ties, never choose by antecedent outcome."""
require(pn.dtype == np.int32 and first_pn.dtype == np.int32 and pn.shape == first_pn.shape,
'PN identity/first-tick schema mismatch')
require(np.all(np.diff(pn) > 0) and np.isin(design, pn).all(), 'Invalid PN/design IDs')
require(np.all((first_pn >= -1) & (first_pn < steps)), 'PN first tick outside trial')
require(first_kc == -1 or 0 <= first_kc < steps, 'KC first tick outside trial')
non_design = ~np.isin(pn, design)
observed = first_pn[non_design & (first_pn >= 0)]
earliest = int(observed.min()) if len(observed) else -1
cohort = pn[non_design & (first_pn == earliest)] if earliest >= 0 else np.empty(0, np.int32)
cutoff = int(first_kc) if first_kc >= 0 else int(steps)
return dict(cohort=cohort, earliest_tick=earliest, pre_kc_stop_exclusive=cutoff,
pre_kc_window_censored=first_kc < 0,
matched_stop_exclusive=min(cutoff, earliest) if earliest >= 0 else cutoff,
cohort_before_first_KC=bool(earliest >= 0 and first_kc >= 0 and earliest < first_kc),
cohort_relation_to_first_KC=('no_non_design_spike' if earliest < 0 else
'KC_unobserved' if first_kc < 0 else 'before' if earliest < first_kc else
'same_tick' if earliest == first_kc else 'after'))
def predecessor_counts(incoming_ticks, incoming_weights, source_tick, previous_source_tick):
"""Strict temporal antecedents. Old-reset history is reported separately."""
ticks, weights = np.asarray(incoming_ticks), np.asarray(incoming_weights)
require(ticks.ndim == weights.ndim == 1 and ticks.shape == weights.shape,
'Predecessor arrays differ')
require(np.isfinite(weights).all() and np.all(np.diff(ticks) >= 0), 'Bad predecessor sequence')
require(-1 <= previous_source_tick < source_tick, 'Bad previous/source spike order')
prior = ticks < source_tick
surviving = prior & (ticks > previous_source_tick)
return dict(all_prior_design_PN_events=int(prior.sum()),
since_last_reset_design_PN_events=int(surviving.sum()),
since_last_reset_nonzero_design_PN_events=int(np.count_nonzero(surviving & (weights != 0))),
since_last_reset_positive_events=int(np.count_nonzero(surviving & (weights > 0))),
since_last_reset_negative_events=int(np.count_nonzero(surviving & (weights < 0))),
since_last_reset_zero_events=int(np.count_nonzero(surviving & (weights == 0))),
since_last_reset_positive_weight_SI=float(np.maximum(weights[surviving], 0).sum(dtype=np.longdouble)),
since_last_reset_negative_abs_weight_SI=float(np.maximum(-weights[surviving], 0).sum(dtype=np.longdouble)),
prior_reset_excluded_events=int(np.count_nonzero(prior & ~surviving)))
def source_labels(global_ids, role, design_mask):
result = role[global_ids].astype(object).copy()
alpn = result == 'ALPN'
result[alpn & design_mask[global_ids]] = 'design_ALPN'
result[alpn & ~design_mask[global_ids]] = 'non_design_ALPN'
return result
def file_budget(root):
size = sum(p.stat().st_size for p in root.rglob('*') if p.is_file())
require(size <= BUDGET, 'Origin audit exceeds 2 GiB; no truncation/resampling allowed')
return size
class Sink:
def __init__(self, path, schema, budget_root):
import pyarrow.parquet as pq
require(not path.exists(), 'Output exists: ' + str(path))
self.writer = pq.ParquetWriter(path, schema, compression='zstd')
self.schema, self.rows, self.root = schema, 0, budget_root
def write(self, columns):
import pyarrow as pa
table = pa.Table.from_pydict(columns, schema=self.schema)
if table.num_rows:
self.writer.write_table(table)
self.rows += table.num_rows
file_budget(self.root)
def close(self):
self.writer.close()
file_budget(self.root)
def event_schema():
import pyarrow as pa
return pa.schema([
('edge_id', pa.int64()), ('source_neuron_idx', pa.int32()), ('target_neuron_idx', pa.int32()),
('source_body_id', pa.uint64()), ('target_body_id', pa.uint64()),
('source_spike_tick', pa.int32()), ('arrival_tick', pa.int32()),
('source_event_key', pa.int64()), ('weight_SI', pa.float64()), ('weight_sign', pa.int32()),
('source_group', pa.string()), ('accepted', pa.bool_()), ('ref_at_arrival', pa.int32()),
('target_previous_spike_tick', pa.int32()), ('before_first_any_KC', pa.bool_())])
class ALLNObserver:
"""Save every accepted incoming event, including frozen-zero edges."""
def __init__(self, targets, incoming, role, design_mask, body, steps, cutoff, sink):
self.targets, self.incoming = targets, incoming
self.role, self.design, self.body = role, design_mask, body
self.steps, self.cutoff, self.sink = steps, cutoff, sink
self.last = np.full(len(targets), -1, np.int32)
self.callbacks = self.due = self.accepted = self.rejected = 0
self.groups = defaultdict(lambda: dict(due=0, accepted=0, rejected=0, positive=0, negative=0, zero=0))
self.design_ticks = defaultdict(list)
self.design_weights = defaultdict(list)
def __call__(self, tick, stage):
require(tick == self.callbacks and tick < self.steps, 'ALLN callback coverage mismatch')
require(np.all(stage['h32'] == 0) and np.all(stage['delta_si'] == 0), 'ALLN received graded APL')
require(not np.any(stage['request']), 'CAL request targeted ALLN rather than PN')
require(bit_equal(stage['g'], stage['g_pre_apl']), 'Unexpected ALLN post-APL change')
due, accepted = stage['due_edge_indices'], stage['arrival_accepted']
require(np.array_equal(accepted, stage['ref_after_reset'][self.incoming.dst_local[due]] == 0),
'ALLN arrival acceptance differs from post-reset ref guard')
require(tick >= DELAY or not len(due), 'Premature ALLN event')
self.last[stage['fired']] = tick
labels = source_labels(self.incoming.src_global[due], self.role, self.design)
weight = self.incoming.weights[due]
for label in np.unique(labels):
mask = labels == label
item = self.groups[str(label)]
item['due'] += int(mask.sum())
item['accepted'] += int(np.count_nonzero(mask & accepted))
item['rejected'] += int(np.count_nonzero(mask & ~accepted))
for name, select in (('positive', weight > 0), ('negative', weight < 0), ('zero', weight == 0)):
item[name] += int(np.count_nonzero(mask & accepted & select))
selected = due[accepted]
src = self.incoming.src_global[selected]
local = self.incoming.dst_local[selected]
dst = self.targets[local]
w = self.incoming.weights[selected]
selected_labels = labels[accepted]
count = len(selected)
self.sink.write(dict(edge_id=self.incoming.edge_id[selected], source_neuron_idx=src,
target_neuron_idx=dst, source_body_id=self.body[src], target_body_id=self.body[dst],
source_spike_tick=np.full(count, tick - DELAY, np.int32),
arrival_tick=np.full(count, tick, np.int32),
source_event_key=np.int64(tick - DELAY) * N + src.astype(np.int64),
weight_SI=w, weight_sign=np.sign(w).astype(np.int32), source_group=selected_labels,
accepted=np.ones(count, bool), ref_at_arrival=stage['ref_after_reset'][local],
target_previous_spike_tick=self.last[local], before_first_any_KC=np.full(count, tick < self.cutoff, bool)))
# At most all accepted design-PN events are retained in RAM. Full event
# rows are already streamed; this compact index never expands path pairs.
for target in np.unique(dst[selected_labels == 'design_ALPN']):
mask = (dst == target) & (selected_labels == 'design_ALPN')
self.design_ticks[int(target)].extend([tick] * int(mask.sum()))
self.design_weights[int(target)].extend(w[mask].tolist())
self.callbacks += 1
self.due += len(due)
self.accepted += count
self.rejected += int(np.count_nonzero(~accepted))
def report(self):
require(self.callbacks == self.steps and self.due == self.accepted + self.rejected,
'Incomplete ALLN replay observer')
require(self.accepted == self.sink.rows, 'ALLN event sink row mismatch')
return dict(callbacks=self.callbacks, ALLN_targets=len(self.targets), due_events=self.due,
accepted_events=self.accepted, rejected_events=self.rejected,
source_groups=dict(self.groups), graded_APL_h_zero=True,
accepted_zero_weight_rows_preserved=True)
def _window_add(stats, frame, mask, labels):
for label in np.unique(labels[mask]):
chosen = mask & (labels == label)
weights = frame.weight_SI.to_numpy()[chosen]
item = stats.setdefault(str(label), dict(events=0, positive_events=0, negative_events=0,
positive_weight_SI=0.0, negative_abs_weight_SI=0.0))
item['events'] += len(weights)
item['positive_events'] += int(np.count_nonzero(weights > 0))
item['negative_events'] += int(np.count_nonzero(weights < 0))
item['positive_weight_SI'] += float(np.maximum(weights, 0).sum(dtype=np.longdouble))
item['negative_abs_weight_SI'] += float(np.maximum(-weights, 0).sum(dtype=np.longdouble))
def _read_npz(path):
with np.load(path, allow_pickle=False) as file:
return {key: file[key].copy() for key in file.files}
def main():
# Lazy import makes all selection/join helpers testable without a server.
import pyarrow as pa
import pyarrow.parquet as pq
from context import (ROOT, EXP, AG, AG_MODEL, AI, AI_MODEL, AH_MODEL, COEFF,
load_module, read, save, digest, check)
check()
require(read(AI / 'audit/final-status.json')['status'] == 'PASS-CONDITIONAL-PN-APL-SOURCE-AUDIT',
'AI is not successfully sealed')
require(read(AI / 'audit/final-status.json')['pass_all'] is True, 'AI seal failed')
outroot = EXP / 'origin-audit'
outroot.mkdir(parents=True, exist_ok=False)
receipts = []
try:
sealed_roots = (AI, AI_MODEL, AG, AG_MODEL, AH_MODEL)
sealed_inventory = {root: {r['path']: r for r in read(root / 'manifest.json')['files']}
for root in sealed_roots}
verified_hashes = {}
def input_hash(path):
path = Path(path)
if path in verified_hashes:
return verified_hashes[path]
actual = digest(path)
for root in sealed_roots:
if path.is_relative_to(root) and path.name not in ('manifest.json', 'checksums.sha256'):
key = path.relative_to(root).as_posix()
require(key in sealed_inventory[root], 'Input absent from frozen seal: ' + str(path))
frozen = sealed_inventory[root][key]
require(actual == frozen['sha256'] and path.stat().st_size == frozen['bytes'],
'Input differs from sealed digest/bytes: ' + str(path))
break
verified_hashes[path] = actual
return actual
# Verify entry-point metadata and code before importing the frozen code.
for path in (AI / 'protocol/inventory.json', AI / 'protocol/neuron-source-groups.parquet',
AG / 'protocol/calibration-cases.json', AH_MODEL / 'replay_core.py',
AI_MODEL / 'validate_events.py', AG_MODEL / 'calibration.py'):
input_hash(path)
inventory = [row for row in read(AI / 'protocol/inventory.json')
if row['kind'] == 'calibration' and row['repeat'] == 0]
require(len(inventory) == 60 and len({r['case'] for r in inventory}) == 60,
'Expected exactly the fixed 60 distinct CAL r0 inputs')
reference_cases = read(AG / 'protocol/calibration-cases.json')
require({r['case'] for r in inventory} == {r['name'] for r in reference_cases},
'AI CAL inventory differs from AG CAL inventory')
specifications = {r['case']: read(AG / 'cases' / r['case'] / 'spec.json') for r in inventory}
cells = {(s['mode'], s['odor_id'], s['seed']) for s in specifications.values()}
odors = sorted({s['odor_id'] for s in specifications.values()})
require(len(odors) == 6 and cells == {(m, o, seed) for m in MODES for o in odors for seed in SEEDS},
'CAL identity grid incomplete or includes repeats/heldout')
meta_path = AI / 'protocol/neuron-source-groups.parquet'
meta = pd.read_parquet(meta_path)
require(len(meta) == N and np.array_equal(meta.neuron_idx.to_numpy(), np.arange(N)),
'Metadata global identity mismatch')
require(meta.audit_source_group.notna().all() and meta.body_id.dtype == np.uint64,
'Invalid frozen role/body metadata')
role, body = meta.audit_source_group.to_numpy(), meta.body_id.to_numpy()
alln = np.flatnonzero(role == 'ALLN').astype(np.int32)
pn = np.load(AG / 'apl-bundle/pn_globals.npy')
kc = np.load(AG / 'apl-bundle/kc_globals.npy')
apl = np.load(AG / 'apl-bundle/apl_globals.npy')
require(len(alln) == 420 and len(pn) == 686 and len(kc) == 4064 and len(apl) == 2,
'Frozen role population sizes differ')
require(np.array_equal(pn, np.flatnonzero(role == 'ALPN').astype(np.int32)), 'ALPN role IDs differ')
require((meta.iloc[alln].annotation_class == 'ALLN').all(), 'ALLN role is not exact annotation class')
graph = AG / 'graphs/full'
arrays = {k: np.load(graph / (k + '.npy'), mmap_mode='r')
for k in ('nodes', 'src', 'dst', 'edge_id', 'weights')}
require(bit_equal(arrays['nodes'], np.arange(N, dtype=np.int32)), 'Full graph global node order differs')
require(len(arrays['edge_id']) == E, 'Full graph edge count differs')
for start in range(0, E, 1000000):
stop = min(E, start + 1000000)
require(np.array_equal(arrays['edge_id'][start:stop], np.arange(start, stop, dtype=np.int64)),
'Full graph edge IDs not canonical identity')
core = load_module('frozen_ah_alln_replay_aj', AH_MODEL / 'replay_core.py')
validate = load_module('sealed_ai_event_validation_aj', AI_MODEL / 'validate_events.py')
cal = load_module('sealed_ag_state_validation_aj', AG_MODEL / 'calibration.py')
lookup = np.full(N, -1, np.int32)
lookup[alln] = np.arange(len(alln), dtype=np.int32)
selected = np.flatnonzero(lookup[arrays['dst']] >= 0)
src = arrays['src'][selected]
weights = arrays['weights'][selected].copy()
weights[np.isin(src, apl)] = 0.0
incoming = core.IncomingIndex(src, lookup[arrays['dst'][selected]], arrays['edge_id'][selected], weights, len(alln))
# Only incoming PN graph views are required to independently validate AI rows.
pnedges = np.flatnonzero(np.isin(arrays['dst'], pn))
pnweights = arrays['weights'][pnedges].copy()
pnweights[np.isin(arrays['src'][pnedges], apl)] = 0.0
shared_inputs = [meta_path, COEFF, AH_MODEL / 'replay_core.py', AI_MODEL / 'validate_events.py',
AG_MODEL / 'calibration.py', AI / 'manifest.json', AI / 'checksums.sha256',
AG / 'manifest.json', AG / 'checksums.sha256']
shared_inputs += [graph / (key + '.npy') for key in arrays]
shared_inputs += [AG / 'apl-bundle' / (name + '_globals.npy') for name in ('pn', 'kc', 'apl')]
save(outroot / 'shared-inputs.json', [dict(path=str(p.relative_to(ROOT)), bytes=p.stat().st_size,
sha256=input_hash(p)) for p in shared_inputs])
save(outroot / 'inventory.json', [dict(**r, mode=specifications[r['case']]['mode'],
odor_id=specifications[r['case']]['odor_id'], seed=specifications[r['case']]['seed']) for r in inventory])
for row in inventory:
name = row['case']
require(Path(name).name == name and '/' not in name and '\\' not in name, 'Invalid case path')
case = AG / 'cases' / name
source = AG / 'gpu' / (name + '-r0')
require(ROOT / row['source'] == source, 'Inventory source differs from fixed CAL r0 source')
previous = AI / 'trials' / (name + '-r0')
dest = outroot / name
dest.mkdir(exist_ok=False)
spec = specifications[name]
for path in (source / 'state.npz', source / 'summary.json', case / 'spec.json', case / 'requests.npz',
case / 'pn-drive.npz', previous / 'accepted-prefirst-edge-events.parquet',
previous / 'per-pn.parquet', previous / 'summary.json'):
input_hash(path)
require(spec['steps'] == 1000 and spec['duration_ms'] == 100.0, 'CAL duration changed')
require(not (case / 'arrays.npz').exists(), 'Unexpected CAL initial-state override')
source_summary = read(source / 'summary.json')
require(source_summary['complete'] is True and source_summary['completed_ticks'] == spec['steps'],
'Incomplete source simulation')
ai_summary = read(previous / 'summary.json')
require(ai_summary['pass_all'] is True and all(ai_summary['exact'].values()), 'AI exactness failed')
state = _read_npz(source / 'state.npz')
event_keys = cal.validate_state(state, N, spec['steps'])
requests = _read_npz(case / 'requests.npz')
drive = _read_npz(case / 'pn-drive.npz')
design = np.sort(drive['targets'])
require(design.dtype == np.int32 and len(np.unique(design)) == len(design) and
np.isin(design, pn).all(), 'Design identities invalid')
require(len(design) == spec['target_count'] and drive['rates_hz'].dtype == np.float32 and
drive['rates_hz'].shape == design.shape and np.all(drive['rates_hz'] > 0),
'Static PN design count/rate schema differs')
require(np.isin(requests['neuron_idx'], design).all(), 'Request outside static odor design')
design_mask = np.zeros(N, bool)
design_mask[design] = True
first_pn = first_ticks(state['step'], state['neuron_idx'], pn)
observed_kc = first_ticks(state['step'], state['neuron_idx'], kc)
first_kc = int(observed_kc[observed_kc >= 0].min()) if np.any(observed_kc >= 0) else -1
selection = select_cohort(pn, first_pn, design, first_kc, spec['steps'])
cohort = selection.pop('cohort')
cohort_mask = np.zeros(N, bool)
cohort_mask[cohort] = True
never_mask = np.zeros(N, bool)
never_mask[pn[first_pn < 0]] = True
saved_pn = pd.read_parquet(previous / 'per-pn.parquet')
require(np.array_equal(saved_pn.neuron_idx, pn) and np.array_equal(saved_pn.first_spike_tick, first_pn),
'Sealed AI PN first-spike identity mismatch')
cutoff = selection['pre_kc_stop_exclusive']
window_stats = {key: {} for key in ('cohort_pre_KC_and_pre_first', 'never_active_pre_KC',
'never_active_non_design_pre_KC', 'never_active_matched_cohort_window',
'never_active_non_design_matched_cohort_window')}
outgoing_chunks = []
cohort_chunks = []
rows_seen, previous_event_key = 0, -1
counts_per_pn = np.zeros(len(pn), np.int64)
for batch in pq.ParquetFile(previous / 'accepted-prefirst-edge-events.parquet').iter_batches(batch_size=65536):
frame = batch.to_pandas()
columns = {k: frame[k].to_numpy() for k in validate.COLUMN_DTYPES}
audit = validate.validate_batch(columns, edge_ids=arrays['edge_id'][pnedges],
srcglobals=arrays['src'][pnedges], dstglobals=arrays['dst'][pnedges], weights=pnweights,
target_globals=pn, target_first=first_pn, source_event_keys=event_keys)
key = frame.arrival_tick.to_numpy().astype(np.int64) * E + frame.edge_id.to_numpy()
require(not len(key) or (key[0] > previous_event_key and np.all(np.diff(key) > 0)),
'AI event order/uniqueness violated across batches')
if len(key):
previous_event_key = int(key[-1])
require(np.all(frame.arrival_tick.to_numpy() < spec['steps']), 'AI event beyond trial')
labels = source_labels(frame.source_neuron_idx.to_numpy(), role, design_mask)
require(np.array_equal(frame.source_group, role[frame.source_neuron_idx.to_numpy()]), 'AI source role differs')
require(np.array_equal(frame.target_is_design, design_mask[frame.target_neuron_idx.to_numpy()]),
'AI target design differs')
targets = frame.target_neuron_idx.to_numpy()
pre = frame.arrival_tick.to_numpy() < cutoff
matched = frame.arrival_tick.to_numpy() < selection['matched_stop_exclusive']
cohort_sel = cohort_mask[targets] & pre
masks = dict(cohort_pre_KC_and_pre_first=cohort_sel,
never_active_pre_KC=never_mask[targets] & pre,
never_active_non_design_pre_KC=never_mask[targets] & ~design_mask[targets] & pre,
never_active_matched_cohort_window=never_mask[targets] & matched,
never_active_non_design_matched_cohort_window=never_mask[targets] & ~design_mask[targets] & matched)
for key, mask in masks.items():
_window_add(window_stats[key], frame, mask, labels)
# The fixed earliest cohort normally makes these small, but the
# on-disk ceiling remains hard and no row may be sampled away.
if np.any(cohort_sel):
selected_frame = frame.loc[cohort_sel].copy()
selected_frame['audit_source_group'] = labels[cohort_sel]
cohort_chunks.append(selected_frame)
out_sel = cohort_sel & (role[frame.source_neuron_idx.to_numpy()] == 'ALLN')
if np.any(out_sel):
outgoing_chunks.append(frame.loc[out_sel].copy())
counts_per_pn += audit['per_target_count']
rows_seen += audit['count']
require(rows_seen == ai_summary['observer']['first_arrival_record_count'], 'AI antecedent rows incomplete')
require(np.array_equal(counts_per_pn, saved_pn[[g + '_accepted_nonzero_before_first_edges'
for g in ai_summary['observer']['source_group_names']]].sum(axis=1).to_numpy()),
'AI per-PN antecedent completeness differs')
saved_event_schema = pq.ParquetFile(previous / 'accepted-prefirst-edge-events.parquet').schema_arrow
cohort_table = (pd.concat(cohort_chunks, ignore_index=True) if cohort_chunks else
pa.Table.from_batches([], schema=saved_event_schema).to_pandas().assign(audit_source_group=pd.Series(dtype=str)))
cohort_table.to_parquet(dest / 'cohort-all-source-antecedents.parquet', index=False, compression='zstd')
output = (pd.concat(outgoing_chunks, ignore_index=True) if outgoing_chunks else
pa.Table.from_batches([], schema=saved_event_schema).to_pandas())
sink = Sink(dest / 'ALLN-all-accepted-incoming.parquet', event_schema(), outroot)
observer = ALLNObserver(alln, incoming, role, design_mask, body, spec['steps'], cutoff, sink)
try:
result = core.replay(np.full(len(alln), spec['model']['resting_potential_mv'], np.float64) * np.float64(.001),
np.zeros(len(alln), np.float64), read(COEFF), spec['steps'], alln, requests,
dict(step=state['step'], neuron_idx=state['neuron_idx']), incoming,
np.zeros((spec['steps'], len(alln)), np.float32), observer)
finally:
sink.close()
observed = np.isin(state['neuron_idx'], alln)
exact = {k: bit_equal(result[k], state[k][alln]) for k in ('v', 'g', 'ref', 'counts')}
exact.update({k: bit_equal(result[k], state[k][observed]) for k in ('step', 'neuron_idx')})
require(all(exact.values()), 'Conditional ALLN replay differs from full AG events/final states: ' + str(exact))
with (dest / 'ALLN-replay-final.npz').open('xb') as stream:
np.savez_compressed(stream, **result)
join_rows = []
alln_source_spikes = {int(node): state['step'][state['neuron_idx'] == node]
for node in np.unique(output.source_neuron_idx.to_numpy())}
for record in output.to_dict('records'):
target = int(record['source_neuron_idx']) # Intermediate ALLN.
tick = int(record['source_spike_tick'])
spikes = alln_source_spikes[target]
pos = int(np.searchsorted(spikes, tick))
require(pos < len(spikes) and spikes[pos] == tick, 'ALLN outgoing source event absent')
last = int(spikes[pos - 1]) if pos else -1
predecessors = predecessor_counts(np.asarray(observer.design_ticks[target], np.int32),
np.asarray(observer.design_weights[target], np.float64), tick, last)
join_rows.append(dict(**record, ALLN_previous_spike_tick=last,
ALLN_source_event_key=tick * N + target, source_body_id=int(body[target]),
target_body_id=int(body[int(record['target_neuron_idx'])]), **predecessors,
witness_status=('timed_design_PN_antecedent_since_last_reset' if
predecessors['since_last_reset_nonzero_design_PN_events'] else 'no_direct_design_PN_antecedent_since_last_reset')))
# Add declared columns even if the fixed cohort has no ALLN antecedent.
join_extra = dict(ALLN_previous_spike_tick='int32', ALLN_source_event_key='int64',
source_body_id='uint64', target_body_id='uint64', all_prior_design_PN_events='int64',
since_last_reset_design_PN_events='int64', since_last_reset_positive_events='int64',
since_last_reset_nonzero_design_PN_events='int64',
since_last_reset_negative_events='int64', since_last_reset_zero_events='int64',
since_last_reset_positive_weight_SI='float64', since_last_reset_negative_abs_weight_SI='float64',
prior_reset_excluded_events='int64', witness_status='object')
witnesses = pd.DataFrame(join_rows) if join_rows else output.copy()
for key, dtype in join_extra.items():
witnesses[key] = witnesses[key].astype(dtype) if key in witnesses else pd.Series(dtype=dtype)
witnesses.to_parquet(dest / 'ALLN-to-cohort-PN-witness-join.parquet', index=False, compression='zstd')
per_pn = meta.iloc[pn].copy()
per_pn['first_spike_tick'] = first_pn
per_pn['design_target'] = design_mask[pn]
per_pn['earliest_non_design_cohort'] = cohort_mask[pn]
per_pn['never_active'] = first_pn < 0
per_pn.to_parquet(dest / 'PN-population-and-cohort.parquet', index=False, compression='zstd')
per_alln = meta.iloc[alln].copy()
per_alln['first_spike_tick'] = result['first_spike']
per_alln['spike_count'] = result['counts']
per_alln.to_parquet(dest / 'ALLN-population.parquet', index=False, compression='zstd')
witness_rows = int((witnesses.since_last_reset_nonzero_design_PN_events > 0).sum())
inputs = [source / 'state.npz', source / 'summary.json', case / 'spec.json', case / 'requests.npz',
case / 'pn-drive.npz', previous / 'accepted-prefirst-edge-events.parquet',
previous / 'per-pn.parquet', previous / 'summary.json']
report = dict(case=name, repeat=0, mode=spec['mode'], odor_id=spec['odor_id'], seed=spec['seed'],
pass_all=True, exact=exact, ALLN_replay=observer.report(), first_any_KC_tick=first_kc,
**selection, cohort_neurons=int(len(cohort)), cohort_neuron_idx=cohort.tolist(),
cohort_body_id=[int(x) for x in body[cohort]], never_active_PN=int(never_mask.sum()),
never_active_non_design_PN=int(np.count_nonzero(never_mask & ~design_mask)),
window_source_accounting=window_stats, AI_antecedent_rows_checked=rows_seen,
cohort_ALLN_outgoing_events=len(output), outgoing_events_with_since_reset_design_antecedent=witness_rows,
outgoing_events_without_since_reset_design_antecedent=len(output) - witness_rows,
compact_join_pair_count=int(witnesses.since_last_reset_nonzero_design_PN_events.sum()),
no_direct_antecedent_is_not_unexplained_wholebrain=True,
join_contract='same trial; incoming.target_neuron_idx=outgoing.source_neuron_idx; incoming.source_group=design_ALPN; '
'incoming.weight_SI != 0; '
'outgoing.ALLN_previous_spike_tick < incoming.arrival_tick < outgoing.source_spike_tick; '
'outgoing.arrival_tick < cohort first spike and first-any-KC cutoff; both legs exactly 18 ticks; '
'all incoming rows accepted at ref=0; signs retained; zero legs not causal transmission',
accounting_scope='Accepted nominal SI weight sums, not membrane contribution, necessity or unique cause; '
'prior-reset antecedents retained only as history; since-reset nonzero weights can still cancel/round away',
window_scope='Cohort first-spike antecedents are censored at their first spike; never-active pre-KC windows are longer. '
'Use matched_cohort_window for duration-matched comparisons. If KC never fires, full trial is marked censored.',
new_brain_simulation=False, GPU_used=False, candidate_generated=False,
CAL_independent_ALLN_per_tick_history_available=False,
input_sha256={str(p.relative_to(ROOT)): input_hash(p) for p in inputs})
save(dest / 'summary.json', report)
save(dest / 'output-hashes.json', [dict(path=p.name, bytes=p.stat().st_size, sha256=digest(p))
for p in sorted(dest.iterdir()) if p.is_file()])
receipts.append(dict(case=name, repeat=0, mode=spec['mode'], odor_id=spec['odor_id'],
seed=spec['seed'], ALLN_targets=420, exact=exact, pass_all=True,
summary_sha256=digest(dest / 'summary.json')))
file_budget(outroot)
print(name, '420 ALLN conditional exact; earliest cohort', len(cohort), 'PASS', flush=True)
require(len(receipts) == 60, 'Incomplete origin audit inventory')
save(outroot / 'summary.json', dict(pass_all=True, status='PASS-CONDITIONAL-EARLIEST-AL-ORIGIN-AUDIT',
trials=receipts, fixed_CAL_r0_count=60, ALLN_per_trial=420, new_brain_simulation=False,
GPU_used=False, no_cartesian_witness_expansion=True, truncated=False,
unique_cause_proven=False, necessity_proven=False, output_budget_bytes=BUDGET,
meaning='Exact conditional ALLN reconstruction and timed anatomical antecedent audit, not a biological causal gate'))
file_budget(outroot)
except Exception as error:
save(outroot / 'failure.json', dict(pass_all=False, error=type(error).__name__, message=str(error),
traceback=traceback.format_exc(), completed=receipts, new_brain_simulation=False))
raise
if __name__ == '__main__':
main()
models/male-cns-lif/apl-timing-lab-v1/calibration.py
+368
View File
@@ -0,0 +1,368 @@
"""Read-only two-arm CAL aggregation using the sealed, full original 4L gate.
No trajectory is generated here. Representation failure never selects an arm
or suppresses the other arm. Timing checks use direct slices, not either
runtime's queue, and old scientific outputs are never modified.
"""
from __future__ import annotations
import csv
import hashlib
import importlib.util
import json
from pathlib import Path
import sys
import numpy as np
LEGACY_SHA256 = "edb9cb2d890471fd07afd88c959ef3540b0cd40af490fac3f26396b5e7f5bf9c"
STATE_DTYPES = {"step": np.int32, "neuron_idx": np.int32, "counts": np.int32,
"v": np.float64, "g": np.float64, "ref": np.int32}
APL_FILES = {"apl-drive.float32.npy": 2, "apl-state.float32.npy": 2,
"apl-inhibition.float32.npy": 4064}
REQUEST_FIELDS = ("tick", "neuron_idx", "attempted", "accepted", "rejected")
_DEFAULT_OUTPUT = object()
def require(ok, message):
if not ok:
raise ValueError(message)
def read_json(path):
return json.loads(Path(path).read_text(encoding="utf-8"))
def file_hash(path):
value = hashlib.sha256()
with Path(path).open("rb") as stream:
for block in iter(lambda: stream.read(8 << 20), b""):
value.update(block)
return value.hexdigest()
def raw_hash(value):
return hashlib.sha256(np.ascontiguousarray(value).tobytes()).hexdigest()
def bit_equal(a, b):
return bool(a.dtype == b.dtype and a.shape == b.shape and
np.array_equal(np.ascontiguousarray(a).view(np.uint8),
np.ascontiguousarray(b).view(np.uint8)))
def load_legacy(path=None):
"""Load the sealed pure functions, without rebinding any old global/path."""
if path is None:
from context import AG_MODEL
path = AG_MODEL / "calibration.py"
path = Path(path)
require(file_hash(path) == LEGACY_SHA256, "Sealed AG calibration source changed")
spec = importlib.util.spec_from_file_location("sealed_ag_calibration_for_aj", path)
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
return module
def _jsonl(path):
with Path(path).open(encoding="utf-8") as stream:
return [json.loads(line) for line in stream if line.strip()]
def _request_rows(path):
with Path(path).open(encoding="utf-8", newline="") as stream:
reader = csv.DictReader(stream)
require(tuple(reader.fieldnames or ()) == REQUEST_FIELDS, "Request schema mismatch")
return [{key: int(row[key]) for key in REQUEST_FIELDS} for row in reader]
def _component(directory):
gpu, oracle = directory / "apl-component.json", directory / "apl-worker-handshake.json"
require(gpu.is_file() != oracle.is_file(), "Exactly one backend APL provenance file is required")
return read_json(gpu) if gpu.is_file() else read_json(oracle)["handshake"]
def control_identity(newdir, olddir, case):
"""Compare delay0 scientific evidence with sealed AG, for fixtures or CAL.
case is the NEW case directory. Summary paths, budgets, build/wall times and
new adapter metadata are deliberately not compared. Missing evidence raises;
numerical differences return pass_all=False with each failed check retained.
"""
newdir, olddir, case = Path(newdir), Path(olddir), Path(case)
spec = read_json(case / "spec.json")
require(type(spec.get("apl_delay_steps")) is int and spec["apl_delay_steps"] == 0,
"Control identity is defined only for explicit delay0")
fresh, old = read_json(newdir / "summary.json"), read_json(olddir / "summary.json")
require(all(row.get("complete") is True and row.get("pass_all") is True for row in (fresh, old)),
"Cannot compare incomplete/failed control evidence")
require(all(row.get("steps") == spec["steps"] and row.get("completed_ticks") == spec["steps"] and
row.get("case", row.get("name")) == spec["name"] for row in (fresh, old)),
"Control case identity/completion changed")
checks = {}
with np.load(newdir / "state.npz", allow_pickle=False) as aa, np.load(olddir / "state.npz", allow_pickle=False) as bb:
require(set(aa.files) == set(bb.files) == set(STATE_DTYPES), "Control state schema mismatch")
for name, dtype in STATE_DTYPES.items():
first, second = aa[name], bb[name]
require(first.dtype == second.dtype == dtype and first.ndim == second.ndim == 1 and
np.isfinite(first).all() and np.isfinite(second).all(), "Invalid control state: " + name)
checks["state/" + name] = bit_equal(first, second)
for name, width in APL_FILES.items():
first, second = (np.load(folder / name, mmap_mode="r", allow_pickle=False) for folder in (newdir, olddir))
require(first.dtype == second.dtype == np.float32 and first.shape == second.shape == (spec["steps"], width)
and np.isfinite(first).all() and np.isfinite(second).all(), "Invalid control APL trace: " + name)
checks[name] = bit_equal(first, second)
for name in ("tick-state-hashes.jsonl", "queue.jsonl"):
first, second = _jsonl(newdir / name), _jsonl(olddir / name)
require(len(first) == len(second) == spec["steps"] and all(
[row.get("tick") for row in rows] == list(range(spec["steps"])) for rows in (first, second)),
"Incomplete control tick record: " + name)
checks[name] = first == second
checks["requests.csv"] = _request_rows(newdir / "requests.csv") == _request_rows(olddir / "requests.csv")
checks["APL_component"] = _component(newdir) == _component(olddir)
require(isinstance(old.get("effective_weights_raw_sha256"), str) and
len(old["effective_weights_raw_sha256"]) == 64, "Missing old effective-weight evidence")
checks["effective_weights_raw_sha256"] = fresh.get("effective_weights_raw_sha256") == old["effective_weights_raw_sha256"]
for key in ("nodes", "edges", "precision", "apl_fast_edges_zeroed", "kc_targets_present"):
require(key in fresh and key in old, "Missing control summary scientific field: " + key)
checks["summary/" + key] = fresh[key] == old[key]
if spec["record_history"]:
for prefix in ("", "preAPL-"):
for field, dtype in (("v", "float64"), ("g", "float64"), ("ref", "int32")):
name = f"{prefix}{field}-history.{dtype}.npy" if not prefix else f"{prefix}{field}.{dtype}.npy"
first, second = (np.load(folder / name, mmap_mode="r", allow_pickle=False) for folder in (newdir, olddir))
checks[name] = bit_equal(first, second)
# CPU oracle also saves actual effective Brian weights. GPU has only the
# raw effective-weight digest, already compared above.
if (newdir / "weights.npy").exists() or (olddir / "weights.npy").exists():
require((newdir / "weights.npy").is_file() and (olddir / "weights.npy").is_file(),
"Control backend weight-file presence differs")
checks["weights.npy"] = bit_equal(np.load(newdir / "weights.npy", mmap_mode="r"),
np.load(olddir / "weights.npy", mmap_mode="r"))
return {"pass_all": bool(all(checks.values())), "case": spec["name"], "checks": checks,
"scope": "Six state arrays, all tick hashes/queues/requests, generated APL, component, effective weights and fixture pre/post histories",
"adapter_paths_budgets_and_runtime_metadata_compared": False}
def validate_timing(output, spec, trial):
"""Independently verify raw timing files and recorded audit, without a queue."""
output = Path(output)
steps, delay = spec["steps"], spec["apl_delay_steps"]
require(type(delay) is int and delay in (0, 18), "Invalid timing arm")
generated = trial["apl"]["apl-inhibition.float32.npy"]
applied = np.load(output / "apl-applied.float32.npy", mmap_mode="r", allow_pickle=False)
pending = np.load(output / "apl-pending-tail.float32.npy", mmap_mode="r", allow_pickle=False)
for name, value, shape in (("generated", generated, (steps, 4064)),
("applied", applied, (steps, 4064)), ("pending", pending, (delay, 4064))):
require(value.dtype == np.float32 and value.shape == shape and np.isfinite(value).all() and
np.all(value >= 0), "Invalid timing array " + name)
warmup, consumed, prefix = min(steps, delay), max(steps - delay, 0), max(delay - steps, 0)
require(np.all(applied[:warmup].view(np.uint32) == 0) and np.all(pending[:prefix].view(np.uint32) == 0),
"Timing initial padding must be positive zero")
require(bit_equal(applied[warmup:], generated[:consumed]) and bit_equal(pending[prefix:], generated[consumed:]),
"Timing transport is not an exact finite generated-sequence partition")
audit = read_json(output / "apl-timing-audit.json")
summary = read_json(output / "summary.json")
expected = {"pass_all": True, "apl_delay_steps": delay, "steps": steps,
"pending_generated_count": min(steps, delay), "pending_initial_zero_prefix": prefix,
"pending_application_tick_start": steps, "pending_application_tick_stop_exclusive": steps + delay,
"pending_generated_tick_start": consumed, "pending_generated_tick_stop_exclusive": steps,
"initial_padding_positive_zero": True, "sample_bits_preserved": True, "brain_tail_advanced": False,
"generated_trace": "apl-inhibition.float32.npy", "applied_trace": "apl-applied.float32.npy",
"pending_trace": "apl-pending-tail.float32.npy", "pending_raw_sha256": raw_hash(pending),
"pending_queue_drained": delay == 0, "finite_generated_sequence_only": True,
"future_apl_decay_or_feedback_generated": False, "closed_loop_branch_integrals_claimed_equal": False,
"biological_delay_measurement_claimed": False}
final_state = trial["apl"]["apl-state.float32.npy"][-1]
expected.update(final_apl_state_float32=final_state.tolist(),
final_apl_state_u32_bits=final_state.view(np.uint32).tolist(),
final_apl_state_nonzero=bool(np.any(final_state != 0)))
for key, value in expected.items():
require(key in audit and audit[key] == value and type(audit[key]) is type(value),
"Timing audit disagrees with independent data: " + key)
require(summary.get("apl_delay_steps") == delay and summary.get("apl_timing") == audit,
"Timing audit differs from GPU summary")
return {"applied": applied, "pending": pending, "audit": audit,
"report": {"pass_all": True, "apl_delay_steps": delay, "steps": steps,
"generated_applied_pending_bits_exact": True, "pending_generated_count": min(steps, delay),
"pending_initial_zero_prefix": prefix, "brain_tail_advanced": False}}
def timing_repeat(first, repeat, legacy):
result = legacy.exact_repeat(first, repeat)
result["checks"].update(apl_applied=bit_equal(first["timing"]["applied"], repeat["timing"]["applied"]),
apl_pending=bit_equal(first["timing"]["pending"], repeat["timing"]["pending"]),
timing_audit=first["timing"]["audit"] == repeat["timing"]["audit"])
result["pass_all"] = bool(all(result["checks"].values()))
result["scope"] += "; plus delayed applied h, finite pending tail and timing audit"
return result
def _case_spec(case, old_case, row, delay):
spec = read_json(case / "spec.json")
old = read_json(old_case / "spec.json")
require(type(spec.get("apl_delay_steps")) is int and spec["apl_delay_steps"] == delay,
"Case arm/delay mismatch")
require(set(spec) == set(old) | {"apl_delay_steps"}, "Unexpected case fields")
require(all(spec[key] == value for key, value in old.items() if key not in ("budget_root", "output_budget_bytes")),
"A scientific case field changed in addition to APL output delay")
require(spec["name"] == row["name"] and spec["kind"] == "calibration" and spec["steps"] == 1000 and
spec["duration_ms"] == 100.0 and spec["dt_ms"] == .1 and spec["training"] is False and
all(spec[key] == row[key] for key in ("mode", "odor_id", "odor_index", "seed")), "Invalid fixed CAL case")
for name in ("requests.npz", "pn-drive.npz"):
require(file_hash(case / name) == file_hash(old_case / name), "Frozen CAL input bytes changed: " + name)
return spec
def _read_new(legacy, output, case, spec, kc_globals, cfg):
trial = legacy.read_trial(output, case, spec, kc_globals, cfg["neurons"], cfg["edges"])
trial["timing"] = validate_timing(output, spec, trial)
return trial
def _provenance(output, name, repeat):
return {"case": name, "repeat": repeat, "sha256": {fname: file_hash(output / fname) for fname in
("state.npz", "summary.json", "tick-state-hashes.jsonl", "queue.jsonl", "requests.csv", "apl-component.json",
*APL_FILES, "apl-applied.float32.npy", "apl-pending-tail.float32.npy", "apl-timing-audit.json")}}
def collect_arm(arm, delay, records, odors, legacy, cfg, exp, ag, kc_globals):
failures, runs, rows, repeats, controls, provenance, timings, modes = [], {}, [], [], [], [], [], {}
for row in records:
name, first, spec = row["name"], None, None
case = exp / "cases" / arm / name
try:
spec = _case_spec(case, ag / "cases" / name, row, delay)
except Exception as error:
failures.append(dict(stage="case_spec", case=name, type=type(error).__name__, message=str(error)))
for repeat_index in ([0, 1] if row["seed"] == legacy.ORIGINAL_SEEDS[0] else [0]):
output = exp / "gpu" / arm / (name + "-r" + str(repeat_index))
try:
require(spec is not None, "Case spec unavailable")
trial = _read_new(legacy, output, case, spec, kc_globals, cfg)
require(trial["record"]["requested"] == row["requests"], "Inventory request count changed")
timings.append(dict(case=name, repeat=repeat_index, **trial["timing"]["report"]))
provenance.append(_provenance(output, name, repeat_index))
if repeat_index == 0:
first = trial
runs[(row["mode"], row["odor_id"], row["seed"])] = {key: first[key] for key in
("kc_counts", "expected_external_events", "external_events")}
rows.append(first["record"])
else:
require(first is not None, "Baseline unavailable for exact repeat")
report = dict(case=name, **timing_repeat(first, trial, legacy))
repeats.append(report)
require(report["pass_all"], "Exact repetition differs: " + ", ".join(
key for key, value in report["checks"].items() if not value))
if delay == 0:
control = control_identity(output, ag / "gpu" / (name + "-r" + str(repeat_index)), case)
controls.append(dict(repeat=repeat_index, **control))
require(control["pass_all"], "Delay0 differs from sealed AG: " + ", ".join(
key for key, value in control["checks"].items() if not value))
except Exception as error:
failures.append(dict(stage="raw_trial" if repeat_index == 0 else "repeat_trial", case=name,
repeat=repeat_index, type=type(error).__name__, message=str(error)))
# Each complete mode is still described even if the other mode/arm failed.
for mode in legacy.ORIGINAL_MODES:
needed = {(mode, odor, seed) for odor in odors for seed in legacy.ORIGINAL_SEEDS}
if needed.issubset(runs):
try:
modes[mode] = legacy.summarize_mode(mode, odors, legacy.ORIGINAL_SEEDS, runs, cfg["gates"])
except Exception as error:
failures.append(dict(stage="representation_calculation", mode=mode, type=type(error).__name__, message=str(error)))
numerical = bool(len(runs) == 60 and len(timings) == 72 and not any(
row["stage"] != "representation_calculation" for row in failures))
repeat_pass = bool(len(repeats) == 12 and all(row["pass_all"] for row in repeats))
control_pass = bool(delay != 0 or (len(controls) == 72 and all(row["pass_all"] for row in controls)))
representation = bool(len(modes) == 2 and all(row["gate"] for row in modes.values()))
execution = bool(numerical and repeat_pass and control_pass)
return {"arm": arm, "apl_delay_steps": delay, "pass_all": bool(execution and representation),
"execution_gate_pass": execution, "numerical_integrity_pass": numerical, "exact_repeat_pass": repeat_pass,
"control_identity_applicable": delay == 0, "delay0_control_identity_pass": control_pass if delay == 0 else None,
"original_representation_gate_pass": representation, "expected_base_trials": 60, "valid_base_trials": len(runs),
"expected_repeats": 12, "checked_repeats": len(repeats), "checked_timing_trials": len(timings),
"modes": modes, "runs": rows, "repeats": repeats, "timing_checks": timings,
"control_checks": controls, "failures": failures, "input_output_provenance": provenance}
def summarize(out_json=_DEFAULT_OUTPUT):
"""Default writes new evidence exclusively; out_json=None is pure recomputation."""
from context import EXP, AG, ROOT, CFG, check
target = EXP / "calibration/summary.json" if out_json is _DEFAULT_OUTPUT else (
None if out_json is None else Path(out_json))
require(target is None or not target.exists(), "Refusing existing calibration summary")
arms, failures, odors, encoder = {}, [], [], None
try:
check()
legacy = load_legacy()
require(CFG["delays_steps"] == [0, 18], "Fixed timing arms changed")
records = read_json(EXP / "protocol/calibration-cases.json")
require(file_hash(EXP / "protocol/calibration-cases.json") == file_hash(AG / "protocol/calibration-cases.json"),
"Original CAL inventory bytes changed")
odors = legacy.validate_inventory(records, CFG)
encoder = read_json(EXP / "protocol/encoder-audit.json")
require(file_hash(EXP / "protocol/encoder-audit.json") == file_hash(AG / "protocol/encoder-audit.json"),
"Frozen encoder audit changed")
require(encoder["panel_count"] == 12 and encoder["cal_count"] == 6 and
encoder["seeds"] == list(legacy.ORIGINAL_SEEDS) and encoder["requested_before_any_dynamics"] is True and
encoder["formal_AB_run"] is False and encoder["heldout_run"] is False, "Invalid frozen encoder scope")
import pandas as pd
panel = pd.read_parquet(ROOT / "experiments/phase4l-olfactory-grounding/odor-encoder/frozen-odor-panel.parquet")
require(len(panel) == 12 and set(panel[panel.split == "calibration"].odor_id) == legacy.ORIGINAL_CAL_IDS and
float(np.median(panel.mapped_response_sum)) == encoder["median_mapped_response_sum"],
"Static original 12-panel median/identities changed; no heldout trajectories are read")
kc = np.load(AG / "apl-bundle/kc_globals.npy", allow_pickle=False)
require(kc.dtype == np.int32 and kc.shape == (4064,) and np.all(np.diff(kc) > 0) and
np.all((kc >= 0) & (kc < CFG["neurons"])), "Invalid frozen KC identity")
for delay in (0, 18):
arm = "delay" + str(delay)
try:
arms[arm] = collect_arm(arm, delay, records, odors, legacy, CFG, EXP, AG, kc)
except Exception as error:
failures.append(dict(stage="arm", arm=arm, type=type(error).__name__, message=str(error)))
except Exception as error:
failures.append(dict(stage="prerequisites", type=type(error).__name__, message=str(error)))
for arm, data in arms.items():
failures.extend(dict(arm=arm, **row) for row in data["failures"])
complete = set(arms) == {"delay0", "delay18"}
numerical = bool(complete and all(row["numerical_integrity_pass"] for row in arms.values()) and not failures)
repeated = bool(complete and all(row["exact_repeat_pass"] for row in arms.values()))
control = bool("delay0" in arms and arms["delay0"]["delay0_control_identity_pass"])
represented = bool(complete and all(row["original_representation_gate_pass"] for row in arms.values()))
execution = bool(numerical and repeated and control)
result = {"phase": "4AJ", "pass_all": bool(execution and represented), "execution_gate_pass": execution,
"numerical_integrity_pass": numerical, "exact_repeat_pass": repeated, "delay0_control_identity_pass": control,
"original_representation_gate_pass": represented,
"original_representation_gate_pass_by_arm": {arm: row["original_representation_gate_pass"] for arm, row in arms.items()},
"expected_base_trials_per_arm": 60, "expected_repeats_per_arm": 12,
"expected_delay0_AG_comparisons": 72, "arms": arms, "failures": failures, "odor_ids": odors,
"encoder_audit": encoder, "legacy_calibration_sha256": LEGACY_SHA256,
"scope": "Both fixed timing arms, full original 4L CAL gate; no heldout or learning claim",
"representation_failure_is_not_numerical_failure": True,
"selection_or_tuning_performed": False, "both_arms_retained_regardless_of_representation": True,
"closed_loop_generated_h_integrals_forced_equal": False,
"pending_is_finite_generated_sequence_not_complete_future_decay": True,
"candidate_generated": False, "training": False, "heldout_dynamic_access": False,
"shared_apl_component_oracle_limit": True, "independent_oracle_equivalence_claim": False}
if target is not None:
outputs = {target: result}
for arm, data in arms.items():
directory = target.parent / arm
outputs[directory / "summary.json"] = data
for mode, metrics in data["modes"].items():
outputs[directory / (mode + ".json")] = metrics
require(all(not path.exists() for path in outputs), "Refusing existing arm/mode summary")
for path, value in outputs.items():
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("x", encoding="utf-8") as stream:
json.dump(value, stream, ensure_ascii=False, indent=2, allow_nan=False)
stream.write("\n")
return result
if __name__ == "__main__":
result = summarize()
print(json.dumps({key: result[key] for key in ("pass_all", "execution_gate_pass",
"delay0_control_identity_pass", "original_representation_gate_pass_by_arm")}), flush=True)
models/male-cns-lif/apl-timing-lab-v1/checksums.sha256
+22
View File
@@ -0,0 +1,22 @@
d96aef0107f7b0791d16e475b749da418b8a38d2c56a94e0779de0cd6581a7de PROTOCOL.md
7201871477a29ede31c7d6c6bf1c92700da3db5505e67ccfa7cb43126b8464b9 al_origin_audit.py
fb15ddeaa108305c8de697bbae36b23ab0414a7c58db51c66dce9feb4a048a0d calibration.py
9010f070fba9b7c9cf2d86187ee2abc57f3ac606ebc08a42baad4f3922b51734 compare_timing.py
0b720e23aa7988b2b443e14951082b1ce9b061a183b7f858c0e0f3c0fc6fb25e config.json
06d4994af3729d653263391ff3a3604f7c919e78562604e5c86acfa292d60ba0 context.py
0a2dbf9e5f889e9dd3491d036e6f92c17417a84482c5e1e49134c0993f7d68f2 finalize.py
af1975ba5e5e700208c0a598d3cbe648a3cf10885c6231239b6e4b28f76d173b gpu_timing.py
39d8c47c94bc2faddf016864a43e7715907fb987f56f7b27098060a88851f6be oracle_timing.py
bba71bcd52dcb78d9ae0fa93a8a43173a4669ecef038dce18ce7d82edd9bce50 prepare.py
d93e4fe2df542ce5a0f0adb5aceada583dc8c921f0159f426e50428d79d67a1d run_case.py
6f26c626acf9ef8e6fae7902af0cbf0bd176eb322529ca8aa203a9700716c889 run_suite.py
46f380dac340acc91f226968bb493475a31777f8290fb20f9145a97c9009d845 shift_audit.py
4b75a39d36090b0a66cc7ad4b370ac785713fb58c366df2efcc55152d612fc2b test_al_origin_audit.py
307a2848bf6d05aaa8105bb3558be2a74425894f96a4a2bd20af8b82cb8b6e25 test_calibration.py
cbae93c993ac9aa99bcce68ae2131c7f938f8a63e89779f8d3d79e36bf268dee test_protocol.py
bfc0d9e17410c2c52ea1d9a5259c92964f16b118dbae8cecad258697b67e4bdc tests/__init__.py
71876b36eef88c198aa8e833af547dbe772001c71b010707b656c165c8b54031 tests/test_compare_timing.py
4a375444dc732c3538df8aa4b81e43b5418754fcebebb0d02c16faad03d43195 tests/test_timing_queue.py
54df6d4818e2a0c26e96fb0aaf5d9b71cd964981706da937562d48af79aca14e timing_queue.py
97d8a240b08baaca935f9c0278978883005dcd6fdbb4ca0d948074ef79fd349c verify_frozen.py
52b05dd1ff30b6b51ff960fe95904532f5c147731a83f0b5c31f98ca7e53c216 manifest.json
models/male-cns-lif/apl-timing-lab-v1/compare_timing.py
+611
View File
@@ -0,0 +1,611 @@
"""Fail-closed byte-exact audit of fixed-delay shared-APL closed-loop runs.
Comparison is independent of either backend's stored PASS. No tolerance is
used, wall-clock performance fields are excluded, and reports contain no wall
timestamp. Scientific delay/tick metadata are checked explicitly.
"""
from __future__ import annotations
import csv
import hashlib
import json
import math
from pathlib import Path
import re
import struct
import numpy as np
N_GLOBAL = 166700
STATE_KEYS = ('step', 'neuron_idx', 'counts', 'v', 'g', 'ref')
REQUEST_FIELDS = ('tick', 'neuron_idx', 'attempted', 'accepted', 'rejected')
HASH_FIELDS = {'tick', 'v_sha256', 'g_sha256', 'ref_sha256'}
QUEUE_FIELDS = {'tick', 'count', 'accepted_count', 'ordered_edge_sha256',
'accepted_mask_sha256', 'joint_sha256'}
BUNDLE_FILES = ('input_matrix.npy', 'output_matrix.npy', 'kc_globals.npy', 'apl_globals.npy')
COEFFICIENTS = ('gd', 'c0', 'cg', 'cv', 'threshold', 'reset', 'seed_volt')
def require(condition, message):
if not condition:
raise ValueError(message)
def file_hash(path):
h = hashlib.sha256()
with Path(path).open('rb') as source:
for block in iter(lambda: source.read(8 << 20), b''):
h.update(block)
return h.hexdigest()
def raw_hash(array, dtype=None):
return hashlib.sha256(np.ascontiguousarray(array, dtype=dtype).tobytes()).hexdigest()
def save(path, report):
if path is None:
return
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
with path.open('x', encoding='utf-8') as target:
json.dump(report, target, indent=2, sort_keys=True, allow_nan=False)
target.write('\n')
def _read_json(path, used):
used.add(Path(path))
return json.loads(Path(path).read_text(encoding='utf-8'))
def _read_jsonl(path, used):
used.add(Path(path))
with Path(path).open(encoding='utf-8') as source:
return [json.loads(line) for line in source if line.strip()]
def _read_array(path, used):
used.add(Path(path))
return np.load(path, mmap_mode='r', allow_pickle=False)
def _check(checks, name, condition):
checks[name] = bool(condition)
def _bit_mask(a, b):
if a.dtype.kind == 'f':
unsigned = np.dtype('uint'+str(a.dtype.itemsize*8))
return np.ascontiguousarray(a).view(unsigned) != np.ascontiguousarray(b).view(unsigned)
return a != b
def _compare_array(name, a, b, checks, differences, dtype=None, shape=None):
a, b = np.asarray(a), np.asarray(b)
schema = a.shape == b.shape and a.dtype == b.dtype and \
(dtype is None or a.dtype == np.dtype(dtype)) and (shape is None or a.shape == shape)
_check(checks, name+'.schema', schema)
if not schema:
differences[name] = {'left_shape': list(a.shape), 'right_shape': list(b.shape),
'left_dtype': str(a.dtype), 'right_dtype': str(b.dtype), 'expected_shape': None if shape is None else list(shape),
'expected_dtype': None if dtype is None else str(np.dtype(dtype))}
_check(checks, name+'.bitwise_exact', False)
return False
finite = bool(np.isfinite(a).all() and np.isfinite(b).all())
_check(checks, name+'.finite', finite)
mask = _bit_mask(a, b)
exact = not bool(np.any(mask))
_check(checks, name+'.bitwise_exact', exact)
if not exact or not finite:
flat = np.flatnonzero(mask.ravel())
detail = {'mismatching_elements': int(len(flat)),
'first_positions': [list(map(int, np.unravel_index(int(index), a.shape))) for index in flat[:5]],
'left_nonfinite': int(np.count_nonzero(~np.isfinite(a))),
'right_nonfinite': int(np.count_nonzero(~np.isfinite(b)))}
if finite and a.size:
error = np.abs(a.astype(np.float64)-b.astype(np.float64))
detail.update(max_abs_error=float(error.max()), mean_abs_error=float(error.mean()))
if a.dtype.kind == 'f' and len(flat):
unsigned = np.dtype('uint'+str(a.dtype.itemsize*8))
av = np.ascontiguousarray(a).view(unsigned).ravel()
bv = np.ascontiguousarray(b).view(unsigned).ravel()
detail['first_bit_patterns'] = [{'left': hex(int(av[index])), 'right': hex(int(bv[index]))} for index in flat[:5]]
differences[name] = detail
return exact and finite
def _sha(value):
return isinstance(value, str) and re.fullmatch('[0-9a-f]{64}', value) is not None
def _read_requests(path, steps, used):
used.add(Path(path))
with Path(path).open(encoding='utf-8', newline='') as source:
reader = csv.DictReader(source)
require(tuple(reader.fieldnames or ()) == REQUEST_FIELDS, 'Request CSV schema mismatch: '+str(path))
rows = [{key: int(row[key]) for key in REQUEST_FIELDS} for row in reader]
previous = (-1, -1)
for row in rows:
key = row['tick'], row['neuron_idx']
require(previous < key and 0 <= key[0] < steps and 0 <= key[1] < N_GLOBAL,
'Request rows must be sorted, unique, and in range: '+str(path))
require(row['attempted'] == 1 and row['accepted'] in (0, 1) and row['rejected'] == 1-row['accepted'],
'Request accounting is incomplete: '+str(path))
previous = key
return rows
def _load_run(directory, role, n, edges, steps, record_history, name, delay, used, checks):
directory = Path(directory)
summary = _read_json(directory/'summary.json', used)
_check(checks, role+'.summary_complete', summary.get('complete') is True and summary.get('pass_all') is True and
summary.get('steps') == steps and summary.get('completed_ticks') == steps and
summary.get('nodes') == n and summary.get('edges') == edges and summary.get('precision') == 'float64')
is_cpu = role == 'cpu'
_check(checks, role+'.case_name', summary.get('name' if is_cpu else 'case') == name)
_check(checks, role+'.history_mode', summary.get('history_recorded' if is_cpu else 'record_history') is record_history)
_check(checks, role+'.apl_precision', summary.get('APL_precision' if is_cpu else 'apl_precision') == 'float32')
_check(checks, role+'.no_error_marker', not (directory/'error.json').exists())
used.add(directory/'state.npz')
with np.load(directory/'state.npz', allow_pickle=False) as source:
require(set(source.files) == set(STATE_KEYS), 'State NPZ keys changed: '+str(directory))
state = {key: source[key].copy() for key in STATE_KEYS}
for key, dtype in (('v', 'float64'), ('g', 'float64'), ('ref', 'int32'), ('counts', 'int32')):
require(state[key].shape == (n,) and state[key].dtype == np.dtype(dtype), 'Invalid final '+key+': '+str(directory))
require(state['step'].dtype == state['neuron_idx'].dtype == np.int32 and state['step'].ndim == 1 and
state['step'].shape == state['neuron_idx'].shape, 'Invalid sparse event schema: '+str(directory))
require(bool(np.all((state['step'] >= 0) & (state['step'] < steps))) and
bool(np.all((state['neuron_idx'] >= 0) & (state['neuron_idx'] < N_GLOBAL))), 'Sparse event range error')
event_keys = state['step'].astype(np.int64)*N_GLOBAL+state['neuron_idx']
_check(checks, role+'.events_sorted_unique', len(event_keys) < 2 or np.all(np.diff(event_keys) > 0))
_check(checks, role+'.count_nonnegative_total', np.all(state['counts'] >= 0) and int(state['counts'].sum()) == len(event_keys))
_check(checks, role+'.ref_range', np.all((state['ref'] >= 0) & (state['ref'] <= 22)))
_check(checks, role+'.reported_events', summary.get('spikes' if is_cpu else 'total_spikes') == len(event_keys))
hashes = _read_jsonl(directory/'tick-state-hashes.jsonl', used)
require(len(hashes) == steps and [row.get('tick') for row in hashes] == list(range(steps)), 'Incomplete tick hash coverage')
require(all(set(row) == HASH_FIELDS and all(_sha(row[key]) for key in HASH_FIELDS-{'tick'}) for row in hashes),
'Invalid tick state hash schema')
queues = _read_jsonl(directory/'queue.jsonl', used)
require(len(queues) == steps and [row.get('tick') for row in queues] == list(range(steps)), 'Incomplete queue coverage')
require(all(set(row) == QUEUE_FIELDS and type(row['count']) is int and type(row['accepted_count']) is int and
0 <= row['accepted_count'] <= row['count'] <= edges and
all(_sha(row[key]) for key in QUEUE_FIELDS-{'tick', 'count', 'accepted_count'}) for row in queues),
'Invalid queue row schema/count/hash')
for key in ('v', 'g', 'ref'):
_check(checks, role+'.last_hash_'+key, hashes[-1][key+'_sha256'] == raw_hash(state[key], '<i4' if key == 'ref' else '<f8'))
requests = _read_requests(directory/'requests.csv', steps, used)
requested, accepted = len(requests), sum(row['accepted'] for row in requests)
_check(checks, role+'.request_summary', summary.get('requests' if is_cpu else 'requested') == requested and
summary.get('applicable_requests' if is_cpu else 'accepted_requests') == accepted and
summary.get('ref_rejected_requests' if is_cpu else 'rejected_requests') == requested-accepted)
apl = {field: _read_array(directory/('apl-'+field+'.float32.npy'), used) for field in ('drive', 'state', 'inhibition', 'applied')}
for field, width in (('drive', 2), ('state', 2), ('inhibition', 4064), ('applied', 4064)):
require(apl[field].dtype == np.float32 and apl[field].shape == (steps, width), 'Invalid APL '+field+' history schema')
_check(checks, role+'.apl_positive_'+field, np.all(apl[field] >= 0))
pending = _read_array(directory/'apl-pending-tail.float32.npy', used)
require(pending.dtype == np.float32 and pending.shape == (delay, 4064), 'Invalid pending APL queue schema')
timing = _read_json(directory/'apl-timing-audit.json', used)
_check(checks, role+'.delay_summary', type(summary.get('apl_delay_steps')) is int and summary['apl_delay_steps'] == delay)
histories, pre = {}, {}
for field, dtype in (('v', 'float64'), ('g', 'float64'), ('ref', 'int32')):
files = (directory/(field+'-history.'+dtype+'.npy'), directory/('preAPL-'+field+'.'+dtype+'.npy'))
if record_history:
histories[field], pre[field] = (_read_array(path, used) for path in files)
require(all(array.dtype == np.dtype(dtype) and array.shape == (steps, n) for array in (histories[field], pre[field])),
'Small pre/post history schema mismatch')
_check(checks, role+'.history_hashes_'+field, all(row[field+'_sha256'] == raw_hash(histories[field][tick])
for tick, row in enumerate(hashes)))
_check(checks, role+'.history_final_'+field, not np.any(_bit_mask(histories[field][-1], state[field])))
else:
_check(checks, role+'.no_dense_'+field, not any(path.exists() for path in files))
if is_cpu:
component = _read_json(directory/'apl-worker-handshake.json', used)['handshake']
else:
component = _read_json(directory/'apl-component.json', used)
_check(checks, role+'.component_summary_consistent', summary.get('apl_component') == component)
return dict(directory=directory, summary=summary, state=state, hashes=hashes, queues=queues,
requests=requests, apl=apl, pending=pending, timing=timing,
histories=histories, pre=pre, component=component)
def _pair_outputs(left, right, checks, differences, n, steps, record_history):
for field in STATE_KEYS:
shape = None if field in ('step', 'neuron_idx') else (n,)
dtype = 'float64' if field in ('v', 'g') else 'int32'
_compare_array('state.'+field, left['state'][field], right['state'][field], checks, differences, dtype, shape)
for field, width in (('drive', 2), ('state', 2), ('inhibition', 4064), ('applied', 4064)):
_compare_array('apl.'+field, left['apl'][field], right['apl'][field], checks, differences, 'float32', (steps, width))
_compare_array('apl.pending', left['pending'], right['pending'], checks, differences, 'float32', left['pending'].shape)
for field in ('hashes', 'queues', 'requests', 'component'):
same = left[field] == right[field]
_check(checks, field+'.parsed_exact', same)
if not same:
if isinstance(left[field], list):
first = next((i for i, (a, b) in enumerate(zip(left[field], right[field])) if a != b), None)
differences[field] = {'left_count': len(left[field]), 'right_count': len(right[field]),
'first_differing_row': first,
'left_row': None if first is None else left[field][first],
'right_row': None if first is None else right[field][first]}
else:
differences[field] = {'left': left[field], 'right': right[field]}
if record_history:
for stage in ('histories', 'pre'):
for field, dtype in (('v', 'float64'), ('g', 'float64'), ('ref', 'int32')):
_compare_array(stage+'.'+field, left[stage][field], right[stage][field], checks, differences, dtype, (steps, n))
def _queue_digest(tick, ids, accepted):
ids = np.ascontiguousarray(ids, dtype='<i8').tobytes()
mask = np.ascontiguousarray(accepted, dtype=np.uint8).tobytes()
return {'tick': tick, 'count': len(mask), 'accepted_count': int(np.count_nonzero(accepted)),
'ordered_edge_sha256': hashlib.sha256(ids).hexdigest(),
'accepted_mask_sha256': hashlib.sha256(mask).hexdigest(),
'joint_sha256': hashlib.sha256(b'phase4af-queue-v1\0'+struct.pack('<qq', tick, len(mask))+ids+mask).hexdigest()}
def _independent_event_accounting(run, role, graph, requested, steps, checks, differences):
nodes, ptr = graph['nodes'], graph['ptr']
n = len(nodes)
state = run['state']
local = np.searchsorted(nodes, state['neuron_idx'])
require(np.all(local < n) and np.array_equal(nodes[local], state['neuron_idx']), 'Actual spike outside graph identity')
_check(checks, role+'.events_per_neuron_counts', np.array_equal(np.bincount(local, minlength=n), state['counts']))
actual_request_pairs = [(row['tick'], row['neuron_idx']) for row in run['requests']]
_check(checks, role+'.requests_match_frozen_input', actual_request_pairs == requested)
require(actual_request_pairs == requested, 'Actual request population/time differs from frozen input')
bounds = np.searchsorted(state['step'], np.arange(steps+1))
request_ticks = np.asarray([row['tick'] for row in run['requests']], np.int32)
rbounds = np.searchsorted(request_ticks, np.arange(steps+1))
request_nodes = np.asarray([row['neuron_idx'] for row in run['requests']], np.int32)
request_local = np.searchsorted(nodes, request_nodes)
require(np.all(request_local < n) and np.array_equal(nodes[request_local], request_nodes), 'Request outside graph')
last = np.full(n, -1_000_000_000, np.int64)
degrees = np.diff(ptr)
expected_counts = np.zeros(steps, np.int64)
arrival_tick = state['step'].astype(np.int64)+18
valid = arrival_tick < steps
np.add.at(expected_counts, arrival_tick[valid], degrees[local[valid]])
_check(checks, role+'.delayed_all_edges_count', np.array_equal(expected_counts, [row['count'] for row in run['queues']]))
request_ok, delivered_ok, ref_ok, queue_ok, reset_ok = True, True, True, True, True
first = {}
for tick in range(steps):
fired = local[bounds[tick]:bounds[tick+1]]
lo, hi = rbounds[tick:tick+2]
targets = request_local[lo:hi]
accepted = (tick-last[targets]) >= 22
for index, ok in zip(range(lo, hi), accepted):
row = run['requests'][index]
if row['accepted'] != int(ok) or row['rejected'] != int(not ok):
request_ok = False
first.setdefault('request_ref_mismatch', {'tick': tick, 'neuron_idx': row['neuron_idx']})
if not np.isin(targets[accepted], fired).all() or np.isin(targets[~accepted], fired).any():
delivered_ok = False
first.setdefault('request_delivery_mismatch', tick)
last[fired] = tick
ref = np.maximum(22-(tick-last), 0).astype(np.int32)
if raw_hash(ref, '<i4') != run['hashes'][tick]['ref_sha256']:
ref_ok = False
first.setdefault('ref_hash_mismatch', tick)
if run['pre']:
if len(fired) and not np.array_equal(np.ascontiguousarray(run['pre']['g'][tick, fired]).view(np.uint64), np.zeros(len(fired), np.uint64)):
reset_ok = False
first.setdefault('pre_apl_spike_reset_mismatch', tick)
due = local[bounds[tick-18]:bounds[tick-17]] if tick >= 18 else np.empty(0, np.int64)
chunks = [np.arange(ptr[source], ptr[source+1], dtype=np.int64) for source in due]
indices = np.concatenate(chunks) if chunks else np.empty(0, np.int64)
expected_queue = _queue_digest(tick, graph['edge_id'][indices], (ref[graph['dst'][indices]] == 0).astype(np.uint8))
if expected_queue != run['queues'][tick]:
queue_ok = False
first.setdefault('queue_reconstruction_mismatch', tick)
_check(checks, role+'.request_refractory_gate', request_ok)
_check(checks, role+'.accepted_spikes_rejected_absent', delivered_ok)
_check(checks, role+'.all_ref_hashes_from_actual_events', ref_ok)
if run['pre']:
_check(checks, role+'.small_queue_independent_reconstruction', queue_ok)
_check(checks, role+'.spike_g_reset_before_apl', reset_ok)
if first:
differences[role+'.independent_accounting'] = first
def _check_apl_application(run, role, nodes, kc, checks, differences):
if not run['pre']:
return
common, local_kc, bundle_kc = np.intersect1d(nodes, kc, assume_unique=True, return_indices=True)
_compare_array(role+'.apl_keeps_voltage', run['pre']['v'], run['histories']['v'], checks, differences, 'float64')
_compare_array(role+'.apl_keeps_ref', run['pre']['ref'], run['histories']['ref'], checks, differences, 'int32')
expected = np.array(run['pre']['g'], copy=True)
delta = run['apl']['applied'][:, bundle_kc].astype(np.float64)*np.float64(.001)
expected[:, local_kc] = expected[:, local_kc]-delta
_compare_array(role+'.apl_unconditional_g_write', expected, run['histories']['g'], checks, differences, 'float64')
def _timing_sequence_checks(generated, applied, pending, delay, role, checks, differences):
"""Independent slice-based reconstruction; does not import either queue."""
require(type(delay) is int and delay in (0, 18), 'Invalid fixed APL delay')
for key, value in (('generated', generated), ('applied', applied), ('pending', pending)):
require(isinstance(value, np.ndarray) and value.ndim == 2 and value.dtype == np.float32,
'Invalid timing array '+key)
steps, width = generated.shape
require(width > 0 and applied.shape == (steps, width) and pending.shape == (delay, width),
'Timing sequence dimensions do not partition the declared window')
_check(checks, role+'.timing_finite_nonnegative',
all(np.isfinite(value).all() and np.all(value >= 0) for value in (generated, applied, pending)))
consumed = max(steps-delay, 0)
expected_applied = np.zeros_like(generated)
if consumed:
expected_applied[delay:] = generated[:consumed]
expected_pending = np.zeros((delay, width), dtype=np.float32)
generated_pending = min(steps, delay)
if generated_pending:
expected_pending[delay-generated_pending:] = generated[consumed:]
_compare_array(role+'.fixed_delay_applied', applied, expected_applied, checks, differences,
'float32', (steps, width))
_compare_array(role+'.fixed_delay_pending', pending, expected_pending, checks, differences,
'float32', (delay, width))
_check(checks, role+'.finite_sequence_row_conservation', consumed+generated_pending == steps)
def _timing_metadata_checks(run, role, delay, steps, checks):
"""Validate declared limits against actual queue and generated APL state."""
timing, summary = run['timing'], run['summary']
count, prefix = min(steps, delay), max(delay-steps, 0)
expected = {'apl_delay_steps': delay, 'steps': steps, 'pending_generated_count': count,
'generated_rows': steps, 'applied_generated_rows': max(steps-delay, 0),
'pending_initial_zero_prefix': prefix, 'pending_application_tick_start': steps,
'pending_application_tick_stop_exclusive': steps+delay,
'pending_generated_tick_start': max(steps-delay, 0),
'pending_generated_tick_stop_exclusive': steps}
_check(checks, role+'.timing_integer_metadata',
all(type(timing.get(key)) is int and timing[key] == value for key, value in expected.items()))
_check(checks, role+'.pending_hash_metadata', timing.get('pending_raw_sha256') == raw_hash(run['pending'], '<f4'))
_check(checks, role+'.pending_shape_metadata', timing.get('pending_shape') == [delay, 4064])
_check(checks, role+'.pending_summary_counts',
type(summary.get('pending_generated_count')) is int and summary['pending_generated_count'] == count and
type(summary.get('pending_initial_zero_prefix')) is int and summary['pending_initial_zero_prefix'] == prefix and
summary.get('pending_tail_shape') == [delay, 4064] and
summary.get('future_generated_decay_tail_included') is False)
state = np.asarray(run['apl']['state'][-1], dtype=np.float32)
state_values, state_bits = timing.get('final_apl_state_float32'), timing.get('final_apl_state_u32_bits')
valid_state = isinstance(state_values, list) and len(state_values) == 2 and isinstance(state_bits, list) and len(state_bits) == 2
if valid_state:
try:
value = np.asarray(state_values, dtype=np.float32)
valid_state = np.array_equal(value.view(np.uint32), state.view(np.uint32)) and \
all(type(x) is int for x in state_bits) and state_bits == state.view(np.uint32).tolist()
except (ValueError, TypeError, OverflowError):
valid_state = False
_check(checks, role+'.final_apl_state_metadata', valid_state and
timing.get('final_apl_state_nonzero') is bool(np.any(state != 0)))
if role == 'cpu':
_check(checks, role+'.no_extra_brain_or_apl_ticks',
type(timing.get('brain_steps_after_window')) is int and timing['brain_steps_after_window'] == 0 and
type(timing.get('APL_generation_steps_after_window')) is int and timing['APL_generation_steps_after_window'] == 0 and
timing.get('future_generated_decay_tail_included') is False and
summary.get('future_generated_decay_tail_included') is False and
summary.get('APL_worker_ticks') == steps)
_check(checks, role+'.timing_summary_metadata',
summary.get('pending_generated_count') == count and
summary.get('pending_initial_zero_prefix') == prefix and
summary.get('pending_tail_shape') == [delay, 4064])
else:
_check(checks, role+'.no_extra_brain_or_apl_ticks',
timing.get('brain_tail_advanced') is False and
timing.get('future_apl_decay_or_feedback_generated') is False and
timing.get('finite_generated_sequence_only') is True and
timing.get('closed_loop_branch_integrals_claimed_equal') is False and
timing.get('biological_delay_measurement_claimed') is False)
_check(checks, role+'.timing_summary_metadata', summary.get('apl_timing') == timing)
def _finish(report, checks, differences, used, out_json):
report['checks'] = {name: bool(value) for name, value in checks.items()}
report['differences'] = differences
report['pass_all'] = bool(checks) and all(checks.values()) and 'error' not in report
report['failed_checks'] = sorted(name for name, value in checks.items() if not value)
report['evidence_sha256'] = {str(path): file_hash(path) for path in sorted(used) if path.is_file()}
save(out_json, report)
return report
def compare(case_dir, gpu_dir, cpu_dir, out_json=None):
"""Strict independent closed-loop comparison; out_json=None is read-only."""
case_dir, gpu_dir, cpu_dir = Path(case_dir), Path(gpu_dir), Path(cpu_dir)
if out_json is not None:
require(not Path(out_json).exists(), 'Refusing existing comparison report')
checks, differences, used = {}, {}, set()
report = {'kind': 'shared-APL-component closed-loop byte-exact comparison', 'pass_all': False,
'case_dir': str(case_dir), 'gpu_dir': str(gpu_dir), 'cpu_dir': str(cpu_dir),
'tolerance': None, 'signed_zero_included': True, 'timing_fields_compared': False,
'independent_CPU_all_components_claim': False, 'finite_generated_sequence_accounting_only': True,
'cross_delay_closed_loop_integrals_claimed_equal': False, 'biological_delay_measurement_claimed': False,
'scientific_apl_delay_fields_compared': True}
try:
spec = _read_json(case_dir/'spec.json', used)
steps, history = spec['steps'], spec['record_history']
delay = spec['apl_delay_steps']
require(type(delay) is int and delay in (0, 18), 'Explicit fixed delay 0 or 18 required')
require(type(steps) is int and steps > 0 and type(history) is bool and spec['dt_ms'] == .1,
'Invalid fixed case clock/history schema')
graph_dir, bundle_dir = Path(spec['graph_dir']), Path(spec['apl_bundle_dir'])
graph = {field: _read_array(graph_dir/(field+'.npy'), used) for field in
('nodes', 'edge_id', 'src', 'dst', 'ptr', 'signed_raw', 'weights')}
nodes = graph['nodes']
n, edges = len(nodes), len(graph['edge_id'])
require(n > 0 and edges > 0 and nodes.dtype == np.int32 and nodes.ndim == 1 and np.all((nodes >= 0) & (nodes < N_GLOBAL)) and
np.all(np.diff(nodes) > 0), 'Invalid canonical node identities')
require(graph['ptr'].dtype == np.int64 and graph['ptr'].shape == (n+1,) and graph['ptr'][0] == 0 and
graph['ptr'][-1] == edges and np.all(np.diff(graph['ptr']) >= 0), 'Invalid CSR')
for key, dtype in (('edge_id', 'int64'), ('src', 'int32'), ('dst', 'int32'), ('signed_raw', 'int64'), ('weights', 'float64')):
require(graph[key].dtype == np.dtype(dtype) and graph[key].shape == (edges,), 'Invalid graph '+key)
require(np.array_equal(graph['src'], np.repeat(np.arange(n, dtype=np.int32), np.diff(graph['ptr']))) and
np.all((graph['dst'] >= 0) & (graph['dst'] < n)), 'Invalid source/destination row mapping')
require(history == (n < 512) and (n < 512 or n == N_GLOBAL), 'Invalid small/full history declaration')
require(np.all(graph['edge_id'] >= 0) and np.all(np.diff(graph['edge_id']) > 0),
'Original canonical edge IDs must be unique and increasing')
if n == N_GLOBAL:
require(edges == 25582938 and np.array_equal(nodes, np.arange(n, dtype=np.int32)) and
np.array_equal(graph['edge_id'], np.arange(edges, dtype=np.int64)), 'Full canonical graph identity changed')
bundle = {name: _read_array(bundle_dir/name, used) for name in BUNDLE_FILES}
kc, apl_ids = bundle['kc_globals.npy'], bundle['apl_globals.npy']
require(kc.dtype == apl_ids.dtype == np.int32 and kc.shape == (4064,) and apl_ids.shape == (2,), 'Invalid APL/KC identities')
require(np.all((kc >= 0) & (kc < N_GLOBAL)) and np.all(np.diff(kc) > 0) and
np.all((apl_ids >= 0) & (apl_ids < N_GLOBAL)) and np.all(np.diff(apl_ids) > 0) and
not np.intersect1d(kc, apl_ids).size, 'APL/KC identities overlap or are invalid')
for key, shape in (('input_matrix.npy', (2, N_GLOBAL)), ('output_matrix.npy', (2, 4064))):
require(bundle[key].dtype == np.float32 and bundle[key].shape == shape and
np.isfinite(bundle[key]).all() and np.all(bundle[key] >= 0), 'Invalid normalized APL matrix')
gpu = _load_run(gpu_dir, 'gpu', n, edges, steps, history, spec['name'], delay, used, checks)
cpu = _load_run(cpu_dir, 'cpu', n, edges, steps, history, spec['name'], delay, used, checks)
_pair_outputs(gpu, cpu, checks, differences, n, steps, history)
used.add(case_dir/'requests.npz')
with np.load(case_dir/'requests.npz', allow_pickle=False) as source:
require(set(source.files) == {'tick', 'neuron_idx'} and source['tick'].dtype == source['neuron_idx'].dtype == np.int32,
'Fixed request NPZ schema changed')
require(source['tick'].shape == source['neuron_idx'].shape and source['tick'].ndim == 1, 'Invalid sparse request shape')
requested = list(zip(map(int, source['tick']), map(int, source['neuron_idx'])))
for role, run in (('gpu', gpu), ('cpu', cpu)):
_independent_event_accounting(run, role, graph, requested, steps, checks, differences)
_check_apl_application(run, role, nodes, kc, checks, differences)
_timing_sequence_checks(run['apl']['inhibition'], run['apl']['applied'], run['pending'],
delay, role, checks, differences)
_timing_metadata_checks(run, role, delay, steps, checks)
# Independently derive the effective clone. Original graph weights stay read-only.
cpu_weights = _read_array(cpu_dir/'weights.npy', used)
require(cpu_weights.dtype == np.float64 and cpu_weights.shape == (edges,), 'CPU effective weight schema mismatch')
expected_hash, base_hash, mask_hash = hashlib.sha256(), hashlib.sha256(), hashlib.sha256()
weight_ok, mask_count = True, 0
for start in range(0, edges, 1 << 20):
stop = min(edges, start+(1 << 20))
original = np.asarray(graph['weights'][start:stop])
require(np.isfinite(original).all(), 'Nonfinite original weights')
raw_expected = graph['signed_raw'][start:stop]*(.275*.001)
require(not np.any(_bit_mask(original, raw_expected)), 'Original SI weights differ from unaltered signed contacts')
mask = np.isin(nodes[graph['src'][start:stop]], apl_ids)
expected = original.copy()
expected[mask] = np.float64(0)
mask_count += int(mask.sum())
expected_hash.update(np.ascontiguousarray(expected, dtype='<f8').tobytes())
base_hash.update(np.ascontiguousarray(original, dtype='<f8').tobytes())
mask_hash.update(np.ascontiguousarray(graph['edge_id'][start:stop][mask], dtype='<i8').tobytes())
weight_ok &= not np.any(_bit_mask(expected, cpu_weights[start:stop]))
expected_sha = expected_hash.hexdigest()
_check(checks, 'cpu.effective_weights_independent_bytes', weight_ok)
for role, run in (('gpu', gpu), ('cpu', cpu)):
_check(checks, role+'.effective_weights_raw_sha', run['summary'].get('effective_weights_raw_sha256') == expected_sha)
_check(checks, role+'.apl_fast_edge_count', run['summary'].get('apl_fast_edges_zeroed') == mask_count)
_check(checks, role+'.kc_target_count', run['summary'].get('kc_targets_present') == len(np.intersect1d(nodes, kc)))
mask_audit = _read_json(cpu_dir/'apl-mask-audit.json', used)
_check(checks, 'cpu.mask_provenance', mask_audit['count'] == mask_count and
mask_audit['ordered_original_edge_id_sha256'] == mask_hash.hexdigest() and
mask_audit['base_weight_raw_bytes_sha256'] == base_hash.hexdigest() and
mask_audit['effective_weights_raw_sha256'] == expected_sha and mask_audit['base_graph_files_modified'] is False)
case_hashes = {name: file_hash(case_dir/name) for name in ('spec.json', 'requests.npz')}
if (case_dir/'arrays.npz').exists():
case_hashes['arrays.npz'] = file_hash(case_dir/'arrays.npz')
used.add(case_dir/'arrays.npz')
baseline_coeff = _read_json(Path(spec['coefficients_path']), used)
actual_coeff = _read_json(cpu_dir/'coefficients.json', used)
_check(checks, 'cpu.actual_author_coefficients_exact', all(float(actual_coeff[key]).hex() == float(baseline_coeff[key]).hex()
for key in COEFFICIENTS))
audit = _read_json(cpu_dir/'input-audit.json', used)
expected_graph_hashes = {name: file_hash(graph_dir/name) for name in
tuple(field+'.npy' for field in graph)+('completeness.csv', 'connectivity.parquet')}
bundle_hashes = {name: file_hash(bundle_dir/name) for name in BUNDLE_FILES}
_check(checks, 'cpu.read_actual_case', audit['spec'] == spec and audit['case_hashes'] == case_hashes)
_check(checks, 'cpu.graph_hashes', audit['graph_hashes'] == expected_graph_hashes)
_check(checks, 'cpu.bundle_hashes', audit['bundle_hashes'] == bundle_hashes)
_check(checks, 'cpu.no_candidate_trace_input', audit.get('candidate_trace_used_as_input') is False and
cpu['summary'].get('candidate_trace_used_as_input') is False)
_check(checks, 'cpu.worker_source_hash', audit['legacy_apl_worker_sha256'] == file_hash(spec['legacy_apl_worker']))
_check(checks, 'cpu.coefficient_input_hash', audit['coefficients_sha256'] == file_hash(spec['coefficients_path']))
gpu_hashes = dict(case_hashes, **{'base_weights.npy': file_hash(graph_dir/'weights.npy'),
'coefficients.json': file_hash(spec['coefficients_path'])})
_check(checks, 'gpu.input_hashes', gpu['summary']['input_sha256'] == gpu_hashes)
for role, run in (('gpu', gpu), ('cpu', cpu)):
component = run['component']
_check(checks, role+'.actual_component_bundle_bytes', component['files_sha256'] == bundle_hashes and
component['array_sha256'] == {name: raw_hash(value) for name, value in bundle.items()})
_check(checks, role+'.component_dimensions_clock', component.get('ok') is True and
component['protocol'] == 'phase4ag-apl-v1' and component['n_neurons'] == N_GLOBAL and
component['n_apl'] == 2 and component['n_kc'] == 4064 and component['dt_ms'] == .1)
decay = np.float32(math.exp(-.1/20.0))
gain = np.float32(.01)
_check(checks, role+'.frozen_apl_constants', component['decay_f32_bits'] == f'{int(decay.view(np.uint32)):08x}' and
component['gain_f32_bits'] == f'{int(gain.view(np.uint32)):08x}')
env = component['environment']
_check(checks, role+'.component_precision_environment', env['cuda_matmul_allow_tf32'] is False and
env['float32_matmul_precision'] == 'highest' and env['precision_configuration_changed_by_component'] is False and
'4090' in env['device_name'])
environment = _read_json(cpu_dir/'environment.json', used)
_check(checks, 'cpu.pinned_environment', environment['packages']['brian2'] == '2.5.1' and
environment['packages']['numpy'] == '1.22.3' and environment['packages']['sympy'] == '1.12' and
environment['Torch_imported_into_Brian'] is False and environment['author_precision'] == 'float64')
queue_audit = _read_json(cpu_dir/'queue-audit.json', used)
_check(checks, 'cpu.queue_observer_coverage', queue_audit['peek_calls'] == queue_audit['advance_calls'] == steps and
queue_audit['same_peek_object_returned'] is True and queue_audit['zero_weight_and_refractory_rejected_edges_retained'] is True)
_check(checks, 'cpu.sidecar_complete', cpu['summary']['APL_worker_exit_status'] == 0 and cpu['summary']['APL_worker_ticks'] == steps)
report.update(case=spec['name'], steps=steps, nodes=n, edges=edges, record_history=history, apl_delay_steps=delay,
total_spikes=len(gpu['state']['step']), active_neurons=int(np.count_nonzero(gpu['state']['counts'])),
requested=len(requested), accepted_requests=sum(row['accepted'] for row in gpu['requests']),
rejected_requests=sum(row['rejected'] for row in gpu['requests']),
arriving_edges=sum(row['count'] for row in gpu['queues']),
accepted_arriving_edges=sum(row['accepted_count'] for row in gpu['queues']),
apl_fast_edges_zeroed=mask_count, effective_weights_raw_sha256=expected_sha,
component_environment=gpu['component']['environment'],
limits=['APL Torch primitives are shared; original AST component validation is a separate required gate.',
'Small cases preserve and independently hash all pre/post states; full cases compare all-tick instrumented state hashes without saving dense whole-brain histories.',
'For full cases intermediate finite checks are performed by the instrumented backends; this file independently inspects final states and all APL traces.',
'Fixed-delay checks conserve this run\'s finite generated sample sequence, including pending initial zeros; no tail brain advancement or equality of different closed-loop branches is claimed.',
'This is exact agreement for the declared trajectory, not physiological validity or learning.'])
except Exception as error:
report['error'] = {'type': type(error).__name__, 'message': str(error)}
return _finish(report, checks, differences, used, out_json)
def compare_gpu_repeats(first, second, out_json=None):
"""Repeat equality only; the first run's independent oracle gate is separate."""
first, second = Path(first), Path(second)
if out_json is not None:
require(not Path(out_json).exists(), 'Refusing existing repeat report')
checks, differences, used = {}, {}, set()
report = {'kind': 'fresh-GPU-run exact repeat comparison', 'first': str(first), 'second': str(second),
'pass_all': False, 'tolerance': None, 'signed_zero_included': True, 'timing_fields_compared': False,
'scientific_apl_delay_fields_compared': True,
'scope': 'Scientific outputs, recording integrity and input hashes; independent case/graph audit belongs to compare()'}
try:
summary = _read_json(first/'summary.json', used)
n, edges, steps, name, history = (summary[key] for key in ('nodes', 'edges', 'steps', 'case', 'record_history'))
delay = summary['apl_delay_steps']
require(type(delay) is int and delay in (0, 18), 'Explicit repeat APL delay required')
require(type(n) is int and type(edges) is int and type(steps) is int and n > 0 and edges > 0 and steps > 0 and
type(history) is bool and history == (n < 512), 'Invalid repeat dimensions')
left = _load_run(first, 'first_gpu', n, edges, steps, history, name, delay, used, checks)
right = _load_run(second, 'second_gpu', n, edges, steps, history, name, delay, used, checks)
_pair_outputs(left, right, checks, differences, n, steps, history)
for role, run in (('first_gpu', left), ('second_gpu', right)):
_timing_sequence_checks(run['apl']['inhibition'], run['apl']['applied'], run['pending'],
delay, role, checks, differences)
_timing_metadata_checks(run, role, delay, steps, checks)
stable = ('case', 'complete', 'pass_all', 'steps', 'completed_ticks', 'nodes', 'edges', 'precision',
'apl_precision', 'state_units', 'total_spikes', 'requested', 'accepted_requests', 'rejected_requests',
'arriving_edges', 'accepted_arriving_edges', 'counts_match_sparse_events', 'all_finite', 'record_history',
'apl_fast_edges_zeroed', 'kc_targets_present', 'effective_weights_raw_sha256', 'apl_schedule',
'apl_si_conversion', 'build', 'sealed_ae_backend_sha256', 'learning', 'input_sha256',
'apl_delay_steps', 'apl_timing', 'source_provenance', 'pending_generated_count',
'pending_initial_zero_prefix', 'pending_tail_shape', 'future_generated_decay_tail_included')
for key in stable:
_check(checks, 'summary.'+key, key in left['summary'] and key in right['summary'] and
left['summary'][key] == right['summary'][key])
if not checks['summary.'+key]:
differences['summary.'+key] = {'first': left['summary'].get(key), 'second': right['summary'].get(key)}
report.update(case=name, steps=steps, nodes=n, edges=edges, record_history=history, apl_delay_steps=delay,
total_spikes=len(left['state']['step']),
explicit_exclusions=['wall_seconds', 'diagnostic_simulation_seconds', 'build_or_cache_seconds',
'peak_allocated_vram_bytes', 'peak_reserved_vram_bytes'])
except Exception as error:
report['error'] = {'type': type(error).__name__, 'message': str(error)}
return _finish(report, checks, differences, used, out_json)
models/male-cns-lif/apl-timing-lab-v1/config.json
+24
View File
@@ -0,0 +1,24 @@
{
"phase": "4AJ",
"name": "apl-timing-lab-v1",
"human_authorization": "允许独立新模型分支测试动力学假设",
"gpu_uuid": "GPU-79f3a85d-adce-7833-7003-6488140ca596",
"neurons": 166700,
"edges": 25582938,
"raw_weight_sum": 124177617,
"delays_steps": [0, 18],
"dt_ms": 0.1,
"apl": {"tau_ms": 20.0, "gain": 0.01, "normalization": 1.0},
"fixtures": ["zero", "pn_pulse", "kc_ref_boundary", "two_sources", "arrival_collision", "natural_request", "apl_only", "high_fanin"],
"fixture_repeats": 3,
"calibration": {"split": "calibration", "odor_count": 6, "modes": ["physiological", "matched_energy"], "seeds": [51101,51102,51103,51104,51105], "duration_ms": 100.0, "median_scope": "original frozen 12-odor panel", "full_oracle_probes": ["cal-physiological-00-s51101", "cal-matched_energy-00-s51101"], "repeat_first_seed_each_odor": true},
"gates": {"sparse_min_fraction": 0.03, "sparse_max_fraction": 0.09, "max_fraction_strict": 0.5, "min_fraction_strict": 0.0, "minimum_separable_odor_pairs": 3, "same_jaccard_gt_between": true, "between_euclidean_gt_within": true, "both_modes_required": true, "finite_required": true, "repeat_exact_required": true},
"output_budget_bytes": 25769803776,
"origin_audit_budget_bytes": 2147483648,
"training": false,
"heldout_dynamic_access": false,
"candidate_generation": false,
"hypothesis": "Only delay the entire generated graded-APL h by 18 exact steps; zero delay is a fresh replay of sealed AG control. Engineering scheduling parity, not measured MaleCNS physiology. No change to I/O, gain, state update, fast-APL zero mask, neuron coefficients, inputs or biological observation window.",
"tail_contract": "pending tail is the next D queue slots, shape(D,4064); if T<D its initial zero prefix is not generated future output. Do not advance the brain or APL after the original window. Closed-loop h totals are not forced to match.",
"selection_policy": "Both arms and all fixed CAL cases retained regardless of representation outcome. Stop later scientific execution on numerical/oracle/integrity failure. No scan, tuning, favorable seed or result selection."
}
models/male-cns-lif/apl-timing-lab-v1/context.py
+92
View File
@@ -0,0 +1,92 @@
"""Independent APL scheduling hypothesis; every earlier artifact is read-only."""
import hashlib
import importlib.util
import json
import sys
from pathlib import Path
ROOT = Path('/data/digital-drosophila')
MODEL = Path(__file__).resolve().parent
EXP = ROOT / 'experiments/phase4aj-apl-timing'
AI_MODEL = ROOT / 'models/male-cns-lif/pn-recruitment-audit-v1'
AI = ROOT / 'experiments/phase4ai-pn-recruitment-audit'
AH_MODEL = ROOT / 'models/male-cns-lif/kc-drive-audit-v1'
AG_MODEL = ROOT / 'models/male-cns-lif/author-apl-lab-v2'
AG = ROOT / 'experiments/phase4ag-author-apl-r2'
META = ROOT / 'datasets/male-cns-v1.0/processed/neuron-only/neuron-index/unified-metadata.parquet'
COEFF = ROOT / 'experiments/phase4af-author-fullgraph/oracle/no_input/coefficients.json'
CFG = json.loads((MODEL / 'config.json').read_text(encoding='utf-8'))
def load_module(name, path):
spec = importlib.util.spec_from_file_location(name, path)
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
spec.loader.exec_module(module)
return module
def digest(path):
h = hashlib.sha256()
with Path(path).open('rb') as f:
for block in iter(lambda: f.read(8 << 20), b''):
h.update(block)
return h.hexdigest()
def read(path):
return json.loads(Path(path).read_text(encoding='utf-8'))
def save(path, value):
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
with path.open('x', encoding='utf-8') as f:
json.dump(value, f, indent=2, ensure_ascii=False, allow_nan=False)
f.write('\n')
def sources():
return {p.relative_to(MODEL).as_posix(): digest(p) for p in sorted(MODEL.rglob('*')) if p.is_file()
and '__pycache__' not in p.parts and p.name not in ('manifest.json', 'checksums.sha256')}
def integrity():
previous = load_module('sealed_ai_context_aj', AI_MODEL / 'context.py')
result = previous.integrity()
sys.path.insert(0, str(ROOT / 'models/male-cns-lif/terminal-scope-diagnostic-v1'))
from diagnostic import manifest_check
for path in (AI_MODEL, AI):
result[str(path.relative_to(ROOT))] = manifest_check(path, digest)
assert all(r['pass'] for r in result.values())
return result
def check():
registered = read(EXP / 'protocol/preregistered.json')
assert registered['sources'] == sources(), 'Registered sources changed'
for row in registered['inputs']:
assert digest(ROOT / row['path']) == row['sha256'], row['path']
for row in read(EXP / 'protocol/prepared-hashes.json'):
assert digest(EXP / row['path']) == row['sha256'], row['path']
return registered
def budget():
size = sum(p.stat().st_size for p in EXP.rglob('*') if p.is_file())
assert size < CFG['output_budget_bytes'], 'Output budget exceeded; preserve partial work'
return size
def seal(root):
rows = [dict(path=str(p.relative_to(root)), bytes=p.stat().st_size, sha256=digest(p))
for p in sorted(root.rglob('*')) if p.is_file() and '__pycache__' not in p.parts]
save(root / 'manifest.json', dict(root=str(root), files=rows))
with (root / 'checksums.sha256').open('x') as f:
for row in rows:
f.write(row['sha256'] + ' ' + row['path'] + '\n')
f.write(digest(root / 'manifest.json') + ' manifest.json\n')
def bit_equal(a, b):
return a.dtype == b.dtype and a.shape == b.shape and a.tobytes() == b.tobytes()
models/male-cns-lif/apl-timing-lab-v1/finalize.py
+156
View File
@@ -0,0 +1,156 @@
"""Recheck raw evidence and seal this explicit scheduling hypothesis, not learning."""
from context import ROOT, MODEL, EXP, AG, CFG, check, save, read, digest, integrity, budget, seal
def optional(path):
return read(path) if path.exists() else None
def main():
from compare_timing import compare, compare_gpu_repeats
from calibration import control_identity, summarize
check(); budget()
arms = ['delay' + str(delay) for delay in CFG['delays_steps']]
expected_cases = CFG['fixtures'] + CFG['calibration']['full_oracle_probes']
comparisons, repeats, controls, components = {}, {}, {}, {}
for arm in arms:
for name in expected_cases:
path = EXP / 'audit' / f'compare-{arm}-{name}.json'
if path.exists():
actual = compare(EXP / 'cases' / arm / name, EXP / 'gpu' / arm / (name + '-r0'), EXP / 'oracle' / arm / name, None)
assert actual == read(path), 'Changed raw oracle evidence: ' + arm + '/' + name
comparisons[arm + '/' + name] = actual['pass_all']
path = EXP / 'component-audit' / arm / (name + '.json')
if path.exists():
report = read(path)
assert set(report.get('input_sha256', {})) == {'state.npz', 'summary.json',
'apl-drive.float32.npy', 'apl-state.float32.npy', 'apl-inhibition.float32.npy'}
for filename, sha in report['input_sha256'].items():
assert digest(EXP / 'gpu' / arm / (name + '-r0') / filename) == sha
assert set(report.get('matrix_checks', {})) == {'input_matrix', 'output_matrix', 'input_raw', 'output_raw',
'kc_globals', 'apl_globals', 'pn_globals', 'apl_out_ids', 'input_scale', 'output_scale'}
assert set(report.get('ast_replay_bitwise_exact', {})) == {'drive', 'state', 'inhibition'}
assert set(report.get('float64_measurement', {})) == {'drive', 'state', 'inhibition'}
actual_pass = bool(not report.get('error') and all(report['matrix_checks'].values()) and
all(report['ast_replay_bitwise_exact'].values()) and report['theoretical_bound']['applicable'] and
all(metric['bound_violations'] == 0 for metric in report['float64_measurement'].values()))
assert report['pass_all'] == actual_pass
components[arm + '/' + name] = actual_pass
for name in CFG['fixtures']:
for repeat in range(1, CFG['fixture_repeats']):
path = EXP / 'audit' / f'repeat-{arm}-{name}-r{repeat}.json'
if path.exists():
actual = compare_gpu_repeats(EXP / 'gpu' / arm / (name + '-r0'), EXP / 'gpu' / arm / f'{name}-r{repeat}', None)
assert actual == read(path), 'Changed raw repeat evidence'
repeats[f'{arm}/{name}-r{repeat}'] = actual['pass_all']
for name in expected_cases:
path = EXP / 'audit' / f'control-{name}.json'
if path.exists():
actual = control_identity(EXP / 'gpu/delay0' / (name + '-r0'), AG / 'gpu' / (name + '-r0'), EXP / 'cases/delay0' / name)
assert actual == read(path), 'Changed delay0 control evidence'
controls[name] = actual['pass_all']
origin = optional(EXP / 'origin-audit/summary.json')
cal_records = read(EXP / 'protocol/calibration-cases.json')
origin_complete = False
if origin:
expected_origin = {row['name']: row for row in cal_records}
assert len(expected_origin) == len(cal_records) == 60
assert len(origin['trials']) == 60 and {row['case'] for row in origin['trials']} == set(expected_origin)
for row in origin['trials']:
original = expected_origin[row['case']]
assert row['repeat'] == 0 and row['ALLN_targets'] == 420 and row['pass_all']
assert set(row['exact']) == {'step', 'neuron_idx', 'counts', 'v', 'g', 'ref'} and all(row['exact'].values())
assert all(row[key] == original[key] for key in ('mode', 'odor_id', 'seed'))
origin_complete = True
for row in origin['trials']:
trial = EXP / 'origin-audit' / row['case']
assert digest(trial / 'summary.json') == row['summary_sha256']
for payload in read(trial / 'output-hashes.json'):
assert digest(trial / payload['path']) == payload['sha256']
for filename, sha in read(trial / 'summary.json')['input_sha256'].items():
assert digest(ROOT / filename) == sha
shift = optional(EXP / 'audit/shift-gate.json')
shift_complete = False
if shift:
expected_shift = {(name, delay) for name in CFG['fixtures'] + [row['name'] for row in cal_records] for delay in CFG['delays_steps']}
assert len(shift['records']) == len(expected_shift) == 136
assert {(row['case'], row['delay_steps']) for row in shift['records']} == expected_shift
assert all(row['pass_all'] for row in shift['records'])
shift_complete = True
from shift_audit import transport
import numpy as np
for row in shift['records']:
source = AG / 'gpu' / (row['case'] + '-r0') / 'apl-inhibition.float32.npy'
assert digest(source) == row['input_sha256']
_, _, actual = transport(np.load(source, mmap_mode='r', allow_pickle=False), row['delay_steps'])
assert all(row[key] == value for key, value in actual.items())
cal = optional(EXP / 'calibration/summary.json')
if cal is not None:
assert summarize(out_json=None) == cal, 'CAL scientific evidence changed'
expected = {arm + '/' + name for arm in arms for name in expected_cases}
expected_repeats = {f'{arm}/{name}-r{repeat}' for arm in arms for name in CFG['fixtures'] for repeat in range(1, CFG['fixture_repeats'])}
numerical = bool(origin_complete and origin['pass_all'] and shift_complete and shift['pass_all'] and
set(comparisons) == set(components) == expected and set(repeats) == expected_repeats and
set(controls) == set(expected_cases) and all(comparisons.values()) and all(components.values()) and
all(repeats.values()) and all(controls.values()))
gates = {name: optional(EXP / 'audit' / (name + '-gate.json')) for name in ('fixtures', 'probes', 'calibration')}
numerical = numerical and all(gates[name] and gates[name]['pass_all'] for name in ('fixtures', 'probes'))
cal_execution = bool(cal and cal['execution_gate_pass'] and gates['calibration'] and gates['calibration']['pass_all'])
treatment_pass = bool(cal and cal.get('arms', {}).get('delay18', {}).get('original_representation_gate_pass', False))
baseline_pass = bool(cal and cal.get('arms', {}).get('delay0', {}).get('original_representation_gate_pass', False))
frozen = read(EXP / 'regression/frozen-five.json')
after = integrity()
save(EXP / 'regression/old-integrity-after.json', after)
oldpass = bool(frozen['pass_all'] and all(row['pass'] for row in after.values()))
if not oldpass:
status = 'FAIL-FROZEN-INTEGRITY-OR-REGRESSION'
elif numerical and cal_execution:
status = 'PASS-FIXED-CAL-APL-TIMING-HYPOTHESIS' if treatment_pass else 'FAIL-FIXED-CAL-APL-TIMING-HYPOTHESIS'
elif (any(not passed for group in (comparisons, components, repeats, controls) for passed in group.values()) or
any(report is not None and not report['pass_all'] for report in (origin, shift, *gates.values())) or
cal is not None and not cal['execution_gate_pass']):
status = 'FAIL-NUMERICAL-OR-CONTROL-GATE'
else:
status = 'INCOMPLETE-NUMERICAL-OR-CAL-EXECUTION'
measurements = []
for path in sorted((EXP / 'gpu').glob('*/*/summary.json')):
summary = read(path)
measurements.append(dict(arm=path.parent.parent.name, run=path.parent.name,
**{key: summary[key] for key in ('case', 'total_spikes', 'accepted_requests', 'nodes', 'edges',
'diagnostic_simulation_seconds', 'peak_allocated_vram_bytes')}))
result = dict(status=status, numerical_gate_pass=bool(numerical), full_CAL_execution_gate_pass=cal_execution,
delay0_representation_pass=baseline_pass, delay18_representation_pass=treatment_pass,
origin_audit=origin, fixed_trace_shift=shift, gates=gates, comparisons=comparisons,
repeats=repeats, controls=controls, components=components, calibration=cal,
stopped=[dict(path=str(path.relative_to(EXP)), report=read(path)) for path in sorted((EXP / 'audit').glob('stopped-*.json'))],
old_frozen_integrity_and_five_reference_pass=oldpass, measurements=measurements,
unchanged_APL_parameters=CFG['apl'], explicit_hypothesis=CFG['hypothesis'],
delay_steps=CFG['delays_steps'], biological_delay_measured=False,
APL_nonKC_fast_edges_unreplaced=1212, shared_APL_arithmetic_oracle_limitation=True,
whole_brain_equal_inhibition_integral_claim=False, universal_equivalence_claim=False,
heldout_generalization_tested=False, learning_achieved=False, candidate_generated=False,
payload_bytes_before_manifest=budget(),
next_gate='Independent seeds and novel-odor representation; then credit specificity and functional learning' if numerical and cal_execution and treatment_pass
else 'Report the first failed gate for this fixed hypothesis; no tuning or conditioning promotion')
save(EXP / 'audit/final-status.json', result)
with (EXP / 'REPORT.md').open('x', encoding='utf-8') as stream:
stream.write('\n'.join(['# Phase 4AJ — fixed graded-APL timing branch', '', f'Status: `{status}`.', '',
f'Numerical gate: {numerical}. Complete both CAL arms: {cal_execution}.',
f'Original representation gate: delay0={baseline_pass}; delay18={treatment_pass}.',
'Only generated h application time changes: zero versus 18 steps. No brain gain, input, learning or mask change.',
'This is an engineering scheduling hypothesis, not measured MaleCNS APL delay. It delays PN- and KC-derived h together.',
'Finite stored generated samples are preserved in applied output or pending tail; the brain is not advanced to flush the tail.',
'Closed-loop generated h and accepted requests can differ between arms; equal total inhibition is not assumed.',
'Independent Brian LIF and candidate share legacy Torch APL arithmetic; component and separate delay-queue audits qualify this limit.',
'Original fast APL mask still leaves 1,212 non-KC edges unreplaced. No silent scope change.',
'The origin audit identifies timed real-edge antecedents, not unique or necessary biological causes.',
f'Frozen checksums and five reference trials: {oldpass}.',
'Costs include diagnostic synchronization, IPC, hashes and tracing; these are not production runtime benchmarks.',
'No heldout trajectories, conditioning, learned model, reward interface, game, decoder or candidate.',
'Next gate: ' + result['next_gate'], '']))
check(); seal(MODEL); seal(EXP)
print(status, flush=True)
if __name__ == '__main__':
main()
models/male-cns-lif/apl-timing-lab-v1/gpu_timing.py
+453
View File
@@ -0,0 +1,453 @@
"""Phase 4AJ isolated fixed-delay graded-APL timing branch.
Copied from sealed AG-r2 gpu_apl.py; only dynamics change is a 0/18-step
float32 output delay before the unchanged unconditional end-stage g write.
The sealed LegacyAPL and AE CUDA operators are loaded explicitly, unmodified.
Generated h is still logged separately. No extra brain/tail ticks are run.
This is engineering schedule parity, not a measured biological delay claim.
"""
from functools import lru_cache
import csv
import difflib
import importlib.util
import hashlib
import json
import math
from pathlib import Path
import time
import traceback
import sys
import numpy as np
import torch
from timing_queue import audit_sequence
SOURCE_ROOT = Path(__file__).resolve().parent
AE_MODEL = SOURCE_ROOT.parent / "author-runtime-lab-v1"
AF_MODEL = SOURCE_ROOT.parent / "author-fullgraph-lab-v1"
AG_MODEL = SOURCE_ROOT.parent / "author-apl-lab-v2"
N_NEURONS = 166700
SEALED_SHA256 = {
"ag_gpu": "69f5d893f47dafff4ef7c3186c3df0b9014666b32c8f4e943b57016bec92b18a",
"ag_worker": "21ceec65fe42581dc55872010e0025d660e80358905f1cf4df1dadf341a3b291",
"ae_backend": "03758d07572f4b44d272059d9308fac534268ad201b57bdd77d69fee95e67278",
"af_queue": "d72003d98f0f1624b12126eff50bd361c43d888f4a64d90ba9247a58976f1961",
}
COEFFICIENT_NAMES = ("gd", "c0", "cg", "cv", "threshold", "reset", "seed_volt")
REQUEST_FIELDS = ("tick", "neuron_idx", "attempted", "accepted", "rejected")
def require(condition, message):
if not condition:
raise ValueError(message)
def write_json(path, value):
with Path(path).open("x", encoding="utf-8") as output:
json.dump(value, output, indent=2, allow_nan=False)
output.write("\n")
def raw_hash(value, dtype):
return hashlib.sha256(np.ascontiguousarray(value, dtype=dtype).tobytes()).hexdigest()
def file_sha256(path):
digest = hashlib.sha256()
with Path(path).open("rb") as source:
for block in iter(lambda: source.read(8 << 20), b""):
digest.update(block)
return digest.hexdigest()
def load_source(path, name, expected_hash):
"""Load a sealed source under a unique name; never patch its globals."""
path = Path(path)
require(file_sha256(path) == expected_hash, "Sealed source changed: " + str(path))
spec = importlib.util.spec_from_file_location(name, path)
require(spec is not None and spec.loader is not None, "Cannot load sealed module")
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
try:
spec.loader.exec_module(module)
except BaseException:
sys.modules.pop(name, None)
raise
return module
@lru_cache(maxsize=1)
def sealed_apl():
return load_source(AG_MODEL / "legacy_apl_worker.py", "phase4aj_sealed_ag_worker",
SEALED_SHA256["ag_worker"])
def source_audit(out_dir):
sources = {"ag_gpu": AG_MODEL / "gpu_apl.py", "ag_worker": AG_MODEL / "legacy_apl_worker.py",
"ae_backend": AE_MODEL / "gpu_backend.py", "af_queue": AF_MODEL / "oracle_full.py"}
actual = {key: file_sha256(path) for key, path in sources.items()}
require(actual == SEALED_SHA256, "Sealed implementation hash mismatch")
diff_path = out_dir / "sealed-gpu-source.diff"
with diff_path.open("x", encoding="utf-8", newline="\n") as output:
output.writelines(difflib.unified_diff(
sources["ag_gpu"].read_text(encoding="utf-8").splitlines(keepends=True),
Path(__file__).read_text(encoding="utf-8").splitlines(keepends=True),
fromfile="sealed-author-apl-lab-v2/gpu_apl.py",
tofile="new-apl-timing-lab-v1/gpu_timing.py"))
audit = {"sealed_source_sha256": actual, "candidate_sha256": file_sha256(__file__),
"source_diff_sha256": file_sha256(diff_path),
"module_globals_patched": False,
"only_dynamics_change": "fixed zero-initialized 0/18-step h32 delay before unchanged end g write",
"diagnostic_additions": ["generated/applied/pending trace split", "fixed-sequence bitwise audit",
"source provenance and final APL state disclosure"]}
write_json(out_dir / "source-audit.json", audit)
return audit
@lru_cache(maxsize=1)
def sealed_backend():
return load_source(AE_MODEL / "gpu_backend.py", "phase4aj_sealed_ae_backend", SEALED_SHA256["ae_backend"])
@lru_cache(maxsize=1)
def queue_function():
return load_source(AF_MODEL / "oracle_full.py", "phase4aj_sealed_af_queue", SEALED_SHA256["af_queue"]).queue_summary
@lru_cache(maxsize=4)
def load_graph(graph_directory):
root = Path(graph_directory)
require(root.is_absolute(), "graph_dir must be absolute")
graph = {}
for name, dtype in (("nodes", np.int32), ("edge_id", np.int64), ("src", np.int32),
("dst", np.int32), ("signed_raw", np.int64), ("ptr", np.int64),
("weights", np.float64)):
value = np.load(root / (name + ".npy"), mmap_mode="r", allow_pickle=False)
require(value.dtype == dtype and value.ndim == 1, "Graph dtype/rank mismatch: " + name)
graph[name] = value
nodes, edges = graph["nodes"], graph["edge_id"]
n, e = len(nodes), len(edges)
require(0 < n <= N_NEURONS and bool(np.all((nodes >= 0) & (nodes < N_NEURONS))) and
bool(np.all(np.diff(nodes) > 0)), "Global node IDs must be sorted unique")
require(n < 512 or (n == N_NEURONS and np.array_equal(nodes, np.arange(N_NEURONS, dtype=np.int32))),
"Only small fixtures or the intact frozen full graph are permitted")
require(n != N_NEURONS or e == 25582938, "Full graph edge count changed")
for name in ("src", "dst", "signed_raw", "weights"):
require(graph[name].shape == (e,), "Graph edge array length mismatch: " + name)
ptr = graph["ptr"]
require(ptr.shape == (n + 1,) and ptr[0] == 0 and ptr[-1] == e and
bool(np.all(np.diff(ptr) >= 0)), "Invalid graph CSR")
require(np.array_equal(graph["src"], np.repeat(np.arange(n, dtype=np.int32), np.diff(ptr))),
"Source order does not match CSR")
# Global edge IDs need not equal local row indices in a fixture.
for start in range(0, e, 1 << 20):
stop = min(start + (1 << 20), e)
require(bool(np.all((graph["dst"][start:stop] >= 0) & (graph["dst"][start:stop] < n))),
"Local destination out of range")
require(bool(np.isfinite(graph["weights"][start:stop]).all()), "Nonfinite base SI weights")
require(np.array_equal(np.sign(graph["weights"][start:stop]), np.sign(graph["signed_raw"][start:stop])),
"Base weight sign/zero pattern differs from frozen raw contacts")
lo = max(0, start - 1)
require(bool(np.all(edges[start:stop] >= 0)) and bool(np.all(np.diff(edges[lo:stop]) > 0)),
"Global edge IDs must retain canonical increasing source/edge order")
return graph
def load_inputs(case_dir):
spec = json.loads((case_dir / "spec.json").read_text(encoding="utf-8"))
for name in ("name", "steps", "dt_ms", "duration_ms", "model", "graph_dir", "apl",
"apl_bundle_dir", "coefficients_path", "record_history", "apl_delay_steps"):
require(name in spec, "Missing case field: " + name)
require(type(spec["apl_delay_steps"]) is int and spec["apl_delay_steps"] in (0, 18),
"apl_delay_steps must be integer 0 or 18")
steps = spec["steps"]
require(type(steps) is int and 0 < steps <= 1000 and float(spec["dt_ms"]) == .1 and
float(spec["duration_ms"]) == steps * .1, "Unexpected steps/dt/duration")
require(type(spec["record_history"]) is bool, "record_history must be Boolean")
require(spec["apl"] == {"tau_ms": 20, "gain": .01, "normalization": 1}, "Frozen APL parameters changed")
for name in ("graph_dir", "apl_bundle_dir", "coefficients_path"):
require(Path(spec[name]).is_absolute(), "Input path must be absolute: " + name)
model = spec["model"]
for name in ("resting_potential_mv", "threshold_mv", "reset_potential_mv", "synaptic_delay_ms",
"refractory_ms", "w_syn_mv_per_anatomical_synapse"):
require(name in model and math.isfinite(float(model[name])), "Invalid model field: " + name)
require(float(model["synaptic_delay_ms"]) == 1.8 and float(model["refractory_ms"]) == 2.2 and
float(model["w_syn_mv_per_anatomical_synapse"]) == .275, "Frozen delay/ref/W_syn changed")
coeff = json.loads(Path(spec["coefficients_path"]).read_text(encoding="utf-8"))
for name in COEFFICIENT_NAMES:
require(name in coeff and math.isfinite(float(coeff[name])), "Invalid author coefficient: " + name)
require(coeff["threshold"] == float(model["threshold_mv"]) * .001 and
coeff["reset"] == float(model["reset_potential_mv"]) * .001, "Coefficient threshold/reset mismatch")
graph = load_graph(str(Path(spec["graph_dir"]).resolve()))
n = len(graph["nodes"])
require(spec["record_history"] == (n < 512), "Fixtures need dense history; full graph must not duplicate dense histories")
with np.load(case_dir / "requests.npz", allow_pickle=False) as source:
require(set(source.files) == {"tick", "neuron_idx"}, "Request sparse schema changed")
ticks, requested = source["tick"].copy(), source["neuron_idx"].copy()
require(ticks.dtype == np.int32 and requested.dtype == np.int32 and ticks.ndim == 1 and
ticks.shape == requested.shape, "Invalid sparse request dtype/shape")
require(bool(np.all((ticks >= 0) & (ticks < steps))), "Request tick outside trial")
if len(ticks) > 1:
require(bool(np.all((ticks[1:] > ticks[:-1]) |
((ticks[1:] == ticks[:-1]) & (requested[1:] > requested[:-1])))),
"Requests must be sorted unique by tick/global neuron")
local = np.searchsorted(graph["nodes"], requested)
require(bool(np.all(local < n)) and np.array_equal(graph["nodes"][local], requested),
"Requested neuron absent from graph")
initial = {"initial_v_mv": np.full(n, float(model["resting_potential_mv"]), dtype=np.float64),
"initial_g_mv": np.zeros(n, dtype=np.float64)}
initial_path = case_dir / "arrays.npz"
if initial_path.exists():
with np.load(initial_path, allow_pickle=False) as source:
require(set(source.files).issubset(initial), "Unexpected initial-state array")
for name in source.files:
value = source[name]
require(value.dtype == np.float64 and value.shape == (n,) and bool(np.isfinite(value).all()),
"Invalid initial state: " + name)
initial[name] = value.copy()
return spec, graph, coeff, ticks, requested, local, initial
def new_array(path, dtype, shape):
require(not Path(path).exists(), "Refusing existing output: " + str(path))
return np.lib.format.open_memmap(path, mode="w+", dtype=dtype, shape=shape)
def run_gpu(case_dir, out_dir, build_root):
"""Run fresh candidate state; never consume oracle/candidate APL traces.
pass_all denotes internal recording/finite integrity only. Independent
comparison and the original complete representation gate are separate.
"""
case_dir, out_dir = Path(case_dir), Path(out_dir)
require(not out_dir.exists(), "Refusing existing output directory: " + str(out_dir))
out_dir.mkdir(parents=True, exist_ok=False)
start, tick, completed = time.perf_counter(), -1, 0
maps = {}
try:
source_provenance = source_audit(out_dir)
spec, graph, coeff, request_ticks, request_nodes, request_local, initial = load_inputs(case_dir)
require(torch.cuda.is_available() and torch.cuda.device_count() == 1 and
"4090" in torch.cuda.get_device_name(), "Only the declared single visible RTX4090 is permitted")
ext, build = sealed_backend().build_extension(build_root)
build_done = time.perf_counter()
apl = sealed_apl().LegacyAPL(spec["apl_bundle_dir"], spec["dt_ms"])
write_json(out_dir / "apl-component.json", apl.metadata)
device = apl.device
torch.cuda.reset_peak_memory_stats(device)
n, e, steps = len(graph["nodes"]), len(graph["edge_id"]), spec["steps"]
def upload(value, dtype):
return torch.from_numpy(np.array(value, copy=True, order="C")).to(device=device, dtype=dtype).contiguous()
indptr, dst = upload(graph["ptr"], torch.int64), upload(graph["dst"], torch.int32)
global_nodes_gpu = upload(graph["nodes"], torch.int64)
base_weights = upload(graph["weights"], torch.float64)
weights = base_weights.clone()
del base_weights
local_apl = np.flatnonzero(np.isin(graph["nodes"], apl.apl_globals)).astype(np.int64)
apl_out_mask = np.isin(graph["src"], local_apl)
if bool(np.any(apl_out_mask)):
weights[upload(apl_out_mask, torch.bool)] = 0.0
effective_hash = hashlib.sha256()
for offset in range(0, e, 1 << 20):
effective_hash.update(weights[offset:offset + (1 << 20)].cpu().numpy().astype("<f8", copy=False).tobytes())
common_kc, local_kc, full_kc = np.intersect1d(graph["nodes"], apl.kc_globals, assume_unique=True, return_indices=True)
kc_local_gpu, kc_full_gpu = upload(local_kc, torch.int64), upload(full_kc, torch.int64)
v = upload(initial["initial_v_mv"] * .001, torch.float64)
g = upload(initial["initial_g_mv"] * .001, torch.float64)
ref = torch.zeros(n, device=device, dtype=torch.int32)
counts = torch.zeros(n, device=device, dtype=torch.int32)
requests = torch.zeros(n, device=device, dtype=torch.uint8)
ring = torch.zeros((19, n), device=device, dtype=torch.uint8)
apl_delay = spec["apl_delay_steps"]
apl_ring = torch.zeros((apl_delay, 4064), device=device, dtype=torch.float32)
maps["apl_applied"] = new_array(out_dir / "apl-applied.float32.npy", np.float32, (steps, 4064))
for name, width in (("drive", 2), ("state", 2), ("inhibition", 4064)):
maps["apl_" + name] = new_array(out_dir / ("apl-" + name + ".float32.npy"), np.float32, (steps, width))
if spec["record_history"]:
for name, dtype in (("v", np.float64), ("g", np.float64), ("ref", np.int32)):
maps[name] = new_array(out_dir / (name + "-history." + np.dtype(dtype).name + ".npy"), dtype, (steps, n))
maps["pre_" + name] = new_array(out_dir / ("preAPL-" + name + "." + np.dtype(dtype).name + ".npy"), dtype, (steps, n))
event_steps, event_nodes = [], []
accepted_requests_total, due_total, accepted_due_total = 0, 0, 0
queue_summary = queue_function()
simulation_start = time.perf_counter()
with (out_dir / "queue.jsonl").open("x", encoding="utf-8", buffering=1) as queue_output, \
(out_dir / "tick-state-hashes.jsonl").open("x", encoding="utf-8", buffering=1) as hash_output, \
(out_dir / "requests.csv").open("x", encoding="utf-8", newline="", buffering=1) as request_output:
writer = csv.DictWriter(request_output, fieldnames=REQUEST_FIELDS)
writer.writeheader()
for tick in range(steps):
requests.zero_()
begin, end = np.searchsorted(request_ticks, (tick, tick + 1))
if end > begin:
requested_local_gpu = upload(request_local[begin:end], torch.int64)
requests[requested_local_gpu] = 1
accepted = (ref[requested_local_gpu] <= 1).to(torch.uint8).cpu().numpy()
accepted_requests_total += int(accepted.sum())
writer.writerows({"tick": tick, "neuron_idx": int(node), "attempted": 1,
"accepted": int(ok), "rejected": 1 - int(ok)}
for node, ok in zip(request_nodes[begin:end], accepted))
spikes = ring[tick % 19]
ext.state_step(v, g, ref, requests, spikes, counts,
*[float(coeff[name]) for name in COEFFICIENT_NAMES], 22)
expanded_global = np.empty(0, dtype=np.int64)
accepted_cpu = np.empty(0, dtype=np.uint8)
if tick >= 18:
due = ring[(tick - 18) % 19]
sources = torch.nonzero(due, as_tuple=False).flatten().contiguous()
if sources.numel():
degrees = indptr[sources + 1] - indptr[sources]
offsets = (torch.cumsum(degrees, dim=0) - degrees).contiguous()
edge_count = int(degrees.sum().item())
expanded = torch.empty(edge_count, device=device, dtype=torch.int64)
ext.fill_edges(indptr, sources, offsets, expanded)
if edge_count:
destination = dst[expanded]
# Actual active fanout order, BEFORE stable target grouping.
expanded_global = graph["edge_id"][expanded.cpu().numpy()].copy()
accepted_cpu = (ref[destination.long()] == 0).to(torch.uint8).cpu().numpy().copy()
order = torch.argsort(destination, stable=True)
ordered_edges, ordered_targets = expanded[order].contiguous(), destination[order]
boundary = torch.ones(edge_count, device=device, dtype=torch.bool)
boundary[1:] = ordered_targets[1:] != ordered_targets[:-1]
starts = torch.nonzero(boundary, as_tuple=False).flatten().contiguous()
targets = ordered_targets[starts].contiguous()
ext.apply_arrivals(ordered_edges, starts, targets, weights, g, ref)
queue = queue_summary(tick, expanded_global, accepted_cpu)
queue_output.write(json.dumps(queue, allow_nan=False) + "\n")
due_total += queue["count"]
accepted_due_total += queue["accepted_count"]
if spec["record_history"]:
for name, value in (("v", v), ("g", g), ("ref", ref)):
maps["pre_" + name][tick] = value.cpu().numpy()
if n == N_NEURONS:
full_spikes = spikes
else:
apl.spikes.zero_()
apl.spikes[global_nodes_gpu] = spikes
full_spikes = apl.spikes
drive, apl_state, h = apl.advance(full_spikes)
if apl_delay == 0:
applied_h = h # Exact legacy identity; no extra arithmetic.
else:
slot = tick % apl_delay
applied_h = apl_ring[slot].clone()
apl_ring[slot].copy_(h)
# Mixed precision contract: widen f32 h EXACTLY, THEN multiply
# in f64, THEN subtract in f64. Do not guard refractory KCs.
delta_si = applied_h[kc_full_gpu].to(torch.float64) * 0.001
g[kc_local_gpu] = g[kc_local_gpu] - delta_si
apl_cpu = {"drive": drive.cpu().numpy().copy(), "state": apl_state.cpu().numpy().copy(),
"inhibition": h.cpu().numpy().copy(), "applied": applied_h.cpu().numpy().copy()}
for name, value in apl_cpu.items():
maps["apl_" + name][tick] = value
actual = {"v": v.cpu().numpy().copy(), "g": g.cpu().numpy().copy(),
"ref": ref.cpu().numpy().copy()}
finite = all(bool(np.isfinite(value).all()) for value in list(actual.values()) + list(apl_cpu.values()))
row = {"tick": tick, **{name + "_sha256": raw_hash(value, "<i4" if name == "ref" else "<f8")
for name, value in actual.items()}}
hash_output.write(json.dumps(row, allow_nan=False) + "\n")
if spec["record_history"]:
for name, value in actual.items():
maps[name][tick] = value
fired_local = torch.nonzero(spikes, as_tuple=False).flatten().cpu().numpy()
if len(fired_local):
event_steps.append(np.full(len(fired_local), tick, dtype=np.int32))
event_nodes.append(graph["nodes"][fired_local].copy())
completed += 1
if not finite:
with (out_dir / "first-failure-state.npz").open("xb") as target:
np.savez_compressed(target, tick=np.array(tick, dtype=np.int32), **actual,
**{"apl_" + name: value for name, value in apl_cpu.items()})
require(False, "Nonfinite state; first failure saved")
torch.cuda.synchronize(device)
def joined(values):
return np.concatenate(values).astype(np.int32, copy=False) if values else np.empty(0, dtype=np.int32)
state = {"step": joined(event_steps), "neuron_idx": joined(event_nodes), "counts": counts.cpu().numpy(),
"v": v.cpu().numpy(), "g": g.cpu().numpy(), "ref": ref.cpu().numpy()}
with (out_dir / "state.npz").open("xb") as target:
np.savez_compressed(target, **state)
event_locals = np.searchsorted(graph["nodes"], state["neuron_idx"])
counts_exact = np.array_equal(np.bincount(event_locals, minlength=n), state["counts"])
require(counts_exact and completed == steps, "Sparse events/counts/completed ticks mismatch")
for value in maps.values():
value.flush()
# Save the queue state, not an invented future APL/brain decay trace.
if apl_delay:
head = steps % apl_delay
ring_cpu = apl_ring.cpu().numpy()
pending = np.concatenate((ring_cpu[head:], ring_cpu[:head]), axis=0)
else:
pending = np.empty((0, 4064), dtype=np.float32)
with (out_dir / "apl-pending-tail.float32.npy").open("xb") as target:
np.save(target, pending, allow_pickle=False)
timing = audit_sequence(maps["apl_inhibition"], maps["apl_applied"], pending, apl_delay)
final_apl_state = apl.state.cpu().numpy().copy()
timing.update({
"generated_rows": steps,
"applied_generated_rows": max(steps - apl_delay, 0),
"pending_shape": [apl_delay, 4064],
"pending_application_tick_start": steps,
"pending_application_tick_stop_exclusive": steps + apl_delay,
"pending_generated_tick_start": max(0, steps - apl_delay),
"pending_generated_tick_stop_exclusive": steps,
"generated_trace": "apl-inhibition.float32.npy",
"applied_trace": "apl-applied.float32.npy",
"pending_trace": "apl-pending-tail.float32.npy",
"pending_raw_sha256": raw_hash(pending, "<f4"),
"final_apl_state_float32": final_apl_state.tolist(),
"final_apl_state_u32_bits": final_apl_state.view(np.uint32).tolist(),
"final_apl_state_nonzero": bool(np.any(final_apl_state != 0)),
"pending_queue_drained": apl_delay == 0,
"finite_generated_sequence_only": True,
"future_apl_decay_or_feedback_generated": False,
"closed_loop_branch_integrals_claimed_equal": False,
"biological_delay_measurement_claimed": False,
})
write_json(out_dir / "apl-timing-audit.json", timing)
summary = {"case": spec["name"], "complete": True, "pass_all": True,
"pass_all_scope": "internal finite/recording/event-count integrity only; independent equivalence not claimed",
"steps": steps, "completed_ticks": completed, "nodes": n, "edges": e, "precision": "float64",
"apl_precision": "float32", "state_units": "SI volts", "total_spikes": len(state["step"]),
"requested": len(request_nodes), "accepted_requests": accepted_requests_total,
"rejected_requests": len(request_nodes) - accepted_requests_total,
"arriving_edges": due_total, "accepted_arriving_edges": accepted_due_total,
"counts_match_sparse_events": bool(counts_exact), "all_finite": True,
"record_history": spec["record_history"], "apl_fast_edges_zeroed": int(apl_out_mask.sum()),
"kc_targets_present": len(common_kc), "effective_weights_raw_sha256": effective_hash.hexdigest(),
"apl_component": apl.metadata, "apl_schedule": "end, after threshold/reset and guarded arrivals, unconditional all KC",
"apl_si_conversion": "applied h32 -> float64 -> multiply float64 0.001 -> separate float64 subtraction",
"apl_delay_steps": apl_delay, "apl_timing": timing, "source_provenance": source_provenance,
"pending_generated_count": min(steps, apl_delay),
"pending_initial_zero_prefix": max(apl_delay - steps, 0),
"pending_tail_shape": [apl_delay, 4064], "future_generated_decay_tail_included": False,
"build": build, "sealed_ae_backend_sha256": file_sha256(AE_MODEL / "gpu_backend.py"),
"shared_apl_component_oracle_limitation": True, "oracle_validation_claim": False,
"learning": False, "performance_runtime_claim": False, "cpu_sync_each_tick": True,
"build_or_cache_seconds": build_done - start,
"diagnostic_simulation_seconds": time.perf_counter() - simulation_start,
"wall_seconds": time.perf_counter() - start,
"peak_allocated_vram_bytes": int(torch.cuda.max_memory_allocated(device)),
"peak_reserved_vram_bytes": int(torch.cuda.max_memory_reserved(device)),
"input_sha256": {"spec.json": file_sha256(case_dir / "spec.json"),
"requests.npz": file_sha256(case_dir / "requests.npz"),
"base_weights.npy": file_sha256(Path(spec["graph_dir"]) / "weights.npy"),
"coefficients.json": file_sha256(spec["coefficients_path"])}}
if (case_dir / "arrays.npz").exists():
summary["input_sha256"]["arrays.npz"] = file_sha256(case_dir / "arrays.npz")
write_json(out_dir / "summary.json", summary)
return summary
except Exception as error:
for value in maps.values():
value.flush()
write_json(out_dir / "error.json", {"complete": False, "pass_all": False,
"tick_in_progress": tick, "completed_ticks": completed,
"type": type(error).__name__, "message": str(error), "traceback": traceback.format_exc()})
raise
models/male-cns-lif/apl-timing-lab-v1/manifest.json
+110
View File
@@ -0,0 +1,110 @@
{
"root": "/data/digital-drosophila/models/male-cns-lif/apl-timing-lab-v1",
"files": [
{
"path": "PROTOCOL.md",
"bytes": 4272,
"sha256": "d96aef0107f7b0791d16e475b749da418b8a38d2c56a94e0779de0cd6581a7de"
},
{
"path": "al_origin_audit.py",
"bytes": 33399,
"sha256": "7201871477a29ede31c7d6c6bf1c92700da3db5505e67ccfa7cb43126b8464b9"
},
{
"path": "calibration.py",
"bytes": 22786,
"sha256": "fb15ddeaa108305c8de697bbae36b23ab0414a7c58db51c66dce9feb4a048a0d"
},
{
"path": "compare_timing.py",
"bytes": 41221,
"sha256": "9010f070fba9b7c9cf2d86187ee2abc57f3ac606ebc08a42baad4f3922b51734"
},
{
"path": "config.json",
"bytes": 2149,
"sha256": "0b720e23aa7988b2b443e14951082b1ce9b061a183b7f858c0e0f3c0fc6fb25e"
},
{
"path": "context.py",
"bytes": 3436,
"sha256": "06d4994af3729d653263391ff3a3604f7c919e78562604e5c86acfa292d60ba0"
},
{
"path": "finalize.py",
"bytes": 11254,
"sha256": "0a2dbf9e5f889e9dd3491d036e6f92c17417a84482c5e1e49134c0993f7d68f2"
},
{
"path": "gpu_timing.py",
"bytes": 27083,
"sha256": "af1975ba5e5e700208c0a598d3cbe648a3cf10885c6231239b6e4b28f76d173b"
},
{
"path": "oracle_timing.py",
"bytes": 55279,
"sha256": "39d8c47c94bc2faddf016864a43e7715907fb987f56f7b27098060a88851f6be"
},
{
"path": "prepare.py",
"bytes": 5970,
"sha256": "bba71bcd52dcb78d9ae0fa93a8a43173a4669ecef038dce18ce7d82edd9bce50"
},
{
"path": "run_case.py",
"bytes": 2855,
"sha256": "d93e4fe2df542ce5a0f0adb5aceada583dc8c921f0159f426e50428d79d67a1d"
},
{
"path": "run_suite.py",
"bytes": 6161,
"sha256": "6f26c626acf9ef8e6fae7902af0cbf0bd176eb322529ca8aa203a9700716c889"
},
{
"path": "shift_audit.py",
"bytes": 2114,
"sha256": "46f380dac340acc91f226968bb493475a31777f8290fb20f9145a97c9009d845"
},
{
"path": "test_al_origin_audit.py",
"bytes": 9998,
"sha256": "4b75a39d36090b0a66cc7ad4b370ac785713fb58c366df2efcc55152d612fc2b"
},
{
"path": "test_calibration.py",
"bytes": 14734,
"sha256": "307a2848bf6d05aaa8105bb3558be2a74425894f96a4a2bd20af8b82cb8b6e25"
},
{
"path": "test_protocol.py",
"bytes": 10402,
"sha256": "cbae93c993ac9aa99bcce68ae2131c7f938f8a63e89779f8d3d79e36bf268dee"
},
{
"path": "tests/__init__.py",
"bytes": 76,
"sha256": "bfc0d9e17410c2c52ea1d9a5259c92964f16b118dbae8cecad258697b67e4bdc"
},
{
"path": "tests/test_compare_timing.py",
"bytes": 9226,
"sha256": "71876b36eef88c198aa8e833af547dbe772001c71b010707b656c165c8b54031"
},
{
"path": "tests/test_timing_queue.py",
"bytes": 5825,
"sha256": "4a375444dc732c3538df8aa4b81e43b5418754fcebebb0d02c16faad03d43195"
},
{
"path": "timing_queue.py",
"bytes": 4443,
"sha256": "54df6d4818e2a0c26e96fb0aaf5d9b71cd964981706da937562d48af79aca14e"
},
{
"path": "verify_frozen.py",
"bytes": 2725,
"sha256": "97d8a240b08baaca935f9c0278978883005dcd6fdbb4ca0d948074ef79fd349c"
}
]
}
models/male-cns-lif/apl-timing-lab-v1/oracle_timing.py
+890
View File
@@ -0,0 +1,890 @@
"""Phase 4AJ graded-APL output-timing adapter of the sealed Phase 4AG oracle.
Brian has no Torch dependency. A separate, pinned science-Python process
executes the unchanged legacy float32 APL operations from Brian's own spikes.
This is independent LIF/scheduling validation with a shared APL numerical
component, NOT an all-components-independent CPU oracle. Only generated h
transport is changed: h[t-D] is written at end(t), D in {0,18}. The original
Brian equations, anatomical delay, refractory policy and queue proxy stay as
in the sealed source. No future brain or APL generation ticks are run.
"""
from __future__ import annotations
import csv
import hashlib
import importlib.metadata
import json
import math
import os
import platform
import queue
import struct
import subprocess
import sys
import threading
import time
import types
from contextlib import ExitStack
from pathlib import Path
import numpy as np
BLOCK_BYTES = 8 << 20
BASE_ORACLE_PATH = Path("/data/digital-drosophila/models/male-cns-lif/author-apl-lab-v2/oracle_apl.py")
BASE_ORACLE_SHA256 = "357a7d3debb5abc34a8714c0233b456547a154d9d5855d28661b4f01019a80ea"
LEGACY_WORKER_PATH = Path("/data/digital-drosophila/models/male-cns-lif/author-apl-lab-v2/legacy_apl_worker.py")
LEGACY_WORKER_SHA256 = "21ceec65fe42581dc55872010e0025d660e80358905f1cf4df1dadf341a3b291"
PROTOCOL = "phase4ag-apl-v1"
AUTHOR_PATH = Path("/data/digital-drosophila/experiments/phase4ad-author-semantics/external-resources/model.py")
AUTHOR_SHA256 = "fc45837d7122c6ce2a7f3f2f23c515992e4b232aadb919efabb72337fac88e4e"
AE_PATH = Path("/data/digital-drosophila/models/male-cns-lif/author-runtime-lab-v1/oracle_stress.py")
AE_SHA256 = "32eb1c42383068ce52701d2139a376ce155ad7072cde019949f8ca2d55492409"
AF_PATH = Path("/data/digital-drosophila/models/male-cns-lif/author-fullgraph-lab-v1/oracle_full.py")
AF_SHA256 = "d72003d98f0f1624b12126eff50bd361c43d888f4a64d90ba9247a58976f1961"
EVENT_DTYPE = np.dtype([("step", "<i4"), ("neuron_idx", "<i4")])
BUNDLE_FILES = ("input_matrix.npy", "output_matrix.npy", "kc_globals.npy", "apl_globals.npy")
_PROCESS_USED = False
def _check(condition, message):
if not condition:
raise ValueError(message)
def _save(path, value):
with Path(path).open("x", encoding="utf-8") as handle:
json.dump(value, handle, indent=2, sort_keys=True, allow_nan=False)
handle.write("\n")
def _digest(path):
result = hashlib.sha256()
with Path(path).open("rb") as handle:
for block in iter(lambda: handle.read(BLOCK_BYTES), b""):
result.update(block)
return result.hexdigest()
def _array_hash(array, dtype):
return hashlib.sha256(np.ascontiguousarray(array, dtype=dtype).tobytes()).hexdigest()
def _load_checked(path, expected, name):
data = Path(path).read_bytes()
_check(hashlib.sha256(data).hexdigest() == expected, f"Pinned source hash mismatch: {path}")
module = types.ModuleType(name)
module.__file__ = str(path)
exec(compile(data, str(path), "exec"), module.__dict__)
return module
def _python_entry(path):
"""A venv is selected by its invocation path, not the resolved ELF target."""
entry = Path(path).absolute()
_check(entry.is_file(), f"Missing declared Python entry: {entry}")
return entry
def _new_memmap(path, dtype, shape):
_check(not Path(path).exists(), f"Refusing to overwrite output: {path}")
return np.lib.format.open_memmap(path, mode="w+", dtype=dtype, shape=shape)
def _budget(root, limit, reserve=0):
total = 0
for directory, _, filenames in os.walk(root, followlinks=False):
for filename in filenames:
total += (Path(directory) / filename).stat(follow_symlinks=False).st_size
_check(total + int(reserve) <= limit,
f"Output budget exceeded: {total}+{reserve}>{limit}; inputs and duration were not adjusted")
return total
class _APLDelayFIFO:
"""Independent list transport; no Brian/Torch import, arithmetic, or feedback.
Start with D distinct positive-zero vectors. On tick t, append a copy of
generated h[t], then pop the oldest vector. The remaining D vectors are
pending applications at ticks T..T+D-1 after T steps, NOT new future output.
"""
def __init__(self, delay_steps):
_check(type(delay_steps) is int and delay_steps in (0, 18),
"apl_delay_steps must be an explicit integer 0 or 18")
self.delay_steps = delay_steps
self.tick = 0
self.finished = False
self.queue = [np.zeros(4064, dtype=np.float32) for _ in range(delay_steps)]
def step(self, tick, generated):
_check(not self.finished and type(tick) is int and tick == self.tick,
"APL FIFO ticks must be consecutive and cannot follow finish")
value = np.asarray(generated)
_check(value.dtype == np.dtype("float32") and value.shape == (4064,) and
np.isfinite(value).all() and np.all(value >= 0),
"APL FIFO requires one finite nonnegative float32 full-KC vector")
self.queue.append(value.copy())
applied = self.queue.pop(0)
_check(len(self.queue) == self.delay_steps, "APL FIFO length changed")
self.tick += 1
return applied
def finish(self, steps):
_check(not self.finished and type(steps) is int and steps == self.tick,
"APL FIFO did not cover every requested tick")
self.finished = True
if not self.queue:
return np.empty((0, 4064), dtype=np.float32)
return np.stack(self.queue).astype(np.float32, copy=False)
class _PipeReader:
"""One bounded reader thread, with timeout/EOF checks for every frame."""
def __init__(self, pipe, timeout):
self.pipe, self.timeout = pipe, float(timeout)
self.chunks = queue.Queue(maxsize=8)
self.buffer = bytearray()
self.thread = threading.Thread(target=self._pump, daemon=True)
self.thread.start()
def _pump(self):
try:
while True:
chunk = os.read(self.pipe.fileno(), 65536)
self.chunks.put(chunk)
if not chunk:
return
except BaseException as error:
self.chunks.put(error)
def _fill(self, deadline):
remaining = deadline - time.monotonic()
if remaining <= 0:
raise TimeoutError("APL sidecar response timed out")
try:
chunk = self.chunks.get(timeout=remaining)
except queue.Empty as error:
raise TimeoutError("APL sidecar response timed out") from error
if isinstance(chunk, BaseException):
raise RuntimeError("APL sidecar stdout reader failed") from chunk
if not chunk:
raise EOFError("APL sidecar ended before a complete response")
self.buffer.extend(chunk)
def read(self, size):
deadline = time.monotonic() + self.timeout
while len(self.buffer) < size:
self._fill(deadline)
result = bytes(self.buffer[:size])
del self.buffer[:size]
return result
def line(self):
deadline = time.monotonic() + self.timeout
while b"\n" not in self.buffer:
_check(len(self.buffer) < 1 << 20, "APL sidecar handshake exceeds 1 MiB")
self._fill(deadline)
end = self.buffer.index(b"\n") + 1
result = bytes(self.buffer[:end])
del self.buffer[:end]
return result
class _APLWorker:
def __init__(self, python, script, bundle_dir, stderr, timeout=180):
self.command = [str(python), "-B", str(script), "--bundle-dir", str(bundle_dir), "--dt-ms", "0.1"]
self.process = subprocess.Popen(self.command, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=stderr, bufsize=0, shell=False)
self.reader = _PipeReader(self.process.stdout, timeout)
self.tick = 0
try:
self.hello = json.loads(self.reader.line().decode("utf-8"))
_check(self.hello.get("ok") is True, "APL sidecar did not acknowledge initialization")
_check(self.hello.get("protocol") == PROTOCOL, "Unexpected APL sidecar protocol")
for key, value in (("n_neurons", 166700), ("n_apl", 2), ("n_kc", 4064)):
_check(self.hello.get(key) == value, f"APL sidecar changed {key}")
_check(self.hello.get("dt_ms") == .1, "APL sidecar changed dt")
expected_decay = f"{int(np.float32(math.exp(-.1 / 20.)).view(np.uint32)):08x}"
expected_gain = f"{int(np.float32(.01).view(np.uint32)):08x}"
_check(self.hello.get("decay_f32_bits") == expected_decay and self.hello.get("gain_f32_bits") == expected_gain,
"APL sidecar decay/gain differs from the frozen float32 scalars")
_check(self.hello.get("files_sha256") == {name: _digest(bundle_dir / name) for name in BUNDLE_FILES},
"APL sidecar bundle hashes differ from the independently read bundle")
except BaseException:
self.abort()
raise
def step(self, tick, global_spikes):
_check(tick == self.tick, "APL sidecar request ticks must be consecutive")
nodes = np.asarray(global_spikes)
_check(nodes.ndim == 1 and nodes.dtype == np.dtype("int32"), "APL input spikes must be int32 globals")
_check(not len(nodes) or (nodes[0] >= 0 and nodes[-1] < 166700 and np.all(np.diff(nodes) > 0)),
"APL input spikes must be sorted, unique, and in the frozen global range")
packet = struct.pack("<qQ", int(tick), len(nodes)) + np.ascontiguousarray(nodes, dtype="<i4").tobytes()
view = memoryview(packet)
while view:
written = self.process.stdin.write(view)
if not written:
raise BrokenPipeError("APL sidecar stdin failed")
view = view[written:]
self.process.stdin.flush()
response_tick, byte_count = struct.unpack("<qQ", self.reader.read(16))
_check(response_tick == tick and byte_count == (2 + 2 + 4064) * 4,
"APL sidecar frame tick/length mismatch")
values = np.frombuffer(self.reader.read(byte_count), dtype="<f4")
_check(np.isfinite(values).all() and np.all(values >= 0), "APL sidecar returned invalid nonnegative f32 states")
self.tick += 1
return values[:2], values[2:4], values[4:]
def finish(self, steps):
_check(self.tick == steps, "APL sidecar did not process every tick")
self.process.stdin.close()
status = self.process.wait(timeout=30)
_check(status == 0, f"APL sidecar exited unsuccessfully: {status}")
self.reader.thread.join(timeout=5)
_check(not self.reader.thread.is_alive(), "APL stdout did not reach EOF")
extra = bytes(self.reader.buffer)
while not self.reader.chunks.empty():
chunk = self.reader.chunks.get_nowait()
_check(not isinstance(chunk, BaseException), "APL stdout reader failed after the last frame")
extra += chunk
_check(not extra, "APL sidecar emitted unrequested trailing stdout bytes")
self.process.stdout.close()
return status
def abort(self):
if self.process.poll() is None:
self.process.terminate()
try:
self.process.wait(timeout=5)
except subprocess.TimeoutExpired:
self.process.kill()
self.process.wait(timeout=5)
for handle in (self.process.stdin, self.process.stdout):
if handle is not None and not handle.closed:
handle.close()
def _graph_inputs(directory):
arrays = {}
definitions = {"nodes": "int32", "edge_id": "int64", "src": "int32",
"dst": "int32", "signed_raw": "int64", "ptr": "int64", "weights": "float64"}
for name, dtype in definitions.items():
value = np.load(directory / (name + ".npy"), mmap_mode="r", allow_pickle=False)
_check(value.ndim == 1 and value.dtype == np.dtype(dtype), f"Invalid graph array {name}")
arrays[name] = value
nodes, edge_ids = arrays["nodes"], arrays["edge_id"]
n, edges = len(nodes), len(edge_ids)
_check(0 < n <= 166700 and edges > 0, "A case needs real global nodes and explicit nonempty connectivity")
_check(nodes[0] >= 0 and nodes[-1] < 166700 and np.all(np.diff(nodes) > 0), "Invalid global node order")
for name in ("src", "dst", "signed_raw", "weights"):
_check(arrays[name].shape == (edges,), f"Graph array {name} has the wrong shape")
pointer = arrays["ptr"]
_check(pointer.shape == (n + 1,) and pointer[0] == 0 and pointer[-1] == edges and
np.all(np.diff(pointer) >= 0), "Invalid graph CSR pointer")
source_counts = np.zeros(n, np.int64)
identity_edges = True
previous_source, previous_edge = -1, -1
width = BLOCK_BYTES // 8
for begin in range(0, edges, width):
stop = min(edges, begin + width)
source, target, ids = arrays["src"][begin:stop], arrays["dst"][begin:stop], edge_ids[begin:stop]
_check(np.all((source >= 0) & (source < n)) and np.all((target >= 0) & (target < n)),
"Graph edge endpoint is outside the local node array")
_check(np.all(ids >= 0), "Graph edge IDs must be nonnegative")
_check(source[0] > previous_source or (source[0] == previous_source and ids[0] > previous_edge),
"Graph edge blocks are not in canonical source/edge order")
_check(np.all((source[1:] > source[:-1]) |
((source[1:] == source[:-1]) & (ids[1:] > ids[:-1]))),
"Graph edges are not in canonical source/edge order")
previous_source, previous_edge = int(source[-1]), int(ids[-1])
identity_edges &= np.array_equal(ids, np.arange(begin, stop, dtype=np.int64))
source_counts += np.bincount(source, minlength=n)
_check(np.isfinite(arrays["weights"][begin:stop]).all(), "Nonfinite graph weights")
_check(np.array_equal(source_counts, np.diff(pointer)), "CSR pointer and source rows disagree")
if not identity_edges:
_check(len(np.unique(edge_ids)) == edges, "Original edge IDs must be unique")
for name in ("completeness.csv", "connectivity.parquet"):
_check((directory / name).is_file(), f"Missing unchanged author input: {name}")
if n == 166700:
_check(edges == 25582938 and identity_edges and np.array_equal(nodes, np.arange(n, dtype=np.int32)),
"A full probe must use the unchanged complete canonical graph")
return arrays, identity_edges
def _requests(case_dir, nodes, steps):
with np.load(case_dir / "requests.npz", allow_pickle=False) as archive:
_check(set(archive.files) == {"tick", "neuron_idx"}, "Unexpected sparse request schema")
ticks, requested = archive["tick"].copy(), archive["neuron_idx"].copy()
_check(ticks.dtype == requested.dtype == np.dtype("int32") and ticks.ndim == requested.ndim == 1 and
ticks.shape == requested.shape, "Requests must be equal-length int32 vectors")
_check(np.all((ticks >= 0) & (ticks < steps)), "Request time outside the case")
_check(not len(ticks) or np.all((ticks[1:] > ticks[:-1]) |
((ticks[1:] == ticks[:-1]) & (requested[1:] > requested[:-1]))),
"Requests must be sorted and unique by tick/global node")
local = np.searchsorted(nodes, requested)
_check(np.all(local < len(nodes)) and np.array_equal(nodes[local], requested),
"Request references a global neuron absent from this case")
counts = np.bincount(ticks, minlength=steps).astype(np.int32)
pointer = np.concatenate((np.zeros(1, np.int64), np.cumsum(counts, dtype=np.int64)))
return requested, local.astype(np.int32), pointer, counts
class _QueueProxy:
def __init__(self, original, get_tick, available, arrays, identity_edges,
due_counts, spike_ring, delay_steps, handle, records, summarize):
self._original_queue, self._get_tick = original, get_tick
self._available, self._arrays, self._identity_edges = available, arrays, identity_edges
self._due_counts, self._spike_ring, self._delay_steps = due_counts, spike_ring, delay_steps
self._handle, self._records, self._summarize = handle, records, summarize
def peek(self, *args, **kwargs):
original = self._original_queue.peek(*args, **kwargs)
local_edges = np.asarray(original)
tick = self._get_tick()
_check(tick == len(self._records["rows"]), "Queue peek ticks must be consecutive")
_check(local_edges.ndim == 1 and np.all((local_edges >= 0) & (local_edges < len(self._arrays["edge_id"]))),
"Original queue contains an invalid local edge index")
_check(len(local_edges) == self._due_counts[tick % len(self._due_counts)],
"Queue arrivals differ from the fixed-delay outgoing degree of actual spikes")
fired_then = np.zeros(len(self._arrays["nodes"]), dtype=bool)
if tick >= self._delay_steps:
fired_then[self._spike_ring[(tick - self._delay_steps) % len(self._spike_ring)]] = True
def accepted(begin, stop):
part = local_edges[begin:stop]
_check(np.all(fired_then[self._arrays["src"][part]]), "Arrival lacks a presynaptic spike at author delay")
return self._available[self._arrays["dst"][part]]
# Only the observed ID view changes; the queue is returned verbatim.
edge_ids = local_edges if self._identity_edges else self._arrays["edge_id"][local_edges]
summary = self._summarize(tick, edge_ids, accepted)
self._handle.write(json.dumps(summary, sort_keys=True) + "\n")
self._records["rows"].append(summary)
self._records["peek_calls"] += 1
return original
def advance(self, *args, **kwargs):
self._records["advance_calls"] += 1
return self._original_queue.advance(*args, **kwargs)
def __getattr__(self, name):
return getattr(self._original_queue, name)
def run_case(case_dir: Path, out_dir: Path) -> dict:
"""Run one continuous case, with exclusive outputs and no candidate trace input.
Graph inputs are the AF array/author-table schema plus original weights.npy.
requests.npz contains sparse tick/neuron_idx int32 vectors. Optional
arrays.npz contains initial_v_mv/initial_g_mv float64 vectors only.
Full probes write state hashes, sparse events and APL traces, not dense
whole-brain histories. Each case requires a fresh process.
"""
global _PROCESS_USED
_check(not _PROCESS_USED, "Every oracle case requires a fresh process")
_PROCESS_USED = True
case_dir, out_dir = Path(case_dir).resolve(strict=True), Path(out_dir).resolve()
spec = json.loads((case_dir / "spec.json").read_text(encoding="utf-8"))
graph_dir = Path(spec["graph_dir"]).resolve(strict=True)
bundle_dir = Path(spec["apl_bundle_dir"]).resolve(strict=True)
worker_path = Path(spec["legacy_apl_worker"]).resolve(strict=True)
_check(worker_path == LEGACY_WORKER_PATH.resolve(strict=True),
"The APL worker must remain the sealed Phase4AG-v2 worker")
_check(_digest(worker_path) == LEGACY_WORKER_SHA256, "Frozen Phase4AG-v2 APL worker changed")
_check(_digest(BASE_ORACLE_PATH) == BASE_ORACLE_SHA256, "Sealed oracle source baseline changed")
science_python = _python_entry(spec["science_python"])
author_path = Path(spec.get("author_path", AUTHOR_PATH)).resolve(strict=True)
ae_path = Path(spec.get("ae_oracle_path", AE_PATH)).resolve(strict=True)
af_path = Path(spec.get("af_oracle_path", AF_PATH)).resolve(strict=True)
coefficients_path = Path(spec["coefficients_path"]).resolve(strict=True)
steps = int(spec["steps"])
apl_delay_steps = spec["apl_delay_steps"]
_check(type(apl_delay_steps) is int and apl_delay_steps in (0, 18),
"apl_delay_steps must be an explicit integer 0 or 18")
_check(steps == spec["steps"] and 0 < steps <= np.iinfo(np.int32).max, "Invalid number of steps")
_check(float(spec["dt_ms"]) == .1 and np.isclose(float(spec["duration_ms"]), steps * .1, rtol=0, atol=1e-12),
"The frozen APL protocol requires dt_ms=0.1 and a matching duration")
_check(spec.get("precision", "float64") == "float64", "Author state must be float64")
_check(spec.get("training", False) is False and spec.get("poi_used", False) is False,
"Training and original poi are outside this gate")
_check(spec["apl"] == {"tau_ms": 20, "gain": .01, "normalization": 1}, "Frozen APL parameters changed")
record_history = spec["record_history"]
_check(isinstance(record_history, bool), "record_history must be a boolean")
budget_root = Path(spec.get("budget_root", out_dir.parent)).resolve()
budget_limit = int(spec.get("output_budget_bytes", 15 * 1024 ** 3))
_check(budget_limit > 0 and out_dir.is_relative_to(budget_root), "Invalid output budget/root")
out_dir.mkdir(parents=True, exist_ok=False)
mapped_files = []
saved_preferences, old_dt, worker = None, None, None
last_complete_tick = -1
try:
arrays, identity_edges = _graph_inputs(graph_dir)
nodes, n, edges = arrays["nodes"], len(arrays["nodes"]), len(arrays["edge_id"])
_check(n < 166700 or not record_history, "Full probes must not save dense V/g/ref histories")
bundle = {name: np.load(bundle_dir / (name + ".npy"), mmap_mode="r", allow_pickle=False)
for name in ("input_matrix", "output_matrix", "kc_globals", "apl_globals")}
for name, shape in (("input_matrix", (2, 166700)), ("output_matrix", (2, 4064))):
value = bundle[name]
_check(value.dtype == np.dtype("float32") and value.shape == shape and value.flags.c_contiguous,
f"Frozen APL {name} shape/dtype/layout changed")
_check(np.isfinite(value).all() and np.all(value >= 0), f"Invalid positive raw-normalized APL {name}")
for name, count in (("kc_globals", 4064), ("apl_globals", 2)):
value = bundle[name]
_check(value.dtype == np.dtype("int32") and value.shape == (count,) and value[0] >= 0 and
value[-1] < 166700 and np.all(np.diff(value) > 0), f"Frozen APL identity {name} changed")
_check(not np.intersect1d(bundle["kc_globals"], bundle["apl_globals"]).size, "APL and KC identities overlap")
kc_positions = np.searchsorted(nodes, bundle["kc_globals"])
present = kc_positions < n
present[present] &= nodes[kc_positions[present]] == bundle["kc_globals"][present]
kc_local, kc_in_bundle = kc_positions[present], np.flatnonzero(present)
requested_globals, request_local, request_ptr, request_count = _requests(case_dir, nodes, steps)
initial = None
if (case_dir / "arrays.npz").exists():
with np.load(case_dir / "arrays.npz", allow_pickle=False) as archive:
_check(set(archive.files) == {"initial_v_mv", "initial_g_mv"}, "Unexpected initial-state schema")
initial = {name: archive[name].copy() for name in archive.files}
for name, value in initial.items():
_check(value.dtype == np.dtype("float64") and value.shape == (n,) and np.isfinite(value).all(),
f"Invalid diagnostic initial state: {name}")
history_bytes = steps * n * 40 if record_history else 0
_budget(budget_root, budget_limit, history_bytes + steps * (2 + 2 + 4064 + 4064) * 4 +
apl_delay_steps * 4064 * 4 + edges * 8 + (64 << 20))
versions = {name: importlib.metadata.version(name) for name in
("brian2", "numpy", "pandas", "pyarrow", "sympy", "scipy", "cython")}
_save(out_dir / "environment.json", {"python": sys.version, "platform": platform.platform(),
"packages": versions, "codegen": "numpy", "author_precision": "float64",
"APL_precision": "float32 in separate science-Python process", "Torch_imported_into_Brian": False})
for name, required in (("brian2", "2.5.1"), ("numpy", "1.22.3"), ("sympy", "1.12")):
_check(versions[name] == required, f"{name} must remain pinned to {required}")
import brian2 as b2
import pandas as pd
from brian2.core.base import BrianObject
ae = _load_checked(ae_path, AE_SHA256, "phase4ag_sealed_ae_helpers")
af = _load_checked(af_path, AF_SHA256, "phase4ag_sealed_af_helpers")
author = _load_checked(author_path, AUTHOR_SHA256, "phase4ag_unchanged_author")
_check("torch" not in sys.modules, "Torch must not be imported into the pinned Brian process")
comp = pd.read_csv(graph_dir / "completeness.csv", index_col=0)
_check(np.array_equal(comp.index.to_numpy(), nodes) and
np.array_equal(comp["local_index"].to_numpy(), np.arange(n)), "Author CSV neuron identity differs")
del comp
graph_hashes = {name: _digest(graph_dir / name) for name in
tuple(key + ".npy" for key in arrays) + ("completeness.csv", "connectivity.parquet")}
bundle_hashes = {name: _digest(bundle_dir / name) for name in BUNDLE_FILES}
case_hashes = {name: _digest(case_dir / name) for name in ("spec.json", "requests.npz")}
if initial is not None:
case_hashes["arrays.npz"] = _digest(case_dir / "arrays.npz")
worker_hash = _digest(worker_path)
coefficient_hash = _digest(coefficients_path)
baseline_coefficients = json.loads(coefficients_path.read_text(encoding="utf-8"))
if "legacy_apl_worker_sha256" in spec:
_check(worker_hash == spec["legacy_apl_worker_sha256"], "APL worker differs from preregistered source")
_save(out_dir / "input-audit.json", {"spec": spec, "case_hashes": case_hashes,
"graph_hashes": graph_hashes, "bundle_hashes": bundle_hashes,
"author_sha256": AUTHOR_SHA256, "ae_sha256": AE_SHA256, "af_sha256": AF_SHA256,
"legacy_apl_worker_sha256": worker_hash, "coefficients_sha256": coefficient_hash,
"copied_oracle_source": str(BASE_ORACLE_PATH), "copied_oracle_sha256": BASE_ORACLE_SHA256,
"apl_delay_steps": apl_delay_steps, "only_scientific_change": "transport generated graded-APL h before the unchanged end write",
"candidate_trace_used_as_input": False,
"scope": "shared-APL-component closed-loop oracle, not all-components-independent CPU equivalence"})
saved_preferences = {key: b2.prefs[key] for key in
("codegen.target", "core.default_float_dtype", "legacy.refractory_timing")}
old_dt = b2.defaultclock.dt
b2.start_scope()
b2.prefs["codegen.target"] = "numpy"
b2.prefs["core.default_float_dtype"] = np.float64
b2.prefs["legacy.refractory_timing"] = False
b2.defaultclock.dt = .1 * b2.ms
parameters = dict(author.default_params)
_save(out_dir / "parameter-checks.json", ae._parameters_match(parameters, spec["model"], b2))
neu, syn, spikes = author.create_model(graph_dir / "completeness.csv", graph_dir / "connectivity.parquet", parameters)
_check(len(neu) == n and len(syn) == edges, "Author loaded an unexpected graph size")
_check(all(np.dtype(neu.variables[key].dtype) == np.float64 for key in ("v", "g")) and
np.dtype(syn.variables["w"].dtype) == np.float64, "Author states/weights are not float64")
effective = _new_memmap(out_dir / "weights.npy", "<f8", (edges,))
mapped_files.append(effective)
width, w_unit, mask_count = BLOCK_BYTES // 8, float(parameters["w_syn"] / b2.volt), 0
mask_hasher = hashlib.sha256()
base_hasher = hashlib.sha256()
effective_hasher = hashlib.sha256()
for begin in range(0, edges, width):
stop = min(edges, begin + width)
np.testing.assert_array_equal(np.asarray(syn.i[begin:stop]), arrays["src"][begin:stop])
np.testing.assert_array_equal(np.asarray(syn.j[begin:stop]), arrays["dst"][begin:stop])
original = np.asarray(syn.w_[begin:stop], dtype=np.float64).copy()
raw_expected = arrays["signed_raw"][begin:stop] * w_unit
_check(np.array_equal(original.view(np.uint64), raw_expected.view(np.uint64)) and
np.array_equal(original.view(np.uint64), arrays["weights"][begin:stop].view(np.uint64)),
"Actual original Brian weights differ from unchanged signed raw or AF base weights")
base_hasher.update(np.ascontiguousarray(original, dtype="<f8").tobytes())
mask = np.isin(nodes[arrays["src"][begin:stop]], bundle["apl_globals"])
mask_count += int(mask.sum())
mask_hasher.update(np.ascontiguousarray(arrays["edge_id"][begin:stop][mask], dtype="<i8").tobytes())
original[mask] = np.float64(0.0)
effective_hasher.update(np.ascontiguousarray(original, dtype="<f8").tobytes())
effective[begin:stop] = original
syn.w_[begin:stop] = original
if n == 166700:
_check(mask_count == 5845, "Full legacy APL fast-edge mask must contain exactly 5845 existing edges")
effective.flush()
_save(out_dir / "apl-mask-audit.json", {"count": mask_count, "ordered_original_edge_id_sha256": mask_hasher.hexdigest(),
"base_weight_raw_bytes_sha256": base_hasher.hexdigest(), "effective_weights_file_sha256": _digest(out_dir / "weights.npy"),
"effective_weights_raw_sha256": effective_hasher.hexdigest(),
"rule": "zero +0 all existing fast edges whose source GLOBAL neuron is in apl_globals; no graph edge removal",
"graded_targets": "only KC identities present in this LIF case", "KC_targets_in_case": len(kc_local),
"base_graph_files_modified": False, "fullgraph_required_mask_count": 5845})
if initial is not None:
neu.v = initial["initial_v_mv"] * b2.mV
neu.g = initial["initial_g_mv"] * b2.mV
v_raw = neu.variables["v"].get_value()
g_raw = neu.variables["g"].get_value()
last_raw = neu.variables["lastspike"].get_value()
available_raw = neu.variables["not_refractory"].get_value()
histories = {}
pre_apl_histories = {}
if record_history:
for name, dtype in (("v", "float64"), ("g", "float64"), ("ref", "int32")):
histories[name] = _new_memmap(out_dir / f"{name}-history.{dtype}.npy", np.dtype(dtype), (steps, n))
mapped_files.append(histories[name])
pre_apl_histories[name] = _new_memmap(out_dir / f"preAPL-{name}.{dtype}.npy", np.dtype(dtype), (steps, n))
mapped_files.append(pre_apl_histories[name])
apl_history = {}
for name, count in (("drive", 2), ("state", 2), ("inhibition", 4064)):
apl_history[name] = _new_memmap(out_dir / f"apl-{name}.float32.npy", "<f4", (steps, count))
mapped_files.append(apl_history[name])
applied_history = _new_memmap(out_dir / "apl-applied.float32.npy", "<f4", (steps, 4064))
mapped_files.append(applied_history)
apl_fifo = _APLDelayFIFO(apl_delay_steps)
_save(out_dir / "output-schema.json", {"history_recorded": record_history, "history_shape": [steps, n],
"state_hash_file": "tick-state-hashes.jsonl", "state_hash_encoding": "SHA256 of contiguous little-endian raw f64 V/g and i32 ref",
"APL_dtype": "float32", "APL_h_units": "mV; gain already applied in float32",
"APL_SI_conversion": "h.astype(float64) * float64(0.001), then separate float64 g subtraction",
"APL_write_schedule": "end, order=0, after original reset, unconditional raw g write; recorder end/order=1",
"APL_input": "Brian's own actual global spikes, never candidate spikes or candidate inhibition",
"APL_shapes": {"drive": [steps, 2], "state": [steps, 2], "inhibition": [steps, 4064],
"applied": [steps, 4064], "pending_tail": [apl_delay_steps, 4064]},
"apl_delay_steps": apl_delay_steps,
"APL_transport": "applied[t]=generated_h[t-D] for t>=D, positive zero otherwise; separate independent list FIFO",
"APL_inhibition_file_means": "generated h at the actual closed-loop Brian spike tick, not the delayed applied value",
"APL_pending_file": "apl-pending-tail.float32.npy",
"APL_pending_scope": "next D scheduled applications from recorded h plus unconsumed initial zero slots; not future generated decay",
"APL_pending_initial_zero_prefix": max(apl_delay_steps - steps, 0),
"APL_pending_generated_count": min(steps, apl_delay_steps),
"request_columns": ["tick", "neuron_idx", "attempted", "accepted", "rejected"],
"queue_protocol": "sealed Phase4AF queue_summary; original global edge IDs, including zero-weight/ref-rejected arrivals",
"state_npz": ["step", "neuron_idx", "counts", "v", "g", "ref"],
"complete_marker": "summary.json complete=true only after worker exit and all checks"})
delay_steps = int(round(float(parameters["t_dly"] / b2.ms) / .1))
ref_steps = int(round(float(parameters["t_rfc"] / b2.ms) / .1))
_check(delay_steps == 18 and ref_steps == 22, "Author delay/refractory changed")
queue_len = delay_steps + 1
due_counts = np.zeros(queue_len, np.int64)
spike_ring = [np.empty(0, np.int32) for _ in range(queue_len)]
outdegree = np.diff(arrays["ptr"])
request_seen = np.zeros(steps, np.uint8)
apl_seen = np.zeros(steps, np.uint8)
applicable_count = np.zeros(steps, np.int32)
rejected_count = np.zeros(steps, np.int32)
spike_count = np.zeros(steps, np.int32)
total_counts = np.zeros(n, np.int32)
records = {"rows": [], "peek_calls": 0, "advance_calls": 0, "installations": []}
last_ticks = np.zeros(n, np.int64)
final_ref = np.zeros(n, np.int32)
permitted = np.empty(0, np.int32)
total_events = 0
def tick_now():
return int(round(float(neu.clock.t / b2.second) / .0001))
def current_ref(tick):
seen = np.isfinite(last_raw) & (last_raw >= 0)
last_ticks.fill(0)
last_ticks[seen] = np.rint(last_raw[seen] / .0001).astype(np.int64)
result = np.where(seen, np.maximum(ref_steps - (tick - last_ticks), 0), 0).astype(np.int32)
_check(np.array_equal(result == 0, available_raw), "Ref history disagrees with author not_refractory")
return result
with ExitStack() as stack:
stderr = stack.enter_context((out_dir / "apl-worker.stderr.log").open("xb"))
worker = _APLWorker(science_python, worker_path, bundle_dir, stderr, float(spec.get("worker_timeout_seconds", 180)))
_save(out_dir / "apl-worker-handshake.json", {"command": worker.command, "handshake": worker.hello})
queue_file = stack.enter_context((out_dir / "queue.jsonl").open("x", encoding="utf-8"))
event_file = stack.enter_context((out_dir / "events.bin").open("xb"))
hash_file = stack.enter_context((out_dir / "tick-state-hashes.jsonl").open("x", encoding="utf-8"))
tick_file = stack.enter_context((out_dir / "ticks.jsonl").open("x", encoding="utf-8"))
requests_file = stack.enter_context((out_dir / "requests.csv").open("x", encoding="utf-8", newline=""))
request_writer = csv.DictWriter(requests_file, fieldnames=["tick", "neuron_idx", "attempted", "accepted", "rejected"])
request_writer.writeheader()
def request_operation():
nonlocal permitted
tick = tick_now()
_check(0 <= tick < steps and not request_seen[tick], "Invalid/duplicate common request tick")
request_seen[tick] = 1
begin, stop = request_ptr[tick:tick + 2]
local = request_local[begin:stop]
available = available_raw[local].copy()
permitted = local[available]
applicable_count[tick] = len(permitted)
rejected_count[tick] = len(local) - len(permitted)
for node, accepted in zip(requested_globals[begin:stop], available):
request_writer.writerow({"tick": tick, "neuron_idx": int(node), "attempted": 1,
"accepted": int(accepted), "rejected": int(not accepted)})
if len(permitted):
neu.v[permitted] = parameters["v_th"] + 1 * b2.mV
def apl_operation():
tick = tick_now()
_check(0 <= tick < steps and not apl_seen[tick] and request_seen[tick], "Invalid/duplicate APL tick")
active = np.asarray(neu.spikes, dtype=np.int32)
_check(not len(active) or np.all(np.diff(active) > 0), "Brian actual spikes are not canonical/unique")
_check(np.all(g_raw[active] == 0.0), "APL ran before the original author g reset")
if record_history:
pre_apl_histories["v"][tick] = v_raw
pre_apl_histories["g"][tick] = g_raw
pre_apl_histories["ref"][tick] = current_ref(tick)
drive, state, inhibition = worker.step(tick, np.asarray(nodes[active], dtype=np.int32))
apl_history["drive"][tick] = drive
apl_history["state"][tick] = state
apl_history["inhibition"][tick] = inhibition
applied = apl_fifo.step(tick, inhibition)
applied_history[tick] = applied
# These are separate array operations. Only h transport is new.
# Do not fuse conversion/subtraction or add a refractory guard.
delta_si = applied[kc_in_bundle].astype(np.float64) * np.float64(.001)
g_raw[kc_local] = g_raw[kc_local] - delta_si
apl_seen[tick] = 1
def record_end():
nonlocal last_complete_tick, total_events, final_ref
tick = tick_now()
_check(tick == last_complete_tick + 1 and tick < steps and apl_seen[tick], "End tick is incomplete/out of order")
_check(len(records["rows"]) == tick + 1, "End state lacks queue observation")
_check(np.isfinite(v_raw).all() and np.isfinite(g_raw).all(), "Nonfinite author/APL state")
final_ref = current_ref(tick)
if record_history:
histories["v"][tick], histories["g"][tick], histories["ref"][tick] = v_raw, g_raw, final_ref
active = np.asarray(neu.spikes, dtype=np.int32)
_check(np.isin(permitted, active).all(), "An applicable request did not produce a real spike")
spike_count[tick] = len(active)
total_counts[active] += 1
if len(active):
events = np.empty(len(active), EVENT_DTYPE)
events["step"], events["neuron_idx"] = tick, nodes[active]
event_file.write(memoryview(events).cast("B"))
total_events += len(active)
spike_ring[tick % queue_len] = active.copy()
due_counts[(tick + delay_steps) % queue_len] = int(outdegree[active].sum(dtype=np.int64))
hash_file.write(json.dumps({"tick": tick, "v_sha256": _array_hash(v_raw, "<f8"),
"g_sha256": _array_hash(g_raw, "<f8"), "ref_sha256": _array_hash(final_ref, "<i4")}, sort_keys=True) + "\n")
tick_file.write(json.dumps({"tick": tick, "spikes": len(active), "requests": int(request_count[tick]),
"applicable_requests": int(applicable_count[tick]), "ref_rejected_requests": int(rejected_count[tick]),
"event_sha256": _array_hash(nodes[active], "<i4"),
"due_arrivals": records["rows"][tick]["count"], "accepted_arrivals": records["rows"][tick]["accepted_count"],
"APL_drive_sha256": _array_hash(apl_history["drive"][tick], "<f4"),
"APL_state_sha256": _array_hash(apl_history["state"][tick], "<f4"),
"APL_inhibition_sha256": _array_hash(apl_history["inhibition"][tick], "<f4"),
"APL_applied_sha256": _array_hash(applied_history[tick], "<f4"),
"APL_applied_generated_tick": tick - apl_delay_steps if tick >= apl_delay_steps else None}, sort_keys=True) + "\n")
_budget(budget_root, budget_limit, reserve=1 << 20)
last_complete_tick = tick
if (tick + 1) % 100 == 0 or tick + 1 == steps:
for mapped in mapped_files:
mapped.flush()
for handle in (queue_file, event_file, hash_file, tick_file, requests_file):
handle.flush()
print(json.dumps({"oracle": spec["name"], "completed_ticks": tick + 1,
"spikes": total_events, "APL_worker_ticks": worker.tick}), flush=True)
class QueueInstaller(BrianObject):
def __init__(self):
super().__init__(clock=neu.clock, when="end", order=1_000_000, name="zz_apl_queue_installer")
def before_run(self, run_namespace):
installed = []
for code in syn.pre.code_objects:
if "_queue" not in code.namespace or "_queue.peek()" not in str(code.code):
continue
original = code.namespace["_queue"]
if isinstance(original, _QueueProxy):
original = original._original_queue
code.namespace["_queue"] = _QueueProxy(original, tick_now, available_raw, arrays, identity_edges,
due_counts, spike_ring, delay_steps, queue_file, records, af.queue_summary)
installed.append(code.name)
_check(len(installed) == 1, "Expected one original anatomical queue consumer")
records["installations"].append(installed[0])
objects = [neu, syn, spikes,
b2.NetworkOperation(request_operation, clock=neu.clock, when="before_thresholds", order=0, name="oracle_common_requests"),
b2.NetworkOperation(apl_operation, clock=neu.clock, when="end", order=0, name="oracle_legacy_apl_unconditional"),
b2.NetworkOperation(record_end, clock=neu.clock, when="end", order=1, name="oracle_after_apl_record"), QueueInstaller()]
network = b2.Network(*objects)
_save(out_dir / "schedule.json", {"schedule": list(network.schedule), "summary": str(network.scheduling_summary())})
network.run(0 * b2.ms, namespace={})
_save(out_dir / "compiled-code.json", ae._code_records(network))
compiled_coefficients = ae._coefficients(neu, parameters, b2, .0001)
coefficient_keys = ("gd", "c0", "cg", "cv", "threshold", "reset", "seed_volt")
_check(all(float(compiled_coefficients[key]).hex() == float(baseline_coefficients[key]).hex()
for key in coefficient_keys), "Compiled Brian coefficients differ from sealed AF")
_check(last_complete_tick == -1 and worker.tick == 0 and not records["rows"] and not request_seen.any(),
"Zero-duration compilation unexpectedly advanced state")
network.run(steps * .1 * b2.ms, namespace={})
_save(out_dir / "generated-code.json", ae._code_records(network))
worker_status = worker.finish(steps)
worker = None
_check(last_complete_tick == steps - 1 and np.all(request_seen == 1) and np.all(apl_seen == 1), "Incomplete trial")
pending_tail = apl_fifo.finish(steps)
warmup = min(steps, apl_delay_steps)
consumed = max(steps - apl_delay_steps, 0)
initial_zero_prefix = max(apl_delay_steps - steps, 0)
_check(np.all(applied_history[:warmup].view(np.uint32) == 0),
"Delayed APL warmup must contain positive zeros")
_check(np.array_equal(applied_history[warmup:].view(np.uint32),
apl_history["inhibition"][:consumed].view(np.uint32)),
"Applied h is not the exact delayed generated float32 sequence")
_check(pending_tail.shape == (apl_delay_steps, 4064) and pending_tail.dtype == np.float32,
"Pending APL tail shape/dtype changed")
_check(np.all(pending_tail[:initial_zero_prefix].view(np.uint32) == 0),
"Short-window pending prefix must contain unconsumed initial positive zeros")
_check(np.array_equal(pending_tail[initial_zero_prefix:].view(np.uint32),
apl_history["inhibition"][consumed:].view(np.uint32)),
"Pending tail does not exactly partition the finite generated h sequence")
with (out_dir / "apl-pending-tail.float32.npy").open("xb") as handle:
np.save(handle, pending_tail, allow_pickle=False)
_save(out_dir / "apl-timing-audit.json", {
"apl_delay_steps": apl_delay_steps, "steps": steps,
"generated_rows": steps, "applied_generated_rows": consumed,
"pending_generated_count": min(steps, apl_delay_steps),
"initial_zero_prefix": initial_zero_prefix,
"pending_initial_zero_prefix": initial_zero_prefix,
"pending_shape": [apl_delay_steps, 4064],
"pending_application_tick_start": steps,
"pending_application_tick_stop_exclusive": steps + apl_delay_steps,
"pending_generated_tick_start": consumed,
"pending_generated_tick_stop_exclusive": steps,
"positive_zero_warmup_exact": True, "transport_shift_exact": True,
"finite_generated_sequence_partition_exact": True,
"pending_raw_sha256": _array_hash(pending_tail, "<f4"),
"pending_file_sha256": _digest(out_dir / "apl-pending-tail.float32.npy"),
"brain_steps_after_window": 0, "APL_generation_steps_after_window": 0,
"brain_tail_advanced": False,
"final_apl_state_float32": np.asarray(apl_history["state"][-1], dtype=np.float32).tolist(),
"final_apl_state_u32_bits": np.asarray(apl_history["state"][-1], dtype=np.float32).view(np.uint32).tolist(),
"final_apl_state_nonzero": bool(np.any(apl_history["state"][-1] != 0)),
"future_generated_decay_tail_included": False,
"future_apl_decay_or_feedback_generated": False,
"closed_loop_branch_integrals_claimed_equal": False,
"delay_implementation": "independent list FIFO; no candidate or timing_queue imports",
"meaning": "Tail contains only already generated h and any remaining initial-zero padding; never a complete future closed-loop or decay tail",
})
_check(records["peek_calls"] == records["advance_calls"] == steps, "Incomplete queue audit")
_check(total_events == int(spikes.num_spikes) and np.array_equal(total_counts, np.asarray(spikes.count[:])),
"Sparse events disagree with original SpikeMonitor")
_check(np.array_equal(request_count, applicable_count + rejected_count), "Incomplete request accounting")
_check(np.all(np.asarray(neu.rfc[:] / b2.ms) == float(parameters["t_rfc"] / b2.ms)), "Author refractory duration changed")
for begin in range(0, edges, width):
stop = min(edges, begin + width)
_check(np.array_equal(np.asarray(syn.w_[begin:stop]).view(np.uint64), effective[begin:stop].view(np.uint64)),
"Effective synaptic weights changed during the trial")
event_values = np.memmap(out_dir / "events.bin", dtype=EVENT_DTYPE, mode="r", shape=(total_events,)) if total_events else np.empty(0, EVENT_DTYPE)
_check((out_dir / "events.bin").stat().st_size == total_events * 8, "Sparse event byte count mismatch")
for begin in range(0, total_events, width):
stop = min(total_events, begin + width)
spike_local = np.asarray(spikes.i[begin:stop], dtype=np.int32)
np.testing.assert_array_equal(event_values["neuron_idx"][begin:stop], nodes[spike_local])
times = np.rint(np.asarray(spikes.t[begin:stop] / b2.second) / .0001).astype(np.int32)
np.testing.assert_array_equal(event_values["step"][begin:stop], times)
_budget(budget_root, budget_limit, total_events * 8 + n * 24 + (8 << 20))
with (out_dir / "state.npz").open("xb") as handle:
np.savez_compressed(handle, step=event_values["step"], neuron_idx=event_values["neuron_idx"],
counts=total_counts, v=v_raw, g=g_raw, ref=final_ref)
with (out_dir / "tick-summary.npz").open("xb") as handle:
np.savez_compressed(handle, request_count=request_count, applicable_count=applicable_count,
rejected_count=rejected_count, spike_count=spike_count,
arrival_count=np.asarray([row["count"] for row in records["rows"]], np.int64),
accepted_arrival_count=np.asarray([row["accepted_count"] for row in records["rows"]], np.int64))
coefficients = ae._coefficients(neu, parameters, b2, .0001)
keys = ("gd", "c0", "cg", "cv", "threshold", "reset", "seed_volt")
_check(all(float(coefficients[key]).hex() == float(baseline_coefficients[key]).hex() for key in keys),
"Actual Brian coefficients differ from sealed AF coefficients")
_save(out_dir / "coefficients.json", coefficients)
_save(out_dir / "queue-audit.json", {"peek_calls": records["peek_calls"], "advance_calls": records["advance_calls"],
"installations": records["installations"], "same_peek_object_returned": True,
"original_edge_ids_not_sorted_or_filtered": True, "zero_weight_and_refractory_rejected_edges_retained": True,
"post_threshold_pre_synaptic_guard_observed": True, "forwarding_contract": "sealed AE/AF design",
"new_unwrapped_repeat_performed": False, "limit": "Current wrapper was not independently rerun without the queue proxy"})
for path, expected in ((author_path, AUTHOR_SHA256), (ae_path, AE_SHA256), (af_path, AF_SHA256),
(worker_path, worker_hash), (BASE_ORACLE_PATH, BASE_ORACLE_SHA256)):
_check(_digest(path) == expected, f"Source changed during the run: {path}")
_check(_digest(coefficients_path) == coefficient_hash, "Sealed coefficient reference changed during the run")
for directory, hashes in ((graph_dir, graph_hashes), (bundle_dir, bundle_hashes), (case_dir, case_hashes)):
for name, expected in hashes.items():
_check(_digest(directory / name) == expected, f"Read-only input changed during the run: {name}")
for mapped in mapped_files:
mapped.flush()
_check("torch" not in sys.modules, "Pinned Brian process unexpectedly imported Torch")
output_bytes = _budget(budget_root, budget_limit, 1 << 20)
summary = {"name": spec["name"], "complete": True, "precision": "float64", "APL_precision": "float32",
"backend": "shared-APL-component closed-loop Brian2 oracle", "steps": steps, "completed_ticks": last_complete_tick + 1,
"nodes": n, "edges": edges, "spikes": total_events, "requests": int(request_count.sum()),
"applicable_requests": int(applicable_count.sum()), "ref_rejected_requests": int(rejected_count.sum()),
"APL_fast_edges_zeroed": mask_count, "APL_worker_exit_status": worker_status, "APL_worker_ticks": steps,
"apl_fast_edges_zeroed": mask_count, "effective_weights_raw_sha256": effective_hasher.hexdigest(),
"kc_targets_present": len(kc_local), "apl_delay_steps": apl_delay_steps,
"APL_transport_shift_exact": True, "APL_finite_sequence_partition_exact": True,
"pending_generated_count": min(steps, apl_delay_steps),
"initial_zero_prefix": max(apl_delay_steps - steps, 0),
"pending_initial_zero_prefix": max(apl_delay_steps - steps, 0),
"pending_tail_shape": [apl_delay_steps, 4064], "future_generated_decay_tail_included": False,
"APL_applied_sha256": _digest(out_dir / "apl-applied.float32.npy"),
"APL_pending_tail_sha256": _digest(out_dir / "apl-pending-tail.float32.npy"),
"APL_timing_audit_sha256": _digest(out_dir / "apl-timing-audit.json"),
"candidate_trace_used_as_input": False, "independent_CPU_all_components_claim": False,
"history_recorded": record_history, "single_continuous_run": True, "no_training": True, "no_poi": True,
"source_and_inputs_unchanged": True, "pass_all": True,
"pass_all_scope": "Oracle construction, protocol and internal self-checks only; candidate comparison is a separate gate",
"scope": "Only this fixed trajectory; legacy Torch APL is a shared numerical component, not an independent CPU APL implementation",
"output_budget_bytes": budget_limit, "output_bytes_before_summary": output_bytes,
"state_hashes_sha256": _digest(out_dir / "tick-state-hashes.jsonl"), "queue_sha256": _digest(out_dir / "queue.jsonl"),
"events_sha256": _digest(out_dir / "events.bin"), "weights_sha256": _digest(out_dir / "weights.npy")}
_save(out_dir / "summary.json", summary)
return summary
except Exception as error:
for mapped in mapped_files:
mapped.flush()
_save(out_dir / "error.json", {"complete": False, "last_complete_tick": last_complete_tick,
"type": type(error).__name__, "message": str(error), "partial_files_preserved": True,
"inputs_or_duration_adjusted": False})
raise
finally:
if worker is not None:
worker.abort()
if saved_preferences is not None:
for key, value in saved_preferences.items():
b2.prefs[key] = value
b2.defaultclock.dt = old_dt
def run_oracle(case_dir: Path, out_dir: Path) -> dict:
"""Public timing-adapter alias; preserves the sealed run_case call shape."""
return run_case(case_dir, out_dir)
def main():
import argparse
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("case_dir", type=Path)
parser.add_argument("out_dir", type=Path)
args = parser.parse_args()
result = run_oracle(args.case_dir, args.out_dir)
print(json.dumps(result, sort_keys=True, allow_nan=False), flush=True)
if not result.get("complete") or not result.get("pass_all"):
raise SystemExit(1)
if __name__ == "__main__":
main()
models/male-cns-lif/apl-timing-lab-v1/prepare.py
+91
View File
@@ -0,0 +1,91 @@
"""Register a two-arm timing intervention before new trajectory outcomes."""
from datetime import datetime, timezone
import difflib
import platform
import shutil
import subprocess
import sys
import numpy as np
from context import (ROOT, MODEL, EXP, AG, AG_MODEL, AI, AI_MODEL, AH_MODEL, META,
COEFF, CFG, digest, read, save, sources, integrity, budget)
def copy_exact(src, dst):
dst.parent.mkdir(parents=True, exist_ok=True)
with src.open('rb') as a, dst.open('xb') as b:
shutil.copyfileobj(a, b)
assert digest(src) == digest(dst)
def main():
assert not EXP.exists() or not any(EXP.iterdir()), 'Never reuse a populated experiment'
assert shutil.disk_usage(EXP.parent).free > CFG['output_budget_bytes'] + 10 * 1024**3
assert read(AI / 'audit/final-status.json')['status'] == 'PASS-CONDITIONAL-PN-APL-SOURCE-AUDIT'
assert CFG['apl'] == read(AG_MODEL / 'config.json')['apl'] and CFG['delays_steps'] == [0, 18]
for sub in ('protocol', 'cases', 'gpu', 'oracle', 'audit', 'logs', 'regression', 'component-audit', 'calibration', 'extension-build'):
(EXP / sub).mkdir(parents=True, exist_ok=False)
command = [sys.executable, '-B', '-m', 'unittest', 'discover', '-v', '-s', str(MODEL), '-p', 'test_*.py']
tested = subprocess.run(command, cwd=MODEL, capture_output=True, text=True)
with (EXP / 'regression/unit-tests.log').open('x') as f:
f.write(tested.stdout + tested.stderr)
save(EXP / 'regression/unit-tests.json', dict(command=command, exit_code=tested.returncode,
pass_all=tested.returncode == 0, sha256=digest(EXP / 'regression/unit-tests.log')))
assert tested.returncode == 0, 'Unit tests failed; stop before registration'
inputs = [AI / 'manifest.json', AI / 'checksums.sha256', AI_MODEL / 'manifest.json',
AG / 'manifest.json', AG_MODEL / 'manifest.json', META, COEFF,
AH_MODEL / 'replay_core.py', AG_MODEL / 'legacy_apl_worker.py', AG_MODEL / 'gpu_apl.py',
AG_MODEL / 'oracle_apl.py', AG_MODEL / 'compare.py', AG_MODEL / 'calibration.py',
AG_MODEL / 'validate_components.py', AI / 'protocol/neuron-source-groups.parquet']
save(EXP / 'protocol/preregistered.json', dict(timestamp_utc=datetime.now(timezone.utc).isoformat(),
sources=sources(), config=CFG, inputs=[dict(path=str(p.relative_to(ROOT)), sha256=digest(p)) for p in inputs],
parent_commit=subprocess.check_output(['git', '-C', str(ROOT), 'rev-parse', 'HEAD'], text=True).strip(),
before_new_trajectories=True, prior_CAL_and_AI_outcomes_known=True,
single_factor='generated graded-APL h applied at t versus t+18; nothing else',
fixed_input_counts_and_amplitudes=True, actual_accepted_requests_may_change_due_to_closed_loop=True,
formal_learning_claim=False, oracle_limit='Independent Brian LIF state with shared legacy Torch APL arithmetic; separate component audit',
delay_biological_measurement=False, whole_brain_equal_total_inhibition_claim=False,
timing_test_no_full_future_tail_claim=True, source_diffs_are_review_material_not_equivalence_proof=True,
scope='Origin audit -> exact shift conservation -> two-arm fixtures -> fullgraph oracle probes -> complete two-arm fixed CAL -> frozen regression',
python=sys.version, numpy=np.__version__, machine=platform.machine()))
save(EXP / 'regression/old-integrity-before.json', integrity())
for old, new in (('gpu_apl.py', 'gpu_timing.py'), ('oracle_apl.py', 'oracle_timing.py'), ('compare.py', 'compare_timing.py')):
difference = ''.join(difflib.unified_diff((AG_MODEL / old).read_text().splitlines(True),
(MODEL / new).read_text().splitlines(True), fromfile=str(AG_MODEL / old), tofile=str(MODEL / new)))
with (EXP / 'protocol' / (new + '.diff')).open('x') as f:
f.write(difference)
records = read(AG / 'protocol/calibration-cases.json')
assert len(records) == 60 and len({r['name'] for r in records}) == 60
copy_exact(AG / 'protocol/calibration-cases.json', EXP / 'protocol/calibration-cases.json')
copy_exact(AG / 'protocol/encoder-audit.json', EXP / 'protocol/encoder-audit.json')
names = CFG['fixtures'] + [r['name'] for r in records]
inventory = []
for delay in CFG['delays_steps']:
arm = 'delay' + str(delay)
for sub in ('cases', 'gpu', 'oracle', 'component-audit', 'calibration'):
(EXP / sub / arm).mkdir(exist_ok=False)
for name in names:
src = AG / 'cases' / name
dst = EXP / 'cases' / arm / name
dst.mkdir(exist_ok=False)
spec = read(src / 'spec.json')
old_spec = dict(spec)
spec.update(apl_delay_steps=delay, budget_root=str(EXP), output_budget_bytes=CFG['output_budget_bytes'])
save(dst / 'spec.json', spec)
for key in set(old_spec) - {'budget_root', 'output_budget_bytes'}:
assert old_spec[key] == spec[key]
files = []
for fname in ('requests.npz', 'arrays.npz', 'pn-drive.npz'):
if (src / fname).exists():
copy_exact(src / fname, dst / fname)
files.append(dict(file=fname, sha256=digest(src / fname)))
inventory.append(dict(arm=arm, delay_steps=delay, name=name, kind=spec['kind'],
source_case=str(src.relative_to(ROOT)), unchanged_input_files=files))
save(EXP / 'protocol/case-inventory.json', inventory)
save(EXP / 'protocol/prepared-hashes.json', [dict(path=str(p.relative_to(EXP)), sha256=digest(p), bytes=p.stat().st_size)
for sub in ('protocol', 'cases') for p in sorted((EXP / sub).rglob('*')) if p.is_file() and p.name != 'prepared-hashes.json'])
budget()
print('REGISTERED 2 delays, 8 fixtures + 60 CAL each; exact original inputs; no new trajectory yet', flush=True)
if __name__ == '__main__':
main()
models/male-cns-lif/apl-timing-lab-v1/run_case.py
+59
View File
@@ -0,0 +1,59 @@
"""Exclusive-output fresh-process runner; original cases, one science GPU job."""
import argparse
import json
import os
import urllib.request
from context import ROOT, MODEL, EXP, AG, AG_MODEL, CFG, check, budget, read, load_module
def preflight():
os.environ['CUDA_VISIBLE_DEVICES'] = CFG['gpu_uuid']
os.environ['CUDA_HOME'] = '/usr/local/cuda'
os.environ['PATH'] = str(ROOT / 'models/male-cns-lif/.venv/bin') + ':/usr/local/cuda/bin:' + os.environ['PATH']
os.environ['MAX_JOBS'] = '2'
os.environ['TORCH_CUDA_ARCH_LIST'] = '8.9'
for port in (8188, 8189, 8190):
with urllib.request.urlopen(f'http://192.168.10.51:{port}/queue', timeout=5) as response:
queue = json.load(response)
assert not queue['queue_running'] and not queue['queue_pending'], 'ComfyUI busy: defer, never evict'
def main():
parser = argparse.ArgumentParser()
parser.add_argument('backend', choices=['gpu', 'oracle', 'component'])
parser.add_argument('arm', choices=['delay0', 'delay18'])
parser.add_argument('case')
parser.add_argument('--repeat', type=int, default=0)
args = parser.parse_args()
assert '/' not in args.case and '..' not in args.case and 0 <= args.repeat < 3
check(); budget()
assert read(EXP / 'origin-audit/summary.json')['pass_all']
assert read(EXP / 'audit/shift-gate.json')['pass_all']
preflight()
case = EXP / 'cases' / args.arm / args.case
assert case.is_dir() and read(case / 'spec.json')['apl_delay_steps'] == int(args.arm[5:])
if args.case not in CFG['fixtures']:
assert read(EXP / 'audit/fixtures-gate.json')['pass_all'], 'Fixture gate required before fullgraph execution'
if args.case not in CFG['calibration']['full_oracle_probes'] or args.repeat > 0:
assert read(EXP / 'audit/probes-gate.json')['pass_all'], 'Both fullgraph probe gates required before CAL'
out = EXP / 'gpu' / args.arm / f'{args.case}-r{args.repeat}'
if args.backend == 'gpu':
from gpu_timing import run_gpu
result = run_gpu(case, out, EXP / 'extension-build')
elif args.backend == 'oracle':
assert args.repeat == 0
from oracle_timing import run_case
result = run_case(case, EXP / 'oracle' / args.arm / args.case)
else:
assert args.repeat == 0
component = load_module('sealed_ag_component_aj', AG_MODEL / 'validate_components.py')
result = component.validate(out, AG / 'apl-bundle', EXP / 'component-audit' / args.arm / (args.case + '.json'))
budget()
print(json.dumps(dict(backend=args.backend, arm=args.arm, case=args.case, repeat=args.repeat,
complete=result.get('complete'), pass_all=result.get('pass_all')), allow_nan=False), flush=True)
if not result.get('pass_all', False):
raise SystemExit(1)
if __name__ == '__main__':
main()
models/male-cns-lif/apl-timing-lab-v1/run_suite.py
+108
View File
@@ -0,0 +1,108 @@
"""Strict ordered numerical gates; both complete CAL arms even if representation fails."""
import argparse
import subprocess
import time
import traceback
from context import ROOT, MODEL, EXP, AG, CFG, check, save, read, budget
SCIENCE = ROOT / 'models/male-cns-lif/.venv/bin/python'
ORACLE = ROOT / 'experiments/phase4ad-author-semantics/environment/.venv/bin/python'
def child(backend, arm, name, repeat=0):
log = EXP / 'logs' / f'{backend}-{arm}-{name}-r{repeat}.log'
command = [str(ORACLE if backend == 'oracle' else SCIENCE), '-B', str(MODEL / 'run_case.py'),
backend, arm, name, '--repeat', str(repeat)]
print('START', backend, arm, name, repeat, flush=True)
started = time.perf_counter()
with log.open('x', encoding='utf-8') as stream:
result = subprocess.run(command, stdout=stream, stderr=subprocess.STDOUT, timeout=1800, check=False)
print('FINISH', backend, arm, name, repeat, 'exit', result.returncode,
'wall', round(time.perf_counter() - started, 2), flush=True)
if result.returncode:
raise RuntimeError(f'{backend} {arm}/{name} failed; preserve and inspect {log}')
def gate_case(arm, name, repeats):
from compare_timing import compare, compare_gpu_repeats
from calibration import control_identity
child('gpu', arm, name)
for repeat in range(1, repeats):
child('gpu', arm, name, repeat)
child('oracle', arm, name)
result = compare(EXP / 'cases' / arm / name, EXP / 'gpu' / arm / (name + '-r0'),
EXP / 'oracle' / arm / name, EXP / 'audit' / f'compare-{arm}-{name}.json')
assert result['pass_all'], f'Closed-loop exact comparison failed: {arm}/{name}'
for repeat in range(1, repeats):
report = compare_gpu_repeats(EXP / 'gpu' / arm / (name + '-r0'),
EXP / 'gpu' / arm / f'{name}-r{repeat}', EXP / 'audit' / f'repeat-{arm}-{name}-r{repeat}.json')
assert report['pass_all'], f'Exact repeat failed: {arm}/{name}/{repeat}'
if arm == 'delay0':
report = control_identity(EXP / 'gpu' / arm / (name + '-r0'), AG / 'gpu' / (name + '-r0'),
EXP / 'cases' / arm / name)
save(EXP / 'audit' / f'control-{name}.json', report)
assert report['pass_all'], f'Zero-delay control changed relative to sealed AG: {name}'
child('component', arm, name)
return dict(arm=arm, case=name, pass_all=True, gpu_repeats=repeats)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('stage', choices=['fixtures', 'probes', 'calibration'])
args = parser.parse_args()
check(); budget()
receipt = EXP / 'audit' / (args.stage + '-gate.json')
assert not receipt.exists()
assert read(EXP / 'origin-audit/summary.json')['pass_all']
assert read(EXP / 'audit/shift-gate.json')['pass_all']
arms = ['delay' + str(delay) for delay in CFG['delays_steps']]
try:
if args.stage == 'fixtures':
rows = [gate_case(arm, name, CFG['fixture_repeats']) for arm in arms for name in CFG['fixtures']]
save(receipt, dict(pass_all=True, cases=rows, scope='Both arms; fixed real-edge induced fixtures only'))
elif args.stage == 'probes':
assert read(EXP / 'audit/fixtures-gate.json')['pass_all']
rows = [gate_case(arm, name, 1) for arm in arms for name in CFG['calibration']['full_oracle_probes']]
save(receipt, dict(pass_all=True, cases=rows, scope='Both arms; same two original first-CAL fullgraph cases'))
else:
assert read(EXP / 'audit/fixtures-gate.json')['pass_all']
assert read(EXP / 'audit/probes-gate.json')['pass_all']
from calibration import summarize, control_identity
from compare_timing import compare_gpu_repeats
records = read(EXP / 'protocol/calibration-cases.json')
assert len(records) == 60
for arm in arms:
for record in records:
if record['name'] not in CFG['calibration']['full_oracle_probes']:
child('gpu', arm, record['name'])
name = record['name']
if arm == 'delay0':
control = control_identity(EXP / 'gpu' / arm / (name + '-r0'),
AG / 'gpu' / (name + '-r0'), EXP / 'cases' / arm / name)
save(EXP / 'audit' / f'cal-control-{name}-r0.json', control)
assert control['pass_all'], 'Zero-delay CAL control diverged: ' + name
if record['seed'] == CFG['calibration']['seeds'][0]:
child('gpu', arm, record['name'], 1)
repeat = compare_gpu_repeats(EXP / 'gpu' / arm / (name + '-r0'),
EXP / 'gpu' / arm / (name + '-r1'), EXP / 'audit' / f'cal-repeat-{arm}-{name}.json')
assert repeat['pass_all'], 'CAL exact repeat diverged: ' + arm + '/' + name
if arm == 'delay0':
control = control_identity(EXP / 'gpu' / arm / (name + '-r1'),
AG / 'gpu' / (name + '-r1'), EXP / 'cases' / arm / name)
save(EXP / 'audit' / f'cal-control-{name}-r1.json', control)
assert control['pass_all'], 'Zero-delay CAL repeat control diverged: ' + name
result = summarize()
save(receipt, dict(pass_all=result['execution_gate_pass'],
representation_pass_all=result['pass_all'], scope='Complete both fixed CAL arms; not heldout or learning',
summary='calibration/summary.json'))
assert result['execution_gate_pass'], 'CAL numerical/control/repeat gate failed'
except Exception as error:
save(EXP / 'audit' / ('stopped-' + args.stage + '.json'), dict(stage=args.stage,
type=type(error).__name__, message=str(error), traceback=traceback.format_exc(),
later_stages_not_permitted=True, partial_outputs_preserved=True, parameters_changed=False))
raise
budget()
if __name__ == '__main__':
main()
models/male-cns-lif/apl-timing-lab-v1/shift_audit.py
+44
View File
@@ -0,0 +1,44 @@
"""Fixed sealed-trace transport check; no closed-loop dynamics or gain search."""
import hashlib
import numpy as np
from context import AG, EXP, CFG, check, read, save, digest, budget
from timing_queue import TimingQueue, audit_sequence
def transport(generated, delay):
if generated.ndim != 2 or generated.dtype != np.float32:
raise ValueError('Expected stored float32 generated h')
queue = TimingQueue(delay, generated.shape[1])
applied = np.empty_like(generated)
for tick, row in enumerate(generated):
applied[tick] = queue.push(row, tick)
pending = queue.pending()
report = audit_sequence(generated, applied, pending, delay)
report.update(queue.metadata())
report['applied_bytes_sha256'] = hashlib.sha256(applied.tobytes()).hexdigest()
report['pending_bytes_sha256'] = hashlib.sha256(pending.tobytes()).hexdigest()
return applied, pending, report
def main():
check()
assert read(EXP / 'origin-audit/summary.json')['pass_all']
names = CFG['fixtures'] + [r['name'] for r in read(EXP / 'protocol/calibration-cases.json')]
records = []
for name in names:
path = AG / 'gpu' / (name + '-r0') / 'apl-inhibition.float32.npy'
generated = np.load(path, mmap_mode='r', allow_pickle=False)
assert generated.shape == (read(AG / 'cases' / name / 'spec.json')['steps'], 4064)
for delay in CFG['delays_steps']:
_, _, report = transport(generated, delay)
records.append(dict(case=name, delay_steps=delay, input_sha256=digest(path), **report))
save(EXP / 'audit/shift-gate.json', dict(pass_all=all(r['pass_all'] for r in records),
records=records, input_trace_count=len(names), no_new_brain_trajectory=True,
no_future_APL_advance=True, closed_loop_equal_integral_claim=False,
preservation='Every generated finite-window sample retained bitwise in applied or chronological pending slots; initial padding is +0'))
budget()
print('PASS: 68 fixed generated traces, 2 exact shifts; no new brain trajectories', flush=True)
if __name__ == '__main__':
main()
models/male-cns-lif/apl-timing-lab-v1/test_al_origin_audit.py
+196
View File
@@ -0,0 +1,196 @@
"""Pure synthetic selection, temporal joining, and streaming observer tests."""
import unittest
from types import SimpleNamespace
import importlib.util
import ast
from pathlib import Path
import numpy as np
import pandas as pd
from al_origin_audit import (select_cohort, predecessor_counts, first_ticks,
source_labels, ALLNObserver, _window_add, bit_equal)
class MemorySink:
def __init__(self):
self.rows = 0
self.calls = []
def write(self, data):
self.rows += len(data['edge_id'])
self.calls.append({k: np.asarray(v).copy() for k, v in data.items()})
class OriginTests(unittest.TestCase):
def test_main_does_not_shadow_callable_helpers(self):
tree = ast.parse((Path(__file__).parent / 'al_origin_audit.py').read_text(encoding='utf-8'))
helpers = {node.name for node in tree.body if isinstance(node, (ast.FunctionDef, ast.ClassDef))}
main = next(node for node in tree.body if isinstance(node, ast.FunctionDef) and node.name == 'main')
assigned = {node.id for node in ast.walk(main) if isinstance(node, ast.Name) and isinstance(node.ctx, ast.Store)}
self.assertEqual(helpers & assigned, set())
def test_all_earliest_ties_and_static_design(self):
ids = np.arange(5, dtype=np.int32)
result = select_cohort(ids, np.array([1, 5, 5, 8, -1], np.int32),
np.array([0], np.int32), 20, 30)
np.testing.assert_array_equal(result['cohort'], [1, 2])
self.assertEqual(result['matched_stop_exclusive'], 5)
self.assertEqual(result['cohort_relation_to_first_KC'], 'before')
def test_no_non_design_activity_does_not_choose_never_active(self):
result = select_cohort(np.arange(3, dtype=np.int32), np.array([0, -1, -1], np.int32),
np.array([0], np.int32), 20, 30)
self.assertEqual(len(result['cohort']), 0)
self.assertEqual(result['earliest_tick'], -1)
def test_same_tick_is_not_before(self):
result = select_cohort(np.arange(2, dtype=np.int32), np.array([0, 20], np.int32),
np.array([0], np.int32), 20, 30)
self.assertFalse(result['cohort_before_first_KC'])
self.assertEqual(result['cohort_relation_to_first_KC'], 'same_tick')
def test_KC_absent_is_censored(self):
result = select_cohort(np.arange(2, dtype=np.int32), np.array([0, 20], np.int32),
np.array([0], np.int32), -1, 30)
self.assertTrue(result['pre_kc_window_censored'])
self.assertEqual(result['pre_kc_stop_exclusive'], 30)
self.assertFalse(result['cohort_before_first_KC'])
def test_after_KC_retains_cohort_without_selecting_a_different_one(self):
result = select_cohort(np.arange(2, dtype=np.int32), np.array([0, 20], np.int32),
np.array([0], np.int32), 10, 30)
self.assertEqual(result['cohort'].tolist(), [1])
self.assertEqual(result['cohort_relation_to_first_KC'], 'after')
self.assertEqual(result['matched_stop_exclusive'], 10)
def test_temporal_join_excludes_both_boundaries_and_prior_reset(self):
result = predecessor_counts(np.array([3, 5, 6, 9, 10, 11]),
np.array([1., 2., -3., 4., 5., 6.]), 10, 5)
self.assertEqual(result['all_prior_design_PN_events'], 4)
self.assertEqual(result['since_last_reset_design_PN_events'], 2)
self.assertEqual(result['prior_reset_excluded_events'], 2)
self.assertEqual(result['since_last_reset_positive_weight_SI'], 4.)
self.assertEqual(result['since_last_reset_negative_abs_weight_SI'], 3.)
def test_zero_weight_not_transmission(self):
result = predecessor_counts(np.array([2]), np.array([0.]), 3, -1)
self.assertEqual(result['since_last_reset_design_PN_events'], 1)
self.assertEqual(result['since_last_reset_nonzero_design_PN_events'], 0)
def test_no_direct_antecedent_valid_empty(self):
result = predecessor_counts(np.array([], np.int32), np.array([], np.float64), 20, -1)
self.assertEqual(result['since_last_reset_design_PN_events'], 0)
def test_reject_bad_temporal_order(self):
with self.assertRaises(ValueError):
predecessor_counts(np.array([3, 1]), np.ones(2), 10, 5)
with self.assertRaises(ValueError):
predecessor_counts(np.array([3]), np.ones(1), 10, 10)
def test_first_ticks_missing_and_repeated_events(self):
values = first_ticks(np.array([1, 3, 3], np.int32), np.array([2, 1, 2], np.int32),
np.arange(4, dtype=np.int32), n=4)
np.testing.assert_array_equal(values, [-1, 3, 1, -1])
def test_exact_role_mapping_unknown_stays_unknown(self):
labels = source_labels(np.arange(4), np.array(['ALPN', 'ALPN', 'ALLN', 'unclassified']),
np.array([True, False, False, False]))
self.assertEqual(labels.tolist(), ['design_ALPN', 'non_design_ALPN', 'ALLN', 'unclassified'])
def test_window_aggregation_preserves_sign_and_exclusion(self):
stats = {}
_window_add(stats, pd.DataFrame({'weight_SI': [1., -2., 9.]}),
np.array([True, True, False]), np.array(['ALLN'] * 3))
self.assertEqual(stats['ALLN']['events'], 2)
self.assertEqual(stats['ALLN']['positive_weight_SI'], 1.)
self.assertEqual(stats['ALLN']['negative_abs_weight_SI'], 2.)
def test_bit_equal_rejects_signed_zero(self):
self.assertFalse(bit_equal(np.array([0.]), np.array([-0.])))
@staticmethod
def observer():
sink = MemorySink()
incoming = SimpleNamespace(src_global=np.array([0, 1, 2], np.int32),
dst_local=np.array([0, 0, 1], np.int32), edge_id=np.arange(3, dtype=np.int64),
weights=np.array([.1, 0., -.2], np.float64))
observer = ALLNObserver(np.array([3, 4], np.int32), incoming,
np.array(['ALPN', 'APL', 'ALLN', 'ALLN', 'ALLN']),
np.array([True, False, False, False, False]), np.arange(5, dtype=np.uint64),
19, 19, sink)
return observer, sink
@staticmethod
def stage(due, accepted, ref=None):
return dict(h32=np.zeros(2, np.float32), delta_si=np.zeros(2), request=np.zeros(2, bool),
g=np.zeros(2), g_pre_apl=np.zeros(2), due_edge_indices=np.asarray(due, np.int64),
arrival_accepted=np.asarray(accepted, bool),
ref_after_reset=np.zeros(2, np.int32) if ref is None else ref,
fired=np.zeros(2, bool))
def test_observer_all_accepted_signs_and_zeros(self):
observer, sink = self.observer()
for tick in range(18):
observer(tick, self.stage([], []))
observer(18, self.stage([0, 1, 2], [True, True, True]))
report = observer.report()
self.assertEqual(sink.rows, 3)
self.assertEqual(report['source_groups']['APL']['zero'], 1)
self.assertEqual(sink.calls[-1]['source_spike_tick'].tolist(), [0, 0, 0])
self.assertEqual(observer.design_ticks[3], [18])
self.assertEqual(sink.calls[-1]['weight_sign'].tolist(), [1, 0, -1])
def test_observer_ref_guard_and_no_rejected_records(self):
observer, sink = self.observer()
for tick in range(18):
observer(tick, self.stage([], []))
stage = self.stage([0, 1, 2], [False, False, True], np.array([22, 0], np.int32))
stage['fired'][0] = True
observer(18, stage)
self.assertEqual(sink.rows, 1)
self.assertEqual(observer.report()['rejected_events'], 2)
self.assertEqual(observer.last.tolist(), [18, -1])
def test_observer_bad_acceptance_raises(self):
observer, _ = self.observer()
for tick in range(18):
observer(tick, self.stage([], []))
with self.assertRaises(ValueError):
observer(18, self.stage([0], [True], np.array([22, 0], np.int32)))
def test_observer_rejects_unexpected_input_and_graded_APL(self):
observer, _ = self.observer()
stage = self.stage([], [])
stage['request'][0] = True
with self.assertRaises(ValueError):
observer(0, stage)
stage['request'][0] = False
stage['h32'][0] = 1.
with self.assertRaises(ValueError):
observer(0, stage)
def test_frozen_replay_core_synthetic_integration(self):
module_spec = importlib.util.spec_from_file_location('origin_test_frozen_core',
Path(__file__).resolve().parent.parent / 'kc-drive-audit-v1/replay_core.py')
core = importlib.util.module_from_spec(module_spec)
module_spec.loader.exec_module(core)
incoming = core.IncomingIndex(np.array([0], np.int32), np.array([0], np.int32),
np.array([0], np.int64), np.array([1.], np.float64), 2)
sink = MemorySink()
observer = ALLNObserver(np.array([3, 4], np.int32), incoming,
np.array(['ALPN', 'APL', 'ALLN', 'ALLN', 'ALLN']),
np.array([True, False, False, False, False]), np.arange(5, dtype=np.uint64), 40, 30, sink)
coeff = dict(gd=.98, c0=-.00052, cg=.005, cv=.99, threshold=-.045, reset=-.052, seed_volt=-.044)
result = core.replay(np.full(2, -.052), np.zeros(2), coeff, 40, np.array([3, 4], np.int32),
dict(tick=np.array([], np.int32), neuron_idx=np.array([], np.int32)),
dict(step=np.array([0], np.int32), neuron_idx=np.array([0], np.int32)), incoming,
np.zeros((40, 2), np.float32), observer)
self.assertEqual(observer.report()['accepted_events'], 1)
self.assertGreater(int(result['first_spike'][0]), 18)
self.assertEqual(int(result['first_spike'][1]), -1)
self.assertEqual(observer.design_ticks[3], [18])
nonempty = [call for call in sink.calls if len(call['edge_id'])]
self.assertEqual(nonempty[0]['target_previous_spike_tick'].tolist(), [-1])
if __name__ == '__main__':
unittest.main()
models/male-cns-lif/apl-timing-lab-v1/test_calibration.py
+244
View File
@@ -0,0 +1,244 @@
"""Synthetic CPU/file-fixture tests only; no real records, Torch, or Brian."""
import copy
import json
from pathlib import Path
import tempfile
import types
import unittest
from unittest.mock import patch
import numpy as np
import pandas as pd
import calibration as cal
HERE = Path(__file__).resolve().parent
LEGACY = cal.load_legacy(HERE.parent / "author-apl-lab-v2/calibration.py")
CFG = json.loads((HERE / "config.json").read_text(encoding="utf-8"))
def save_json(path, value):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(value, allow_nan=False), encoding="utf-8")
def records():
odors = sorted(LEGACY.ORIGINAL_CAL_IDS)
return [dict(name=f"cal-{mode}-{index:02d}-s{seed}", mode=mode, odor_index=index,
odor_id=odor, seed=seed, requests=0)
for mode in LEGACY.ORIGINAL_MODES for index, odor in enumerate(odors) for seed in LEGACY.ORIGINAL_SEEDS]
def timing_files(directory, delay, steps=20):
generated = np.arange(steps * 4064, dtype=np.float32).reshape(steps, 4064) / np.float32(10000)
stream = np.concatenate((np.zeros((delay, 4064), np.float32), generated))
applied, pending = stream[:steps].copy(), stream[steps:].copy()
state = np.arange(steps * 2, dtype=np.float32).reshape(steps, 2)
np.save(directory / "apl-applied.float32.npy", applied)
np.save(directory / "apl-pending-tail.float32.npy", pending)
audit = dict(pass_all=True, apl_delay_steps=delay, steps=steps, pending_generated_count=min(steps, delay),
pending_initial_zero_prefix=max(delay-steps, 0), pending_application_tick_start=steps,
pending_application_tick_stop_exclusive=steps+delay, pending_generated_tick_start=max(0, steps-delay),
pending_generated_tick_stop_exclusive=steps, initial_padding_positive_zero=True, sample_bits_preserved=True,
brain_tail_advanced=False, generated_trace="apl-inhibition.float32.npy", applied_trace="apl-applied.float32.npy",
pending_trace="apl-pending-tail.float32.npy", pending_raw_sha256=cal.raw_hash(pending),
pending_queue_drained=delay == 0, finite_generated_sequence_only=True,
future_apl_decay_or_feedback_generated=False, closed_loop_branch_integrals_claimed_equal=False,
biological_delay_measurement_claimed=False, final_apl_state_float32=state[-1].tolist(),
final_apl_state_u32_bits=state[-1].view(np.uint32).tolist(), final_apl_state_nonzero=bool(np.any(state[-1] != 0)))
save_json(directory / "apl-timing-audit.json", audit)
save_json(directory / "summary.json", dict(apl_delay_steps=delay, apl_timing=audit))
trial = dict(apl={"apl-inhibition.float32.npy": generated, "apl-state.float32.npy": state})
return dict(steps=steps, apl_delay_steps=delay), trial
def control_files(root, record_history=True):
new, old, case = root / "new", root / "old", root / "case"
for directory in (new, old, case):
directory.mkdir()
spec = dict(name="synthetic", steps=3, apl_delay_steps=0, record_history=record_history)
save_json(case / "spec.json", spec)
for index, directory in enumerate((new, old)):
summary = dict(case="synthetic", steps=3, completed_ticks=3, complete=True, pass_all=True,
nodes=2, edges=3, precision="float64", apl_fast_edges_zeroed=1, kc_targets_present=1,
effective_weights_raw_sha256="a" * 64, build_wall_seconds=index,
input_sha256={"spec.json": str(index)}, output_budget_bytes=123+index)
save_json(directory / "summary.json", summary)
state = {key: np.zeros(0 if key in ("step", "neuron_idx") else 2, dtype) for key, dtype in cal.STATE_DTYPES.items()}
np.savez(directory / "state.npz", **state)
for name, width in cal.APL_FILES.items():
np.save(directory / name, np.zeros((3, width), np.float32))
for name in ("tick-state-hashes.jsonl", "queue.jsonl"):
(directory / name).write_text("\n".join(json.dumps(dict(tick=t, same=True)) for t in range(3)), encoding="utf-8")
(directory / "requests.csv").write_text(",".join(cal.REQUEST_FIELDS) + "\n", encoding="utf-8")
save_json(directory / "apl-component.json", dict(ok=True, environment={"synthetic": True}))
if record_history:
for key, dtype in (("v", "float64"), ("g", "float64"), ("ref", "int32")):
for name in (f"{key}-history.{dtype}.npy", f"preAPL-{key}.{dtype}.npy"):
np.save(directory / name, np.zeros((3, 2), dtype))
return new, old, case
def fake_trial(output, row, delay):
state = {name: np.zeros(1, dtype) for name, dtype in cal.STATE_DTYPES.items()}
counts = np.zeros(1000, np.int32)
if delay == 18:
counts[row["odor_index"] * 50:(row["odor_index"] + 1) * 50] = 1
return dict(state=state, apl={name: np.zeros((1, width), np.float32) for name, width in cal.APL_FILES.items()},
hashes=[], queues=[], requests=[], component={}, kc_counts=counts, expected_external_events=10., external_events=0,
record=dict(case=row["name"], requested=0), output=output,
timing=dict(applied=np.zeros((1, 4064), np.float32), pending=np.zeros((delay, 4064), np.float32),
audit=dict(synthetic=True), report=dict(pass_all=True)))
class CalibrationTimingTests(unittest.TestCase):
def test_all_original_inventory_and_pair_specific_gates_are_reused(self):
rows = records()
odors = LEGACY.validate_inventory(rows, CFG)
runs = {(r["mode"], r["odor_id"], r["seed"]): fake_trial(Path("synthetic"), r, 18) for r in rows}
report = LEGACY.summarize_mode(LEGACY.ORIGINAL_MODES[0], odors, LEGACY.ORIGINAL_SEEDS, runs, CFG["gates"])
self.assertTrue(report["gate"])
self.assertEqual((report["within_distance_count"], report["between_distance_count"], report["pair_count"]), (60, 75, 15))
self.assertTrue(all(r["within_distance_count"] == 20 and r["between_distance_count"] == 5 for r in report["pairs"]))
self.assertFalse(hasattr(LEGACY, "EXP"))
with self.assertRaises(ValueError):
LEGACY.validate_inventory(rows[:-1], CFG)
def test_direct_slices_cover_zero_delay_boundary_and_short_window(self):
for delay in (0, 18):
for steps in (1, 17, 18, 19, 37):
with self.subTest(delay=delay, steps=steps), tempfile.TemporaryDirectory() as tmp:
spec, trial = timing_files(Path(tmp), delay, steps)
result = cal.validate_timing(Path(tmp), spec, trial)
try:
self.assertTrue(result["report"]["pass_all"])
self.assertEqual(result["pending"].shape, (delay, 4064))
finally:
result["applied"]._mmap.close()
result["pending"]._mmap.close()
def test_tail_or_application_bit_corruption_fails(self):
for filename in ("apl-applied.float32.npy", "apl-pending-tail.float32.npy"):
with self.subTest(filename=filename), tempfile.TemporaryDirectory() as tmp:
root = Path(tmp); spec, trial = timing_files(root, 18)
value = np.load(root / filename)
value[-1, 1] = np.nextafter(value[-1, 1], np.float32(np.inf))
np.save(root / filename, value)
with self.assertRaisesRegex(ValueError, "partition"):
cal.validate_timing(root, spec, trial)
def test_negative_zero_padding_and_fabricated_future_audit_fail(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp); spec, trial = timing_files(root, 18, 1)
value = np.load(root / "apl-applied.float32.npy")
value[0, 0] = np.float32(-0.)
np.save(root / "apl-applied.float32.npy", value)
with self.assertRaisesRegex(ValueError, "positive zero"):
cal.validate_timing(root, spec, trial)
for key, value in (("brain_tail_advanced", True), ("final_apl_state_nonzero", False),
("pending_initial_zero_prefix", 1), ("pass_all", 1)):
with self.subTest(key=key), tempfile.TemporaryDirectory() as tmp:
root = Path(tmp); spec, trial = timing_files(root, 18, 20)
audit = cal.read_json(root / "apl-timing-audit.json"); audit[key] = value
save_json(root / "apl-timing-audit.json", audit)
with self.assertRaisesRegex(ValueError, "audit disagrees"):
cal.validate_timing(root, spec, trial)
def test_delay0_control_ignores_runtime_and_budget_metadata(self):
with tempfile.TemporaryDirectory() as tmp:
new, old, case = control_files(Path(tmp))
result = cal.control_identity(new, old, case)
self.assertTrue(result["pass_all"])
self.assertFalse(result["adapter_paths_budgets_and_runtime_metadata_compared"])
self.assertIn("preAPL-g.float64.npy", result["checks"])
def test_control_signed_zero_history_and_missing_trace_fail(self):
with tempfile.TemporaryDirectory() as tmp:
new, old, case = control_files(Path(tmp))
with np.load(new / "state.npz") as archive:
state = {key: archive[key].copy() for key in archive.files}
state["v"][0] = -0.
np.savez(new / "state.npz", **state)
result = cal.control_identity(new, old, case)
self.assertFalse(result["checks"]["state/v"])
self.assertFalse(result["pass_all"])
with tempfile.TemporaryDirectory() as tmp:
new, old, case = control_files(Path(tmp))
np.save(new / "preAPL-g.float64.npy", np.ones((3, 2), np.float64))
self.assertFalse(cal.control_identity(new, old, case)["pass_all"])
(new / "apl-drive.float32.npy").unlink()
with self.assertRaises(FileNotFoundError):
cal.control_identity(new, old, case)
def test_control_rejects_delay18_and_incomplete_results(self):
with tempfile.TemporaryDirectory() as tmp:
new, old, case = control_files(Path(tmp), False)
spec = cal.read_json(case / "spec.json"); spec["apl_delay_steps"] = 18
save_json(case / "spec.json", spec)
with self.assertRaisesRegex(ValueError, "only for explicit delay0"):
cal.control_identity(new, old, case)
spec["apl_delay_steps"] = 0; save_json(case / "spec.json", spec)
summary = cal.read_json(new / "summary.json"); summary["complete"] = False
save_json(new / "summary.json", summary)
with self.assertRaisesRegex(ValueError, "incomplete"):
cal.control_identity(new, old, case)
def test_repeat_includes_applied_pending_and_audit(self):
trial = fake_trial(Path("synthetic"), records()[0], 18)
self.assertTrue(cal.timing_repeat(trial, copy.deepcopy(trial), LEGACY)["pass_all"])
for key in ("applied", "pending", "audit"):
bad = copy.deepcopy(trial)
if key == "audit": bad["timing"][key]["synthetic"] = False
else: bad["timing"][key][0, 0] = 1
self.assertFalse(cal.timing_repeat(trial, bad, LEGACY)["pass_all"])
def test_both_arms_keep_full_inventory_despite_control_repr_failure(self):
rows = records(); odors = LEGACY.validate_inventory(rows, CFG)
by_name = {row["name"]: row for row in rows}
calls = []
def read_new(legacy, output, case, spec, kc, cfg):
calls.append(str(output))
return fake_trial(output, by_name[case.name], spec["apl_delay_steps"])
with patch.object(cal, "_case_spec", side_effect=lambda case, old, row, delay: dict(apl_delay_steps=delay)), \
patch.object(cal, "_read_new", side_effect=read_new), \
patch.object(cal, "_provenance", return_value={}), \
patch.object(cal, "control_identity", return_value=dict(case="synthetic", pass_all=True, checks={})):
reports = [cal.collect_arm("delay"+str(delay), delay, rows, odors, LEGACY, CFG,
Path("synthetic-exp"), Path("synthetic-ag"), np.arange(4064, dtype=np.int32)) for delay in (0,18)]
self.assertEqual(len(calls), 144)
self.assertTrue(all(r["execution_gate_pass"] for r in reports))
self.assertFalse(reports[0]["original_representation_gate_pass"])
self.assertTrue(reports[1]["original_representation_gate_pass"])
self.assertEqual([r["valid_base_trials"] for r in reports], [60, 60])
self.assertEqual([r["checked_repeats"] for r in reports], [12, 12])
self.assertEqual(len(reports[0]["control_checks"]), 72)
def test_none_recomputation_is_deterministic_and_never_writes(self):
rows = records(); odors = sorted(LEGACY.ORIGINAL_CAL_IDS)
panel = pd.DataFrame(dict(split=["calibration"]*6+["heldout"]*6,
odor_id=odors+["static-only-"+str(n) for n in range(6)], mapped_response_sum=[1.]*12))
encoder = dict(panel_count=12, cal_count=6, seeds=list(LEGACY.ORIGINAL_SEEDS),
requested_before_any_dynamics=True, formal_AB_run=False, heldout_run=False, median_mapped_response_sum=1.)
with tempfile.TemporaryDirectory() as tmp:
ctx = types.ModuleType("context")
ctx.EXP, ctx.AG, ctx.ROOT, ctx.CFG, ctx.check = Path(tmp)/"exp", Path(tmp)/"ag", Path(tmp), CFG, lambda: None
def fake_arm(arm, *args):
return dict(arm=arm, numerical_integrity_pass=True, exact_repeat_pass=True,
delay0_control_identity_pass=True if arm == "delay0" else None,
original_representation_gate_pass=False, failures=[], modes={"physiological": {"gate": False}})
with patch.dict("sys.modules", {"context": ctx}), patch.object(cal, "load_legacy", return_value=LEGACY), \
patch.object(cal, "read_json", side_effect=lambda p: rows if Path(p).name == "calibration-cases.json" else encoder), \
patch.object(cal, "file_hash", return_value="same"), patch.object(pd, "read_parquet", return_value=panel), \
patch.object(cal.np, "load", return_value=np.arange(4064, dtype=np.int32)), \
patch.object(cal, "collect_arm", side_effect=fake_arm):
first, second = cal.summarize(None), cal.summarize(None)
self.assertEqual(first, second)
self.assertTrue(first["execution_gate_pass"])
self.assertFalse(first["pass_all"])
json.dumps(first, allow_nan=False)
self.assertEqual(list(Path(tmp).iterdir()), [])
if __name__ == "__main__":
unittest.main()
models/male-cns-lif/apl-timing-lab-v1/test_protocol.py
+199
View File
@@ -0,0 +1,199 @@
"""Synthetic orchestration/producer-contract checks; never launch a brain."""
import ast
import json
from pathlib import Path
import sys
import tempfile
from types import ModuleType
import unittest
from unittest.mock import patch
import numpy as np
import compare_timing
import oracle_timing
import run_suite
from shift_audit import transport
from timing_queue import audit_sequence
HERE = Path(__file__).resolve().parent
def fake_calibration_records():
return [dict(name=f'cal-{mode}-{odor:02d}-s{seed}', mode=mode, odor_id='odor'+str(odor), seed=seed)
for mode in ('physiological', 'matched_energy') for odor in range(6)
for seed in range(51101, 51106)]
def oracle_timing_dict(delay, generated, pending, final_state):
"""Evaluate only the actual producer's explicit metadata dict, no run call."""
tree = ast.parse((HERE / 'oracle_timing.py').read_text(encoding='utf-8'))
calls = [n for n in ast.walk(tree) if isinstance(n, ast.Call) and isinstance(n.func, ast.Name)
and n.func.id == '_save' and len(n.args) > 1 and isinstance(n.args[1], ast.Dict)
and any(isinstance(s, ast.Constant) and s.value == 'apl-timing-audit.json'
for s in ast.walk(n.args[0]))]
if len(calls) != 1:
raise AssertionError('Expected exactly one explicit oracle timing-metadata producer')
steps = len(generated)
namespace = dict(np=np, min=min, max=max, bool=bool, apl_delay_steps=delay, steps=steps,
consumed=max(steps-delay, 0), initial_zero_prefix=max(delay-steps, 0), pending_tail=pending,
apl_history={'state': np.asarray([final_state], np.float32)}, out_dir=Path('.'),
_array_hash=oracle_timing._array_hash, _digest=lambda _: '0'*64)
return eval(compile(ast.Expression(calls[0].args[1]), '<only metadata literal>', 'eval'), namespace)
class ProtocolTests(unittest.TestCase):
def test_two_independent_queues_match_both_arms_and_short_windows(self):
for delay in (0, 18):
for steps in (0, 1, 17, 18, 19, 37):
with self.subTest(delay=delay, steps=steps):
generated = ((np.arange(steps * 4064).reshape(steps, 4064) % 31)
.astype(np.float32) * np.float32(.001))
if steps:
generated[0, 0] = np.float32(-0.)
applied, pending, report = transport(generated, delay)
independent = oracle_timing._APLDelayFIFO(delay)
expected = np.empty_like(generated)
for tick, row in enumerate(generated):
expected[tick] = independent.step(tick, row)
tail = independent.finish(steps)
np.testing.assert_array_equal(applied.view(np.uint32), expected.view(np.uint32))
np.testing.assert_array_equal(pending.view(np.uint32), tail.view(np.uint32))
self.assertTrue(report['pass_all'])
def test_queue_rejects_off_by_one_and_lost_pending(self):
values = np.ones((19, 2), np.float32)
applied, pending, _ = transport(values, 18)
corrupt = applied.copy()
corrupt[17] = 1
with self.assertRaises(ValueError):
audit_sequence(values, corrupt, pending, 18)
with self.assertRaises(ValueError):
audit_sequence(values, applied, pending[:-1], 18)
def test_actual_oracle_metadata_matches_compare_contract(self):
for delay in (0, 18):
for steps in (1, 17, 18, 19, 37):
with self.subTest(delay=delay, steps=steps):
generated = np.full((steps, 4064), .25, np.float32)
_, pending, _ = transport(generated, delay)
final = np.array([.125, -0.], np.float32)
timing = oracle_timing_dict(delay, generated, pending, final)
summary = dict(pending_generated_count=min(delay, steps),
pending_initial_zero_prefix=max(delay-steps, 0), pending_tail_shape=[delay, 4064],
future_generated_decay_tail_included=False, APL_worker_ticks=steps)
run = dict(timing=timing, summary=summary, pending=pending,
apl={'state': np.asarray([final], np.float32)})
checks = {}
compare_timing._timing_metadata_checks(run, 'cpu', delay, steps, checks)
self.assertTrue(all(checks.values()), checks)
def test_bad_metadata_must_fail_without_fitted_tolerance(self):
generated = np.ones((3, 4064), np.float32)
_, pending, _ = transport(generated, 18)
final = np.array([.125, 0.], np.float32)
timing = oracle_timing_dict(18, generated, pending, final)
timing['pending_initial_zero_prefix'] += 1
run = dict(timing=timing, summary={}, pending=pending, apl={'state': np.asarray([final])})
checks = {}
compare_timing._timing_metadata_checks(run, 'cpu', 18, 3, checks)
self.assertFalse(checks['cpu.timing_integer_metadata'])
def _suite(self, stage, *, failed_read=None, execution=True, representation=False,
control=True, repeat=True):
records = fake_calibration_records()
saved, children = {}, []
def read(path):
path = str(path).replace('\\', '/')
if path.endswith('calibration-cases.json'):
return records
return {'pass_all': not (failed_read and path.endswith(failed_read))}
fake = ModuleType('calibration')
fake.summarize = lambda: dict(execution_gate_pass=execution, pass_all=representation)
fake.control_identity = lambda *args: dict(pass_all=control)
with tempfile.TemporaryDirectory() as directory:
with patch.object(run_suite, 'EXP', Path(directory)), patch.object(run_suite, 'check'), \
patch.object(run_suite, 'budget'), patch.object(run_suite, 'read', side_effect=read), \
patch.object(run_suite, 'save', side_effect=lambda path, result: saved.update({Path(path).name: result})), \
patch.object(run_suite, 'child', side_effect=lambda *args: children.append(args)), \
patch.object(run_suite, 'gate_case', side_effect=lambda arm, name, repeats:
dict(arm=arm, case=name, pass_all=True, gpu_repeats=repeats)), \
patch.object(compare_timing, 'compare_gpu_repeats', return_value=dict(pass_all=repeat)), \
patch.dict(sys.modules, {'calibration': fake}), patch.object(sys, 'argv', ['run_suite.py', stage]):
error = None
try:
run_suite.main()
except AssertionError as caught:
error = caught
return saved, children, error
def test_fixture_gate_covers_both_arms_all_cases(self):
saved, children, error = self._suite('fixtures')
self.assertIsNone(error)
self.assertEqual(children, [])
rows = saved['fixtures-gate.json']['cases']
self.assertEqual(len(rows), 16)
self.assertEqual({r['arm'] for r in rows}, {'delay0', 'delay18'})
self.assertTrue(all(r['gpu_repeats'] == 3 for r in rows))
def test_probe_stage_cannot_bypass_fixture_failure(self):
saved, children, error = self._suite('probes', failed_read='fixtures-gate.json')
self.assertIsNotNone(error)
self.assertEqual(children, [])
self.assertNotIn('probes-gate.json', saved)
def test_CAL_cannot_bypass_probe_failure(self):
saved, children, error = self._suite('calibration', failed_read='probes-gate.json')
self.assertIsNotNone(error)
self.assertEqual(children, [])
self.assertNotIn('calibration-gate.json', saved)
def test_representation_failure_does_not_select_or_stop_an_arm(self):
saved, children, error = self._suite('calibration', execution=True, representation=False)
self.assertIsNone(error)
self.assertEqual(len(children), 140) # (60 - 2 already-probed + 12 repeat) per arm.
self.assertEqual({row[1] for row in children}, {'delay0', 'delay18'})
self.assertTrue(saved['calibration-gate.json']['pass_all'])
self.assertFalse(saved['calibration-gate.json']['representation_pass_all'])
for arm in ('delay0', 'delay18'):
self.assertEqual(sum(row[1] == arm and len(row) == 4 and row[3] == 1 for row in children), 12)
def test_CAL_numerical_failure_stops_later_execution(self):
saved, _, error = self._suite('calibration', execution=False, representation=False)
self.assertIsNotNone(error)
self.assertFalse(saved['calibration-gate.json']['pass_all'])
self.assertTrue(saved['stopped-calibration.json']['later_stages_not_permitted'])
def test_control_divergence_stops_before_next_trial_or_other_arm(self):
saved, children, error = self._suite('calibration', control=False)
self.assertIsNotNone(error)
# The first r0 is a completed fullgraph probe; do not even run its r1.
self.assertEqual(children, [])
self.assertNotIn('calibration-gate.json', saved)
self.assertTrue(saved['stopped-calibration.json']['later_stages_not_permitted'])
def test_repeat_divergence_stops_before_next_trial_or_other_arm(self):
saved, children, error = self._suite('calibration', repeat=False)
self.assertIsNotNone(error)
self.assertEqual(len(children), 1)
self.assertEqual(children[0][1], 'delay0')
self.assertEqual(children[0][-1], 1)
self.assertNotIn('calibration-gate.json', saved)
self.assertTrue(saved['stopped-calibration.json']['later_stages_not_permitted'])
def test_origin_failure_prevents_any_new_fixture_run(self):
saved, children, error = self._suite('fixtures', failed_read='origin-audit/summary.json')
self.assertIsNotNone(error)
self.assertEqual(saved, {})
self.assertEqual(children, [])
def test_shift_failure_prevents_any_new_fixture_run(self):
saved, children, error = self._suite('fixtures', failed_read='shift-gate.json')
self.assertIsNotNone(error)
self.assertEqual(saved, {})
self.assertEqual(children, [])
if __name__ == '__main__':
unittest.main()
models/male-cns-lif/apl-timing-lab-v1/tests/__init__.py
+1
View File
@@ -0,0 +1 @@
"""Discoverable synthetic tests; no scientific execution during imports."""
models/male-cns-lif/apl-timing-lab-v1/tests/test_compare_timing.py
+169
View File
@@ -0,0 +1,169 @@
"""Synthetic array-only comparator tests: no backend or scientific inputs."""
from pathlib import Path
import copy
import sys
import unittest
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
import compare_timing as compare
def traces(delay, steps, width=3):
generated = np.arange(steps * width, dtype=np.float32).reshape(steps, width) * np.float32(.125)
full = np.concatenate((np.zeros((delay, width), dtype=np.float32), generated))
return generated, full[:steps].copy(), full[steps:].copy()
def sequence_result(generated, applied, pending, delay):
checks, differences = {}, {}
compare._timing_sequence_checks(generated, applied, pending, delay, 'test', checks, differences)
return checks, differences
def metadata_run(delay, steps, cpu=False):
generated, applied, pending = traces(delay, steps, 4064)
state = np.tile(np.array([.25, 1.125], np.float32), (steps, 1))
timing = dict(apl_delay_steps=delay, steps=steps, pending_generated_count=min(steps, delay),
generated_rows=steps, applied_generated_rows=max(steps-delay, 0), pending_shape=[delay, 4064],
pending_initial_zero_prefix=max(delay-steps, 0),
pending_application_tick_start=steps, pending_application_tick_stop_exclusive=steps+delay,
pending_generated_tick_start=max(steps-delay, 0), pending_generated_tick_stop_exclusive=steps,
pending_raw_sha256=compare.raw_hash(pending, '<f4'),
final_apl_state_float32=state[-1].tolist(),
final_apl_state_u32_bits=state[-1].view(np.uint32).tolist(), final_apl_state_nonzero=True,
brain_tail_advanced=False, future_apl_decay_or_feedback_generated=False,
finite_generated_sequence_only=True, closed_loop_branch_integrals_claimed_equal=False,
biological_delay_measurement_claimed=False, brain_steps_after_window=0,
APL_generation_steps_after_window=0, future_generated_decay_tail_included=False)
summary = dict(apl_timing=copy.deepcopy(timing), pending_generated_count=min(steps, delay),
pending_initial_zero_prefix=max(delay-steps, 0), pending_tail_shape=[delay, 4064],
future_generated_decay_tail_included=False, APL_worker_ticks=steps)
return dict(apl={'inhibition': generated, 'applied': applied, 'state': state}, pending=pending,
timing=timing, summary=summary)
class TimingComparisonTests(unittest.TestCase):
def test_delay_zero_and_18_partition(self):
for delay in (0, 18):
for steps in (1, 7, 18, 19, 54):
with self.subTest(delay=delay, steps=steps):
checks, differences = sequence_result(*traces(delay, steps), delay)
self.assertTrue(all(checks.values()), differences)
def test_one_tick_shift_rejected_independently(self):
generated, applied, pending = traces(18, 40)
applied[18:] = generated[1:23]
checks, _ = sequence_result(generated, applied, pending, 18)
self.assertFalse(checks['test.fixed_delay_applied.bitwise_exact'])
def test_pending_wrong_chronology_rejected(self):
generated, applied, pending = traces(18, 40)
checks, _ = sequence_result(generated, applied, pending[::-1].copy(), 18)
self.assertFalse(checks['test.fixed_delay_pending.bitwise_exact'])
def test_initial_zero_must_be_positive_bits(self):
generated, applied, pending = traces(18, 4)
pending.view(np.uint32)[0, 0] = 0x80000000
checks, _ = sequence_result(generated, applied, pending, 18)
self.assertFalse(checks['test.fixed_delay_pending.bitwise_exact'])
generated, applied, pending = traces(18, 22)
applied.view(np.uint32)[0, 0] = 0x80000000
checks, _ = sequence_result(generated, applied, pending, 18)
self.assertFalse(checks['test.fixed_delay_applied.bitwise_exact'])
def test_generated_signed_zero_preserved(self):
generated, applied, pending = traces(0, 1)
generated.view(np.uint32)[0, 0] = 0x80000000
applied[:] = generated
checks, _ = sequence_result(generated, applied, pending, 0)
self.assertTrue(all(checks.values()))
applied.view(np.uint32)[0, 0] = 0
checks, _ = sequence_result(generated, applied, pending, 0)
self.assertFalse(checks['test.fixed_delay_applied.bitwise_exact'])
def test_schema_and_delay_fail_closed(self):
generated, applied, pending = traces(18, 4)
for delay in (True, 1, 18.0):
with self.assertRaises(ValueError):
sequence_result(generated, applied, pending, delay)
with self.assertRaises(ValueError):
sequence_result(generated.astype(np.float64), applied, pending, 18)
with self.assertRaises(ValueError):
sequence_result(generated, applied, pending[-4:], 18)
def test_nonfinite_or_negative_generated_fails_even_if_shift_matches(self):
for value in (np.inf, np.nan, -1):
generated, applied, pending = traces(0, 2)
generated[0, 0] = value
applied[:] = generated
checks, _ = sequence_result(generated, applied, pending, 0)
self.assertFalse(checks['test.timing_finite_nonnegative'])
def test_both_backend_metadata_valid(self):
for role in ('gpu', 'cpu', 'first_gpu', 'second_gpu'):
for delay, steps in ((0, 25), (18, 7), (18, 25)):
run = metadata_run(delay, steps, cpu=role == 'cpu')
checks = {}
compare._timing_metadata_checks(run, role, delay, steps, checks)
self.assertTrue(all(checks.values()), checks)
def test_metadata_rejects_wrong_count_tail_advance_and_final_state(self):
for role in ('gpu', 'cpu'):
for change in ('count', 'tail', 'final', 'prefix', 'hash'):
run = metadata_run(18, 25, cpu=role == 'cpu')
if change == 'count':
run['timing']['pending_generated_count'] = 17
elif change == 'tail':
run['timing']['brain_steps_after_window' if role == 'cpu' else 'brain_tail_advanced'] = 1
elif change == 'final':
run['timing']['final_apl_state_u32_bits'][0] += 1
elif change == 'prefix':
run['timing']['pending_initial_zero_prefix'] = False
else:
run['timing']['pending_raw_sha256'] = '0' * 64
checks = {}
compare._timing_metadata_checks(run, role, 18, 25, checks)
self.assertFalse(all(checks.values()), (role, change))
def test_g_write_uses_applied_and_is_unconditional_during_ref(self):
nodes, kc = np.array([10, 12], np.int32), np.array([10], np.int32)
v = np.full((3, 2), -.06, np.float64)
ref = np.array([[22, 0], [21, 0], [20, 0]], np.int32)
before = np.array([[0, -.001], [-0., .001], [.002, -.003]], np.float64)
applied = np.array([[.25], [0], [1]], np.float32)
after = before.copy()
after[:, 0] -= applied[:, 0].astype(np.float64) * np.float64(.001)
run = dict(pre={'v': v, 'g': before, 'ref': ref}, histories={'v': v.copy(), 'g': after, 'ref': ref.copy()},
apl={'applied': applied, 'inhibition': np.full((3, 1), 99, np.float32)})
checks, differences = {}, {}
compare._check_apl_application(run, 'test', nodes, kc, checks, differences)
self.assertTrue(all(checks.values()), differences)
run['histories']['g'] = before.copy() # Hypothetical refractory guard must fail.
checks, differences = {}, {}
compare._check_apl_application(run, 'test', nodes, kc, checks, differences)
self.assertFalse(checks['test.apl_unconditional_g_write.bitwise_exact'])
def test_pair_comparison_includes_applied_and_pending(self):
def output():
generated, applied, pending = traces(18, 20)
return dict(state={'step': np.empty(0, np.int32), 'neuron_idx': np.empty(0, np.int32),
'counts': np.zeros(1, np.int32), 'ref': np.zeros(1, np.int32),
'v': np.zeros(1, np.float64), 'g': np.zeros(1, np.float64)},
apl={'drive': np.zeros((20, 2), np.float32), 'state': np.zeros((20, 2), np.float32),
'inhibition': np.pad(generated, ((0, 0), (0, 4061))),
'applied': np.pad(applied, ((0, 0), (0, 4061)))},
pending=np.pad(pending, ((0, 0), (0, 4061))),
hashes=[], queues=[], requests=[], component={}, histories={}, pre={})
left, right = output(), output()
right['apl']['applied'][0, 0] = 1
right['pending'][0, 0] += 1
checks, differences = {}, {}
compare._pair_outputs(left, right, checks, differences, 1, 20, False)
self.assertFalse(checks['apl.applied.bitwise_exact'])
self.assertFalse(checks['apl.pending.bitwise_exact'])
if __name__ == '__main__':
unittest.main()
models/male-cns-lif/apl-timing-lab-v1/tests/test_timing_queue.py
+129
View File
@@ -0,0 +1,129 @@
"""Synthetic CPU-only checks; no Torch, data bundle, or scientific trial."""
from pathlib import Path
import sys
import unittest
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from timing_queue import TimingQueue, audit_sequence
class TimingQueueTests(unittest.TestCase):
def sequence(self, delay, steps, width=3):
generated = np.arange(1, steps * width + 1, dtype=np.float32).reshape(steps, width)
queue = TimingQueue(delay, width)
applied = np.empty_like(generated)
for tick, row in enumerate(generated):
applied[tick] = queue.push(row, tick=tick)
return queue, generated, applied, queue.pending()
def test_delay_zero_identity(self):
queue, generated, applied, pending = self.sequence(0, 40)
self.assertTrue(np.array_equal(generated.view(np.uint32), applied.view(np.uint32)))
self.assertEqual(pending.shape, (0, 3))
self.assertTrue(audit_sequence(generated, applied, pending, 0)["pass_all"])
self.assertEqual(queue.metadata()["pending_generated_count"], 0)
def test_delay_18_boundary(self):
_, generated, applied, pending = self.sequence(18, 39)
self.assertTrue(np.all(applied[:18].view(np.uint32) == 0))
np.testing.assert_array_equal(applied[18], generated[0])
np.testing.assert_array_equal(applied[36], generated[18])
np.testing.assert_array_equal(pending, generated[-18:])
audit_sequence(generated, applied, pending, 18)
def test_short_sequence_keeps_initial_pending_slots(self):
for steps in (0, 1, 7, 17, 18, 19):
with self.subTest(steps=steps):
queue, generated, applied, pending = self.sequence(18, steps)
meta = audit_sequence(generated, applied, pending, 18)
self.assertEqual(pending.shape, (18, 3))
self.assertEqual(meta["pending_initial_zero_prefix"], max(18 - steps, 0))
self.assertEqual(meta["pending_generated_count"], min(18, steps))
self.assertEqual(queue.steps, steps)
self.assertTrue(np.all(pending[:max(18 - steps, 0)].view(np.uint32) == 0))
def test_no_alias_to_input_or_return_or_pending(self):
queue = TimingQueue(18, 1)
row = np.array([3], dtype=np.float32)
first = queue.push(row)
row[0] = 9
first[0] = 8
pending = queue.pending()
pending[:] = 12
for _ in range(17):
queue.push(np.zeros(1, dtype=np.float32))
self.assertEqual(queue.push(np.zeros(1, dtype=np.float32))[0], 3)
def test_zero_delay_return_owned(self):
row = np.array([2], dtype=np.float32)
value = TimingQueue(0, 1).push(row)
value[0] = 99
self.assertEqual(row[0], 2)
def test_signed_zero_subnormal_and_extreme_bits(self):
row = np.array([0x80000000, 1, 0x7f7fffff], dtype=np.uint32).view(np.float32)
for delay in (0, 18):
queue = TimingQueue(delay, 3)
generated = np.stack([row.copy() for _ in range(23)])
applied = np.stack([queue.push(x) for x in generated])
audit_sequence(generated, applied, queue.pending(), delay)
def test_reject_invalid_configuration(self):
for delay in (True, 1, -18, 18.0, np.int32(18)):
with self.assertRaises(ValueError):
TimingQueue(delay)
for width in (0, -1, True, 2.0):
with self.assertRaises(ValueError):
TimingQueue(18, width)
def test_reject_invalid_sample_without_advance(self):
queue = TimingQueue(18, 2)
bad = [np.ones(2, dtype=np.float64), np.ones((1, 2), dtype=np.float32),
np.ones(3, dtype=np.float32), [1, 2],
np.array([np.nan, 1], dtype=np.float32), np.array([np.inf, 1], dtype=np.float32),
np.array([-1, 1], dtype=np.float32)]
for row in bad:
with self.assertRaises(ValueError):
queue.push(row)
self.assertEqual(queue.steps, 0)
def test_tick_must_be_consecutive(self):
queue = TimingQueue(0, 1)
row = np.zeros(1, dtype=np.float32)
for tick in (1, True, -1, .0):
with self.assertRaises(ValueError):
queue.push(row, tick=tick)
queue.push(row, tick=0)
with self.assertRaises(ValueError):
queue.push(row, tick=0)
self.assertEqual(queue.steps, 1)
def test_audit_detects_shift_or_initial_negative_zero(self):
_, generated, applied, pending = self.sequence(18, 40)
applied[18, 0] += 1
with self.assertRaises(ValueError):
audit_sequence(generated, applied, pending, 18)
_, generated, applied, pending = self.sequence(18, 40)
applied.view(np.uint32)[0, 0] = 0x80000000
with self.assertRaises(ValueError):
audit_sequence(generated, applied, pending, 18)
def test_audit_rejects_invalid_schema(self):
_, generated, applied, pending = self.sequence(18, 2)
for g, a, p in ((generated.astype(np.float64), applied, pending),
(generated, applied[:1], pending), (generated, applied, pending[:2])):
with self.assertRaises(ValueError):
audit_sequence(g, a, p, 18)
def test_finite_sequence_sum_preserved_without_brain_tail(self):
queue, generated, applied, pending = self.sequence(18, 53)
np.testing.assert_array_equal(applied.sum(axis=0, dtype=np.float64) + pending.sum(axis=0, dtype=np.float64),
generated.sum(axis=0, dtype=np.float64))
self.assertEqual(queue.steps, 53)
self.assertFalse(queue.metadata()["brain_tail_advanced"])
if __name__ == "__main__":
unittest.main()
models/male-cns-lif/apl-timing-lab-v1/timing_queue.py
+87
View File
@@ -0,0 +1,87 @@
"""Pure NumPy reference for the fixed 0/18-step generated-APL output delay.
No brain dynamics, gain, float arithmetic or implicit tail advancement occurs
here. Pending rows are the next D application slots, including initial +0
padding if fewer than D samples have been generated.
"""
import numpy as np
class TimingQueue:
def __init__(self, delay_steps, width=4064):
if type(delay_steps) is not int or delay_steps not in (0, 18):
raise ValueError("delay_steps must be integer 0 or 18")
if type(width) is not int or width <= 0:
raise ValueError("width must be a positive integer")
self.delay_steps = delay_steps
self.width = width
self.steps = 0
self._ring = np.zeros((delay_steps, width), dtype=np.float32)
def push(self, generated, tick=None):
"""Return an owned f32 application row, without changing its bits."""
if tick is not None and (type(tick) is not int or tick != self.steps):
raise ValueError("tick must be the next consecutive integer tick")
if not isinstance(generated, np.ndarray) or generated.dtype != np.float32:
raise ValueError("generated must be a float32 ndarray")
if generated.shape != (self.width,) or not np.isfinite(generated).all():
raise ValueError("generated shape or finiteness invalid")
if np.any(generated < 0):
raise ValueError("generated inhibition must be nonnegative")
if self.delay_steps == 0:
applied = generated.copy()
else:
slot = self.steps % self.delay_steps
applied = self._ring[slot].copy()
self._ring[slot] = generated
self.steps += 1
return applied
def pending(self):
"""Owned rows in next-application order; always shape (D, width)."""
if self.delay_steps == 0:
return self._ring.copy()
head = self.steps % self.delay_steps
return np.concatenate((self._ring[head:], self._ring[:head]), axis=0)
def metadata(self):
return {"apl_delay_steps": self.delay_steps, "steps": self.steps,
"pending_generated_count": min(self.steps, self.delay_steps),
"pending_initial_zero_prefix": max(self.delay_steps - self.steps, 0),
"pending_application_tick_start": self.steps,
"pending_application_tick_stop_exclusive": self.steps + self.delay_steps,
"brain_tail_advanced": False}
def audit_sequence(generated, applied, pending, delay_steps):
"""Bitwise finite-sequence accounting; not a closed-loop integral claim.
Checks concat(applied, pending) == concat(D initial +0 rows, generated).
This preserves signed zeros and every stored sample, without assuming the
generated sequences of two different closed-loop branches are identical.
"""
if type(delay_steps) is not int or delay_steps not in (0, 18):
raise ValueError("delay_steps must be integer 0 or 18")
for name, value in (("generated", generated), ("applied", applied), ("pending", pending)):
if not isinstance(value, np.ndarray) or value.dtype != np.float32 or value.ndim != 2:
raise ValueError(name + " must be a rank-2 float32 ndarray")
if not np.isfinite(value).all() or np.any(value < 0):
raise ValueError(name + " must be finite nonnegative inhibition")
if generated.shape != applied.shape or pending.shape != (delay_steps, generated.shape[1]):
raise ValueError("Sequence shapes disagree")
steps = len(generated)
# Rowwise comparison avoids allocating a second T-by-K trace.
for tick in range(steps + delay_steps):
actual = applied[tick] if tick < steps else pending[tick - steps]
actual_bits = actual.view(np.uint32)
if tick < delay_steps:
ok = np.all(actual_bits == 0)
else:
ok = np.array_equal(actual_bits, generated[tick - delay_steps].view(np.uint32))
if not ok:
raise ValueError("Fixed-delay bitwise accounting failed at application tick " + str(tick))
return {"pass_all": True, "apl_delay_steps": delay_steps, "steps": steps,
"pending_generated_count": min(steps, delay_steps),
"pending_initial_zero_prefix": max(delay_steps - steps, 0),
"initial_padding_positive_zero": True, "sample_bits_preserved": True,
"brain_tail_advanced": False}
models/male-cns-lif/apl-timing-lab-v1/verify_frozen.py
+55
View File
@@ -0,0 +1,55 @@
"""Run unchanged frozen reference into new output; never rewrite old evidence."""
import os
import sys
import time
import numpy as np
import pandas as pd
import yaml
from context import ROOT, EXP, CFG, save, check, bit_equal
def main():
check()
assert not (EXP / 'regression/frozen-five.json').exists()
rtpath = ROOT / 'models/male-cns-lif/deterministic-runtime-v2'
pc = yaml.safe_load((rtpath / 'phase4p.yaml').read_text())
os.environ['CUDA_VISIBLE_DEVICES'] = CFG['gpu_uuid']
os.environ['CUDA_HOME'] = '/usr/local/cuda'
os.environ['PATH'] = str(ROOT / 'models/male-cns-lif/.venv/bin') + ':/usr/local/cuda/bin:' + os.environ['PATH']
os.environ['MAX_JOBS'] = '2'
os.environ['TORCH_CUDA_ARCH_LIST'] = '8.9'
sys.path.insert(0, str(ROOT / 'models/male-cns-lif/terminal-scope-diagnostic-v1'))
from diagnostic import queues
queues()
sys.path.insert(0, str(ROOT / 'models/male-cns-lif'))
sys.path.insert(0, str(rtpath))
import torch
import phase4p
assert torch.cuda.mem_get_info()[0] > 2 * 1024**3
torch.cuda.reset_peak_memory_stats()
started = time.perf_counter()
runtime = phase4p.CanonicalRuntime(pc, rtpath, EXP)
rows = []
for spec in pc['reference_trials']:
queues()
base = ROOT / 'experiments/phase4p-deterministic-runtime/phase4c-v2/reference' / spec['name']
events = pd.read_parquet(base / 'spike-events.parquet')
reference = dict(step=events.step.to_numpy(), neuron_idx=events.neuron_idx.to_numpy(),
counts=np.load(base / 'per-neuron-spike-counts.int32.npy'),
v=np.load(base / 'final-voltage.float32.npy'), g=np.load(base / 'final-g.float32.npy'))
result = runtime.run(np.flatnonzero(runtime.pops[spec['population']]), spec['rate_hz'], spec['seed'], spec['duration_ms'])
row = dict(trial=spec['name'], exact={key: bool(bit_equal(value, result[key])) for key, value in reference.items()},
dtype_shape={key: dict(reference_dtype=str(value.dtype), actual_dtype=str(result[key].dtype),
reference_shape=list(value.shape), actual_shape=list(result[key].shape)) for key, value in reference.items()},
wall_seconds=result['metrics']['wall_seconds'])
rows.append(row)
print(row, flush=True)
passed = all(all(row['exact'].values()) for row in rows)
save(EXP / 'regression/frozen-five.json', dict(pass_all=passed, trials=rows,
wall_seconds=time.perf_counter() - started, peak_allocated_vram=int(torch.cuda.max_memory_allocated()),
oracle='Old frozen deterministic-v2 only; not this independent dynamics branch'))
assert passed
if __name__ == '__main__':
main()