Audit fixed D18 credit and conditional MBON transfer (Phase 4AL)
This commit is contained in:
Digital Drosophila Archive
@@ -0,0 +1,201 @@
|
||||
# Phase 4AL — credit specificity and conditional KC→MBON transfer
|
||||
|
||||
## Purpose and frozen boundary
|
||||
|
||||
This is a diagnostic audit of the already sealed Phase 4AK D18 branch, not
|
||||
conditioning, training, or a new learning candidate. Phase 4AK established its
|
||||
predeclared 100 ms KC representation gate; it did not establish appropriate
|
||||
credit assignment, functional associative memory, or biological fidelity of
|
||||
every omitted mechanism.
|
||||
|
||||
Use all 120 Phase 4AK primary trials and all 24 exact-repeat trials. Keep the
|
||||
two cohorts and their two input modes separate. Repeats check reproducibility;
|
||||
they are not extra independent observations in the five-seed statistics. Do
|
||||
not replace seeds, select favorable odors, choose a best pair or compartment,
|
||||
or inspect old held-out dynamic outcomes to change this design.
|
||||
The inherited matched-energy mode matches expected input, not necessarily the
|
||||
realized event count of every stochastic trial; do not relabel it exact
|
||||
realized-count matching.
|
||||
|
||||
All earlier raw, processed, frozen/reference, model and experiment artifacts
|
||||
remain read-only. Preserve the base CSR, anatomical counts, transmitter signs,
|
||||
LIF parameters, D18 operator, APL settings, eligibility-v2, and existing 14,232
|
||||
plastic-edge mask. All new records belong only to this new audit. No DAN
|
||||
stimulus is injected; no reinforcement is delivered to a brain; no learned
|
||||
`plastic_delta` is written. No decoder, action selection, game, RL, or visual
|
||||
interface is introduced.
|
||||
|
||||
## Three distinct evidence levels
|
||||
|
||||
1. **Exact baseline reconstruction.** Reconstruct target MBONs from the actual
|
||||
frozen full-network events using real edges, original arithmetic/order,
|
||||
initial states, arrivals, refractory acceptance, resets, and applicable APL
|
||||
handling. Require bitwise identity of events, per-neuron counts, and final
|
||||
V/g/ref against Phase 4AK D18. A failed or incomplete reconstruction blocks
|
||||
scientific interpretation of that replay; it is never rescued by plausible
|
||||
plots or by agreement of aggregate firing rates. Complete-audit integrity
|
||||
requires all 144 baseline replays; do not silently drop failed cases and
|
||||
report the remaining subset as the complete predeclared audit.
|
||||
2. **Virtual credit audit.** Replay the unchanged, existing float32 eligibility
|
||||
operator and inspect its state at the fixed tick-900 virtual gate, after
|
||||
that tick's original `mul → add → clamp` sequence. Tick-900 spikes are
|
||||
included. This is shadow accounting only, not dopamine delivery or weight
|
||||
updating; a before/after-update change is a different experiment. Do not
|
||||
treat virtual gate amplitude as measured dopamine concentration.
|
||||
3. **One fixed conditional sensitivity test.** Compare the exact baseline with
|
||||
one temporary replay applying uniform 25% depression to the whole existing
|
||||
plastic-edge mask. This uses the pre-existing maximum allowed fractional
|
||||
depression as a single declared diagnostic intervention. It is not a
|
||||
fraction scan, an eligibility-trained delta, a selected successful memory,
|
||||
or permission to enlarge the mask. The base weights remain unchanged; only
|
||||
the diagnostic effective amplitudes change, preserving edge identity and
|
||||
transmitter sign. On an independent incoming-weight copy, apply
|
||||
`weights[plastic] *= 0.75` throughout the full 100 ms replay, not only after
|
||||
tick 900. All presynaptic spike trains, including those of other MBONs and
|
||||
DANs, remain the original recorded trains. Do not save the overlay as a
|
||||
learned or official model.
|
||||
|
||||
The 25% intervention is **not an upper bound on functional response**. LIF
|
||||
threshold, reset, refractory gating, and recurrent influences make responses
|
||||
nonlinear; the largest permitted fractional depression does not establish the
|
||||
largest attainable change in spikes, latency, or downstream activity.
|
||||
|
||||
## Incoming-drive and overlap accounting
|
||||
|
||||
Decompose actual MBON incoming drive into six mutually exclusive groups:
|
||||
|
||||
- plastic KC→MBON;
|
||||
- non-plastic KC→MBON;
|
||||
- DAN;
|
||||
- MBON;
|
||||
- APL;
|
||||
- other.
|
||||
|
||||
Use the frozen canonical IDs, mask membership and audited metadata, not guessed
|
||||
names or transmitter-to-cell-role substitutions. Preserve unknown role
|
||||
provenance inside the residual `other` group. Establish each edge's group once;
|
||||
do not reclassify it to improve the result. Report accepted/rejected arrivals,
|
||||
signed contributions, and roundoff/reset effects as the implementation makes
|
||||
available. The model's variable `g` is a model state; do not rename it a measured
|
||||
biophysical conductance or dopamine signal. A source's share of accepted drive
|
||||
is not its necessity, sufficiency, or unique causal contribution to a spike.
|
||||
Any ratio with a zero/undefined denominator must remain explicitly undefined.
|
||||
The sole set-similarity convention is empty/empty Jaccard = 1 with an explicit
|
||||
`empty_union` flag. Such pairs are counted separately and excluded from block
|
||||
overlap aggregates; this convention is never evidence of useful credit.
|
||||
|
||||
For each cohort × mode, retain all 15 unordered pairs of its six odors: 60
|
||||
pair records in total. Report two separate three-of-five-seeds memberships for
|
||||
each odor: (1) a strictly positive frozen eligibility trace sampled after the
|
||||
tick-900 update, including tick-900 spikes, and (2) a strictly positive spike
|
||||
count anywhere in the full 100 ms trial. The primary virtual-credit grouping
|
||||
uses the first definition. The whole-window activity grouping is auxiliary;
|
||||
it cannot replace the primary grouping to rescue a result. These are
|
||||
descriptive reliability rules, not biological thresholds, top-K filters, or
|
||||
ways to delete cells/edges from replay. Preserve per-trial activity and
|
||||
eligibility alongside both aggregate memberships.
|
||||
|
||||
Compute overlap on the actual A-active and B-active sets. In particular,
|
||||
`Jaccard(A_active, B_active)` must use their real intersection and union; never
|
||||
substitute the already disjoint sets `A_only` and `B_only`, whose zero
|
||||
intersection is true by construction. Shared, A-only, B-only, and inactive
|
||||
strata are useful descriptive partitions, not proof of stimulus-specific
|
||||
causation. Shared representations can support biological generalization;
|
||||
nonzero B effects are not automatically a defect, nor are they automatically
|
||||
evidence of valid generalization.
|
||||
|
||||
Keep arithmetic/integrity checks separate from scientific results. Report all
|
||||
pairwise and trial-level outcomes without inventing an overall 120-trial
|
||||
biological PASS fraction, post hoc effect threshold, or pooled rescue gate.
|
||||
An absence of MBON spike change may coexist with a subthreshold change and
|
||||
does not mean the entire biological pathway is incapable of plasticity.
|
||||
|
||||
## What a conditional replay can and cannot establish
|
||||
|
||||
The baseline can establish that the saved events and the frozen operator
|
||||
reconstruct the saved MBON trajectory exactly. Virtual eligibility can reveal
|
||||
which real candidate edges would receive credit under that declared rule.
|
||||
The single depression replay can measure how the selected MBONs respond to
|
||||
that fixed intervention **with the external network event history held fixed**.
|
||||
|
||||
It cannot establish closed-loop whole-brain necessity/sufficiency or learned
|
||||
behavior. If an MBON's replayed spikes change while its outgoing effects and
|
||||
the other neurons' events remain clamped to the old trial, MBON→DAN, MBON→MBON,
|
||||
and other feedback are not recomputed self-consistently. Their fixed incoming
|
||||
traces must not be described as counterfactual feedback responses. If the
|
||||
replay instead also clamps the target MBON's original reset/refractory gates,
|
||||
its result is narrower still: conditional drive/state accounting, not an
|
||||
independently recomputed functional spike response. State these scopes
|
||||
explicitly for each reported quantity.
|
||||
|
||||
The anatomical D18 arrival delay is 18 × 0.1 ms = 1.8 ms. It is not the
|
||||
eligibility lifetime, a measured dopamine latency, or the CS–reinforcer
|
||||
association window. A 100 ms observation does not test memory retention,
|
||||
consolidation, extinction, reversal, or behavioral recall. Do not promote any
|
||||
successful replay to a dopamine-plasticity or learning candidate.
|
||||
|
||||
## Primary evidence and non-measured assumptions
|
||||
|
||||
- **Hige et al., Neuron (2015), DOI 10.1016/j.neuron.2015.11.003.** Pairing an
|
||||
odor with selected DAN activation produced odor-specific, compartment-local
|
||||
KC→MBON depression with order dependence. The reported induction did not
|
||||
require postsynaptic MBON spiking; overlapping KC representations predicted
|
||||
reduced specificity/generalization. This supports auditing the correct
|
||||
presynaptic and compartment credit, not requiring MBON spikes to define all
|
||||
plasticity. It does not specify MaleCNS per-edge learning parameters.
|
||||
[Author publication and paper](https://www.janelia.org/publication/heterosynaptic-plasticity-underlies-aversive-olfactory-learning-drosophila).
|
||||
- **Handler et al., Cell (2019), DOI 10.1016/j.cell.2019.05.040.** DopR1/DopR2
|
||||
pathways support opposing plasticity associated with different cue–DAN
|
||||
timing. This establishes why time ordering matters, not that the frozen
|
||||
float32 trace is a measured biochemical state or that one rule applies to
|
||||
every compartment. [Primary paper](https://www.sciencedirect.com/science/article/pii/S0092867419306117).
|
||||
- **Aso and Rubin, eLife (2016), DOI 10.7554/eLife.16135.** DAN cell types differ
|
||||
in memory formation, updating and persistence. PAM/PPL1 labels alone are
|
||||
not a license to assign a universal valence or a common temporal rule.
|
||||
[Primary paper](https://elifesciences.org/articles/16135).
|
||||
- **Aso et al., eLife (2014), DOI 10.7554/eLife.04580.** Different MBON types
|
||||
bias approach/avoidance, with combined activation supporting population
|
||||
integration. Assessing a population response is more appropriate than
|
||||
equating greater weight or total firing with positive memory. The current
|
||||
audit nevertheless has no behavioral decoder or demonstrated behavior.
|
||||
[Primary paper](https://elifesciences.org/articles/04580.pdf).
|
||||
- **Huang et al., Nature (2024), DOI 10.1038/s41586-024-07819-w.** Physiological
|
||||
measurements and a constrained model show heterogeneous, bidirectional
|
||||
PPL1 signaling and feedback interactions among memory units. This motivates
|
||||
explicit closed-loop limitations. Its fitted model weights and plasticity
|
||||
parameters are not MaleCNS anatomical counts or measurements to import into
|
||||
this frozen branch. [Primary paper](https://www.nature.com/articles/s41586-024-07819-w).
|
||||
- **Manoim et al., Current Biology (2022), DOI 10.1016/j.cub.2022.09.007, and
|
||||
Manoim Wolkovitz et al. (2026), DOI 10.1016/j.cub.2026.01.014.** KC lateral
|
||||
axonal mAChR-B modulation contributes to stimulus specificity; subsequent
|
||||
work identifies voltage-dependent modulation relevant to active versus
|
||||
inactive KCs. Sparse coding plus a timed reinforcer is therefore not, by
|
||||
itself, proof of appropriate credit. These studies do not prove that an
|
||||
omitted receptor mechanism caused any result in this fixed model, and do
|
||||
not authorize a sign, gain, threshold, or receptor-parameter change.
|
||||
[2022 paper](https://www.sciencedirect.com/science/article/pii/S0960982222014518);
|
||||
[2026 paper](https://www.sciencedirect.com/science/article/pii/S096098222600014X).
|
||||
|
||||
The frozen eligibility operator, tick-900 virtual gate, three-of-five activity
|
||||
summary, 25% uniform diagnostic depression, clamped event history, and chosen
|
||||
observation window are engineering/model assumptions. None is relabelled as a
|
||||
MaleCNS single-cell or synaptic measurement by citing the papers above.
|
||||
|
||||
## Deferred functional-memory controls — not executed here
|
||||
|
||||
A later explicitly authorized learning experiment would need pre/post CS+
|
||||
and CS− responses, reciprocal cue assignment, independent evaluation seeds,
|
||||
and reinforcement-free probes with updates disabled. CS-only, DAN-only,
|
||||
unpaired/yoked, D(t)=0, and frozen/delta=0 conditions distinguish associative
|
||||
effects from exposure, transient state, adaptation and unrelated modulation.
|
||||
Real DAN-only stimulation is not universally expected to have zero effect;
|
||||
that prediction depends on the declared model and compartment. Retention
|
||||
requires its own observation interval, and any model-specific output effect
|
||||
must remain distinct from a behavioral-memory claim. No such controls are
|
||||
silently replaced by virtual credit or the 25% sensitivity test.
|
||||
|
||||
Google/DeepMind's [FlyBody work](https://www.nature.com/articles/s41586-025-09029-4)
|
||||
provides body physics and artificial RL controllers, not evidence for this
|
||||
KC→MBON credit rule. Google's [MaleCNS author resource](https://research.google/blog/a-connectomics-milestone-mapping-the-complete-male-fruit-fly-brain/)
|
||||
supports the anatomical resource, not measured dopamine kinetics or successful
|
||||
learning in this model. Neither permits bypassing the present mechanism audit.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Preparation record
|
||||
|
||||
The first preparation invocation stopped before experiment creation or
|
||||
registration: the ordinary account could not create a child of the protected
|
||||
experiments directory. No trajectory, credit calculation or conditional replay
|
||||
was executed. The fix provisions only the new, empty Phase 4AL directory with
|
||||
the ordinary account as owner; preparation permits an absent or empty directory
|
||||
but still refuses any populated experiment. No old directory permission is
|
||||
changed. This is an execution-environment correction, not a scientific change.
|
||||
|
||||
Local synthetic tests initially had no Torch installation and explicitly
|
||||
skipped the Torch arithmetic comparison. Server testing includes this comparison
|
||||
and preparation rejects skipped tests.
|
||||
@@ -0,0 +1,15 @@
|
||||
cda2d810eada83ea33c96c1ce9d16102d1af22d1862bce84c47f0e7415863eca EVIDENCE.md
|
||||
8259428a56721ba8174039846623998ef11a1d8e9e14f123dfe0a0e4849c71ee PREPARATION.md
|
||||
b996e175731a920ad19e2c8989bd267cdbb71c14aa4558a8f1ef7486454c4b44 config.json
|
||||
1626873baa224fab5d427a1fab1d6f840f14814152914ac4c2d6aa7a4f0bb1f6 context.py
|
||||
e03d3e1f64328058273e8364990e7a2c7f1527a31822ea7594a9e8914aa43497 credit.py
|
||||
9356587aa8811a3db50daee9bca22cf4114e06c6669cc80c69cd0dd2224b0130 finalize.py
|
||||
825e9e60cb579479ddb2860a36f8dee0715b9b44d624cbad59ce0d279550d17f mbon_accounting.py
|
||||
9e7532bc4a564b0a0c0384ba2f3a7bfc0559c16bb256510dde7c97cce9036c0c prepare.py
|
||||
3ece589f0f6ce1bff2cde94494e896a58fa4154106e613ee7e60549094304071 run_audit.py
|
||||
62432fe7b068eafae99396c4a9231adf8ef8e9d340ab64517f618fc794e7fd78 summarize.py
|
||||
4c473ce709ed9496b7ce1b5bedd6964c780715ff41dd54977489fbd2b99f998e test_credit.py
|
||||
23c4e9138b326e1a84fed88d771a6c2044a6abf72ecc7a1b829f604bcdde215d test_integration.py
|
||||
d192dd357db96e628c21b9bcde617eaf6c6bc1efe5920ec55cc9cc1e4b549454 test_mbon_accounting.py
|
||||
85442fcefa93c64a2fac7f0a4b60ce505ddb9edab7a63cb0e25cb20f013d3850 verify_frozen.py
|
||||
72e4b4e2c53c258939ca355f31eeba2cd6381e126a61efeea651d8bac8df5328 manifest.json
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"phase": "4AL",
|
||||
"purpose": "read-only D18 credit and conditional MBON transfer audit; not learning",
|
||||
"neurons": 166700,
|
||||
"edges": 25582938,
|
||||
"candidate_edges": 14232,
|
||||
"primary_trials": 120,
|
||||
"technical_repeats": 24,
|
||||
"steps": 1000,
|
||||
"dt_ms": 0.1,
|
||||
"virtual_gate_tick": 900,
|
||||
"kc_trace_tau_ms": 100.0,
|
||||
"trace_cap": 1.0,
|
||||
"stable_seed_count": 3,
|
||||
"conditional_depression_fraction": 0.25,
|
||||
"groups": ["plastic_KC", "nonplastic_KC", "DAN", "MBON", "APL", "other"],
|
||||
"workers": 4,
|
||||
"output_budget_bytes": 4294967296,
|
||||
"training": false,
|
||||
"candidate_generation": false,
|
||||
"gate_meaning": "Post-update tick 900 virtual assay, not a DAN or dopamine measurement",
|
||||
"conditional_meaning": "Recorded all-source events clamped, target MBON spikes independently predicted; not whole-brain counterfactual",
|
||||
"no_new_overall_credit_threshold": true
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
"""New read-only audit envelope. All old payloads are hash-anchored inputs."""
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.dont_write_bytecode = True
|
||||
ROOT = Path('/data/digital-drosophila')
|
||||
MODEL = Path(__file__).resolve().parent
|
||||
EXP = ROOT / 'experiments/phase4al-mb-credit-transfer'
|
||||
AK = ROOT / 'experiments/phase4ak-apl-timing-validation'
|
||||
AK_MODEL = ROOT / 'models/male-cns-lif/apl-timing-validation-v1'
|
||||
H = ROOT / 'experiments/phase4h-dopamine-plasticity'
|
||||
AG = ROOT / 'experiments/phase4ag-author-apl-r2'
|
||||
CORE = ROOT / 'models/male-cns-lif/kc-drive-audit-v1/replay_core.py'
|
||||
META = ROOT / 'datasets/male-cns-v1.0/processed/neuron-only/neuron-index/unified-metadata.parquet'
|
||||
COEFF = ROOT / 'experiments/phase4af-author-fullgraph/oracle/no_input/coefficients.json'
|
||||
CFG = json.loads((MODEL / 'config.json').read_text(encoding='utf-8'))
|
||||
PINS = {
|
||||
'models/male-cns-lif/runtime-credit-v1/phase4o_event_replay.py': 'b169937bab495c77d8ee6c1013877e7500e7354140d9fcc90595d984238ba44e',
|
||||
'models/male-cns-lif/phase4l.py': 'e97d58d8c1f44920d0e953f8f5831ddc86fa0198784e86460e315de4116ac553',
|
||||
'models/male-cns-lif/apl-timing-validation-v1/manifest.json': '3206a8666718b4a99e09b8384c87dd25895b5d7038d23fad7b93e40379183ccb',
|
||||
'experiments/phase4ak-apl-timing-validation/manifest.json': '25eb7e467eb17c3fa83ffee2e4e82b754f161fd8891e515bed23eeb500c5656d',
|
||||
'models/male-cns-lif/apl-timing-validation-v1/context.py': '5e4e500d5ec777bb8ae7ff77ac9a7279bfe995b1a17be2baededdbe065c98b73',
|
||||
'experiments/phase4ak-apl-timing-validation/protocol/validation-cases.json': '50d7b3aa7b0235e8f7bc1e21ef693cd5c554272ef3bd205c72f61e8e07ad1fd9',
|
||||
'experiments/phase4ak-apl-timing-validation/audit/final-status.json': 'c49181086ecbd19511b9f1e783afea8f66d0f74348a7b70d7f6c5fa99b4337fc',
|
||||
'models/male-cns-lif/kc-drive-audit-v1/replay_core.py': 'aa3187cb23712c09ab603e27e60424294db92de8086d60a95cf7a629d0521fdc',
|
||||
'models/male-cns-lif/mb-physiology-v1/phase4j.py': 'abb4574da85f00bd9ef61024c65993ef81d42651ff8d8d63c15dd69c6222471e',
|
||||
'models/male-cns-lif/mb-physiology-v1/phase4j.yaml': 'b35e4332a91b8489f69348cedf5ac675e9c3902c7e8c7beec5d053773c6529a3',
|
||||
'experiments/phase4h-dopamine-plasticity/manifests/checksums.sha256': '1ea0394fd0477bb40d656ed0854108964f3809d633b5561bed9de43cda630ba1',
|
||||
'datasets/male-cns-v1.0/processed/neuron-only/neuron-index/unified-metadata.parquet': '922af16846a065c3dbfca7c11146b83ca217fcbd7392edc745f444a04eae8c63'
|
||||
}
|
||||
|
||||
|
||||
def digest(path):
|
||||
h = hashlib.sha256()
|
||||
with Path(path).open('rb') as f:
|
||||
for block in iter(lambda: f.read(8 << 20), b''):
|
||||
h.update(block)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def read(path):
|
||||
return json.loads(Path(path).read_text(encoding='utf-8'))
|
||||
|
||||
|
||||
def save(path, value):
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open('x', encoding='utf-8') as f:
|
||||
json.dump(value, f, indent=2, ensure_ascii=False, allow_nan=False)
|
||||
f.write('\n')
|
||||
|
||||
|
||||
def write_npz(path, **values):
|
||||
import numpy as np
|
||||
with Path(path).open('xb') as f:
|
||||
np.savez_compressed(f, **values)
|
||||
|
||||
|
||||
def load_module(name, path, expected_hash):
|
||||
data = Path(path).read_bytes()
|
||||
assert hashlib.sha256(data).hexdigest() == expected_hash, str(path)
|
||||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module
|
||||
exec(compile(data, str(path), 'exec'), module.__dict__)
|
||||
return module
|
||||
|
||||
|
||||
def sources():
|
||||
return {p.relative_to(MODEL).as_posix(): digest(p) for p in sorted(MODEL.rglob('*'))
|
||||
if p.is_file() and '__pycache__' not in p.parts and p.name not in ('manifest.json', 'checksums.sha256')}
|
||||
|
||||
|
||||
def integrity():
|
||||
previous = load_module('sealed_ak_context_al', AK_MODEL / 'context.py',
|
||||
PINS['models/male-cns-lif/apl-timing-validation-v1/context.py'])
|
||||
result = previous.integrity()
|
||||
sys.path.insert(0, str(ROOT / 'models/male-cns-lif/terminal-scope-diagnostic-v1'))
|
||||
from diagnostic import manifest_check
|
||||
for path in (AK_MODEL, AK):
|
||||
result[path.relative_to(ROOT).as_posix()] = manifest_check(path, digest)
|
||||
assert all(row['pass'] for row in result.values())
|
||||
return result
|
||||
|
||||
|
||||
def check():
|
||||
reg = read(EXP / 'protocol/preregistered.json')
|
||||
assert reg['sources'] == sources() and reg['config'] == CFG
|
||||
for row in reg['inputs']:
|
||||
assert digest(ROOT / row['path']) == row['sha256'], row['path']
|
||||
for row in read(EXP / 'protocol/prepared-hashes.json'):
|
||||
assert digest(EXP / row['path']) == row['sha256'], row['path']
|
||||
return reg
|
||||
|
||||
|
||||
def bit_equal(a, b):
|
||||
return a.dtype == b.dtype and a.shape == b.shape and a.tobytes() == b.tobytes()
|
||||
|
||||
|
||||
def budget():
|
||||
size = sum(p.stat().st_size for p in EXP.rglob('*') if p.is_file())
|
||||
assert size < CFG['output_budget_bytes'], 'Audit output budget exceeded; stop, preserve partials'
|
||||
return size
|
||||
|
||||
|
||||
def seal(root):
|
||||
rows = [dict(path=p.relative_to(root).as_posix(), bytes=p.stat().st_size, sha256=digest(p))
|
||||
for p in sorted(root.rglob('*')) if p.is_file() and '__pycache__' not in p.parts]
|
||||
save(root / 'manifest.json', dict(root=str(root), files=rows))
|
||||
with (root / 'checksums.sha256').open('x', encoding='utf-8') as f:
|
||||
for row in rows:
|
||||
f.write(row['sha256'] + ' ' + row['path'] + '\n')
|
||||
f.write(digest(root / 'manifest.json') + ' manifest.json\n')
|
||||
@@ -0,0 +1,171 @@
|
||||
"""Read-only, float32 TemporalEligibility-v2 trace and fixed A/B identities.
|
||||
|
||||
This module consumes already observed full-graph spike events. It neither
|
||||
simulates neurons nor generates a weight delta. A gate is a virtual readout
|
||||
after the same-tick KC trace update, not evidence of an actual DAN event.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
N_GLOBAL = 166700
|
||||
CATEGORY_CODES = {'inactive': 0, 'A_specific': 1, 'B_specific': 2, 'shared': 3}
|
||||
MAJORITY = 3
|
||||
REPLICATES = 5
|
||||
|
||||
|
||||
def _require(condition, message):
|
||||
if not condition:
|
||||
raise ValueError(message)
|
||||
|
||||
|
||||
def _vector(value, name, dtype):
|
||||
_require(isinstance(value, np.ndarray) and value.ndim == 1 and value.dtype == np.dtype(dtype),
|
||||
name + ' must be a one-dimensional ' + dtype + ' ndarray; no implicit dtype conversion')
|
||||
return value
|
||||
|
||||
|
||||
def _frozen_scalar(value, expected, name):
|
||||
_require(not isinstance(value, (bool, np.bool_)) and isinstance(value, (int, float, np.number)) and
|
||||
np.ndim(value) == 0 and math.isfinite(float(value)) and float(value) == expected,
|
||||
name + ' must retain its frozen value ' + str(expected))
|
||||
|
||||
|
||||
def replay_credit(step, neuron_idx, kc_globals, steps=1000, gate_tick=900,
|
||||
dt_ms=.1, tau_ms=100, cap=1):
|
||||
"""Replay the frozen trace without touching the caller's arrays.
|
||||
|
||||
``step`` and ``neuron_idx`` are equal-length int32 vectors for the complete
|
||||
observed graph (or its KC-only subset), ordered uniquely by (tick, global
|
||||
neuron). Non-KC events are validated then ignored. ``kc_globals`` is sorted
|
||||
unique int32. The trace starts at +0 and, on every tick including tick zero,
|
||||
performs separate float32 multiply, add current spikes, then clamp to 1.
|
||||
|
||||
No 18-tick shift is applied: the recorded KC emission tick is the trace
|
||||
event tick. Gate counts include gate_tick; final counts cover [0, steps).
|
||||
dt/tau/cap are exposed for explicit provenance but may not be changed.
|
||||
Shorter steps/gates are allowed for synthetic contract tests.
|
||||
|
||||
Returns owned trace_gate/trace_final float32[K] and
|
||||
spike_count_gate/spike_count_total int32[K], plus scalar audit metadata.
|
||||
Memory is O(events + KC + global-neuron count), never a dense T-by-K trace.
|
||||
"""
|
||||
ticks = _vector(step, 'step', 'int32')
|
||||
nodes = _vector(neuron_idx, 'neuron_idx', 'int32')
|
||||
kc = _vector(kc_globals, 'kc_globals', 'int32')
|
||||
_require(type(steps) is int and 0 < steps <= np.iinfo(np.int32).max, 'steps must be a positive int32-range integer')
|
||||
_require(type(gate_tick) is int and 0 <= gate_tick < steps, 'gate_tick must lie in [0, steps)')
|
||||
_frozen_scalar(dt_ms, .1, 'dt_ms')
|
||||
_frozen_scalar(tau_ms, 100., 'tau_ms')
|
||||
_frozen_scalar(cap, 1., 'cap')
|
||||
_require(ticks.shape == nodes.shape, 'Event vectors have different lengths')
|
||||
_require(np.all((ticks >= 0) & (ticks < steps)), 'Event tick outside the observed window')
|
||||
_require(np.all((nodes >= 0) & (nodes < N_GLOBAL)), 'Event neuron outside the frozen global graph')
|
||||
_require(len(ticks) < 2 or np.all((ticks[1:] > ticks[:-1]) |
|
||||
((ticks[1:] == ticks[:-1]) & (nodes[1:] > nodes[:-1]))),
|
||||
'Events must be sorted and unique by (tick, global neuron); duplicates are not extra spikes')
|
||||
_require(len(kc) > 0 and kc[0] >= 0 and kc[-1] < N_GLOBAL and np.all(kc[1:] > kc[:-1]),
|
||||
'kc_globals must be nonempty, sorted, unique, in-range global identities')
|
||||
|
||||
lookup = np.full(N_GLOBAL, -1, np.int32)
|
||||
lookup[kc] = np.arange(len(kc), dtype=np.int32)
|
||||
local = lookup[nodes]
|
||||
selected = local >= 0
|
||||
kc_ticks, kc_local = ticks[selected], local[selected]
|
||||
counts_total = np.bincount(kc_local, minlength=len(kc)).astype(np.int32)
|
||||
counts_gate = np.bincount(kc_local[kc_ticks <= gate_tick], minlength=len(kc)).astype(np.int32)
|
||||
decay = np.float32(math.exp(-float(dt_ms) / float(tau_ms)))
|
||||
trace = np.zeros(len(kc), np.float32)
|
||||
trace_gate = None
|
||||
begin = 0
|
||||
for tick in range(steps):
|
||||
# Preserve the source's separate rounding operations; do not substitute
|
||||
# float64 exponentiation by elapsed time or shift spikes to arrivals.
|
||||
np.multiply(trace, decay, out=trace)
|
||||
end = int(np.searchsorted(kc_ticks, tick, side='right'))
|
||||
if end > begin:
|
||||
firing = kc_local[begin:end]
|
||||
trace[firing] = np.add(trace[firing], np.float32(1.0))
|
||||
np.minimum(trace, np.float32(1.0), out=trace)
|
||||
if tick == gate_tick:
|
||||
trace_gate = trace.copy()
|
||||
begin = end
|
||||
_require(begin == len(kc_ticks) and trace_gate is not None and np.isfinite(trace).all(), 'Incomplete or nonfinite trace replay')
|
||||
return {
|
||||
'trace_gate': trace_gate, 'trace_final': trace.copy(),
|
||||
'spike_count_gate': counts_gate, 'spike_count_total': counts_total,
|
||||
'steps': steps, 'gate_tick': gate_tick, 'dt_ms': float(dt_ms), 'tau_ms': float(tau_ms), 'cap': float(cap),
|
||||
'decay_f32_bits': f'{int(decay.view(np.uint32)):08x}',
|
||||
'total_input_events': int(len(ticks)), 'total_KC_events': int(len(kc_ticks)),
|
||||
'KC_events_through_gate': int(counts_gate.sum(dtype=np.int64)),
|
||||
'gate_includes_same_tick_spikes': True, 'event_shift_steps': 0,
|
||||
'gate_is_virtual_readout_not_DAN_activity': True,
|
||||
'weights_modified': False, 'weight_delta_generated': False,
|
||||
}
|
||||
|
||||
|
||||
def _traces(value, name):
|
||||
_require(isinstance(value, np.ndarray) and value.ndim == 2 and value.dtype == np.dtype('float32') and
|
||||
value.shape[0] == REPLICATES and value.shape[1] > 0,
|
||||
name + ' must have exact shape (5, KC) and float32 dtype')
|
||||
_require(np.isfinite(value).all() and np.all((value >= 0) & (value <= 1)),
|
||||
name + ' must contain finite frozen-cap traces in [0, 1]')
|
||||
return value
|
||||
|
||||
|
||||
def _set_summary(active_a, active_b, category):
|
||||
intersection = int(np.count_nonzero(active_a & active_b))
|
||||
union = int(np.count_nonzero(active_a | active_b))
|
||||
return {
|
||||
'total': int(len(category)), 'A_active': int(np.count_nonzero(active_a)),
|
||||
'B_active': int(np.count_nonzero(active_b)), 'intersection': intersection, 'union': union,
|
||||
'active_set_jaccard': float(intersection / union) if union else 1.0,
|
||||
'empty_union': union == 0,
|
||||
'category_counts': {name: int(np.count_nonzero(category == code)) for name, code in CATEGORY_CODES.items()},
|
||||
}
|
||||
|
||||
|
||||
def pair_identity(traces_a, traces_b, edge_source_local):
|
||||
"""Classify fixed paired five-seed A/B traces, without installing plasticity.
|
||||
|
||||
A KC is A-active/B-active if its gate trace is strictly positive in at
|
||||
least 3 of 5 corresponding trials. Categories are int8: inactive=0,
|
||||
A_specific=1, B_specific=2, shared=3. Every candidate edge inherits its
|
||||
source KC's category in the supplied edge order. Multiple edges sharing
|
||||
a KC are retained as separate rows, not independent neural observations.
|
||||
|
||||
``edge_source_local`` is int32[E] indexing the common KC axis of both
|
||||
trace matrices. The caller must bind that axis and the five paired seed
|
||||
rows to the frozen identities; this pure function cannot infer labels.
|
||||
Jaccard uses COMPLETE active A/B sets, not the disjoint specific sets.
|
||||
Empty/empty Jaccard is 1 with empty_union=True, never evidence of useful
|
||||
credit. Compartment stratification and any success gate belong to caller.
|
||||
"""
|
||||
a, b = _traces(traces_a, 'traces_a'), _traces(traces_b, 'traces_b')
|
||||
_require(a.shape == b.shape, 'A and B trace matrices use different KC axes')
|
||||
source = _vector(edge_source_local, 'edge_source_local', 'int32')
|
||||
_require(np.all((source >= 0) & (source < a.shape[1])), 'Candidate edge source is not a valid KC-local index')
|
||||
positive_a = np.count_nonzero(a > np.float32(0), axis=0).astype(np.int32)
|
||||
positive_b = np.count_nonzero(b > np.float32(0), axis=0).astype(np.int32)
|
||||
active_a, active_b = positive_a >= MAJORITY, positive_b >= MAJORITY
|
||||
category = active_a.astype(np.int8) + np.int8(2) * active_b.astype(np.int8)
|
||||
edge_a, edge_b, edge_category = active_a[source].copy(), active_b[source].copy(), category[source].copy()
|
||||
return {
|
||||
'kc_category': category, 'edge_category': edge_category,
|
||||
'kc_active_a': active_a, 'kc_active_b': active_b,
|
||||
'edge_active_a': edge_a, 'edge_active_b': edge_b,
|
||||
'kc_positive_trials_a': positive_a, 'kc_positive_trials_b': positive_b,
|
||||
'summary': {
|
||||
'replicates_per_condition': REPLICATES, 'majority_required': MAJORITY,
|
||||
'positivity_rule': 'float32 trace > 0; no fitted epsilon', 'category_codes': dict(CATEGORY_CODES),
|
||||
'KC': _set_summary(active_a, active_b, category),
|
||||
'edges': _set_summary(edge_a, edge_b, edge_category),
|
||||
'jaccard_scope': 'complete majority-active A and B sets, including shared members',
|
||||
'empty_union_jaccard_convention': 1.0,
|
||||
'edge_counts_are_source_degree_weighted': True,
|
||||
'actual_DAN_credit_claimed': False, 'weights_modified': False, 'weight_delta_generated': False,
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
"""Seal all evidence without mistaking diagnostic coverage for learning."""
|
||||
from context import MODEL, EXP, read, save, digest, check, integrity, budget, seal
|
||||
|
||||
|
||||
def main():
|
||||
check()
|
||||
from summarize import run
|
||||
summary = run(verify_only=True)
|
||||
assert summary['arithmetic_and_coverage_pass'] and summary['primary_trials'] == 120 and summary['exact_repeats'] == 24
|
||||
refs = read(EXP / 'regression/frozen-five.json')
|
||||
assert refs['pass_all'] and len(refs['trials']) == 5 and all(all(r['exact'].values()) for r in refs['trials'])
|
||||
old = integrity()
|
||||
save(EXP / 'regression/old-integrity-after.json', old)
|
||||
status = 'PASS-CONDITIONAL-CREDIT-TRANSFER-AUDIT-NOT-LEARNING'
|
||||
result = dict(status=status, **summary, old_integrity_pass=all(r['pass'] for r in old.values()),
|
||||
frozen_five_reference_pass=refs['pass_all'], payload_bytes_before_manifest=budget(),
|
||||
frozen_eligibility_and_mask_unchanged=True, base_weights_unchanged=True,
|
||||
scientific_summary_sha256=digest(EXP / 'audit/scientific-summary.json'),
|
||||
summary_all_144_baseline_raw_states_and_repeat_pair_metrics_reverified=True,
|
||||
next_required_gate='Only if the diagnostic evidence supports it, separately preregister compartment-specific reinforcement and independent whole-brain evaluation; no automatic conditioning unlock')
|
||||
save(EXP / 'audit/final-status.json', result)
|
||||
with (EXP / 'REPORT.md').open('x', encoding='utf-8') as f:
|
||||
f.write('# Phase 4AL — conditional credit/MBON transfer audit\n\nStatus: `' + status + '`.\n\n')
|
||||
f.write('All 120 primary trials and 24 repeats are retained. Full events/counts/final V/g/ref baseline exactness is required.\n')
|
||||
f.write('Virtual eligibility is sampled after tick 900; no DAN is injected, no delta is learned.\n')
|
||||
f.write('Uniform 25% mask depression is a temporary conditional replay; every source train, including feedback, is clamped.\n')
|
||||
f.write('All 60 odor pair-mode records and all 97 MBON targets are reported; no best pair/parameter selection.\n\n')
|
||||
for name, block in summary['blocks'].items():
|
||||
f.write(f"- {name}: specific KC/edge sets {block['pairs_with_both_specific_KC_and_edge_sets']}/15; conditional event change {block['conditional25_event_changed_trials']}/30; count/first-latency change {block['conditional25_count_or_first_latency_changed_trials']}/30.\n")
|
||||
f.write('\nNo biological learning PASS or candidate. This does not complete the learning goal.\n')
|
||||
f.write('Read EVIDENCE.md in the model for provenance, unmatched biology and conditional/closed-loop distinctions.\n')
|
||||
check(); seal(MODEL); seal(EXP)
|
||||
print(status, flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"root": "/data/digital-drosophila/models/male-cns-lif/mb-credit-transfer-audit-v1",
|
||||
"files": [
|
||||
{
|
||||
"path": "EVIDENCE.md",
|
||||
"bytes": 12770,
|
||||
"sha256": "cda2d810eada83ea33c96c1ce9d16102d1af22d1862bce84c47f0e7415863eca"
|
||||
},
|
||||
{
|
||||
"path": "PREPARATION.md",
|
||||
"bytes": 755,
|
||||
"sha256": "8259428a56721ba8174039846623998ef11a1d8e9e14f123dfe0a0e4849c71ee"
|
||||
},
|
||||
{
|
||||
"path": "config.json",
|
||||
"bytes": 864,
|
||||
"sha256": "b996e175731a920ad19e2c8989bd267cdbb71c14aa4558a8f1ef7486454c4b44"
|
||||
},
|
||||
{
|
||||
"path": "context.py",
|
||||
"bytes": 5478,
|
||||
"sha256": "1626873baa224fab5d427a1fab1d6f840f14814152914ac4c2d6aa7a4f0bb1f6"
|
||||
},
|
||||
{
|
||||
"path": "credit.py",
|
||||
"bytes": 9109,
|
||||
"sha256": "e03d3e1f64328058273e8364990e7a2c7f1527a31822ea7594a9e8914aa43497"
|
||||
},
|
||||
{
|
||||
"path": "finalize.py",
|
||||
"bytes": 2743,
|
||||
"sha256": "9356587aa8811a3db50daee9bca22cf4114e06c6669cc80c69cd0dd2224b0130"
|
||||
},
|
||||
{
|
||||
"path": "mbon_accounting.py",
|
||||
"bytes": 18376,
|
||||
"sha256": "825e9e60cb579479ddb2860a36f8dee0715b9b44d624cbad59ce0d279550d17f"
|
||||
},
|
||||
{
|
||||
"path": "prepare.py",
|
||||
"bytes": 9844,
|
||||
"sha256": "9e7532bc4a564b0a0c0384ba2f3a7bfc0559c16bb256510dde7c97cce9036c0c"
|
||||
},
|
||||
{
|
||||
"path": "run_audit.py",
|
||||
"bytes": 9755,
|
||||
"sha256": "3ece589f0f6ce1bff2cde94494e896a58fa4154106e613ee7e60549094304071"
|
||||
},
|
||||
{
|
||||
"path": "summarize.py",
|
||||
"bytes": 12067,
|
||||
"sha256": "62432fe7b068eafae99396c4a9231adf8ef8e9d340ab64517f618fc794e7fd78"
|
||||
},
|
||||
{
|
||||
"path": "test_credit.py",
|
||||
"bytes": 12924,
|
||||
"sha256": "4c473ce709ed9496b7ce1b5bedd6964c780715ff41dd54977489fbd2b99f998e"
|
||||
},
|
||||
{
|
||||
"path": "test_integration.py",
|
||||
"bytes": 3016,
|
||||
"sha256": "23c4e9138b326e1a84fed88d771a6c2044a6abf72ecc7a1b829f604bcdde215d"
|
||||
},
|
||||
{
|
||||
"path": "test_mbon_accounting.py",
|
||||
"bytes": 9241,
|
||||
"sha256": "d192dd357db96e628c21b9bcde617eaf6c6bc1efe5920ec55cc9cc1e4b549454"
|
||||
},
|
||||
{
|
||||
"path": "verify_frozen.py",
|
||||
"bytes": 3004,
|
||||
"sha256": "85442fcefa93c64a2fac7f0a4b60ce505ddb9edab7a63cb0e25cb20f013d3850"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,267 @@
|
||||
"""Read-only CPU accounting for the sealed AH replay callback.
|
||||
|
||||
Source/sign lanes are conditional, non-feedback bookkeeping, not additional
|
||||
neurons, conductances, energies, or a whole-brain causal intervention. The
|
||||
caller supplies anatomical groups and must independently verify the original
|
||||
events/state and due/pending coverage. No files, Torch, or simulation entry
|
||||
points are used here.
|
||||
"""
|
||||
from collections.abc import Mapping
|
||||
import json
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
DT_MS = .1
|
||||
COEFF_KEYS = ("gd", "c0", "cg", "cv", "threshold", "reset", "seed_volt")
|
||||
FLOAT_KEYS = ("old_v", "old_g", "v_integrated", "g_decayed", "v_threshold",
|
||||
"v_after_reset", "g_after_reset", "v_pre_apl", "g_pre_apl", "delta_si", "v", "g")
|
||||
INT_KEYS = ("ref_before", "ref_after_reset", "ref_pre_apl", "ref")
|
||||
BOOL_KEYS = ("available", "request", "request_accepted", "request_rejected", "fired")
|
||||
|
||||
|
||||
def require(value, message):
|
||||
if not value:
|
||||
raise ValueError(message)
|
||||
|
||||
|
||||
def bit_equal(a, b):
|
||||
return (a.dtype == b.dtype and a.shape == b.shape and
|
||||
np.array_equal(np.ascontiguousarray(a).view(np.uint8), np.ascontiguousarray(b).view(np.uint8)))
|
||||
|
||||
|
||||
class MBONAccounting:
|
||||
"""callback(tick, stage); finish() -> (per_target, per_edge, JSON summary).
|
||||
|
||||
Group arrays have shape (N,G), sign lanes (N,G,2), with sign order positive,
|
||||
negative. Negative increments/integrals retain their negative sign.
|
||||
Histories use (T,N), or (T,N,G,2) for history_group_g. Target indices are
|
||||
incoming.dst_local, not global IDs; the caller joins the verified target
|
||||
list. Float64 lanes retain their own canonical additions; summing lanes
|
||||
is not the runtime's interleaved accumulation, so residuals are reported
|
||||
without correcting them or assigning them to an anatomical source.
|
||||
"""
|
||||
|
||||
def __init__(self, incoming, groups, group_names, coeff, steps):
|
||||
require(isinstance(steps, (int, np.integer)) and not isinstance(steps, (bool, np.bool_))
|
||||
and 0 < steps <= np.iinfo(np.int32).max, "steps must be a positive int32-range integer")
|
||||
require(isinstance(group_names, tuple) and group_names and
|
||||
all(isinstance(x, str) and x for x in group_names) and len(set(group_names)) == len(group_names),
|
||||
"group_names must be a nonempty tuple of unique names")
|
||||
require(isinstance(coeff, Mapping), "coeff must be a mapping")
|
||||
self.coeff = {}
|
||||
for key in COEFF_KEYS:
|
||||
value = coeff.get(key)
|
||||
require(not isinstance(value, (bool, np.bool_)) and
|
||||
isinstance(value, (int, float, np.integer, np.float64)) and np.isfinite(value),
|
||||
"Missing/non-double/nonfinite coefficient: " + key)
|
||||
self.coeff[key] = np.float64(value)
|
||||
require(self.coeff["seed_volt"] > self.coeff["threshold"], "seed_volt must exceed threshold")
|
||||
require("dt_ms" not in coeff or coeff["dt_ms"] == DT_MS, "AH time step is fixed at .1 ms")
|
||||
require(isinstance(incoming.target_count, (int, np.integer)) and 0 < incoming.target_count <= 166700,
|
||||
"Invalid incoming target count")
|
||||
self.n, self.steps, self.names = int(incoming.target_count), int(steps), group_names
|
||||
self.e = len(incoming.edge_id)
|
||||
for name, dtype in (("src_global", "int32"), ("dst_local", "int32"),
|
||||
("edge_id", "int64"), ("weights", "float64")):
|
||||
value = np.asarray(getattr(incoming, name))
|
||||
require(value.dtype == np.dtype(dtype) and value.shape == (self.e,), "Invalid incoming " + name)
|
||||
setattr(self, name, value.copy())
|
||||
require(np.all((self.src_global >= 0) & (self.src_global < 166700)) and
|
||||
np.all(self.src_global[1:] >= self.src_global[:-1]), "Noncanonical incoming sources")
|
||||
require(np.all(self.edge_id >= 0) and np.all(self.edge_id[1:] > self.edge_id[:-1]), "Noncanonical edge IDs")
|
||||
require(np.all((self.dst_local >= 0) & (self.dst_local < self.n)) and
|
||||
np.isfinite(self.weights).all(), "Invalid incoming destination/weight")
|
||||
codes = np.asarray(groups)
|
||||
require(codes.dtype == np.int32 and codes.shape == (self.e,) and
|
||||
np.all((codes >= 0) & (codes < len(self.names))), "groups must be int32 per incoming row")
|
||||
self.groups = codes.copy()
|
||||
self.ticks, self.finished, self.previous = 0, False, None
|
||||
shape = (self.n, len(self.names))
|
||||
self.counts = {key: np.zeros(shape, np.int64) for key in
|
||||
("due_count", "accepted_count", "rejected_count", "accepted_zero_count", "rejected_zero_count")}
|
||||
self.increments = {key: np.zeros(shape, np.float64) for key in
|
||||
("accepted_positive_V", "accepted_negative_V", "rejected_positive_V", "rejected_negative_V")}
|
||||
self.edge_due = np.zeros(self.e, np.int64)
|
||||
self.edge_accepted = np.zeros(self.e, np.int64)
|
||||
self.spike_count = np.zeros(self.n, np.int32)
|
||||
self.first = np.full(self.n, -1, np.int32)
|
||||
self.requests = {key: np.zeros(self.n, np.int64) for key in
|
||||
("request_count", "request_accepted_count", "request_rejected_count")}
|
||||
self.sg = np.zeros(shape + (2,), np.float64)
|
||||
self.sv = np.zeros_like(self.sg)
|
||||
self.bg = np.zeros(self.n, np.float64)
|
||||
self.bv = np.zeros(self.n, np.float64)
|
||||
self.integral_g = np.zeros_like(self.sg)
|
||||
self.integral_v = np.zeros_like(self.sg)
|
||||
self.max_g_residual = np.zeros(self.n, np.float64)
|
||||
self.max_v_residual = np.zeros(self.n, np.float64)
|
||||
self.history_v = np.empty((self.steps, self.n), np.float64)
|
||||
self.history_g = np.empty_like(self.history_v)
|
||||
self.history_ref = np.empty((self.steps, self.n), np.int32)
|
||||
self.history_group_g = np.empty((self.steps,) + self.sg.shape, np.float64)
|
||||
self.history_baseline_g = np.empty_like(self.history_v)
|
||||
self.history_g_residual = np.empty_like(self.history_v)
|
||||
|
||||
def _validate(self, tick, s):
|
||||
require(not self.finished and isinstance(tick, (int, np.integer)) and
|
||||
not isinstance(tick, (bool, np.bool_)) and tick == self.ticks and tick < self.steps,
|
||||
"Missing, duplicate or out-of-order callback")
|
||||
require(isinstance(s, Mapping), "stage must be a mapping")
|
||||
for keys, dtype in ((FLOAT_KEYS, np.float64), (INT_KEYS, np.int32), (BOOL_KEYS, np.bool_)):
|
||||
for key in keys:
|
||||
value = s.get(key)
|
||||
require(isinstance(value, np.ndarray) and value.shape == (self.n,) and value.dtype == dtype
|
||||
and np.isfinite(value).all(), "Missing/invalid stage field: " + key)
|
||||
h = s.get("h32")
|
||||
require(isinstance(h, np.ndarray) and h.dtype == np.float32 and h.shape == (self.n,)
|
||||
and np.all(h == 0) and np.all(s["delta_si"] == 0), "MBON requires zero graded APL h/delta")
|
||||
require(np.all((s["ref_before"] >= 0) & (s["ref_before"] <= 22)), "Invalid refractory value")
|
||||
if self.previous is not None:
|
||||
require(all(bit_equal(s[a], self.previous[b]) for a, b in
|
||||
(("old_v", "v"), ("old_g", "g"), ("ref_before", "ref"))), "Discontinuous observed state")
|
||||
else:
|
||||
require(np.all(s["ref_before"] == 0), "AH replay starts nonrefractory")
|
||||
ref = np.maximum(s["ref_before"] - np.int32(1), np.int32(0))
|
||||
available = ref == 0
|
||||
require(np.array_equal(s["available"], available), "Availability mismatch")
|
||||
require(np.array_equal(s["request_accepted"], s["request"] & available) and
|
||||
np.array_equal(s["request_rejected"], s["request"] & ~available), "Request guard mismatch")
|
||||
c = self.coeff
|
||||
v, g = s["old_v"].copy(), s["old_g"].copy()
|
||||
g[available] = np.multiply(c["gd"], s["old_g"][available])
|
||||
v[available] = np.add(c["c0"], np.add(np.multiply(c["cg"], s["old_g"][available]),
|
||||
np.multiply(c["cv"], s["old_v"][available])))
|
||||
require(bit_equal(v, s["v_integrated"]) and bit_equal(g, s["g_decayed"]), "Integration mismatch")
|
||||
v[s["request_accepted"]] = c["seed_volt"]
|
||||
require(bit_equal(v, s["v_threshold"]), "Request override mismatch")
|
||||
fired = available & (v > c["threshold"])
|
||||
require(np.array_equal(s["fired"], fired), "Strict threshold mismatch")
|
||||
v[fired], g[fired], ref[fired] = c["reset"], np.float64(0), np.int32(22)
|
||||
for key, value in (("v_after_reset", v), ("g_after_reset", g), ("ref_after_reset", ref)):
|
||||
require(bit_equal(s[key], value), "Reset mismatch: " + key)
|
||||
due, accepted = s.get("due_edge_indices"), s.get("arrival_accepted")
|
||||
require(isinstance(due, np.ndarray) and due.dtype == np.int64 and due.ndim == 1 and
|
||||
np.all((due >= 0) & (due < self.e)) and np.all(due[1:] > due[:-1]), "Noncanonical due edge rows")
|
||||
require(isinstance(s.get("due_edge_id"), np.ndarray) and
|
||||
bit_equal(s["due_edge_id"], self.edge_id[due]), "Due edge-ID mismatch")
|
||||
require(isinstance(accepted, np.ndarray) and accepted.dtype == np.bool_ and accepted.shape == due.shape
|
||||
and np.array_equal(accepted, ref[self.dst_local[due]] == 0), "Arrival acceptance mismatch")
|
||||
require(tick >= 18 or not len(due), "Arrival before delay 18")
|
||||
rows = due[accepted]
|
||||
np.add.at(g, self.dst_local[rows], self.weights[rows])
|
||||
for key, value in (("g_pre_apl", g), ("v_pre_apl", v), ("ref_pre_apl", ref), ("v", v), ("ref", ref)):
|
||||
require(bit_equal(s[key], value), "Arrival/final mismatch: " + key)
|
||||
require(bit_equal(s["g"], np.subtract(g, s["delta_si"])), "Final g mismatch")
|
||||
return due, accepted
|
||||
|
||||
def _residual(self, voltage, g):
|
||||
# No repair: anatomical lanes are never contaminated with roundoff.
|
||||
vr = voltage - np.add(self.bv, self.sv.sum(axis=(1, 2)))
|
||||
gr = g - np.add(self.bg, self.sg.sum(axis=(1, 2)))
|
||||
self.max_v_residual = np.maximum(self.max_v_residual, np.abs(vr))
|
||||
self.max_g_residual = np.maximum(self.max_g_residual, np.abs(gr))
|
||||
return gr
|
||||
|
||||
def __call__(self, tick, s):
|
||||
due, accepted = self._validate(tick, s)
|
||||
if self.previous is None:
|
||||
self.bv, self.bg = s["old_v"].copy(), s["old_g"].copy()
|
||||
c, available, fired = self.coeff, s["available"], s["fired"]
|
||||
self.sv[available] = np.add(np.multiply(c["cg"], self.sg[available]),
|
||||
np.multiply(c["cv"], self.sv[available]))
|
||||
self.bv[available] = np.add(c["c0"], np.add(np.multiply(c["cg"], self.bg[available]),
|
||||
np.multiply(c["cv"], self.bv[available])))
|
||||
self.sg[available] = np.multiply(c["gd"], self.sg[available])
|
||||
self.bg[available] = np.multiply(c["gd"], self.bg[available])
|
||||
self._residual(s["v_integrated"], s["g_decayed"])
|
||||
req = s["request_accepted"]
|
||||
self.sv[req], self.bv[req] = 0., c["seed_volt"]
|
||||
self._residual(s["v_threshold"], s["g_decayed"])
|
||||
self.sv[fired], self.sg[fired] = 0., 0.
|
||||
self.bv[fired], self.bg[fired] = c["reset"], 0.
|
||||
self.first[(self.first < 0) & fired] = np.int32(tick)
|
||||
self.spike_count += fired.astype(np.int32)
|
||||
for key, stage_key in (("request_count", "request"), ("request_accepted_count", "request_accepted"),
|
||||
("request_rejected_count", "request_rejected")):
|
||||
self.requests[key] += s[stage_key].astype(np.int64)
|
||||
dst, codes, weights = self.dst_local[due], self.groups[due], self.weights[due]
|
||||
self.edge_due[due] += 1
|
||||
self.edge_accepted[due[accepted]] += 1
|
||||
for key, keep in (("due_count", np.ones(len(due), bool)), ("accepted_count", accepted),
|
||||
("rejected_count", ~accepted), ("accepted_zero_count", accepted & (weights == 0)),
|
||||
("rejected_zero_count", ~accepted & (weights == 0))):
|
||||
np.add.at(self.counts[key], (dst[keep], codes[keep]), 1)
|
||||
for key, keep in (("accepted_positive_V", accepted & (weights > 0)),
|
||||
("accepted_negative_V", accepted & (weights < 0)),
|
||||
("rejected_positive_V", ~accepted & (weights > 0)),
|
||||
("rejected_negative_V", ~accepted & (weights < 0))):
|
||||
np.add.at(self.increments[key], (dst[keep], codes[keep]), weights[keep])
|
||||
nonzero = accepted & (weights != 0)
|
||||
# add.at is sequential for repeated lane indices, preserving every
|
||||
# lane's relative canonical edge order. Never pre-sum edge weights.
|
||||
np.add.at(self.sg, (dst[nonzero], codes[nonzero], (weights[nonzero] < 0).astype(np.int32)), weights[nonzero])
|
||||
gr = self._residual(s["v"], s["g"])
|
||||
self.integral_g += np.multiply(self.sg, np.float64(DT_MS))
|
||||
self.integral_v += np.multiply(self.sv, np.float64(DT_MS))
|
||||
for value in (self.sg, self.sv, self.bg, self.bv, self.integral_g, self.integral_v,
|
||||
self.max_v_residual, self.max_g_residual, *self.increments.values()):
|
||||
require(np.isfinite(value).all(), "Nonfinite source accounting")
|
||||
self.history_v[tick], self.history_g[tick], self.history_ref[tick] = s["v"], s["g"], s["ref"]
|
||||
self.history_group_g[tick] = self.sg
|
||||
self.history_baseline_g[tick], self.history_g_residual[tick] = self.bg, gr
|
||||
self.previous = {key: s[key].copy() for key in ("v", "g", "ref")}
|
||||
self.ticks += 1
|
||||
|
||||
def finish(self):
|
||||
require(not self.finished and self.ticks == self.steps and self.previous is not None,
|
||||
"Observer must cover exactly all ticks before finish")
|
||||
require(np.array_equal(self.counts["due_count"], self.counts["accepted_count"] + self.counts["rejected_count"]),
|
||||
"Group count conservation mismatch")
|
||||
for key, edge in (("due_count", self.edge_due), ("accepted_count", self.edge_accepted)):
|
||||
grouped = np.zeros_like(self.counts[key])
|
||||
np.add.at(grouped, (self.dst_local, self.groups), edge)
|
||||
require(np.array_equal(grouped, self.counts[key]), "Per-edge/group count mismatch")
|
||||
data = {"target_local": np.arange(self.n, dtype=np.int32), "spike_count": self.spike_count.copy(),
|
||||
"first_spike_tick": self.first.copy(), **{key: value.copy() for key, value in self.requests.items()},
|
||||
**{key: value.copy() for key, value in self.counts.items()},
|
||||
**{key: value.copy() for key, value in self.increments.items()},
|
||||
**{"final_" + key: value.copy() for key, value in self.previous.items()},
|
||||
"final_group_g": self.sg.copy(), "final_group_v": self.sv.copy(),
|
||||
"g_integral_V_ms": self.integral_g.sum(axis=2), "g_sign_integral_V_ms": self.integral_g.copy(),
|
||||
"v_sign_integral_V_ms": self.integral_v.copy(),
|
||||
"max_g_lane_residual_V": self.max_g_residual.copy(), "max_v_lane_residual_V": self.max_v_residual.copy(),
|
||||
"history_v": self.history_v.copy(), "history_g": self.history_g.copy(), "history_ref": self.history_ref.copy(),
|
||||
"history_group_g": self.history_group_g.copy(), "history_baseline_g": self.history_baseline_g.copy(),
|
||||
"history_g_residual": self.history_g_residual.copy()}
|
||||
edges = {"edge_id": self.edge_id.copy(), "source_neuron_idx": self.src_global.copy(),
|
||||
"target_local": self.dst_local.copy(), "group": self.groups.copy(), "weight_V": self.weights.copy(),
|
||||
"due_count": self.edge_due.copy(), "accepted_count": self.edge_accepted.copy(),
|
||||
"rejected_count": self.edge_due - self.edge_accepted}
|
||||
groups = {name: {**{key: int(value[:, gi].sum()) for key, value in self.counts.items()},
|
||||
**{key: float(value[:, gi].sum()) for key, value in self.increments.items()},
|
||||
"g_integral_V_ms": float(self.integral_g[:, gi].sum()),
|
||||
"g_positive_integral_V_ms": float(self.integral_g[:, gi, 0].sum()),
|
||||
"g_negative_integral_V_ms": float(self.integral_g[:, gi, 1].sum())}
|
||||
for gi, name in enumerate(self.names)}
|
||||
summary = {"complete": True, "pass_all": True,
|
||||
"pass_all_scope": "Observer callback coverage and internal count/state-accounting checks only",
|
||||
"steps": self.steps, "callback_count": self.ticks, "targets": self.n, "incoming_edges": self.e,
|
||||
"group_names": list(self.names), "sign_order": ["positive", "negative"], "dt_ms": DT_MS,
|
||||
"due_count": int(self.edge_due.sum()), "accepted_count": int(self.edge_accepted.sum()),
|
||||
"rejected_count": int((self.edge_due - self.edge_accepted).sum()), "groups": groups,
|
||||
"max_g_lane_residual_V": float(self.max_g_residual.max(initial=0.)),
|
||||
"max_v_lane_residual_V": float(self.max_v_residual.max(initial=0.)),
|
||||
"residual_policy": "Observed state minus uncorrected float64 lane sum; no fitted tolerance or anatomical reassignment",
|
||||
"integral_contract": "Sum of end-of-tick source g or v lanes times dt_ms; units V*ms, not conductance or energy",
|
||||
"increment_contract": "Nominal accepted/rejected signed per-edge increments; negative fields retain negative sign",
|
||||
"lane_contract": "Observed availability controls decay/integration; observed request/reset clears voltage/source history; fired clears g; canonical accepted adds only",
|
||||
"pending_coverage_checked_by_observer": False, "original_events_verified_by_observer": False,
|
||||
"caller_must_verify_original_events_final_and_queue_coverage": True,
|
||||
"nonfeedback": True, "parameter_changes": False, "learning": False,
|
||||
"functional_transfer_claim": False, "whole_brain_intervention_claim": False}
|
||||
# Includes aggregate overflows: a finite per-target array alone must
|
||||
# never permit an Infinity/NaN JSON report with pass_all=True.
|
||||
json.dumps(summary, allow_nan=False)
|
||||
self.finished = True
|
||||
return data, edges, summary
|
||||
@@ -0,0 +1,136 @@
|
||||
"""Preregister all AK trials before credit/MBON outcomes; never train."""
|
||||
from datetime import datetime, timezone
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import yaml
|
||||
from context import (ROOT, MODEL, EXP, AK, H, AG, META, CORE, COEFF, CFG, PINS,
|
||||
read, save, digest, sources, integrity, write_npz)
|
||||
|
||||
|
||||
def main():
|
||||
assert not EXP.exists() or (EXP.is_dir() and not any(EXP.iterdir())), 'Populated experiment exists: preserve it, do not regenerate'
|
||||
assert shutil.disk_usage(EXP.parent).free > CFG['output_budget_bytes'] + 10 * 1024**3
|
||||
for relative, expected in PINS.items():
|
||||
assert digest(ROOT / relative) == expected, relative
|
||||
assert read(AK / 'audit/final-status.json')['status'] == 'PASS-FROZEN-D18-INDEPENDENT-VALIDATION'
|
||||
assert not CFG['training'] and not CFG['candidate_generation']
|
||||
jc = yaml.safe_load((ROOT / 'models/male-cns-lif/mb-physiology-v1/phase4j.yaml').read_text())
|
||||
j = jc['eligibility_v2']
|
||||
assert j['kc_trace_tau_ms'] == CFG['kc_trace_tau_ms'] and j['trace_cap'] == CFG['trace_cap']
|
||||
assert j['max_fractional_depression'] == CFG['conditional_depression_fraction'] == .25
|
||||
assert jc['direct_kc']['dan_time_ms'] / CFG['dt_ms'] == CFG['virtual_gate_tick']
|
||||
inventory = read(AK / 'protocol/validation-cases.json')
|
||||
assert len(inventory) == CFG['primary_trials']
|
||||
seeds = sorted({r['seed'] for r in inventory})
|
||||
assert len(seeds) == 5
|
||||
planned = []
|
||||
for row in inventory:
|
||||
for repeat in (0, 1) if row['seed'] == seeds[0] else (0,):
|
||||
planned.append(dict(row, repeat=repeat))
|
||||
assert len(planned) == CFG['primary_trials'] + CFG['technical_repeats']
|
||||
inputs = dict(PINS)
|
||||
h_manifest = {line.split(' ', 1)[1]: line.split(' ', 1)[0]
|
||||
for line in (H / 'manifests/checksums.sha256').read_text().splitlines()}
|
||||
static_h = ('plastic-mask/plastic-edge-candidates.parquet', 'plastic-mask/plastic-edge-ids.int64.npy',
|
||||
'plastic-mask/plastic-edge-mask.bool.npy', 'mappings/dan-to-mbon-confirmed.parquet',
|
||||
'mappings/mbon-to-compartment.parquet', 'mappings/kc-to-mbon-actual-edges.parquet')
|
||||
for rel in static_h:
|
||||
assert digest(H / rel) == h_manifest[rel]
|
||||
inputs[(H / rel).relative_to(ROOT).as_posix()] = h_manifest[rel]
|
||||
for row in planned:
|
||||
case = AK / 'cases' / row['cohort'] / row['name']
|
||||
trial = AK / 'gpu' / row['cohort'] / (row['name'] + '-r' + str(row['repeat']))
|
||||
for p in (case / 'spec.json', case / 'requests.npz', trial / 'state.npz', trial / 'summary.json'):
|
||||
inputs[p.relative_to(ROOT).as_posix()] = digest(p)
|
||||
paths = [COEFF] + list((AG / 'graphs/full').glob('*.npy')) + list((AG / 'apl-bundle').glob('*.npy'))
|
||||
raw_path = ROOT / 'datasets/male-cns-v1.0/processed/neuron-only/gpu/csr-weights.int64.npy'
|
||||
for p in paths + [raw_path]:
|
||||
inputs[p.relative_to(ROOT).as_posix()] = digest(p)
|
||||
for name in ('protocol', 'trials', 'audit', 'regression', 'logs', 'pairs'):
|
||||
(EXP / name).mkdir(parents=True, exist_ok=False)
|
||||
test = subprocess.run([sys.executable, '-B', '-m', 'unittest', 'discover', '-v', '-s', str(MODEL)],
|
||||
cwd=MODEL, text=True, capture_output=True)
|
||||
with (EXP / 'regression/unit-tests.log').open('x', encoding='utf-8') as f:
|
||||
f.write(test.stdout + test.stderr)
|
||||
assert test.returncode == 0, 'Unit tests failed; preserve incomplete experiment'
|
||||
assert 'skipped=' not in test.stderr, 'Server arithmetic comparison must not be skipped'
|
||||
save(EXP / 'protocol/preregistered.json', dict(timestamp_utc=datetime.now(timezone.utc).isoformat(),
|
||||
sources=sources(), config=CFG, inputs=[dict(path=k, sha256=v) for k, v in sorted(inputs.items())],
|
||||
parent_commit=subprocess.check_output(['git', '-C', str(ROOT), 'rev-parse', 'HEAD'], text=True).strip(),
|
||||
planned=planned, prior_AK_representation_known=True, before_new_credit_and_MBON_outcomes=True,
|
||||
training=False, learning_achieved=False, virtual_gate_is_dopamine_measurement=False,
|
||||
MBON_recurrent_and_all_other_presynaptic_events_clamped=True,
|
||||
expected_functional_effect_not_a_preregistered_learning_gate=True,
|
||||
python=sys.version, platform=platform.platform(), numpy=np.__version__))
|
||||
save(EXP / 'regression/old-integrity-before.json', integrity())
|
||||
meta = pd.read_parquet(META).sort_values('neuron_idx').reset_index(drop=True)
|
||||
assert np.array_equal(meta.neuron_idx, np.arange(CFG['neurons'], dtype=np.int32))
|
||||
kc = np.flatnonzero(meta.annotation_class.eq('Kenyon_Cell')).astype(np.int32)
|
||||
mbon = np.flatnonzero(meta.annotation_class.eq('MBON')).astype(np.int32)
|
||||
apl = np.flatnonzero(meta.annotation_type.eq('APL')).astype(np.int32)
|
||||
assert len(kc) == 4064 and len(mbon) == 97
|
||||
assert np.array_equal(kc, np.load(AG / 'apl-bundle/kc_globals.npy'))
|
||||
assert np.array_equal(apl, np.load(AG / 'apl-bundle/apl_globals.npy'))
|
||||
folder = AG / 'graphs/full'
|
||||
arrays = {k: np.load(folder / (k + '.npy'), mmap_mode='r') for k in ('nodes', 'src', 'dst', 'edge_id', 'weights')}
|
||||
assert np.array_equal(arrays['nodes'], np.arange(CFG['neurons'], dtype=np.int32))
|
||||
assert np.array_equal(arrays['edge_id'], np.arange(CFG['edges'], dtype=np.int64))
|
||||
raw = np.load(raw_path, mmap_mode='r')
|
||||
candidate = pd.read_parquet(H / static_h[0])
|
||||
ids = np.load(H / static_h[1]); mask = np.load(H / static_h[2])
|
||||
assert ids.dtype == np.int64 and ids.shape == (CFG['candidate_edges'],)
|
||||
assert np.array_equal(candidate.edge_id, ids) and np.array_equal(np.flatnonzero(mask), ids)
|
||||
for prefix, key in (('source', 'src'), ('target', 'dst')):
|
||||
assert np.array_equal(candidate[prefix + '_neuron_idx'], arrays[key][ids])
|
||||
assert np.array_equal(candidate[prefix + '_body_id'], meta.iloc[arrays[key][ids]].body_id)
|
||||
assert np.array_equal(candidate.raw_synapse_count, raw[ids])
|
||||
assert np.isin(candidate.source_neuron_idx, kc).all() and np.isin(candidate.target_neuron_idx, mbon).all()
|
||||
assert (candidate.plasticity_polarity == -1).all() and candidate.target_compartment_verbatim.notna().all()
|
||||
confirmed = pd.read_parquet(H / static_h[3])
|
||||
assert confirmed.exact_compartment_match.all()
|
||||
assert np.array_equal(confirmed.source_compartment_verbatim, confirmed.target_compartment_verbatim)
|
||||
for row in confirmed.itertuples():
|
||||
assert arrays['src'][row.edge_id] == row.source_neuron_idx and arrays['dst'][row.edge_id] == row.target_neuron_idx
|
||||
assert raw[row.edge_id] == row.raw_synapse_count and meta.iloc[row.source_neuron_idx].annotation_class == 'DAN'
|
||||
assert set(candidate.target_neuron_idx).issubset(set(confirmed.target_neuron_idx))
|
||||
compartments = pd.read_parquet(H / static_h[4]).sort_values('neuron_idx').reset_index(drop=True)
|
||||
assert np.array_equal(compartments.neuron_idx, mbon)
|
||||
compartments.to_parquet(EXP / 'protocol/mbon-metadata.parquet', index=False)
|
||||
candidate.to_parquet(EXP / 'protocol/candidate-static.parquet', index=False)
|
||||
lookup = np.full(CFG['neurons'], -1, np.int32); lookup[mbon] = np.arange(len(mbon), dtype=np.int32)
|
||||
selected = np.flatnonzero(lookup[arrays['dst']] >= 0)
|
||||
src, dst = arrays['src'][selected], lookup[arrays['dst'][selected]]
|
||||
weights = arrays['weights'][selected].copy()
|
||||
assert weights.dtype == np.float64 and raw.dtype == np.int64
|
||||
weights[np.isin(src, apl)] = 0.0
|
||||
is_candidate = mask[selected]
|
||||
assert int(is_candidate.sum()) == CFG['candidate_edges'] and (weights[is_candidate] > 0).all()
|
||||
groups = np.full(len(selected), CFG['groups'].index('other'), np.int32)
|
||||
for name, chosen in (('nonplastic_KC', np.isin(src, kc)), ('DAN', meta.iloc[src].annotation_class.eq('DAN').to_numpy()),
|
||||
('MBON', np.isin(src, mbon)), ('APL', np.isin(src, apl)), ('plastic_KC', is_candidate)):
|
||||
groups[chosen] = CFG['groups'].index(name)
|
||||
kcall = pd.read_parquet(H / static_h[5])
|
||||
assert np.array_equal(selected[np.isin(src, kc)], kcall.edge_id.to_numpy())
|
||||
assert np.array_equal(raw[kcall.edge_id.to_numpy()], kcall.raw_synapse_count.to_numpy())
|
||||
write_npz(EXP / 'protocol/incoming.npz', src_global=src, dst_local=dst, edge_id=selected.astype(np.int64),
|
||||
weights=weights, raw_contacts=raw[selected], groups=groups, is_candidate=is_candidate,
|
||||
targets=mbon, kc_globals=kc, edge_source_local=np.searchsorted(kc, candidate.source_neuron_idx).astype(np.int32))
|
||||
save(EXP / 'protocol/graph-audit.json', dict(neurons=len(meta), all_edges=CFG['edges'], KC=len(kc), MBON=len(mbon),
|
||||
all_MBON_incoming_edges=len(selected), all_KC_MBON_edges=len(kcall), candidate_edges=len(ids),
|
||||
candidate_targets=int(candidate.target_neuron_idx.nunique()), confirmed_DAN_MBON_edges=len(confirmed),
|
||||
compartments={str(c): dict(edges=len(g), targets=int(g.target_neuron_idx.nunique()), sources=int(g.source_neuron_idx.nunique()))
|
||||
for c, g in candidate.groupby('target_compartment_verbatim')},
|
||||
fast_APL_incoming_edges_zeroed=int(np.isin(src, apl).sum()), groups={name: int((groups == i).sum()) for i, name in enumerate(CFG['groups'])},
|
||||
mask_all_edges_identity_verified=True, mapping_basis='Frozen explicit annotation token + real DAN-to-MBON edge; not synapse ROI',
|
||||
feedback_neuron_class_not_inferred=True, weight_units='float64 SI volts; not measured conductance'))
|
||||
save(EXP / 'protocol/prepared-hashes.json', [dict(path=p.relative_to(EXP).as_posix(), bytes=p.stat().st_size, sha256=digest(p))
|
||||
for p in sorted((EXP / 'protocol').iterdir()) if p.is_file() and p.name != 'prepared-hashes.json'])
|
||||
print('PREREGISTERED 120 primary +24 repeats; static edge identities validated; no new trajectory or training', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,160 @@
|
||||
"""CPU-only read-only baseline and one predeclared conditional sensitivity."""
|
||||
from concurrent.futures import ProcessPoolExecutor, as_completed
|
||||
from functools import lru_cache
|
||||
import time
|
||||
import traceback
|
||||
import numpy as np
|
||||
from context import (ROOT, EXP, AK, CORE, COEFF, CFG, PINS, read, save, digest,
|
||||
load_module, check, budget, bit_equal, write_npz)
|
||||
from credit import replay_credit
|
||||
from mbon_accounting import MBONAccounting
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def prepared():
|
||||
core = load_module('sealed_ah_core_al', CORE, PINS[CORE.relative_to(ROOT).as_posix()])
|
||||
with np.load(EXP / 'protocol/incoming.npz', allow_pickle=False) as f:
|
||||
data = {k: f[k].copy() for k in f.files}
|
||||
incoming = core.IncomingIndex(data['src_global'], data['dst_local'], data['edge_id'], data['weights'], len(data['targets']))
|
||||
altered = data['weights'].copy()
|
||||
altered[data['is_candidate']] *= np.float64(1.0 - CFG['conditional_depression_fraction'])
|
||||
assert bit_equal(data['weights'][~data['is_candidate']], altered[~data['is_candidate']])
|
||||
assert np.array_equal(np.sign(data['weights']), np.sign(altered))
|
||||
depressed = core.IncomingIndex(data['src_global'], data['dst_local'], data['edge_id'], altered, len(data['targets']))
|
||||
for array in data.values():
|
||||
array.setflags(write=False)
|
||||
return core, data, incoming, depressed, read(COEFF)
|
||||
|
||||
|
||||
def compare_results(base, changed, base_history, changed_history):
|
||||
event_equal = all(bit_equal(base[k], changed[k]) for k in ('step', 'neuron_idx'))
|
||||
count_changes = base['counts'] != changed['counts']
|
||||
latency_changes = base['first_spike'] != changed['first_spike']
|
||||
return dict(event_sequence_changed=not event_equal,
|
||||
spike_count_L1=int(np.abs(base['counts'].astype(np.int64) - changed['counts']).sum()),
|
||||
neurons_with_count_change=int(count_changes.sum()), neurons_with_first_latency_change=int(latency_changes.sum()),
|
||||
baseline_spikes=int(base['counts'].sum()), depressed_spikes=int(changed['counts'].sum()),
|
||||
max_abs_V_difference_SI=float(np.max(np.abs(base_history['history_v'] - changed_history['history_v']))),
|
||||
max_abs_g_difference_SI=float(np.max(np.abs(base_history['history_g'] - changed_history['history_g']))),
|
||||
final_V_max_abs_difference_SI=float(np.max(np.abs(base['v'] - changed['v']))),
|
||||
final_g_max_abs_difference_SI=float(np.max(np.abs(base['g'] - changed['g']))),
|
||||
functional_count_or_first_latency_change=bool(count_changes.any() or latency_changes.any()),
|
||||
result_scope='Conditional clamped-upstream replay only; all target threshold/reset/ref recalculated; not whole-brain or learning')
|
||||
|
||||
|
||||
def audit_trial(row, stage):
|
||||
start = time.perf_counter()
|
||||
key = row['name'] + '-r' + str(row['repeat'])
|
||||
out = EXP / 'trials' / key
|
||||
assert stage in ('baseline', 'conditional25')
|
||||
if stage == 'baseline':
|
||||
out.mkdir(exist_ok=False)
|
||||
else:
|
||||
assert read(EXP / 'audit/baselines-complete.json')['pass_all'], 'All 144 baseline replays must pass first'
|
||||
assert out.is_dir() and not (out / 'summary.json').exists()
|
||||
case = AK / 'cases' / row['cohort'] / row['name']
|
||||
trial = AK / 'gpu' / row['cohort'] / key
|
||||
spec = read(case / 'spec.json')
|
||||
assert spec['steps'] == CFG['steps'] and spec['dt_ms'] == CFG['dt_ms'] and spec['apl_delay_steps'] == 18
|
||||
assert all(spec[k] == row[k] for k in ('name', 'seed', 'mode', 'odor_id'))
|
||||
assert read(trial / 'summary.json')['complete'] and read(trial / 'summary.json')['pass_all']
|
||||
with np.load(trial / 'state.npz', allow_pickle=False) as f:
|
||||
state = {k: f[k].copy() for k in f.files}
|
||||
assert set(state) == {'step', 'neuron_idx', 'counts', 'v', 'g', 'ref'}
|
||||
with np.load(case / 'requests.npz', allow_pickle=False) as f:
|
||||
requests = {k: f[k].copy() for k in f.files}
|
||||
core, data, incoming, depressed, coeff = prepared()
|
||||
targets, steps = data['targets'], spec['steps']
|
||||
assert not np.isin(requests['neuron_idx'], targets).any(), 'Unexpected MBON injection: do not reinterpret it'
|
||||
v0 = np.full(len(targets), spec['model']['resting_potential_mv'], np.float64) * np.float64(.001)
|
||||
g0 = np.zeros(len(targets), np.float64)
|
||||
events = dict(step=state['step'], neuron_idx=state['neuron_idx'])
|
||||
if stage == 'baseline':
|
||||
observer = MBONAccounting(incoming, data['groups'], tuple(CFG['groups']), coeff, steps)
|
||||
result = core.replay(v0, g0, coeff, steps, targets, requests, events, incoming,
|
||||
np.zeros((steps, len(targets)), np.float32), observer)
|
||||
selected = np.isin(state['neuron_idx'], targets)
|
||||
exact = {k: bit_equal(result[k], state[k][targets]) for k in ('counts', 'v', 'g', 'ref')}
|
||||
exact.update({k: bit_equal(result[k], state[k][selected]) for k in ('step', 'neuron_idx')})
|
||||
assert all(exact.values()), 'Baseline MBON divergence: ' + str(exact)
|
||||
bt, be, bs = observer.finish()
|
||||
write_npz(out / 'baseline-replay.npz', **result)
|
||||
write_npz(out / 'baseline-target-accounting.npz', **bt)
|
||||
pending = np.zeros(len(incoming.edge_id), np.int64)
|
||||
for tick in range(steps - 18, steps):
|
||||
due = incoming.due(state['neuron_idx'][state['step'] == tick])
|
||||
pending[due] += 1
|
||||
be['pending_after_window_count'] = pending
|
||||
# Independent source-spike multiplicity covers due plus pending events.
|
||||
source_counts = np.bincount(state['neuron_idx'], minlength=CFG['neurons'])
|
||||
assert np.array_equal(be['due_count'] + pending, source_counts[incoming.src_global])
|
||||
write_npz(out / 'baseline-edge-accounting.npz', **be)
|
||||
save(out / 'baseline-accounting.json', bs)
|
||||
trace = replay_credit(state['step'], state['neuron_idx'], data['kc_globals'],
|
||||
steps=steps, gate_tick=CFG['virtual_gate_tick'], dt_ms=CFG['dt_ms'],
|
||||
tau_ms=CFG['kc_trace_tau_ms'], cap=CFG['trace_cap'])
|
||||
arrays = {k: value for k, value in trace.items() if isinstance(value, np.ndarray)}
|
||||
write_npz(out / 'credit.npz', **arrays)
|
||||
save(out / 'credit.json', {k: value for k, value in trace.items() if not isinstance(value, np.ndarray)})
|
||||
report = dict(**row, pass_all=True, baseline_exact=exact, due_plus_pending_source_events_exact=True,
|
||||
CPU_wall_seconds=time.perf_counter() - start)
|
||||
save(out / 'baseline-summary.json', report)
|
||||
return report
|
||||
with np.load(out / 'baseline-replay.npz', allow_pickle=False) as f:
|
||||
result = {k: f[k].copy() for k in f.files}
|
||||
with np.load(out / 'baseline-target-accounting.npz', allow_pickle=False) as f:
|
||||
bt = {k: f[k].copy() for k in f.files}
|
||||
baseline_report = read(out / 'baseline-summary.json')
|
||||
assert baseline_report['pass_all'] and all(baseline_report['baseline_exact'].values())
|
||||
exact = baseline_report['baseline_exact']
|
||||
d_observer = MBONAccounting(depressed, data['groups'], tuple(CFG['groups']), coeff, steps)
|
||||
changed = core.replay(v0, g0, coeff, steps, targets, requests, events, depressed,
|
||||
np.zeros((steps, len(targets)), np.float32), d_observer)
|
||||
dt, de, ds = d_observer.finish()
|
||||
write_npz(out / 'conditional25-replay.npz', **changed)
|
||||
write_npz(out / 'conditional25-target-accounting.npz', **dt)
|
||||
write_npz(out / 'conditional25-edge-accounting.npz', **de)
|
||||
save(out / 'conditional25-accounting.json', ds)
|
||||
sensitivity = compare_results(result, changed, bt, dt)
|
||||
save(out / 'sensitivity.json', sensitivity)
|
||||
report = dict(**row, pass_all=True, baseline_exact=exact, sensitivity=sensitivity,
|
||||
credit_gate='Virtual tick900 post-update; no DAN injection or plastic update',
|
||||
biological_duration_ms=spec['duration_ms'], CPU_wall_seconds=time.perf_counter() - start,
|
||||
baseline_CPU_wall_seconds=baseline_report['CPU_wall_seconds'],
|
||||
source_sha256={f: digest(trial / f) for f in ('state.npz', 'summary.json')},
|
||||
conditional_weight_copy_only=True, training=False, candidate_generated=False)
|
||||
save(out / 'summary.json', report)
|
||||
save(out / 'output-hashes.json', [dict(path=p.name, bytes=p.stat().st_size, sha256=digest(p))
|
||||
for p in sorted(out.iterdir()) if p.is_file()])
|
||||
return report
|
||||
|
||||
|
||||
def main():
|
||||
reg = check()
|
||||
assert not any((EXP / 'trials').iterdir()), 'No automatic resume/overwrite of diagnostic outputs'
|
||||
complete = []
|
||||
try:
|
||||
for stage in ('baseline', 'conditional25'):
|
||||
complete = []
|
||||
with ProcessPoolExecutor(max_workers=CFG['workers']) as pool:
|
||||
futures = {pool.submit(audit_trial, row, stage): row for row in reg['planned']}
|
||||
for task in as_completed(futures):
|
||||
result = task.result()
|
||||
complete.append(dict(name=result['name'], repeat=result['repeat'], pass_all=True))
|
||||
budget()
|
||||
print(stage, len(complete), result['name'], result['repeat'],
|
||||
round(result['CPU_wall_seconds'], 3), 'CPU s', flush=True)
|
||||
assert len(complete) == 144
|
||||
if stage == 'baseline':
|
||||
save(EXP / 'audit/baselines-complete.json', dict(pass_all=True, trials=complete, primary=120, repeats=24))
|
||||
except Exception as error:
|
||||
save(EXP / 'audit/failure.json', dict(error=type(error).__name__, message=str(error),
|
||||
traceback=traceback.format_exc(), stage=stage, completed=complete))
|
||||
raise
|
||||
assert len(complete) == 144
|
||||
save(EXP / 'audit/replays-complete.json', dict(pass_all=True, trials=complete, primary=120, repeats=24,
|
||||
meaning='All exact conditional baseline reconstructions; not an overall biological or learning PASS'))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,177 @@
|
||||
"""All predeclared pairs and all trial outputs; never choose a winning odor."""
|
||||
from itertools import combinations
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from context import EXP, AK, CFG, read, save as _save, digest, bit_equal, write_npz as _write_npz
|
||||
from credit import pair_identity
|
||||
|
||||
|
||||
def load_arrays(path):
|
||||
with np.load(path, allow_pickle=False) as data:
|
||||
return {key: data[key].copy() for key in data.files}
|
||||
|
||||
|
||||
def metrics(values):
|
||||
values = np.asarray(values, np.float64)
|
||||
if not len(values):
|
||||
return dict(min=None, mean=None, max=None, undefined_reason='No defined denominator observations')
|
||||
return dict(min=float(values.min()), mean=float(values.mean()), max=float(values.max()))
|
||||
|
||||
|
||||
def primary_rows(reg):
|
||||
rows = [r for r in reg['planned'] if r['repeat'] == 0]
|
||||
assert len(rows) == CFG['primary_trials'] and len({r['name'] for r in rows}) == len(rows)
|
||||
return rows
|
||||
|
||||
|
||||
def run(verify_only=False):
|
||||
def save(path, value):
|
||||
if verify_only:
|
||||
assert read(path) == value, 'Summary JSON differs from recomputed evidence: ' + str(path)
|
||||
else:
|
||||
_save(path, value)
|
||||
|
||||
def write_npz(path, **value):
|
||||
if verify_only:
|
||||
old = load_arrays(path)
|
||||
assert set(old) == set(value) and all(bit_equal(old[k], value[k]) for k in old)
|
||||
else:
|
||||
_write_npz(path, **value)
|
||||
|
||||
reg = read(EXP / 'protocol/preregistered.json')
|
||||
assert read(EXP / 'audit/replays-complete.json')['pass_all']
|
||||
data = load_arrays(EXP / 'protocol/incoming.npz')
|
||||
for row in reg['planned']:
|
||||
folder = EXP / 'trials' / (row['name'] + '-r' + str(row['repeat']))
|
||||
hashes = read(folder / 'output-hashes.json')
|
||||
assert {r['path'] for r in hashes} == {p.name for p in folder.iterdir() if p.name != 'output-hashes.json'}
|
||||
for entry in hashes:
|
||||
assert digest(folder / entry['path']) == entry['sha256']
|
||||
original = load_arrays(AK / 'gpu' / row['cohort'] / (row['name'] + '-r' + str(row['repeat'])) / 'state.npz')
|
||||
base = load_arrays(folder / 'baseline-replay.npz')
|
||||
selected = np.isin(original['neuron_idx'], data['targets'])
|
||||
assert all(bit_equal(base[k], original[k][data['targets']]) for k in ('v', 'g', 'ref', 'counts'))
|
||||
assert all(bit_equal(base[k], original[k][selected]) for k in ('step', 'neuron_idx'))
|
||||
candidate = pd.read_parquet(EXP / 'protocol/candidate-static.parquet')
|
||||
target_meta = pd.read_parquet(EXP / 'protocol/mbon-metadata.parquet')
|
||||
primary = primary_rows(reg)
|
||||
repeats = []
|
||||
for row in reg['planned']:
|
||||
if row['repeat'] != 1:
|
||||
continue
|
||||
paths = [EXP / 'trials' / (row['name'] + '-r' + str(i)) for i in (0, 1)]
|
||||
checks = {}
|
||||
for filename in ('baseline-replay.npz', 'baseline-target-accounting.npz', 'baseline-edge-accounting.npz',
|
||||
'conditional25-replay.npz', 'conditional25-target-accounting.npz', 'conditional25-edge-accounting.npz', 'credit.npz'):
|
||||
a, b = (load_arrays(p / filename) for p in paths)
|
||||
checks[filename] = set(a) == set(b) and all(bit_equal(a[k], b[k]) for k in a)
|
||||
for filename in ('baseline-accounting.json', 'conditional25-accounting.json', 'credit.json', 'sensitivity.json'):
|
||||
checks[filename] = read(paths[0] / filename) == read(paths[1] / filename)
|
||||
assert all(checks.values()), 'Repeat accounting differs: ' + row['name']
|
||||
repeats.append(dict(name=row['name'], checks=checks, pass_all=True))
|
||||
assert len(repeats) == CFG['technical_repeats']
|
||||
save(EXP / 'audit/repeat-checks.json', dict(pass_all=True, repeats=repeats))
|
||||
summaries, target_rows, trial_metrics = [], [], []
|
||||
for row in primary:
|
||||
folder = EXP / 'trials' / (row['name'] + '-r0')
|
||||
report = read(folder / 'summary.json')
|
||||
assert report['pass_all'] and all(report['baseline_exact'].values())
|
||||
summaries.append(report)
|
||||
targets = load_arrays(folder / 'baseline-target-accounting.npz')
|
||||
change = load_arrays(folder / 'conditional25-target-accounting.npz')
|
||||
edges = load_arrays(folder / 'baseline-edge-accounting.npz')
|
||||
exc = targets['accepted_positive_V'].sum(axis=1)
|
||||
plastic_exc = targets['accepted_positive_V'][:, 0]
|
||||
area_exc = targets['g_sign_integral_V_ms'][:, :, 0].sum(axis=1)
|
||||
area_plastic = targets['g_sign_integral_V_ms'][:, 0, 0]
|
||||
for i, meta in enumerate(target_meta.itertuples()):
|
||||
entry = dict(name=row['name'], cohort=row['cohort'], mode=row['mode'], odor_id=row['odor_id'], seed=row['seed'],
|
||||
neuron_idx=int(meta.neuron_idx), body_id=int(meta.body_id), annotation_type=meta.annotation_type,
|
||||
compartment_verbatim=meta.mb_compartment_verbatim,
|
||||
candidate_target=bool(np.any(candidate.target_neuron_idx.to_numpy() == meta.neuron_idx)),
|
||||
accepted_total_positive_V=float(exc[i]), accepted_plastic_positive_V=float(plastic_exc[i]),
|
||||
plastic_accepted_positive_fraction=None if exc[i] == 0 else float(plastic_exc[i] / exc[i]),
|
||||
positive_source_g_area_V_ms=float(area_exc[i]), plastic_positive_g_area_V_ms=float(area_plastic[i]),
|
||||
plastic_positive_g_area_fraction=None if area_exc[i] == 0 else float(area_plastic[i] / area_exc[i]),
|
||||
baseline_spikes=int(targets['spike_count'][i]), conditional25_spikes=int(change['spike_count'][i]),
|
||||
baseline_first_tick=int(targets['first_spike_tick'][i]), conditional25_first_tick=int(change['first_spike_tick'][i]),
|
||||
max_abs_V_delta_SI=float(np.max(np.abs(targets['history_v'][:, i] - change['history_v'][:, i]))),
|
||||
max_abs_g_delta_SI=float(np.max(np.abs(targets['history_g'][:, i] - change['history_g'][:, i]))))
|
||||
target_rows.append(entry)
|
||||
pick = data['is_candidate']
|
||||
summary = dict(name=row['name'], cohort=row['cohort'], mode=row['mode'], odor_id=row['odor_id'], seed=row['seed'],
|
||||
plastic_due_events=int(edges['due_count'][pick].sum()), plastic_accepted_events=int(edges['accepted_count'][pick].sum()),
|
||||
plastic_rejected_events=int(edges['rejected_count'][pick].sum()),
|
||||
plastic_pending_events=int(edges['pending_after_window_count'][pick].sum()),
|
||||
plastic_edges_with_accepted_events=int(np.count_nonzero(edges['accepted_count'][pick])),
|
||||
accepted_positive_V=float(exc.sum()), plastic_accepted_positive_V=float(plastic_exc.sum()),
|
||||
plastic_accepted_positive_fraction=None if exc.sum() == 0 else float(plastic_exc.sum() / exc.sum()),
|
||||
plastic_positive_g_area_fraction=None if area_exc.sum() == 0 else float(area_plastic.sum() / area_exc.sum()),
|
||||
**report['sensitivity'])
|
||||
trial_metrics.append(summary)
|
||||
target_table = pd.DataFrame(target_rows)
|
||||
if verify_only:
|
||||
assert target_table.equals(pd.read_parquet(EXP / 'audit/per-MBON-primary.parquet'))
|
||||
else:
|
||||
target_table.to_parquet(EXP / 'audit/per-MBON-primary.parquet', index=False)
|
||||
save(EXP / 'audit/primary-trial-metrics.json', trial_metrics)
|
||||
pair_reports = []
|
||||
for cohort in sorted({r['cohort'] for r in primary}):
|
||||
for mode in sorted({r['mode'] for r in primary}):
|
||||
rows = [r for r in primary if r['cohort'] == cohort and r['mode'] == mode]
|
||||
odors = sorted({r['odor_index'] for r in rows})
|
||||
assert len(rows) == 30 and len(odors) == 6
|
||||
traces = {}; whole = {}; ids = {}
|
||||
for odor in odors:
|
||||
series = sorted((r for r in rows if r['odor_index'] == odor), key=lambda r: r['seed'])
|
||||
assert len(series) == 5
|
||||
ids[odor] = series[0]['odor_id']
|
||||
credit = [load_arrays(EXP / 'trials' / (r['name'] + '-r0') / 'credit.npz') for r in series]
|
||||
traces[odor] = np.stack([c['trace_gate'] for c in credit])
|
||||
whole[odor] = np.stack([(c['spike_count_total'] > 0).astype(np.float32) for c in credit])
|
||||
for a, b in combinations(odors, 2):
|
||||
name = f'{cohort}-{mode}-{a:02d}-vs-{b:02d}'
|
||||
result = dict(cohort=cohort, mode=mode, odor_a=ids[a], odor_b=ids[b], odor_index_a=a, odor_index_b=b)
|
||||
for definition, series in (('virtual_gate900', traces), ('whole_window_auxiliary', whole)):
|
||||
identities = pair_identity(series[a], series[b], data['edge_source_local'])
|
||||
save_arrays = {k: v for k, v in identities.items() if isinstance(v, np.ndarray)}
|
||||
write_npz(EXP / 'pairs' / (name + '-' + definition + '.npz'), **save_arrays)
|
||||
summary = identities['summary']
|
||||
# Historic local existence criterion only, no new overall pass rate.
|
||||
summary['local_specific_KC_and_edges_exist_both_sides'] = all(
|
||||
summary[unit]['category_counts'][side] >= 1 for unit in ('KC', 'edges') for side in ('A_specific', 'B_specific'))
|
||||
summary['compartments'] = {}
|
||||
for comp, group in candidate.groupby('target_compartment_verbatim', sort=True):
|
||||
sub = pair_identity(series[a], series[b], data['edge_source_local'][group.index.to_numpy()])
|
||||
summary['compartments'][str(comp)] = sub['summary']['edges']
|
||||
result[definition] = summary
|
||||
pair_reports.append(result)
|
||||
assert len(pair_reports) == 60
|
||||
save(EXP / 'audit/pair-identities.json', pair_reports)
|
||||
blocks = {}
|
||||
for cohort in sorted({r['cohort'] for r in primary}):
|
||||
for mode in sorted({r['mode'] for r in primary}):
|
||||
trials = [r for r in trial_metrics if r['cohort'] == cohort and r['mode'] == mode]
|
||||
pairs = [p for p in pair_reports if p['cohort'] == cohort and p['mode'] == mode]
|
||||
blocks[cohort + '/' + mode] = dict(trials=len(trials), pairs=len(pairs),
|
||||
pairs_with_both_specific_KC_and_edge_sets=sum(p['virtual_gate900']['local_specific_KC_and_edges_exist_both_sides'] for p in pairs),
|
||||
KC_empty_union_pairs=sum(p['virtual_gate900']['KC']['empty_union'] for p in pairs),
|
||||
edge_empty_union_pairs=sum(p['virtual_gate900']['edges']['empty_union'] for p in pairs),
|
||||
KC_active_set_Jaccard=metrics([p['virtual_gate900']['KC']['active_set_jaccard'] for p in pairs if not p['virtual_gate900']['KC']['empty_union']]),
|
||||
candidate_edge_active_set_Jaccard=metrics([p['virtual_gate900']['edges']['active_set_jaccard'] for p in pairs if not p['virtual_gate900']['edges']['empty_union']]),
|
||||
plastic_accepted_events=metrics([r['plastic_accepted_events'] for r in trials]),
|
||||
plastic_fraction_of_all_MBON_accepted_positive=metrics([r['plastic_accepted_positive_fraction'] for r in trials if r['plastic_accepted_positive_fraction'] is not None]),
|
||||
conditional25_event_changed_trials=sum(r['event_sequence_changed'] for r in trials),
|
||||
conditional25_count_or_first_latency_changed_trials=sum(r['functional_count_or_first_latency_change'] for r in trials),
|
||||
max_abs_V_difference_SI=max(r['max_abs_V_difference_SI'] for r in trials),
|
||||
max_abs_g_difference_SI=max(r['max_abs_g_difference_SI'] for r in trials))
|
||||
outcome = dict(arithmetic_and_coverage_pass=True, primary_trials=120, exact_repeats=24, pairs=60, blocks=blocks,
|
||||
candidate_generated=False, learning_achieved=False, conditioning_executed=False, scientific_overall_PASS_threshold_defined=False,
|
||||
maximum_observed_g_lane_roundoff_residual_SI=max(read(EXP / 'trials' / (r['name'] + '-r0') / 'baseline-accounting.json')['max_g_lane_residual_V'] for r in primary),
|
||||
interpretation='Read-only credit delivery and fixed conditional sensitivity; not closed-loop memory or dopamine measurement')
|
||||
save(EXP / 'audit/scientific-summary.json', outcome)
|
||||
return outcome
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
print(run(), flush=True)
|
||||
@@ -0,0 +1,237 @@
|
||||
"""Pure synthetic credit tests; no brain execution or historical dynamic read."""
|
||||
import ast
|
||||
import hashlib
|
||||
import math
|
||||
from pathlib import Path
|
||||
import types
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
|
||||
from credit import CATEGORY_CODES, N_GLOBAL, pair_identity, replay_credit
|
||||
|
||||
|
||||
SOURCE_PINS = {
|
||||
'phase4o_event_replay.py': 'b169937bab495c77d8ee6c1013877e7500e7354140d9fcc90595d984238ba44e',
|
||||
'phase4l.py': 'e97d58d8c1f44920d0e953f8f5831ddc86fa0198784e86460e315de4116ac553',
|
||||
}
|
||||
|
||||
|
||||
def frozen_tree(filename):
|
||||
"""Only static source bytes are read; never import the old script or its main."""
|
||||
here = Path(__file__).absolute().parent
|
||||
candidates = [here.parent / 'models/male-cns-lif' / filename,
|
||||
here.parent / filename,
|
||||
here.parent / ('runtime-credit-v1' if filename.startswith('phase4o') else 'olfactory-grounding-v1') / filename]
|
||||
path = next((path for path in candidates if path.is_file()), None)
|
||||
if path is None:
|
||||
raise unittest.SkipTest('Frozen arithmetic source not installed at declared static paths: ' + filename)
|
||||
source = path.read_bytes()
|
||||
if hashlib.sha256(source).hexdigest() != SOURCE_PINS[filename]:
|
||||
raise AssertionError('Frozen arithmetic source SHA mismatch: ' + str(path))
|
||||
return ast.parse(source, filename=str(path)), path
|
||||
|
||||
|
||||
def assert_bits(test, first, second):
|
||||
test.assertEqual(first.dtype, second.dtype)
|
||||
test.assertEqual(first.shape, second.shape)
|
||||
np.testing.assert_array_equal(first.view(np.uint8), second.view(np.uint8))
|
||||
|
||||
|
||||
def events(rows):
|
||||
if not rows:
|
||||
return np.empty(0, np.int32), np.empty(0, np.int32)
|
||||
array = np.asarray(rows, np.int32)
|
||||
return array[:, 0].copy(), array[:, 1].copy()
|
||||
|
||||
|
||||
class ReplayTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.kc = np.array([3, 9, 12], np.int32)
|
||||
|
||||
def test_gate_900_includes_same_tick_901_is_final_only_no_18_shift(self):
|
||||
ticks, nodes = events([(882, 12), (900, 3), (901, 9)])
|
||||
result = replay_credit(ticks, nodes, self.kc)
|
||||
self.assertEqual(result['trace_gate'][0], np.float32(1))
|
||||
self.assertEqual(result['trace_gate'][1], np.float32(0))
|
||||
decay = np.float32(math.exp(-.1 / 100))
|
||||
expected = np.float32(1)
|
||||
for _ in range(18):
|
||||
expected = np.float32(expected * decay)
|
||||
self.assertEqual(result['trace_gate'][2].view(np.uint32), expected.view(np.uint32))
|
||||
self.assertTrue(np.all(result['trace_final'] > 0))
|
||||
np.testing.assert_array_equal(result['spike_count_gate'], [1, 0, 1])
|
||||
np.testing.assert_array_equal(result['spike_count_total'], [1, 1, 1])
|
||||
self.assertEqual(result['event_shift_steps'], 0)
|
||||
self.assertTrue(result['gate_is_virtual_readout_not_DAN_activity'])
|
||||
|
||||
def test_spike_cap_resets_recency_not_accumulated_count(self):
|
||||
ticks, nodes = events([(0, 3), (7, 3), (7, 9)])
|
||||
result = replay_credit(ticks, nodes, self.kc, steps=15, gate_tick=7)
|
||||
np.testing.assert_array_equal(result['trace_gate'], np.array([1, 1, 0], np.float32))
|
||||
self.assertEqual(result['trace_final'][0].view(np.uint32), result['trace_final'][1].view(np.uint32))
|
||||
np.testing.assert_array_equal(result['spike_count_total'], [2, 1, 0])
|
||||
|
||||
def test_tick_zero_gate_and_last_tick_have_no_extra_decay(self):
|
||||
ticks, nodes = events([(0, 3), (2, 9)])
|
||||
result = replay_credit(ticks, nodes, self.kc, steps=3, gate_tick=0)
|
||||
self.assertEqual(result['trace_gate'][0], np.float32(1))
|
||||
self.assertEqual(result['trace_final'][1], np.float32(1))
|
||||
self.assertEqual(result['spike_count_gate'].tolist(), [1, 0, 0])
|
||||
|
||||
def test_empty_events_are_positive_zero_with_exact_dtypes(self):
|
||||
ticks, nodes = events([])
|
||||
result = replay_credit(ticks, nodes, self.kc, steps=2, gate_tick=1)
|
||||
for key in ('trace_gate', 'trace_final'):
|
||||
self.assertEqual(result[key].dtype, np.dtype('float32'))
|
||||
self.assertTrue(np.all(result[key].view(np.uint32) == 0))
|
||||
for key in ('spike_count_gate', 'spike_count_total'):
|
||||
self.assertEqual(result[key].dtype, np.dtype('int32'))
|
||||
self.assertFalse(result[key].any())
|
||||
|
||||
def test_non_kc_events_ignored_only_after_full_validation(self):
|
||||
ticks, nodes = events([(0, 2), (0, 3), (1, 4)])
|
||||
for value in (ticks, nodes, self.kc):
|
||||
value.flags.writeable = False
|
||||
before = [value.tobytes() for value in (ticks, nodes, self.kc)]
|
||||
result = replay_credit(ticks, nodes, self.kc, steps=2, gate_tick=1)
|
||||
self.assertEqual(result['total_input_events'], 3)
|
||||
self.assertEqual(result['total_KC_events'], 1)
|
||||
self.assertEqual(before, [value.tobytes() for value in (ticks, nodes, self.kc)])
|
||||
result['trace_gate'][0] = 0
|
||||
self.assertGreater(result['trace_final'][0], 0)
|
||||
|
||||
def test_invalid_event_order_duplicate_ranges_and_dtype(self):
|
||||
bad_rows = [[(-1, 3)], [(1000, 3)], [(0, -1)], [(0, N_GLOBAL)],
|
||||
[(1, 3), (0, 3)], [(0, 9), (0, 3)], [(0, 3), (0, 3)], [(0, 2), (0, 2)]]
|
||||
for rows in bad_rows:
|
||||
with self.subTest(rows=rows), self.assertRaises(ValueError):
|
||||
replay_credit(*events(rows), self.kc)
|
||||
for dtype in (np.int64, np.float32, np.float64):
|
||||
with self.subTest(dtype=dtype), self.assertRaises(ValueError):
|
||||
replay_credit(np.array([], dtype), np.array([], dtype), self.kc)
|
||||
with self.assertRaises(ValueError):
|
||||
replay_credit(np.array([0], np.int32), np.array([], np.int32), self.kc)
|
||||
with self.assertRaises(ValueError):
|
||||
replay_credit(np.array([np.nan]), np.array([3], np.int32), self.kc)
|
||||
|
||||
def test_invalid_kc_identities_and_frozen_parameters(self):
|
||||
empty = np.empty(0, np.int32)
|
||||
for kc in (empty, np.array([9, 3], np.int32), np.array([3, 3], np.int32),
|
||||
np.array([-1], np.int32), np.array([N_GLOBAL], np.int32), self.kc.astype(np.int64)):
|
||||
with self.subTest(kc=kc.tolist()), self.assertRaises(ValueError):
|
||||
replay_credit(empty, empty, kc)
|
||||
for keyword, value in (('steps', 0), ('steps', True), ('gate_tick', -1), ('gate_tick', 1000),
|
||||
('gate_tick', 900.), ('dt_ms', .2), ('tau_ms', 20), ('tau_ms', np.inf),
|
||||
('cap', .5), ('cap', True)):
|
||||
with self.subTest(keyword=keyword, value=value), self.assertRaises(ValueError):
|
||||
replay_credit(empty, empty, self.kc, **{keyword: value})
|
||||
|
||||
def test_independent_frozen_numpy_loop_ast_matches_all_trace_bits(self):
|
||||
tree, path = frozen_tree('phase4o_event_replay.py')
|
||||
loops = [node for node in ast.walk(tree) if isinstance(node, ast.For) and isinstance(node.target, ast.Name)
|
||||
and node.target.id == 'step' and any(isinstance(child, ast.AugAssign) and
|
||||
isinstance(child.target, ast.Name) and child.target.id == 'trace' for child in node.body)]
|
||||
self.assertEqual(len(loops), 1)
|
||||
self.assertEqual(len(loops[0].body), 4)
|
||||
code = compile(ast.fix_missing_locations(ast.Module(body=[loops[0]], type_ignores=[])), str(path), 'exec')
|
||||
ticks, nodes = events([(0, 3), (0, 12), (1, 9), (22, 3), (899, 9), (900, 3), (901, 12), (999, 3)])
|
||||
result = replay_credit(ticks, nodes, self.kc)
|
||||
lookup = {int(node): index for index, node in enumerate(self.kc)}
|
||||
for gate, field in ((900, 'trace_gate'), (999, 'trace_final')):
|
||||
by_step = {}
|
||||
for tick, node in zip(ticks, nodes):
|
||||
by_step.setdefault(int(tick), []).append(lookup[int(node)])
|
||||
namespace = dict(np=np, trace=np.zeros(len(self.kc), np.float32),
|
||||
decay=np.float32(math.exp(-.1 / 100)), by_step=by_step, step_gate=gate)
|
||||
exec(code, namespace)
|
||||
assert_bits(self, result[field], namespace['trace'])
|
||||
|
||||
def test_independent_frozen_torch_cpu_statement_matches_when_available(self):
|
||||
try:
|
||||
import torch
|
||||
except (ImportError, OSError) as error:
|
||||
self.skipTest('Torch CPU unavailable; comparison explicitly not performed: ' + str(error))
|
||||
tree, path = frozen_tree('phase4l.py')
|
||||
expressions = [node for node in ast.walk(tree) if isinstance(node, ast.Expr) and
|
||||
'trace.mul_(trace_decay).add_' in ast.unparse(node)]
|
||||
self.assertEqual(len(expressions), 1)
|
||||
code = compile(ast.fix_missing_locations(ast.Module(body=expressions, type_ignores=[])), str(path), 'exec')
|
||||
ticks, nodes = events([(0, 3), (22, 3), (899, 9), (900, 3), (901, 12), (999, 9)])
|
||||
result = replay_credit(ticks, nodes, self.kc)
|
||||
trace = torch.zeros(len(self.kc), dtype=torch.float32, device='cpu')
|
||||
namespace = dict(trace=trace, trace_decay=math.exp(-.1 / 100), torch=torch,
|
||||
self=types.SimpleNamespace(kc_gpu=torch.tensor(self.kc.astype(np.int64), device='cpu')))
|
||||
gate = None
|
||||
for tick in range(1000):
|
||||
spikes = torch.zeros(13, dtype=torch.bool, device='cpu')
|
||||
active = nodes[ticks == tick]
|
||||
if len(active):
|
||||
spikes[torch.tensor(active.astype(np.int64), device='cpu')] = True
|
||||
namespace['spikes'] = spikes
|
||||
exec(code, namespace)
|
||||
if tick == 900:
|
||||
gate = trace.numpy().copy()
|
||||
assert_bits(self, result['trace_gate'], gate)
|
||||
assert_bits(self, result['trace_final'], trace.numpy())
|
||||
|
||||
|
||||
class IdentityTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.a, self.b = np.zeros((5, 4), np.float32), np.zeros((5, 4), np.float32)
|
||||
self.a[:3, 0] = 1 # A specific
|
||||
self.a[:3, 1] = .1; self.b[:4, 1] = .2 # shared
|
||||
self.b[:3, 2] = .3 # B specific
|
||||
self.a[:2, 3] = 1; self.b[:2, 3] = 1 # no majority
|
||||
self.source = np.array([0, 1, 1, 2, 3], np.int32)
|
||||
|
||||
def test_three_of_five_categories_and_full_set_jaccard(self):
|
||||
result = pair_identity(self.a, self.b, self.source)
|
||||
np.testing.assert_array_equal(result['kc_category'], np.array([1, 3, 2, 0], np.int8))
|
||||
np.testing.assert_array_equal(result['edge_category'], np.array([1, 3, 3, 2, 0], np.int8))
|
||||
self.assertEqual(result['summary']['KC']['active_set_jaccard'], 1 / 3)
|
||||
self.assertEqual(result['summary']['edges']['active_set_jaccard'], .5)
|
||||
self.assertEqual(result['summary']['KC']['category_counts'], {name: 1 for name in CATEGORY_CODES})
|
||||
self.assertTrue(result['summary']['edge_counts_are_source_degree_weighted'])
|
||||
|
||||
def test_tiny_positive_no_epsilon_and_no_weight_update_outputs(self):
|
||||
self.a[:3, 0] = np.nextafter(np.float32(0), np.float32(1))
|
||||
result = pair_identity(self.a, self.b, self.source)
|
||||
self.assertTrue(result['kc_active_a'][0])
|
||||
self.assertFalse(result['summary']['weights_modified'])
|
||||
self.assertFalse(result['summary']['weight_delta_generated'])
|
||||
self.assertFalse(any('delta' in key or 'weight' in key for key in result if key != 'summary'))
|
||||
|
||||
def test_empty_union_and_empty_edge_mapping_explicit(self):
|
||||
result = pair_identity(np.zeros_like(self.a), np.zeros_like(self.b), np.empty(0, np.int32))
|
||||
self.assertTrue(result['summary']['KC']['empty_union'])
|
||||
self.assertEqual(result['summary']['KC']['active_set_jaccard'], 1.)
|
||||
self.assertTrue(result['summary']['edges']['empty_union'])
|
||||
self.assertEqual(result['edge_category'].dtype, np.dtype('int8'))
|
||||
|
||||
def test_readonly_inputs_and_returned_mapping_owns_memory(self):
|
||||
for value in (self.a, self.b, self.source):
|
||||
value.flags.writeable = False
|
||||
before = [value.tobytes() for value in (self.a, self.b, self.source)]
|
||||
result = pair_identity(self.a, self.b, self.source)
|
||||
result['edge_category'][0] = 0
|
||||
self.assertEqual(result['kc_category'][0], 1)
|
||||
self.assertEqual(before, [value.tobytes() for value in (self.a, self.b, self.source)])
|
||||
|
||||
def test_reject_wrong_dtype_shape_nonfinite_negative_cap_and_source(self):
|
||||
for bad in (self.a.astype(np.float64), self.a[:4], np.zeros((5, 0), np.float32)):
|
||||
with self.subTest(shape=bad.shape), self.assertRaises(ValueError):
|
||||
pair_identity(bad, self.b, self.source)
|
||||
for value in (np.nan, np.inf, -.1, 1.01):
|
||||
bad = self.a.copy(); bad[0, 0] = value
|
||||
with self.subTest(value=value), self.assertRaises(ValueError):
|
||||
pair_identity(bad, self.b, self.source)
|
||||
with self.assertRaises(ValueError):
|
||||
pair_identity(self.a, self.b[:, :3], self.source)
|
||||
for source in (np.array([-1], np.int32), np.array([4], np.int32), self.source.astype(np.int64)):
|
||||
with self.subTest(source=source.tolist()), self.assertRaises(ValueError):
|
||||
pair_identity(self.a, self.b, source)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,56 @@
|
||||
"""Synthetic interface and fail-closed schedule checks, no scientific records."""
|
||||
import ast
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
import numpy as np
|
||||
from run_audit import compare_results
|
||||
from summarize import metrics, primary_rows
|
||||
|
||||
|
||||
class IntegrationTests(unittest.TestCase):
|
||||
def test_per_neuron_count_difference_not_cancelled_by_equal_total(self):
|
||||
a = dict(step=np.array([0, 1], np.int32), neuron_idx=np.array([1, 1], np.int32),
|
||||
counts=np.array([2, 0], np.int32), first_spike=np.array([0, -1], np.int32),
|
||||
v=np.array([-.052, -.052], np.float64), g=np.zeros(2, np.float64))
|
||||
b = {k: v.copy() for k, v in a.items()}
|
||||
b['neuron_idx'][:] = 2; b['counts'][:] = [0, 2]; b['first_spike'][:] = [-1, 0]
|
||||
h = dict(history_v=np.zeros((2, 2)), history_g=np.zeros((2, 2)))
|
||||
result = compare_results(a, b, h, h)
|
||||
self.assertEqual(result['spike_count_L1'], 4)
|
||||
self.assertEqual(result['neurons_with_count_change'], 2)
|
||||
self.assertTrue(result['functional_count_or_first_latency_change'])
|
||||
|
||||
def test_event_difference_with_same_counts_and_first_is_retained(self):
|
||||
a = dict(step=np.array([0, 3], np.int32), neuron_idx=np.array([1, 1], np.int32),
|
||||
counts=np.array([2], np.int32), first_spike=np.array([0], np.int32),
|
||||
v=np.zeros(1), g=np.zeros(1))
|
||||
b = {k: v.copy() for k, v in a.items()}; b['step'][1] = 4
|
||||
h = dict(history_v=np.zeros((5, 1)), history_g=np.zeros((5, 1)))
|
||||
result = compare_results(a, b, h, h)
|
||||
self.assertTrue(result['event_sequence_changed'])
|
||||
self.assertFalse(result['functional_count_or_first_latency_change'])
|
||||
|
||||
def test_undefined_denominator_is_not_silently_zero(self):
|
||||
self.assertIsNone(metrics([])['mean'])
|
||||
|
||||
def test_repeat_not_an_independent_primary_trial(self):
|
||||
rows = [dict(name=str(i), repeat=0) for i in range(120)] + [dict(name=str(i), repeat=1) for i in range(24)]
|
||||
self.assertEqual(len(primary_rows(dict(planned=rows))), 120)
|
||||
rows[1]['name'] = '0'
|
||||
with self.assertRaises(AssertionError):
|
||||
primary_rows(dict(planned=rows))
|
||||
|
||||
def test_all_baseline_gate_precedes_conditional_schedule(self):
|
||||
tree = ast.parse((Path(__file__).parent / 'run_audit.py').read_text())
|
||||
main = next(n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name == 'main')
|
||||
loops = [n for n in ast.walk(main) if isinstance(n, ast.For) and isinstance(n.target, ast.Name) and n.target.id == 'stage']
|
||||
self.assertEqual(len(loops), 1)
|
||||
self.assertEqual(ast.literal_eval(loops[0].iter), ('baseline', 'conditional25'))
|
||||
audit = next(n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name == 'audit_trial')
|
||||
source = ast.unparse(audit)
|
||||
self.assertIn("baselines-complete.json", source)
|
||||
self.assertLess(source.index("baselines-complete.json"), source.index('core.replay'))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,179 @@
|
||||
"""Synthetic-only tests; no recorded scientific data, GPU, or file writes."""
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
|
||||
from mbon_accounting import MBONAccounting, bit_equal
|
||||
|
||||
|
||||
_source = Path(__file__).resolve().parents[1] / "kc-drive-audit-v1" / "replay_core.py"
|
||||
_spec = importlib.util.spec_from_file_location("_mbon_test_frozen_ah_replay", _source)
|
||||
ah = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(ah)
|
||||
|
||||
|
||||
def coeff(**changes):
|
||||
values = dict(gd=.5, c0=0., cg=1., cv=1., threshold=100., reset=0., seed_volt=101.)
|
||||
values.update(changes)
|
||||
return values
|
||||
|
||||
|
||||
def events(rows=()):
|
||||
array = np.asarray(rows, np.int32).reshape(-1, 2)
|
||||
return dict(tick=array[:, 0], neuron_idx=array[:, 1])
|
||||
|
||||
|
||||
def graph(weights=(), destinations=None, n=1):
|
||||
e = len(weights)
|
||||
return ah.IncomingIndex(np.arange(10, 10 + e, dtype=np.int32),
|
||||
np.zeros(e, np.int32) if destinations is None else np.array(destinations, np.int32),
|
||||
np.arange(1000, 1000 + e, dtype=np.int64), np.array(weights, np.float64), n)
|
||||
|
||||
|
||||
def run(index, *, groups=None, names=("plastic_KC",), steps=25, constants=None,
|
||||
spikes=(), requests=(), v=None, g=None, observer_hook=None):
|
||||
constants = coeff() if constants is None else constants
|
||||
groups = np.zeros(len(index.edge_id), np.int32) if groups is None else np.array(groups, np.int32)
|
||||
obs = MBONAccounting(index, groups, names, constants, steps)
|
||||
seen = []
|
||||
|
||||
def callback(tick, stage):
|
||||
# Read-only views also catch accidental assignments to callback data.
|
||||
original = {key: value.copy() for key, value in stage.items()}
|
||||
for value in stage.values():
|
||||
value.setflags(write=False)
|
||||
obs(tick, stage)
|
||||
for key in original:
|
||||
if not bit_equal(stage[key], original[key]):
|
||||
raise AssertionError("Observer modified callback field " + key)
|
||||
if observer_hook:
|
||||
observer_hook(tick, stage)
|
||||
seen.append(original)
|
||||
|
||||
result = ah.replay(np.zeros(index.target_count, np.float64) if v is None else np.asarray(v, np.float64),
|
||||
np.zeros(index.target_count, np.float64) if g is None else np.asarray(g, np.float64),
|
||||
constants, steps, np.arange(100, 100 + index.target_count, dtype=np.int32),
|
||||
events(requests), events(spikes), index,
|
||||
np.zeros((steps, index.target_count), np.float32), callback)
|
||||
data, edges, summary = obs.finish()
|
||||
return data, edges, summary, result, seen, obs
|
||||
|
||||
|
||||
class AccountingTests(unittest.TestCase):
|
||||
def test_empty_graph_initial_baseline_and_json_summary(self):
|
||||
data, edge, summary, result, _, _ = run(graph(), steps=3, g=[2.], v=[-.052])
|
||||
self.assertEqual(summary["due_count"], 0)
|
||||
self.assertEqual(len(edge["edge_id"]), 0)
|
||||
np.testing.assert_array_equal(data["history_group_g"], 0.)
|
||||
np.testing.assert_array_equal(data["history_g"], data["history_baseline_g"])
|
||||
self.assertTrue(bit_equal(data["final_g"], result["g"]))
|
||||
json.dumps(summary, allow_nan=False)
|
||||
self.assertFalse(summary["functional_transfer_claim"])
|
||||
self.assertFalse(summary["original_events_verified_by_observer"])
|
||||
|
||||
def test_real_delay_old_g_for_voltage_and_end_tick_integral(self):
|
||||
data, edge, summary, _, _, _ = run(graph([2.]), steps=21, spikes=[(0, 10)])
|
||||
np.testing.assert_array_equal(data["history_g"][:18], 0.)
|
||||
np.testing.assert_array_equal(data["history_g"][18:, 0], [2., 1., .5])
|
||||
np.testing.assert_array_equal(data["history_v"][18:, 0], [0., 2., 3.])
|
||||
self.assertAlmostEqual(data["g_integral_V_ms"][0, 0], .35)
|
||||
np.testing.assert_array_equal(edge["accepted_count"], [1])
|
||||
self.assertEqual(summary["max_g_lane_residual_V"], 0.)
|
||||
|
||||
def test_arrival_rejected_on_fire_and_refractory_release(self):
|
||||
data, edge, summary, result, _, _ = run(graph([2., -1.]), steps=41,
|
||||
spikes=[(t, source) for t in (0, 1, 22) for source in (10, 11)], requests=[(18, 100)])
|
||||
np.testing.assert_array_equal(edge["due_count"], [3, 3])
|
||||
np.testing.assert_array_equal(edge["accepted_count"], [1, 1])
|
||||
np.testing.assert_array_equal(edge["rejected_count"], [2, 2])
|
||||
np.testing.assert_array_equal(data["history_g"][18:40], 0.)
|
||||
self.assertEqual(data["history_g"][40, 0], 1.)
|
||||
self.assertEqual(data["rejected_positive_V"][0, 0], 4.)
|
||||
self.assertEqual(data["rejected_negative_V"][0, 0], -2.)
|
||||
self.assertEqual(data["accepted_negative_V"][0, 0], -1.)
|
||||
np.testing.assert_array_equal(data["spike_count"], result["counts"])
|
||||
self.assertEqual(summary["accepted_count"], 2)
|
||||
|
||||
def test_firing_clears_source_g_and_voltage_lanes(self):
|
||||
data, _, _, result, _, _ = run(graph([2.]), steps=21, constants=coeff(threshold=1., seed_volt=2.),
|
||||
spikes=[(0, 10)])
|
||||
self.assertEqual(data["first_spike_tick"][0], 19)
|
||||
np.testing.assert_array_equal(data["history_group_g"][19:], 0.)
|
||||
np.testing.assert_array_equal(data["final_group_v"], 0.)
|
||||
self.assertEqual(data["history_ref"][19, 0], 22)
|
||||
self.assertEqual(data["history_ref"][20, 0], 21)
|
||||
self.assertTrue(bit_equal(data["final_ref"], result["ref"]))
|
||||
|
||||
def test_group_sign_totals_and_zero_weight_event_counts(self):
|
||||
index = graph([2., -3., 0., 4.], destinations=[0, 0, 1, 1], n=2)
|
||||
data, edge, summary, _, _, _ = run(index, groups=[0, 1, 0, 1], names=("plastic_KC", "other"),
|
||||
steps=19, spikes=[(0, x) for x in range(10, 14)])
|
||||
np.testing.assert_array_equal(data["accepted_count"], [[1, 1], [1, 1]])
|
||||
np.testing.assert_array_equal(data["accepted_zero_count"], [[0, 0], [1, 0]])
|
||||
np.testing.assert_array_equal(data["accepted_positive_V"], [[2., 0.], [0., 4.]])
|
||||
np.testing.assert_array_equal(data["accepted_negative_V"], [[0., -3.], [0., 0.]])
|
||||
self.assertEqual(summary["groups"]["other"]["accepted_negative_V"], -3.)
|
||||
self.assertEqual(int(edge["due_count"].sum()), 4)
|
||||
|
||||
def test_cross_lane_roundoff_is_visible_not_reassigned(self):
|
||||
data, _, summary, _, _, _ = run(graph([1.e16, 1., -1.e16]), groups=[0, 1, 0],
|
||||
names=("plastic_KC", "other"), steps=19, constants=coeff(threshold=1.e30, seed_volt=2.e30),
|
||||
spikes=[(0, 10), (0, 11), (0, 12)])
|
||||
self.assertEqual(data["history_g"][18, 0], 0.)
|
||||
self.assertEqual(data["history_group_g"][18, 0, 1, 0], 1.)
|
||||
self.assertEqual(data["history_g_residual"][18, 0], -1.)
|
||||
self.assertEqual(summary["max_g_lane_residual_V"], 1.)
|
||||
|
||||
def test_callback_and_constructor_inputs_never_modified(self):
|
||||
index = graph([1., -.25])
|
||||
snapshots = {name: getattr(index, name).copy() for name in ("src_global", "dst_local", "edge_id", "weights")}
|
||||
run(index, spikes=[(0, 10), (0, 11)], requests=[(1, 100)])
|
||||
for name, original in snapshots.items():
|
||||
self.assertTrue(bit_equal(getattr(index, name), original))
|
||||
|
||||
def test_incomplete_duplicate_tick_and_repeated_finish_fail(self):
|
||||
index = graph()
|
||||
with self.assertRaises(ValueError):
|
||||
MBONAccounting(index, np.empty(0, np.int32), ("other",), coeff(), 2).finish()
|
||||
_, _, _, _, stages, finished = run(index, steps=2)
|
||||
with self.assertRaises(ValueError):
|
||||
finished.finish()
|
||||
obs = MBONAccounting(index, np.empty(0, np.int32), ("other",), coeff(), 2)
|
||||
obs(0, stages[0])
|
||||
with self.assertRaises(ValueError):
|
||||
obs(0, stages[0])
|
||||
|
||||
def test_bad_dtype_groups_coefficients_and_nonfinite_fail(self):
|
||||
index = graph([1.])
|
||||
for codes in (np.array([0], np.int64), np.array([1], np.int32), np.array([], np.int32)):
|
||||
with self.assertRaises(ValueError):
|
||||
MBONAccounting(index, codes, ("other",), coeff(), 2)
|
||||
for bad in (coeff(gd=np.float32(.5)), coeff(gd=float("nan")), coeff(dt_ms=.2)):
|
||||
with self.assertRaises(ValueError):
|
||||
MBONAccounting(index, np.array([0], np.int32), ("other",), bad, 2)
|
||||
|
||||
def test_stage_guard_apl_and_canonical_order_corruption_fail(self):
|
||||
index = graph([1., 2.])
|
||||
_, _, _, _, stages, _ = run(index, steps=19, spikes=[(0, 10), (0, 11)])
|
||||
for corruption in ("guard", "order", "apl", "state"):
|
||||
obs = MBONAccounting(index, np.array([0, 0], np.int32), ("other",), coeff(), 19)
|
||||
for tick in range(18):
|
||||
obs(tick, stages[tick])
|
||||
stage = {k: v.copy() for k, v in stages[18].items()}
|
||||
if corruption == "guard":
|
||||
stage["arrival_accepted"][0] = False
|
||||
elif corruption == "order":
|
||||
stage["due_edge_indices"] = stage["due_edge_indices"][::-1].copy()
|
||||
elif corruption == "apl":
|
||||
stage["h32"][0] = 1.
|
||||
else:
|
||||
stage["g_pre_apl"][0] += 1.
|
||||
with self.subTest(corruption=corruption), self.assertRaises(ValueError):
|
||||
obs(18, stage)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Rerun old deterministic-v2 references without changing their semantics."""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import yaml
|
||||
from context import ROOT, EXP, check, save, bit_equal
|
||||
|
||||
|
||||
def preflight():
|
||||
os.environ['CUDA_VISIBLE_DEVICES'] = 'GPU-79f3a85d-adce-7833-7003-6488140ca596'
|
||||
os.environ['CUDA_HOME'] = '/usr/local/cuda'
|
||||
os.environ['PATH'] = str(ROOT / 'models/male-cns-lif/.venv/bin') + ':/usr/local/cuda/bin:' + os.environ['PATH']
|
||||
os.environ['MAX_JOBS'] = '2'
|
||||
os.environ['TORCH_CUDA_ARCH_LIST'] = '8.9'
|
||||
for port in (8188, 8189, 8190):
|
||||
with urllib.request.urlopen(f'http://192.168.10.51:{port}/queue', timeout=5) as response:
|
||||
queue = json.load(response)
|
||||
assert not queue['queue_running'] and not queue['queue_pending'], 'Comfy busy: defer, do not evict'
|
||||
|
||||
|
||||
def main():
|
||||
check(); preflight()
|
||||
assert not (EXP / 'regression/frozen-five.json').exists()
|
||||
runtime_path = ROOT / 'models/male-cns-lif/deterministic-runtime-v2'
|
||||
config = yaml.safe_load((runtime_path / 'phase4p.yaml').read_text())
|
||||
sys.path.insert(0, str(ROOT / 'models/male-cns-lif'))
|
||||
sys.path.insert(0, str(runtime_path))
|
||||
import torch
|
||||
import phase4p
|
||||
assert torch.cuda.mem_get_info()[0] > 2 * 1024**3
|
||||
torch.cuda.reset_peak_memory_stats()
|
||||
started = time.perf_counter()
|
||||
runtime = phase4p.CanonicalRuntime(config, runtime_path, EXP)
|
||||
rows = []
|
||||
for spec in config['reference_trials']:
|
||||
preflight()
|
||||
reference_dir = ROOT / 'experiments/phase4p-deterministic-runtime/phase4c-v2/reference' / spec['name']
|
||||
events = pd.read_parquet(reference_dir / 'spike-events.parquet')
|
||||
reference = dict(step=events.step.to_numpy(), neuron_idx=events.neuron_idx.to_numpy(),
|
||||
counts=np.load(reference_dir / 'per-neuron-spike-counts.int32.npy'),
|
||||
v=np.load(reference_dir / 'final-voltage.float32.npy'), g=np.load(reference_dir / 'final-g.float32.npy'))
|
||||
result = runtime.run(np.flatnonzero(runtime.pops[spec['population']]), spec['rate_hz'], spec['seed'], spec['duration_ms'])
|
||||
row = dict(trial=spec['name'], exact={k: bit_equal(v, result[k]) for k, v in reference.items()},
|
||||
dtype_shape={k: dict(reference_dtype=str(v.dtype), actual_dtype=str(result[k].dtype),
|
||||
reference_shape=list(v.shape), actual_shape=list(result[k].shape)) for k, v in reference.items()},
|
||||
wall_seconds=result['metrics']['wall_seconds'])
|
||||
rows.append(row)
|
||||
print(row, flush=True)
|
||||
passed = len(rows) == 5 and all(all(row['exact'].values()) for row in rows)
|
||||
save(EXP / 'regression/frozen-five.json', dict(pass_all=passed, trials=rows,
|
||||
wall_seconds=time.perf_counter() - started, peak_allocated_vram=int(torch.cuda.max_memory_allocated()),
|
||||
scope='Old frozen deterministic-v2 reference only, not D18 equality to old dynamics'))
|
||||
assert passed
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user