folklore GitHub
◆ Research · measured, not claimed

The seven questions, answered.

Folklore is a peer-to-peer layer where agents share prior, verified inference instead of re-deriving it. That makes two bets — it saves cost and compounds quality, and it stays reliable through signed provenance. We tried to falsify both. Here is every question, its measured answer, and the disciplined negatives we kept rather than buried. Three of the seven overturned our own first guess.

RQ1 · reuse → quality
Reuse is safe
Positive
RQ2 · detect bad reuse
OOD gate unsolved
Hard negative
RQ3 · trust / poison
Provenance ~25×
Proven
RQ4 · savings
Beats solo cache
Positive
RQ5 · compounding
Compounds, real
Pilot-pending
RQ6 · benchmarks
Suite shipped
Built
RQ7 · architecture
Hopfield link
Framed
RQ1Positive

When does reusing a prior answer improve quality?

Reuse helps — and never hurts — when the match is made on the question (query↔query), the reused answer was actually correct, and it enters as additive context the model can still reason over. The recalibrated 0.6 match gate turns out to sit in a safe gap: the most similar different-need claims on SciFact top out at cosine 0.544 — below the gate — so a cross-need answer can't even fire.

ExampleTwo engineers ask paraphrases of "does X regulate Y?". The second inherits the first peer's already-verified source instead of paying for a fresh search — same need, near-identical embedding.

Bench: bench-inference-tree-sharing.mjs, run-e1-reuse.mjs · recall@10 73.5%→98.6%, answer-level hurt 0/9

Retrieval recall@10 · NFCorpus
baseline tree-shared 73.5% 98.6% +34% relative · rescued 2015 · hurt 7 / 8000
Sharing the answered-question→verified-doc tree lifts recall with essentially zero regressions.
RQ3Proven

Can cryptographic provenance defend reused context against poison?

Yes — exactly where the baseline is vulnerable. We first saw a null (a frontier model shrugged off the poison) and almost shipped it. It was a measurement artifact: the correct source was pinned in the window. Push the gold out, run a weaker agent under a 75% colluding-Sybil majority, and the baseline is genuinely poisoned (53–64% flip to the lie). The provenance ranker — drop every unsigned passage, keep the attributed-signed source — cuts attack-success ~25×, and stays flat as the poison rate climbs.

ExampleA Sybil cluster floods the context with signed-but-fake "erratum: the finding was retracted" passages. Dropping everything unsigned re-surfaces the one real source; the agent answers correctly.

Bench: BEIR SciFact, Haiku agent / Opus judge, n=82, 2,957 judged cells · eval/RESULTS-LOG.md

Flip-attack-success · weak agent, gold-displaced
T0 baseline T1 ranker 0.59 0.024 ~25× protection · poison-rate-invariant (25→75%)
The defense matters precisely when the baseline is vulnerable; a frontier model with the source present needs no rescue.
RQ4Positive

How much does sharing actually save — versus a good solo cache?

The existential test for being peer-to-peer: does federation beat a single node running a well-tuned semantic cache? At matched error (≤2% false-accept), federated reuse lands +22% recall@1 over the tuned single-node cache, and +55% over cold retrieval. Savings are real but bounded by retrieval precision × hit-rate × the cost of verifying a reused answer.

ExampleYou already run a vCache-style local cache. Adding peers still resolves +22% more queries from memory at the same error budget — the network knows things your single cache never saw.

Bench: bench-vcache-compare.mjs (SciFact, recall@1, matched ≤2% error)

Recall@1 · cold vs solo cache vs federated
cold solo cache federated 47.7% 60.7% 74.0% federated vs solo cache: +22% at matched ≤2% error
If a tuned local cache already captured the value, decentralization wouldn't be justified. It doesn't.
RQ5Positive · pilot-pending

Does shared inference compound across users — reliably?

It compounds. On real SciFact qrels, cooperative correct-resolve rises from 27.9% → 39.3% over isolated peers, at 1.8% false-admit — and real paraphrases separate from spurious matches at AUC 0.998, so peers can safely reuse each other's resolutions. Structurally it's monotone submodular coverage (diminishing returns, not runaway). The one piece still unmeasured is at-scale reliability: a 100-peer, 30-day live pilot.

ExampleOne peer pays to resolve a niche topic once; every other peer inherits it for a federation round-trip instead of a paid web call. The n-th query is increasingly likely to hit context already paid for.

Bench: bench-compounding-real.mjs, bench-paraphrase-sigma.mjs

Correct-resolve · isolated vs cooperative
isolated cooperative 27.9% 39.3% +11.4 pts · false-admit 1.8% · paraphrase sep. AUC 0.998
Knowledge and inference both compound on a real corpus; the pilot is what turns the thesis from likely to proven.
RQ2Hard negative

Can the system tell a reused answer is stale or off-topic?

Not yet — and we tested this to exhaustion rather than claim it. An out-of-distribution admission gate should fire low for a query the graph can't really answer. On a 21,000-node graph it can't: every query finds moderate-cosine neighbours, so the signal sits near chance (AUC ~0.41–0.55). We swept every lever — embedder (MiniLM and a full bge re-embed), token coverage, margin, entropy, retrieval depth — all negative. The one untried lever is a cross-encoder admission score; that's the next real attempt.

ExampleAsk a software-research graph "how do I bake sourdough?" It still returns its five nearest nodes at respectable cosine — the gate can't separate "I can answer this" from "I'm reaching."

Bench: bench-energy-gate.mjs, bench-energy-bge.mjs, bench-ood-signals.mjs

OOD separation (AUC) · every lever ≈ chance
0.7 = usable MiniLM .55 bge re-embed .55 gap .51 broad-margin .47 top-1 .44 margin .43 coverage −.10 None clears the usable bar. Documented, not buried.
A negative we kept: detecting bad reuse isn't solvable at the bi-encoder layer here. Honesty is the result.
RQ6Built

What benchmarks make these answers rigorous?

The harness is the suite. Retrieval quality on BEIR (SciFact / NFCorpus / FiQA), long-horizon memory on LongMemEval and LoCoMo, a templated poison eval (3 attacks × 3 rates × 3 treatments), a recall_any@k metric, and a vCache A/B for the savings question — plus the new ones built for this program: answer-quality-under-reuse, energy-coverage, energy-bge, and the OOD-signal sweep. Discipline throughout: matched comparisons, answers scored against gold qrels (never the gate), and every number labelled simulator vs measured. The open gap: no public benchmark scores cross-peer reuse quality + savings + trust jointly — which is the harness this program starts.

RQ7Framed

What architecture fits shared, provenance-tagged inference?

The longer-term frontier, framed not yet benched. The reranker is already one modern-Hopfield update step — which is softmax attention; personalized-PageRank, EigenTrust, and cache eviction all read as energy-descent fixed points. The hypothesis that ties it to RQ3: long-context models that ingest provenance-tagged retrieved context as first-class evidence should out-calibrate models that see anonymous tokens. The open theory problem is a distributed Hopfield capacity theorem — how much shared memory the network holds before retrieval degrades.

OPENWhat's next

Open questions — where the research goes next

Every question above has a measured answer. These are the falsifiable follow-ups those answers point to — each a concrete experiment, not a wish.

extends RQ2 · the open negativeCan a cross-encoder admission score detect bad reuse where every bi-encoder signal failed?

The one untried OOD lever — joint query–doc scoring, or a margin/entropy signal over the full corpus instead of top-k. The bi-encoder layer is exhausted; this needs the reranker model the sweep didn't have.

extends RQ5 · the only calendar-bound pieceDoes compounding stay reliable across a real, churning 100-peer population over 30 days?

The live web-fallback curve and false-admit tail the simulator can't produce. This is where the thesis stops being a thesis.

extends RQ3 · harden the winDoes the provenance defense survive an adaptive adversary — and a true 7B consumer?

Poison optimized against the trust filter (not templated fixtures), on a real small model rather than the Haiku proxy, plus EigenTrust-style Sybil-resistant reputation layered on the signature.

extends RQ1 · RQ4Does the reuse win survive measured end-to-end quality and wall-clock cost at scale?

Beyond n=20 retrieval recall: real answer-grading, net tokens/latency after verification cost, and a vCache per-prompt error guarantee replacing the static 0.6 gate.

extends RQ6 · the missing harnessBuild the benchmark that scores cross-peer reuse — quality, savings, and trust — jointly.

No public benchmark measures all three at once. That gap is the harness this program starts.

extends RQ7 · the theory frontierA distributed Hopfield capacity theorem — and does provenance-tagged context out-calibrate anonymous tokens?

How much shared memory the network holds before retrieval degrades, and whether long-context models that read provenance as first-class evidence calibrate better than those that see anonymous tokens.

Three of seven falsified our own first guess.

The poisoning "null" was an artifact — provenance actually works (RQ3). The bge re-embed that looked like a fix for the OOD gate didn't survive live retrieval (RQ2). The 0.6 reuse gate we feared was too loose turned out to sit in the safe gap (RQ1). Engineering a benchmark that can kill your own idea, then reporting exactly what it shows, is the whole posture.

Every figure here is a real bench run on real corpora (BEIR qrels, the live graph) — wins and negatives alike. Methods, per-question notes, and raw captures live in /research. Simulator results are labelled as such; the 100-peer pilot is the one calendar-bound piece still ahead.

Folklore · MIT-licensed · home · white paper · github