# **Implementation-Neutral Benchmark Specification for Extreme Vector Compression in Edge Environments** ## **1\. Executive Experimental Recommendation** \[Label: The agent's engineering analysis and recommendations\] The objective of this specification is to design a rigorous, implementation-neutral experimental program to evaluate the efficacy of the TurboQuant algorithmic family and its Rust-based vector search derivative, TurboVec, for deployment within the TinyRustLM ecosystem. TinyRustLM operates under strict edge constraints, utilizing pre-release browser-local architectures dependent on WebAssembly (WASM) and consumer-grade central processing units (CPUs) rather than datacenter-class accelerators. Consequently, the evaluation of extreme vector compression requires a meticulous separation of concerns. This program isolates three distinct engineering problems: static model-weight quantization (which is explicitly excluded from the TurboQuant evaluation paths, as these algorithms target dynamic state), online transformer key-value (KV) cache quantization, and persistent embedding/vector-index quantization1. The core TurboQuant algorithms promise transformative theoretical benefits, including the reduction of KV cache and vector index memory footprints by factors of 4× to 6× without necessitating model fine-tuning1. However, the primary literature surrounding these methods is highly contested. Recent independent analyses reveal profound discrepancies between the theoretical optimality claimed by the authors of TurboQuant and the independently reproduced sub-optimality of its scaling factors when compared to prior art such as EDEN and RaBitQ4. Furthermore, the conventional TurboQuant reliance on dense ![][image1] Gaussian projection matrices poses a severe risk of memory bandwidth starvation and latency degradation on the target WASM hardware. Therefore, this protocol mandates the evaluation of Fast-TurboQuant—a multiplier-free, Fast Walsh-Hadamard Transform (FWHT) variant—as a critical mitigation path for edge environments6. \[Label: Facts that require local TinyRustLM verification\] Because the evaluation team does not have access to the private TinyRustLM source tree, models, proprietary prompts, or evaluation cases, this document serves as a strict blueprint. The local TinyRustLM engineering team must execute this specification precisely, relying solely on empirical measurements derived from the target hardware matrix. The local team must independently verify whether the browser's JavaScript engine garbage collection and WASM SIMD128 polyfills introduce non-deterministic latency spikes that negate the theoretical throughput gains of these compression algorithms. ## **2\. Dated Claim Ledger and Source Identity Table** \[Label: Directly observed public facts\] The following ledger establishes the exact sources, version dates, and numerical claims requiring verification. The retrieval date for all sources is established as July 18, 2026\. The ledger separates author claims from independently reproduced facts to ensure rigorous auditing. | Source ID | Reference / URL | Claim / Observation | Classification Label | | :---- | :---- | :---- | :---- | | 8 | arXiv:2504.19874 | TurboQuant achieves near-optimal distortion rates, bounding MSE within a factor of ![][image2] of the information-theoretic limit. | Claims made by paper or project authors | | 1 | Google Research Blog | TurboQuant KV-cache quantization reduces memory footprint by 6× (down to 3 bits/value) with zero accuracy loss on LongBench, enabling 8× faster attention computation on H100 GPUs. | Claims made by paper or project authors | | 4 | arXiv:2604.18555 (EDEN Note) | ![][image3] is mathematically equivalent to the 2022 EDEN algorithm but sub-optimally fixes the scaling factor to ![][image4]. Biased EDEN with optimized ![][image5] reduces MSE by up to 2.25% at 4-bit over TurboQuant. | Independently reproduced or externally corroborated facts | | 4 | arXiv:2604.18555 (EDEN Note) | ![][image6] uses a 1-bit QJL residual that yields a vector-normalized MSE (vNMSE) of ![][image7], whereas the prior DRIVE estimator converges to ![][image8]. | Independently reproduced or externally corroborated facts | | 13 | RyanCodrai/turbovec | TurboVec achieves 16× compression (2-bit), shrinking a 31 GB float32 dataset to 4 GB, while beating FAISS IndexPQFastScan by 10–19% on ARM NEON. | Claims made by paper or project authors | | 14 | RyanCodrai/turbovec | TurboVec incorporates "TQ+", an empirical pre-calibration step mapping 5/95% quantiles to a canonical Beta marginal to correct low-dimensional coordinate drift. | Directly observed public facts | | 5 | arXiv:2604.19528v2 (RaBitQ vs TQ) | TurboQuant author experiments allegedly tested RaBitQ on a single-core CPU while testing TurboQuant on an A100 GPU. Properly configured RaBitQ outperforms TurboQuant in distance estimation on matched hardware. | Independently reproduced or externally corroborated facts | | 6 | arXiv:2606.21448 (Fast-TurboQuant) | Fast-TurboQuant replaces dense random rotation with a Rademacher phase inversion and FWHT, achieving 19.7× algorithmic speedup and requiring zero hardware multipliers. | Claims made by paper or project authors | ## **3\. Preregistered Hypotheses and Equivalence Margins** \[Label: The agent's engineering analysis and recommendations\] To insulate the evaluation from confirmation bias and retrospective threshold adjustments, the following hypotheses and equivalence margins are preregistered prior to any empirical testing. The definition of success strictly separates mathematical primitives, persistent search, and dynamic inference. **Track A (Mathematical Primitives):** The core mathematical evaluation seeks to determine if the ![][image4] scaling assumption in TurboQuant fundamentally degrades distortion compared to prior art. * **Null Hypothesis (![][image9]):** There is no statistically significant difference in vector-normalized mean squared error (vNMSE) and inner-product variance between ![][image3] (![][image4]), EDEN-biased (![][image10]), and Fast-TurboQuant (FWHT) at equal effective bit-widths. * **Equivalence Margin:** A difference in vNMSE of less than ![][image11] is considered practically equivalent for downstream embedding retrieval and attention logit calculations. **Track B (Persistent Vector Search):** This track evaluates whether the theoretical properties of TurboQuant translate into functional retrieval pipelines inside the public TurboVec implementation13. * **Null Hypothesis (![][image12]):** TurboVec yields identical Recall@10 and p95 query latencies compared to FAISS IndexPQFastScan and the reference RaBitQ implementation at exactly matched persistent memory footprints. * **Equivalence Margin ("Quality-Neutral"):** A degradation in Recall@10 of less than 1.5 percentage points relative to an exact float32 flat scan oracle. * **Superiority Threshold ("Faster"):** A statistically significant reduction in wall-clock query latency of at least 15% on the target edge hardware. **Track C (Transformer KV-Cache Inference):** This track determines if extreme compression sustains autoregressive coherence inside the browser. * **Null Hypothesis (![][image13]):** Generating tokens using a 3.5-bit TurboQuant KV-cache produces identical semantic quality and perplexity as an uncompressed 16-bit baseline and the tuning-free KIVI baseline18. * **Equivalence Margin ("Quality-Neutral"):** Perplexity deviation of ![][image14]; exact token match on ![][image15] of generation steps up to 4,096 tokens; Needle-in-a-Haystack retrieval score drop of ![][image16]. ## **4\. Track A: Vector-Primitive Protocol** \[Label: The agent's engineering analysis and recommendations\] Track A strictly isolates the underlying mathematical transformations to verify distortion rates, bias, and variance away from complex systems. The results from this track govern the validity of the low-level quantization theory but do not independently prove end-to-end viability for TinyRustLM. The protocol demands controlled vector generation across dimensions ![][image17]. The distributions must challenge the quantization assumptions, including uniform hypersphere, standard Gaussian, Laplace, highly anisotropic (modeling KV-cache outliers), sparse, clustered, heavy-tail (modeling un-normalized embeddings), and near-zero adversarial cases. Algorithm implementations must be programmed exactly as specified in their foundational texts to ensure structural parity. The required algorithms include: 1. ![][image3]: Utilizes a dense Gaussian projection, normalizes the vector, and applies a Lloyd-Max codebook parameterized for the Beta distribution, fixing the reconstruction scale to ![][image4]4. 2. ![][image6]: Identical to the above for the first ![][image18] bits, followed by a 1-bit Quantized Johnson-Lindenstrauss (QJL) residual correction to restore unbiased inner products20. 3. EDEN-biased: Employs Gaussian projection and Beta codebooks but applies the analytically optimal biased scale (![][image10]) designed to strictly minimize MSE12. 4. EDEN-unbiased: Applies the corresponding optimal unbiased scale (![][image19])11. 5. Fast-TurboQuant: Replaces the dense matrix projection with a Rademacher phase inversion and a Fast Walsh-Hadamard Transform (FWHT), requiring vectors to be zero-padded to the nearest power of two6. 6. RaBitQ: Utilizes random rotation followed by hypercube projection and 1-bit sign-bit encoding, preserving unbiased distance estimation23. The evaluation must compare these methods at equal effective storage bit-widths (![][image20]). The metrics include reconstruction MSE, angular error, inner-product bias (![][image21]), variance, distance error, and rank inversions against the uncompressed oracles. Every measurement must include all operational overheads: random seeds, codebook loading, matrix application, dimension padding, and scalar normalizations. ## **5\. Track B: Vector-Search Protocol** \[Label: The agent's engineering analysis and recommendations\] Track B evaluates the utility of these mathematical primitives when embedded in persistent search systems. The focus is specifically on the RyanCodrai/turbovec public project and its integration into retrieval-augmented generation (RAG) pipelines without neural network generation complexities13. The testing corpus must utilize public, redistributable datasets spanning low and high dimensions. The protocol requires frozen train/base/query/ground-truth splits. Recommended datasets include DBpedia-OpenAI-3-Large (![][image22]) and GloVe-200 (![][image23])8. The use of exact float32 (f32) oracle neighbors is mandatory to establish the ground-truth recall ranking. The baseline configurations must ensure architectural fairness. Baselines include an exact flat f32/f16 scan, a transparent scalar quantized flat scan (Qdrant SQ int8), FAISS IndexPQFastScan (where the platform supports NEON/AVX2), and the multi-threaded C++ reference implementation of RaBitQ24. TurboVec must be tested in two distinct modes: with its optional "TQ+" empirical pre-calibration enabled, and with all calibration/correction features strictly ablated to evaluate the raw TurboQuant mathematical baseline14. The required metrics encompass Recall@1, 10, and 100, Mean Reciprocal Rank (MRR), Normalized Discounted Cumulative Gain (NDCG), and rank correlation. Latency distributions (p50, p95, p99) must be captured for initial index loading, first-query preparation (which triggers matrix allocations in TurboVec), and steady-state queries. TurboVec claims search-time filtering with zero recall hits14; therefore, a filtered search protocol utilizing a 5% density bitmask must be executed to verify this claim. System measurements must capture build time, first-add time, persistent bytes on disk, and steady-state resident memory. ## **6\. Track C: KV-Cache Protocol** \[Label: The agent's engineering analysis and recommendations\] Track C models the dynamic allocation and scoring of attention keys and values during autoregressive decoding. This environment dictates the feasibility of deploying large context windows inside TinyRustLM running locally in a browser. The protocol mandates the selection of publicly available model families with reproducible tokenizers, chat templates, and unmodified inference paths. The required models are Llama-3.1-8B-Instruct (which utilizes Grouped-Query Attention, GQA, with ![][image24]) and Mistral-7B-Instruct-v0.218. Context lengths must span from 512 tokens through at least 8192 tokens to measure sustained degradation. The comparative baselines must include uncompressed f32/f16 KV caches, standard int8, uniform int4, and the asymmetric 2-bit KIVI methodology18. KIVI operates by quantizing keys per-channel and values per-token, explicitly maintaining a full-precision residual window for the most recent tokens to preserve coherence. The evaluation must also include PolarQuant, which utilizes recursive polar coordinates to eliminate normalization overhead29, and the specific TurboQuant variants (MSE-only and QJL-corrected)31. Track C requires rigorous ablation of Key versus Value compression. Literature indicates that keys dictate attention distribution and are highly sensitive to inner-product variance, whereas values dictate the output vector and require different quantization tolerances18. The evaluation must test TurboQuant's QJL 1-bit residual correction on the Keys, measuring whether the compute overhead of QJL reconstruction outstrips its accuracy benefits in WASM environments31. Numerical metrics must capture cache reconstruction error, pre-softmax attention-logit error, inner-product bias/variance, top-attended-token agreement, and softmax Jensen-Shannon (JS) divergence. Semantic metrics must include long-context retrieval (Needle-in-a-Haystack), distractor resistance, and adherence to structured JSON outputs. All raw outputs must be preserved untouched for blinded human review to evaluate ambiguous semantic regressions. ## **7\. Datasets, Models, Licenses, and Immutable Inputs** \[Label: Directly observed public facts\] The integrity of the reproducibility audit relies on immutable inputs. The exact datasets, models, and licensing parameters must be recorded to ensure that future local TinyRustLM teams can replicate the exact environmental states. * **Models:** Llama-3.1-8B-Instruct is licensed under the Llama 3.1 License. Mistral-7B-Instruct-v0.2 is licensed under the Apache 2.0 License. Both models must be instantiated with fixed random seeds for all generation sampling to isolate the noise introduced strictly by the vector quantization. * **Track B Datasets:** DBpedia-OpenAI-3-Large is distributed under an MIT/CC-BY equivalent, representing high-dimensional, well-behaved text embeddings. GloVe-200 operates under the Open Data Commons Public Domain Dedication and License (ODC-PDDL) and is utilized to represent lower-dimensional drifted word vectors where asymptotic assumptions break down8. The exact corpus hashes (SHA-256) of the subset splits must be frozen and recorded in the evaluation receipts. * **Track C Datasets:** LongBench-V1 and the RULER benchmarks are utilized for long-context tasks. For exact prompting fairness, the specific chat templates baked into the model's tokenizer configuration must be strictly adhered to without ad-hoc adjustments or custom parsing logic. ## **8\. Baseline Fairness and Complete Bit Accounting** \[Label: The agent's engineering analysis and recommendations\] Prior evaluations in the extreme quantization literature have been marred by asymmetric comparisons. Specifically, independent reproduction audits indicate that the original TurboQuant evaluation tested RaBitQ on a single-core CPU with multithreading disabled, while simultaneously testing TurboQuant on an NVIDIA A100 GPU5. This protocol strictly outlaws cross-hardware or cross-thread comparisons; all algorithms must be executed on matched hardware pathways. Complete bit accounting is mandatory. A claim of "4-bit per coordinate" is scientifically invalid if metadata overhead is ignored. The effective bits per coordinate (![][image25]) must be mathematically calculated as: ![][image26] * **TurboQuant/PolarQuant:** The metadata payload includes the 32-bit float ![][image27] norm. If the QJL residual is utilized, the 32-bit residual norm is added34. * **Fast-TurboQuant (FWHT):** Because the Walsh-Hadamard transform requires dimension lengths to be a power of two, padding a ![][image22] vector to ![][image28] introduces massive padding bits that *must* be included in the total storage footprint cost6. * **TurboVec (TQ+):** The global calibration parameters (the shift and scale variables per coordinate) must be accounted for in the static index footprint14. * **KIVI:** The unquantized residual window (e.g., ![][image29] tokens) must be amortized and factored into the total average bytes per token18. ## **9\. Hardware, Browser, Compiler, and Run Controls** \[Label: The agent's engineering analysis and recommendations\] Because TinyRustLM is designed to operate locally inside user endpoints, cloud accelerator benchmarks (e.g., NVIDIA H100 arrays) are explicitly marked as *non-authoritative*. **Hardware Matrix:** 1. **x86-64 CPU:** Intel Xeon Sapphire Rapids or AMD Zen 4 architectures (AVX-512 capable). The power profile must be locked, and C-states disabled to prevent dynamic frequency scaling from corrupting latency timing. 2. **ARM64 CPU:** Apple Silicon (M3/M4) or AWS Graviton 3 (NEON). Performance cores must be explicitly pinned. 3. **Browser Environments:** Chromium-based WASM (V8 engine) and Mozilla Firefox WASM (SpiderMonkey). Testing must cover both scalar WASM execution and SIMD128 enabled paths. The protocol must explicitly document WASM memory page allocations and any garbage collection pauses that interrupt steady-state latency. **Compiler Controls:** The Rust compiler version must be pinned (e.g., 1.80+). Optimization flags must be standardized: opt-level \= 3, lto \= "fat", codegen-units \= 1\. Target features must be explicitly passed (e.g., \+avx2, \+avx512bw, \+simd128). The global allocator must be defined (jemalloc or mimalloc for native binaries; wee\_alloc for WASM artifacts). **Execution State Protocols:** To prevent hidden warmups from obfuscating the true cost of initialization: * **Cold State:** The first query execution immediately after index load, which forces page faults, cache misses, and WASM JIT compilation. * **Warmup State:** 100 discarding queries designed strictly to stabilize the JIT compiler. * **Steady-State:** The subsequent 1,000 queries. Run orders must be randomized to distribute and limit thermal drift bias. ## **10\. Numerical and Semantic Metrics** \[Label: The agent's engineering analysis and recommendations\] A microkernel speedup is not an end-to-end speedup. If the evaluation is timing a SIMD inner-product scan, the timer must enclose the data loading, decoding, coordinate transformations (e.g., the FWHT butterfly network), and heap insertion6. * **Numerical Suite:** 1. Vector reconstruction MSE and angular error relative to the ![][image27] normalized original. 2. Cache reconstruction error (measured via the Frobenius norm of the difference in KV matrices). 3. Attention-logit error: Maximum absolute difference in pre-softmax attention logits. 4. Peak Resident Memory (RSS / WASM Pages) mapped continuously during ingest, generation, and cleanup. * **Semantic Suite:** 1. Source conflict resolution: Determining if the model hallucinates when identical semantic context is presented at the boundary of a quantized versus unquantized KIVI block. 2. Untrusted quoted instructions: Evaluating security prompt adherence under degraded vector precision. 3. Structured output: JSON schema adherence stability and syntactic validity across 100 generations. ## **11\. Statistical Analysis Plan** \[Label: The agent's engineering analysis and recommendations\] A lack of statistical significance does not constitute proof of equivalence. The statistical treatment must be rigorous and predefined. 1. **Repeated independent seeds:** All indexing operations involving random orthogonal matrices (TurboQuant) or Rademacher phase inversions (Fast-TurboQuant) must be repeated across 5 distinct pseudo-random number generator (PRNG) seeds to measure projection variance6. 2. **Significance testing:** For latency, utilize non-parametric bootstrap confidence intervals (10,000 resamples) at the median (p50), 95th, and 99th percentiles, as hardware latency distributions are strictly non-normal and heavily right-skewed. For accuracy and recall metrics, utilize paired Student’s t-tests with a declared alpha of ![][image30]. 3. **Multiple comparisons:** Apply the Benjamini-Hochberg procedure for False Discovery Rate (FDR) control across the multiple KV-cache bit-width permutations to prevent spurious significance. 4. **Effect Size:** Report Cohen's ![][image31] for performance deltas to capture the practical magnitude of any speedups, stripping away statistically significant but practically irrelevant microsecond gains. ## **12\. Raw Evidence and Receipt Schemas** \[Label: The agent's engineering analysis and recommendations\] All experimental trials, including complete failures and Out-Of-Memory (OOM) crashes, are immutable. Hidden warmups are strictly forbidden. Results must be serialized to JSON formats that bind the execution context to the raw measurements. **Receipt Schema Draft (Abridged Narrative Structure):** Every receipt must contain a UUID and an ISO8601 timestamp. The environment block must capture the hardware architecture (e.g., aarch64), CPU model, runtime engine (v8-wasm-simd128), compiler version, and a SHA256 hash of the dependency lockfile. The method\_config block must specify the track (A, B, or C), the precise algorithm variant (e.g., TurboVec\_TQ\_Plus), the mathematical dimension, the calculated effective\_bits\_per\_coord, and the prng\_seed. The measurements block must map the raw latency arrays alongside the aggregated p50/p99 microsecond values, the absolute recall values, and peak memory in megabytes. Finally, an immutable status flag must declare success, timeout, or specific fault modes. ## **13\. Ablation and Fault-Injection Matrix** \[Label: The agent's engineering analysis and recommendations\] To pinpoint the exact sources of quality loss and computational overhead, the local team must perform the following strict structural ablations: 1. **QJL vs. EDEN vs. Raw MSE:** Strip the 1-bit QJL residual correction from ![][image6]. Compare the resulting raw ![][image3] against the EDEN-biased and EDEN-unbiased scaling parameters to independently verify the claims of sub-optimality11. 2. **Dense vs. Structured Projection:** Replace the dense Gaussian rotation matrix with the Fast-TurboQuant Rademacher-FWHT pipeline6. Measure the exact CPU cycle reduction against the potential drop in sub-Gaussian concentration effectiveness at lower dimensions. 3. **TurboVec TQ+ Ablation:** In the turbovec package, systematically disable the pre-calibration (shift/scale) step for the Beta distribution14. Quantify how heavily the reported GloVe 200d accuracy relies on this non-data-oblivious empirical patch. 4. **Outlier Channel Handling:** Force the TurboQuant KV-cache to quantize the known heavy-tail outlier channels (which the KIVI algorithm purposefully protects in full precision19) at the uniform low bit-width, measuring the exact threshold where semantic collapse occurs. ## **14\. No-Hype Claim Vocabulary** \[Label: The agent's engineering analysis and recommendations\] The following restrictive vocabulary applies to all internal reports, dashboards, and final recommendations resulting from this benchmark program: * **"Faster"**: This term may only be used to denote a statistically significant reduction in *end-to-end wall-clock latency* on the target WASM/CPU architecture. A fast SIMD microkernel that requires expensive layout repacking or dense matrix multiplication in a pre-processing wrapper does not qualify as "faster". * **"Quality-Neutral"**: The method falls entirely within the preregistered equivalence margins defined in Section 3\. A lack of statistical difference alone is rejected as proof of neutrality. * **"End-to-End"**: The timing profile explicitly includes WASM-to-JavaScript bridge overhead, memory allocation, index loading from disk, prompt tokenization, the full autoregressive decoding loop, and memory deallocation. * **"Zero Accuracy Loss"**: This is a banned marketing phrase. It must be replaced with "Statistically equivalent downstream accuracy within bounds ![][image32]". ## **15\. Go/No-Go Decision Table** \[Label: The agent's engineering analysis and recommendations\] The ultimate adoption of TurboQuant or TurboVec into the TinyRustLM WASM prototype depends on passing the following strict threshold gates. Failure on any single "No-Go" threshold terminates the adoption pathway. | Criteria | Go Threshold | No-Go Threshold | | :---- | :---- | :---- | | **Peak Memory Reduction** | **![][image33]** relative to f16/f32 baselines. | ![][image34] relative to f16/f32 baselines. | | **Steady Latency (WASM)** | **![][image35]** Uncompressed baseline. | ![][image36] slower than uncompressed baseline. | | **Retrieval Recall (Track B)** | **![][image37]** relative to exact f32 at top-10. | ![][image38] relative to exact f32 at top-10. | | **Semantic Regression (Track C)** | Pass 100% of untrusted quote tests. | Fails source-conflict resolution tests. | | **Implementation Complexity** | Multiplier-free (FWHT) implementable in pure safe Rust. | Requires dense ![][image1] matmuls; needs unsafe blocks bypassing Rust lifetimes. | | **Browser Compatibility** | Runs on standard V8/SpiderMonkey w/ SIMD128. | Requires experimental WASM features (e.g., Memory64) or GPU WebGPU/WebGL dependencies. | ## **16\. Estimated Experiment Cost as Formulas** \[Label: The agent's engineering analysis and recommendations\] The computational and engineering budget for executing this protocol is parameterized by the following formulas: * **Hardware provisioning (Cloud validation for baselines):** ![][image39] *(Where ![][image40] represents the hourly rental rate of the specific instance type).* * **Human evaluation (Blinded Semantic Review):** ![][image41] *(Ensuring ambiguous generation degradation is reviewed by human operators, not automated proxy LLMs).* * **Engineering setup:** ![][image42] *(This accounts for the heavy lifting required to write the FWHT and SIMD polyfills in Rust for WASM execution, ensuring the baselines are fair).* ## **17\. Unknowns That Only Local TinyRustLM Execution Can Resolve** \[Label: Facts that require local TinyRustLM verification\] The public literature, benchmark repositories, and academic debates cannot answer how these methods behave inside the specific, proprietary sandbox of TinyRustLM. The local team must independently resolve the following: 1. **WASM JIT Eviction:** Does the browser's JIT compiler arbitrarily evict the highly unrolled FWHT butterfly networks or the TurboVec SIMD128 kernels during long-context generation, causing sudden, unpredictable latency spikes? 2. **Memory Fragmentation:** Does the dynamic allocation of 3.5-bit packed byte arrays inside the WASM linear memory cause catastrophic heap fragmentation during multi-turn chat sessions compared to standard f16 continuous buffers? 3. **Prompt Sensitivity:** Do the specific proprietary system prompts and JSON-schema constraints utilized by TinyRustLM exhibit unique failure modes under QJL variance or EDEN unbiased scaling? 4. **Thermal Throttling on Fanless Devices:** Does the computational overhead of continuous dequantization and 1-bit residual addition (QJL) generate enough heat to trigger thermal throttling on consumer devices (e.g., MacBook Air, tablets) faster than the memory-bound f16 baseline? ## **18\. Annotated Primary-Source Bibliography with Links and Dates** \[Label: Directly observed public facts\] *All sources accessed and verified on simulated date: July 18, 2026\.* 1. **TurboQuant Main Paper:** Zandieh, A., Daliri, M., Hadian, M., Mirrokni, V. (April 2025). *TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate*. arXiv:2504.19874. ICLR 2026\. \[https://arxiv.org/abs/2504.19874\] *Annotation:* Introduces the core MSE-optimal algorithm and the 1-bit QJL correction for KV caches and vector search. 2. **TurboQuant OpenReview:** ICLR 2026 Forum. \[https://openreview.net/forum?id=tO3ASKZlok\] *Annotation:* Contains peer-review critiques and author responses regarding methodological similarities to RaBitQ and testing methodology disputes. 3. **TurboQuant Google Blog:** Zandieh, A., Mirrokni, V. (March 24, 2026). *TurboQuant: Redefining AI efficiency with extreme compression*. \[https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/\] *Annotation:* Public marketing announcement citing 6x memory reduction and 8x speedup. 4. **TurboVec Public Repository:** Codrai, R. (2026). *RyanCodrai/turbovec*. \[https://github.com/RyanCodrai/turbovec\] and \[https://docs.rs/turbovec/\] *Annotation:* Third-party Rust library implementing the TurboQuant algorithm for persistent indexing, featuring the proprietary "TQ+" calibration. 5. **QJL Paper:** Zandieh, A., Daliri, M., Han, I. (June 2024). *QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead*. arXiv:2406.03482. AAAI 2025\. \[https://arxiv.org/abs/2406.03482\] *Annotation:* Precursor paper introducing the 1-bit residual unbiasing technique used in the ![][image6] architecture. 6. **PolarQuant Paper:** Han, I., Kacham, P., Karbasi, A., Mirrokni, V., Zandieh, A. (February 2025). *PolarQuant: Quantizing KV Caches with Polar Transformation*. arXiv:2502.02617. AISTATS 2026\. \[https://arxiv.org/abs/2502.02617\] *Annotation:* Introduces the recursive polar coordinate mapping to bypass normalization overhead and manage channel-wise outliers. 7. **EDEN Critique Note:** Ben-Basat, R., et al. (April 2026). *A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work*. arXiv:2604.18555. \[https://arxiv.org/abs/2604.18555\] *Annotation:* Formal critique claiming TurboQuant is a mathematically sub-optimal variant of the 2021/2022 DRIVE/EDEN algorithms due to fixing the scaling parameter to ![][image4]. 8. **EDEN Original Paper:** Vargaftik, S., et al. (2022). *EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning*. ICML 2022\. \[https://proceedings.mlr.press/v162/vargaftik22a.html\] *Annotation:* Groundwork for rotation-based quantization using optimal biased and unbiased scaling factors. 9. **RaBitQ Paper:** Gao, J., Long, C. (May 2024). *RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search*. arXiv:2405.12497. SIGMOD 2024/2025. \[https://arxiv.org/abs/2405.12497\] *Annotation:* Precursor method utilizing random rotation and hypercube projection, matching the Alon-Klartag theoretical bounds. 10. **Fast-TurboQuant Paper:** Pereira de Figueiredo, F., et al. (June 2026). *Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach*. arXiv:2606.21448. \[https://arxiv.org/abs/2606.21448\] *Annotation:* Critical architectural paper that replaces dense matrix rotations with ![][image43] Fast Walsh-Hadamard Transforms, vital for mitigating performance bottlenecks on TinyRustLM's CPU/WASM constraints. #### **Works cited** 1. TurboQuant: Redefining AI efficiency with extreme compression \- Google Research, [https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/](https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/) 2. Google TurboQuant: 6x KV Cache Compression for LLM Inference | Spheron Blog, [https://www.spheron.network/blog/google-turboquant-llm-compression-gpu-cloud/](https://www.spheron.network/blog/google-turboquant-llm-compression-gpu-cloud/) 3. TurboQuant: 31GB AI Memory Down to 4GB, Explained | TECHSY, [https://techsy.io/en/blog/google-turboquant-ai-memory-compression](https://techsy.io/en/blog/google-turboquant-ai-memory-compression) 4. \[2604.18555\] A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work \- arXiv, [https://arxiv.org/abs/2604.18555](https://arxiv.org/abs/2604.18555) 5. Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments \- arXiv, [https://arxiv.org/pdf/2604.19528](https://arxiv.org/pdf/2604.19528) 6. \[2606.21448\] Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach, [https://arxiv.org/abs/2606.21448](https://arxiv.org/abs/2606.21448) 7. Fast-TurboQuant A Multiplier-Free Online Vector Quantization Approach \- arXiv, [https://arxiv.org/html/2606.21448v1](https://arxiv.org/html/2606.21448v1) 8. \[2504.19874\] TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate, [https://arxiv.org/abs/2504.19874](https://arxiv.org/abs/2504.19874) 9. TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate \- Hugging Face, [https://huggingface.co/papers/2504.19874](https://huggingface.co/papers/2504.19874) 10. TurboQuant: Google Removed All Overhead From KV compression unlocking near unlimited upside | by Mandar Karhade, MD. PhD. | AI Advances, [https://ai.gopubby.com/turboquant-google-removed-all-overhead-from-kv-compression-unlocking-near-unlimited-upside-42a8f09495e7](https://ai.gopubby.com/turboquant-google-removed-all-overhead-from-kv-compression-unlocking-near-unlimited-upside-42a8f09495e7) 11. A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work \- arXiv, [https://arxiv.org/html/2604.18555v1](https://arxiv.org/html/2604.18555v1) 12. How a 2021 Quantization Algorithm Quietly Outperforms Its 2026 Successor, [https://towardsdatascience.com/how-a-2021-quantization-algorithm-quietly-outperforms-its-2026-successor/](https://towardsdatascience.com/how-a-2021-quantization-algorithm-quietly-outperforms-its-2026-successor/) 13. GitHub \- RyanCodrai/turbovec \- daily.dev, [https://daily.dev/posts/github---ryancodrai-turbovec-vk5u15wft](https://daily.dev/posts/github---ryancodrai-turbovec-vk5u15wft) 14. GitHub \- RyanCodrai/turbovec: A vector index built on TurboQuant, written in Rust with Python bindings, [https://github.com/RyanCodrai/turbovec](https://github.com/RyanCodrai/turbovec) 15. turbovec \- crates.io: Rust Package Registry, [https://crates.io/crates/turbovec](https://crates.io/crates/turbovec) 16. Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments \- arXiv, [https://arxiv.org/html/2604.19528v2](https://arxiv.org/html/2604.19528v2) 17. RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search \- ResearchGate, [https://www.researchgate.net/publication/381022541\_RaBitQ\_Quantizing\_High-Dimensional\_Vectors\_with\_a\_Theoretical\_Error\_Bound\_for\_Approximate\_Nearest\_Neighbor\_Search](https://www.researchgate.net/publication/381022541_RaBitQ_Quantizing_High-Dimensional_Vectors_with_a_Theoretical_Error_Bound_for_Approximate_Nearest_Neighbor_Search) 18. I Added the Most-Cited KV-Cache Baseline to My Mac Quantization Library — Here's What the Numbers Actually Say | by Rajveer Rathod | Jun, 2026 | Medium, [https://medium.com/@rajveer.rathod1301/i-added-the-most-cited-kv-cache-baseline-to-my-mac-quantization-library-heres-what-the-numbers-14d3e7e70016](https://medium.com/@rajveer.rathod1301/i-added-the-most-cited-kv-cache-baseline-to-my-mac-quantization-library-heres-what-the-numbers-14d3e7e70016) 19. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache \- SciSpace, [https://scispace.com/pdf/kivi-a-tuning-free-asymmetric-2bit-quantization-for-kv-cache-23hjqudcav.pdf](https://scispace.com/pdf/kivi-a-tuning-free-asymmetric-2bit-quantization-for-kv-cache-23hjqudcav.pdf) 20. QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead \- arXiv, [https://arxiv.org/abs/2406.03482](https://arxiv.org/abs/2406.03482) 21. Arclabs001/YATQ: Yet Another TurboQuant in PyTorch (YATQ) is a PyTorch implementation of TurboQuant for KV cache compression, following the paper TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate (ICLR 2026). With HuggingFace interface supported. · GitHub, [https://github.com/arclabs001/YATQ](https://github.com/arclabs001/YATQ) 22. EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning \- GitHub, [https://github.com/amitport/EDEN-Distributed-Mean-Estimation](https://github.com/amitport/EDEN-Distributed-Mean-Estimation) 23. RaBitQ: 1-Bit Vector Quantization (Part 1\) | by Dnotitia Inc. | Medium, [https://medium.com/@dnotitia/rabitq-1-bit-vector-quantization-part-1-1495172e88ec](https://medium.com/@dnotitia/rabitq-1-bit-vector-quantization-part-1-1495172e88ec) 24. Beyond the TurboQuant-RaBitQ Debate: Why Vector Quantization Matters for AI Infrastructure Costs \- Milvus, [https://milvus.io/blog/turboquant-rabitq-vector-database-cost.md](https://milvus.io/blog/turboquant-rabitq-vector-database-cost.md) 25. Practical and Asymptotically Optimal Quantization of High-Dimensional Vectors in Euclidean Space for Approximate Nearest Neighbor Search \- arXiv, [https://arxiv.org/html/2409.09913v1](https://arxiv.org/html/2409.09913v1) 26. RaBitQ Library \- Vector Database Group @ NTU, [https://vectordb-ntu.github.io/RaBitQ-Library/](https://vectordb-ntu.github.io/RaBitQ-Library/) 27. Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving \[Experiment, Analysis & Benchmark\] \- arXiv, [https://arxiv.org/html/2607.05399v1](https://arxiv.org/html/2607.05399v1) 28. \[2402.02750\] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache \- arXiv, [https://arxiv.org/abs/2402.02750](https://arxiv.org/abs/2402.02750) 29. What Is Polar-Coordinate Quantization (PolarQuant)? \- JumpCloud, [https://jumpcloud.com/it-index/what-is-polar-coordinate-quantization-polarquant](https://jumpcloud.com/it-index/what-is-polar-coordinate-quantization-polarquant) 30. \[2502.02617\] PolarQuant: Quantizing KV Caches with Polar Transformation \- arXiv, [https://arxiv.org/abs/2502.02617](https://arxiv.org/abs/2502.02617) 31. Productionizing TurboQuant on AMD GPUs for KV-Cache-Bound LLM Inference, [https://rocm.blogs.amd.com/artificial-intelligence/turboquant-vllm-agentic/README.html](https://rocm.blogs.amd.com/artificial-intelligence/turboquant-vllm-agentic/README.html) 32. HyperQuant: A Rate–Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models \- arXiv, [https://arxiv.org/html/2606.23406v1](https://arxiv.org/html/2606.23406v1) 33. TurboQuant and RaBitQ: What the Public Story Gets Wrong | by Jianyang Gao \- Medium, [https://medium.com/@gaojianyang0017/turboquant-and-rabitq-what-the-public-story-gets-wrong-23df83209c22](https://medium.com/@gaojianyang0017/turboquant-and-rabitq-what-the-public-story-gets-wrong-23df83209c22) 34. qjl.py \- scos-lab/turboquant \- GitHub, [https://github.com/scos-lab/turboquant/blob/main/qjl.py](https://github.com/scos-lab/turboquant/blob/main/qjl.py) 35. TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference \- arXiv, [https://arxiv.org/html/2604.19769v1](https://arxiv.org/html/2604.19769v1) 36. fwht \- Fast Walsh-Hadamard transform \- MATLAB \- MathWorks, [https://www.mathworks.com/help/signal/ref/fwht.html](https://www.mathworks.com/help/signal/ref/fwht.html) [image1]: [image2]: [image3]: [image4]: [image5]: [image6]: [image7]: [image8]: [image9]: [image10]: [image11]: [image12]: [image13]: [image14]: [image15]: [image16]: [image17]: [image18]: [image19]: [image20]: [image21]: [image22]: [image23]: [image24]: [image25]: [image26]: [image27]: [image28]: [image29]: [image30]: [image31]: [image32]: [image33]: [image34]: [image35]: [image36]: [image37]: [image38]: [image39]: [image40]: [image41]: [image42]: [image43]: