# **Mathematical Reconstruction, Lineage, and Accuracy Limits of TurboQuant and TurboVec** ## **1\. Executive Findings and Explicit Assumptions** (Agent's engineering analysis) This report presents a comprehensive mathematical reconstruction and architectural evaluation of the "TurboQuant" algorithm family and its practical implementations. The public quantization landscape is currently fractured into three distinct paradigms: theoretical asymptotic derivations (Google Research), adversarial rebuttal literature exposing theoretical redundancies and irreproducible baselines (EDEN, RaBitQ), and practical engineering artifacts prioritizing hardware-aware execution (TurboVec). Applying these methodologies to a browser-local, 0.3B to 2B parameter web-assembly (WASM) runtime environment (TinyRustLM) necessitates the strict decoupling of three distinct engineering problems: static model-weight quantization, online transformer key-value (KV) cache quantization, and persistent embedding retrieval. (Directly observed public fact) The primary literature establishes that randomly rotating high-dimensional vectors on a unit hypersphere induces coordinate independence and predictable marginal distributions, enabling data-oblivious Lloyd-Max scalar quantization1. However, (Independently reproduced or externally corroborated fact) rigorous peer review demonstrates that the foundational TurboQuant\_mse algorithm is mathematically equivalent to a suboptimal, special case of the earlier EDEN/DRIVE framework, restricted by a fixed scale parameter3. Furthermore, the public RyanCodrai/turbovec repository deliberately discards TurboQuant's secondary 1-bit residual correction (QJL) in favor of length-renormalized scoring and empirical per-coordinate calibration (TQ+) to evade finite-dimension failures4. The analysis relies on the following explicit assumptions: 1. **Computational Bounds**: TinyRustLM operates within constrained CPU/WASM environments. Algorithmic components requiring ![][image1] floating-point multipliers (e.g., dense orthogonal rotations) are hostile to edge execution and must be replaced by multiplier-free ![][image2] transforms (e.g., Fast Walsh-Hadamard Transforms) to be viable6. 2. **Architectural Separation**: Methods proven for persistent vector indices do not automatically translate to transient KV-cache tokens due to differing sensitivity to Kullback-Leibler (KL) divergence in attention distributions7. 3. **No Private Verification**: This report relies strictly on public artifacts. Verification of WASM SIMD translations and low-dimensional (![][image3]) asymptotic convergence requires local TinyRustLM integration. ## **2\. Terminology and Identity Verification** To maintain engineering precision, the following identities are established and verified from primary sources: * **TurboQuant (Algorithm)**: (Claim made by paper authors) A data-oblivious vector quantization algorithm proposed by Zandieh et al. (ICLR 2026). It combines random orthogonal rotation, coordinate-wise scalar quantization, and a 1-bit Quantized Johnson-Lindenstrauss (QJL) residual correction1. * **TurboVec (Artifact)**: (Directly observed public fact) A third-party Rust/Python library (RyanCodrai/turbovec) implementing a heavily modified variant of TurboQuant. It is explicitly optimized for vector search, adds TQ+ calibration, uses RaBitQ-style scalar correction, and omits the QJL residual phase4. * **QJL (Quantized Johnson-Lindenstrauss)**: (Claim made by paper authors) A standalone 1-bit random Gaussian projection and sign-quantization method (AAAI 2025\) yielding an unbiased estimator for inner products8. * **PolarQuant**: (Claim made by paper authors) A related KV-cache algorithm (AISTATS 2026\) that abandons Cartesian scalar quantization for a recursive polar coordinate decomposition to bypass normalization overheads9. * **DRIVE / EDEN**: (Externally corroborated fact) Precursor algorithms (NeurIPS 2021, ICML 2022\) utilizing random rotation and scalar quantization for distributed mean estimation. The 2026 literature mathematically proves EDEN encompasses TurboQuant\_mse3. * **RaBitQ**: (Claim made by paper authors) An alternative high-dimensional quantization method utilizing JL transforms and unsigned integer grids, providing strong high-probability error bounds10. * **Fast-TurboQuant**: (Claim made by paper authors) A 2026 hardware-aware architectural modification that replaces TurboQuant's dense matrix multiplication with a Fast Walsh-Hadamard Transform (FWHT) and Rademacher phase inversion6. ## **3\. Source/Version Observation Log** (Directly observed public facts) | Artifact | Version / Publication Date | Identifier / Link | Material Differences & Key Notes | | :---- | :---- | :---- | :---- | | **TurboQuant** | arXiv v1 (Apr 2025\) / ICLR 2026 | 2504.19874 / tO3ASKZlok | Introduces the core \_mse and \_prod algorithm variants. Claims near-optimal distortion and zero indexing time1. | | **QJL Paper** | arXiv v1 (Jun 2024\) / AAAI 2025 | 2406.03482 | Establishes the 1-bit asymmetric unbiased estimator later used as TurboQuant's second stage8. | | **PolarQuant** | arXiv v1 (Feb 2025\) / AISTATS 2026 | 2502.02617 | Replaces Cartesian grids with recursive polar angle mapping (![][image4])9. | | **Fast-TurboQuant** | arXiv v1 (Jun 2026\) | 2606.21448 | Replaces dense ![][image1] rotation with ![][image2] multiplier-free FWHT and Rademacher signs6. | | **DRIVE/EDEN Note** | arXiv v1 (Apr 2026\) | 2604.18555 | Mathematical critique proving TurboQuant\_mse is equivalent to EDEN with a suboptimal fixed scale ![][image5]3. | | **RaBitQ Rebuttal** | arXiv v1 (Apr 2026\) | 2604.19528 | Exposes severe reproducibility failures in TurboQuant's benchmarks, noting TurboQuant disabled multithreading on baseline tests11. | | **TurboVec Crate** | v0.9.0 (Mid-2026) | docs.rs/turbovec | Public Rust artifact. Explicitly diverges from the paper by adding TQ+ 5/95% calibration and length-renormalized scoring5. | ## **4\. Mathematical Notation and Objective Definitions** (Agent's engineering analysis) Quantizing high-dimensional vectors (![][image6]) into a discrete code ![][image7] and generating a reconstruction ![][image8] serves differing mathematical objectives depending on the downstream application. The literature utilizes several competing metrics: 1. **Reconstruction Mean Squared Error (MSE)**: The expected squared Euclidean distance between the original and reconstructed vector, ![][image9]. TurboQuant\_mse and EDEN-biased strictly optimize this metric2. 2. **Inner Product Distortion**: The expected squared error of the dot product estimation, ![][image10]. 3. **Biased vs. Unbiased Estimation**: An estimator is unbiased if the expectation over the quantization noise yields the true value: ![][image11]. MSE-optimal scalar quantizers inherently shrink vector magnitudes, introducing a systematic downward bias16. TurboQuant\_prod, QJL, and EDEN-unbiased attempt to restore this property. 4. **Vector-Normalized MSE (vNMSE)**: A scale-invariant distortion metric used in DRIVE/EDEN analysis, defined as ![][image12]3. 5. **Angular / Cosine Error**: Measures the preservation of directional alignment, independent of magnitude: ![][image13]. 6. **Top-k Rank Preservation (Recall@k)**: An empirical metric evaluating whether the ordinal distance relationships of a database relative to a query ![][image14] survive quantization. 7. **Downstream Task Quality**: Proxies like perplexity, KL divergence of attention logits, or exact-match scores (e.g., Needle-in-a-Haystack). Small MSE does not guarantee low KL divergence7. ## **5\. Full TurboQuant\_mse Derivation** (Agent's engineering analysis) The TurboQuant\_mse algorithm derives its compression by forcing arbitrary input data to conform to a mathematically predictable probability distribution, eliminating the need to train dataset-specific codebooks2. ### **Step 1: Norm Separation** Given an input vector ![][image6], the algorithm separates the magnitude from the direction. The ![][image15] norm ![][image16] is stored explicitly as a 32-bit or 16-bit float. The remaining vector ![][image17] lies on the unit hypersphere ![][image18]. ### **Step 2: Random Rotation** A dense, random orthogonal matrix ![][image19] is generated (typically via the QR decomposition of a standard Gaussian matrix) and fixed globally. The unit vector is rotated: ![][image20]. (Claim made by paper authors) Due to the spherical symmetry of ![][image18] and the orthogonality of ![][image21], the energy of any arbitrary input vector is isotropically distributed. Consequently, the marginal probability density function (PDF) of any single rotated coordinate ![][image22] follows a scaled Beta distribution2: ![][image23] ### **Step 3: The Gaussian Limit and Near-Independence** As the dimension ![][image24], the Beta distribution converges rapidly to a standard Gaussian distribution ![][image25]17. Furthermore, distinct coordinates ![][image26] and ![][image27] become nearly statistically independent. This theoretical decoupling allows the algorithm to abandon complex multi-dimensional product quantization (PQ) in favor of simple, coordinate-wise scalar quantization. ### **Step 4: Lloyd-Max Boundaries and Centroids** Because the distribution ![][image28] is known a priori, optimal quantization boundaries ![][image29] and representation centroids ![][image30] can be computed perfectly offline for a given bit-budget ![][image31] (creating ![][image32] buckets). The continuous k-means (Lloyd-Max) algorithm is executed on the theoretical Beta (or Gaussian) PDF to minimize MSE19: ![][image33] Centroids cluster densely near zero (the peak of the distribution) and spread out toward the tails17. ### **Step 5: Code Assignment, Packing, and Reconstruction** Each coordinate ![][image26] is mapped to the index ![][image34] of the nearest centroid ![][image30]. These ![][image31]\-bit integer codes are tightly bit-packed into byte arrays. During decompression, the codes are used as indices into the offline lookup table to yield ![][image35]. The vector is then inversely rotated (![][image36]) and scaled by the stored norm (![][image37]). ### **Distortion-Rate Result** (Claim made by paper authors) Shannon's source coding theorem establishes that the fundamental lower bound for MSE distortion given a bit budget ![][image38] is proportional to ![][image39]. The authors of TurboQuant prove that their algorithm's expected MSE achieves an upper bound of ![][image40]2. (Agent's engineering analysis) The authors claim this constitutes a "constant-factor gap" (![][image41]) from the absolute theoretical limit17. However, this asymptotic bound assumes infinite dimensions. At lower, finite dimensions (![][image3]), strict coordinate independence fails because ![][image42]. The rigid mathematical bounds loosen, introducing empirical degradation not captured by the theorem. ## **6\. Full TurboQuant\_prod and QJL Derivation** (Agent's engineering analysis) TurboQuant\_mse exhibits a critical flaw for transformer attention: it is biased. Because Lloyd-Max centroids represent the center of mass of a distribution slice, rounding continuous values to these centroids systematically shrinks the ![][image15] norm of the reconstructed vector (![][image43]). Consequently, the estimated inner product ![][image44] is systematically smaller than the true inner product ![][image45]16. ### **The Residual and Bit-Splitting** To construct an unbiased inner-product estimator, TurboQuant\_prod splits the quantization budget. It utilizes ![][image46] bits to perform the standard TurboQuant\_mse encoding, generating a biased estimate ![][image47]. It then calculates the exact quantization residual vector in full precision: ![][image48]21. ### **The 1-Bit QJL Transform** The remaining 1 bit per coordinate is dedicated to the Quantized Johnson-Lindenstrauss (QJL) sketch. 1. A random projection matrix ![][image49] is populated with i.i.d. standard normal entries8. 2. The residual vector is projected and quantized to its sign: ![][image50]8. 3. The full-precision norm of the residual, ![][image51], is saved alongside the vector22. ### **The Asymmetric Unbiased Estimator** (Claim made by paper authors) When querying the database, the query vector ![][image14] remains in full precision (an asymmetric setup). The QJL estimator for the residual inner product is derived as8: ![][image52] Because the sign of a random Gaussian projection probabilistically encodes the angle between vectors, taking the expectation over the random matrix ![][image53] proves the estimator is strictly unbiased: ![][image54]8. The final dot product combines the MSE reconstruction and the residual: ![][image55]. ## **7\. PolarQuant Relationship and Distinctions** (Directly observed public fact) Authored by an overlapping Google Research team (AISTATS 2026), PolarQuant addresses KV-cache memory overhead through coordinate geometry rather than Cartesian residual sketches9. ### **Recursive Polar Transformation** Instead of quantizing ![][image56] and ![][image57] Cartesian coordinates directly, PolarQuant pairs adjacent dimensions and converts them to polar coordinates: ![][image58] and ![][image59]. This process is applied recursively to the resulting radii across ![][image60] hierarchical levels, creating a tree of angles and a single final root radius23. ### **Angle Distributions and Allocation** (Claim made by paper authors) After the initial random preconditioning (rotation), the distributions of these angles are analytically known and tightly concentrated. Level 1 angles are uniformly distributed ![][image61]. At Level ![][image62], the probability density is proportional to ![][image63], which clusters sharply around ![][image64]23. PolarQuant exploits this by allocating more bits to Level 1 (e.g., 4 bits) and fewer bits to highly concentrated upper levels (e.g., 2 bits), avoiding per-block normalization overhead23. (Agent's engineering analysis) **Applicability constraint**: PolarQuant requires heavy nonlinear trigonometric function evaluations (sin, cos, atan2) during decompression to reconstruct the Cartesian inner products24. While datacenter GPUs can hide this latency via fused CUDA kernels, deploying PolarQuant on a TinyRustLM CPU/WASM runtime would induce catastrophic decode latency bottlenecks compared to simple array lookups. ## **8\. Bit Accounting and Memory Formulas** (Agent's engineering analysis) To evaluate memory reduction claims, we must equalize all allocator overheads, metadata, and mathematical constants. Let ![][image65] be the vector dimension and ![][image31] be the nominal bit-width. **TurboQuant\_mse Storage Formula:** * Norm: 1 float32 (4 bytes). * Packed Coordinates: ![][image66] bits \= ![][image67] bytes. * Total per vector: ![][image68] bytes. **TurboQuant\_prod Storage Formula:** * Input Norm: 1 float32 (4 bytes). * Residual Norm: 1 float32 (4 bytes). * Packed MSE Coordinates: ![][image69] bits. * Packed QJL Signs: ![][image70] bits. * Total per vector: ![][image71] bytes. **TurboVec (Public Artifact) Storage Formula:** * Input Norm: 1 float32 (4 bytes). * Length-Renormalization Correction Scalar: 1 float32 (4 bytes)4. * Packed Coordinates: ![][image66] bits. * Total per vector: ![][image71] bytes. **Global Metadata Overhead (Stored once per index):** * Rotation Matrix (![][image21]): ![][image72] bytes (can be compressed if seeded or replaced by FWHT). * Codebook / Centroids: ![][image73] bytes. * TQ+ Calibration (TurboVec only): ![][image65] shifts \+ ![][image65] scales \= ![][image74] bytes5. ### **Effective Memory Table (Bytes per Vector)** *Assuming the TurboVec formula (![][image75]) without allocator block headers.* | Dimension (d) | 2-bit Storage (Bytes) | 4-bit Storage (Bytes) | Float32 Baseline (Bytes) | Effective Compression Ratio (2-bit) | | :---- | :---- | :---- | :---- | :---- | | **64** | 24 | 40 | 256 | \~10.6x | | **128** | 40 | 72 | 512 | \~12.8x | | **200** | 58 | 108 | 800 | \~13.7x | | **256** | 72 | 136 | 1024 | \~14.2x | | **512** | 136 | 264 | 2048 | \~15.0x | | **768** | 200 | 392 | 3072 | \~15.3x | | **1024** | 264 | 520 | 4096 | \~15.5x | | **1536** | 392 | 776 | 6144 | \~15.6x | | **2048** | 520 | 1032 | 8192 | \~15.7x | | **3072** | 776 | 1544 | 12288 | \~15.8x | | **4096** | 1032 | 2056 | 16384 | \~15.8x | (Note: Claims of "16x compression"4 represent the asymptotic limit of the coordinate data alone and ignore the 8-byte constant overhead which degrades ratios at low dimensions.) ## **9\. Transform Implementation Tradeoffs** (Agent's engineering analysis) The rotation matrix ![][image21] is the computational bottleneck of the TurboQuant architecture. Integrating this into TinyRustLM requires careful selection. | Transform Algorithm | Memory Overhead | Encode Compute Cost | Query Compute Cost | Deterministic | Suitability for Edge/WASM (TinyRustLM) | | :---- | :---- | :---- | :---- | :---- | :---- | | **Dense Orthogonal (QR-derived)** | **![][image1]** floats | ![][image1] MADDs | ![][image1] MADDs | Yes | **Poor**. The dense vector-matrix multiplication induces severe cache pressure and latency during token generation6. | | **Gaussian JL Transform** | **![][image1]** floats | ![][image1] MADDs | ![][image1] MADDs | Yes (seeded) | **Poor**. Used by QJL, but lacks exact orthogonality, introducing distance distortion prior to quantization. | | **Fast Walsh-Hadamard \+ Rademacher (Fast-TurboQuant)** | **![][image76]** floats (signs) | ![][image77] Adds | ![][image77] Adds | Yes | **Excellent**. Eliminates hardware multipliers entirely. Requires zero-padding vectors to the nearest power of two, but achieves ideal sub-Gaussian concentration6. | (Externally corroborated fact) Fast-TurboQuant mathematically proves that applying a Rademacher phase inversion (random sign flipping via XOR operations) followed by an in-place FWHT butterfly network fulfills the prerequisite Gaussian coordinate concentration necessary for Lloyd-Max quantization, yielding a 19.7x algorithmic speedup on CPUs6. ## **10\. DRIVE/EDEN, RaBitQ, PQ, and KV-Baseline Comparison** ### **The DRIVE/EDEN Mathematical Critique** (Externally corroborated fact) In April 2026, the authors of the earlier DRIVE (NeurIPS 2021\) and EDEN (ICML 2022\) algorithms published a rigorous note (2604.18555) revealing that TurboQuant is mathematically derivative and suboptimal3. EDEN defines its dequantized, rotated codeword via a tunable scale parameter ![][image53]: ![][image78] * **TurboQuant\_mse** mathematically equates to EDEN where ![][image53] is permanently fixed to ![][image79]. The note proves this fixed choice is strictly suboptimal3. * **EDEN-biased** computes an optimized scale ![][image80] to minimize MSE, outperforming TurboQuant\_mse empirically3. * **EDEN-unbiased** computes ![][image81] to restore expectation unbiasedness (![][image82])3. (Claim made by paper authors) The EDEN critique proves that allocating the full ![][image31] bits to a single EDEN-unbiased scalar yields vastly lower variance than TurboQuant\_prod's strategy of splitting ![][image46] bits for a biased code and 1 bit for a QJL residual. Asymptotically, 1-bit EDEN variance converges to ![][image83], whereas 1-bit QJL variance converges to ![][image84] (2.75x higher). Consequently, a 2-bit EDEN mathematically outperforms a 3-bit TurboQuant\_prod3. ### **RaBitQ Reproducibility Dispute** (Externally corroborated fact) A separate 2026 technical note (2604.19528) evaluating RaBitQ vs. TurboQuant documented severe reproducibility failures in Google's original publication11. RaBitQ quantizes data using a uniform unsigned integer grid and a per-vector rescaling factor ![][image85], allowing direct distance estimation via native integer bitwise arithmetic without decoding27. The investigators found that TurboQuant consistently performs worse than RaBitQ in identical nearest-neighbor setups. Furthermore, the TurboQuant paper artificially handicapped RaBitQ's efficiency metrics by testing it on a single CPU core with multi-threading disabled, while running TurboQuant on an A100 GPU11. ## **11\. Finite-Dimension and Adversarial Analysis** (Agent's engineering analysis) The theoretical elegance of the TurboQuant proof relies on the Central Limit Theorem forcing the rotated Beta distribution to match a perfect Gaussian as ![][image24]17. In practical SLM constraints (![][image86]), this asymptotic assumption degrades, triggering specific adversarial failures: 1. **Anisotropy and Heavy Tails**: Transformer activations naturally develop massive outlier channels. While random rotation smoothly distributes this outlier energy across all coordinates (preventing channel overflow), it violently skews the resulting empirical distribution away from the idealized Gaussian shape. Precomputed Lloyd-Max centroids will fundamentally misalign with this skewed data7. 2. **Zero and Near-Zero Vectors**: The prerequisite norm separation step (![][image87]) causes division-by-zero NaNs or massive noise amplification when handling padded tokens or zeroed matrices. 3. **MSE vs. KL Divergence Disconnect**: Minimizing MSE (the core objective of TurboQuant) is a flawed proxy for language modeling quality. A small ![][image88] error in a reconstructed Key vector that happens to align geometrically with a high-magnitude Query vector will exponentially distort the Softmax attention distribution. Empirical tests demonstrate that implementations achieving superior MSE can simultaneously trigger catastrophic (\>10%) KL divergence failures on small models7. ## **12\. TurboVec Additions and Divergences** (Directly observed public fact) The open-source RyanCodrai/turbovec repository achieved viral adoption precisely by abandoning the unworkable theoretical elements of TurboQuant in favor of applied engineering4. 1. **Omission of QJL**: TurboVec discards the 1-bit QJL residual correction entirely. 2. **Length-Renormalized Scoring**: To resolve the inner-product shrinkage bias caused by scalar quantization, TurboVec pre-calculates a static correction scalar per vector at ingestion: ![][image89]. At search time, the dot product is simply multiplied by this scalar. This restores unbiasedness with zero query-time computational overhead, mirroring strategies from RaBitQ and EDEN4. 3. **TQ+ Empirical Calibration**: To counteract the finite-dimension distribution drift identified in Section 11, TurboVec adds a shift and scale parameter per coordinate. During the first vector ingestion, it maps the empirical 5th and 95th percentiles of the rotated data to fit the idealized ![][image90] Beta range. This calibration locks in permanently after the first batch (preserving the "training-free" claim) and lifts recall significantly on messy embeddings4. 4. **SIMD Lookup Table (LUT) Scoring**: To bypass full decompression at query time, the query vector is rotated, calibrated, and split to form an 8-bit lookup table. The database's packed 4-bit codes are scored directly in registers using AVX-512/AVX2 pshufb or maddubs instructions4. ## **13\. Reference Pseudocode and Golden-Vector Design** (Agent's engineering analysis) The following reference logic fuses Fast-TurboQuant's multiplier-free FWHT with TurboVec's TQ+ calibration and length-renormalized scoring. This represents the optimal configuration for a TinyRustLM WASM implementation. Matrix dimensions must be power-of-two (zero-padded). Python import numpy as np def pad\_to\_pow2(v): d \= len(v) pow2 \= 1 \<\< (d \- 1).bit\_length() return np.pad(v, (0, pow2 \- d), 'constant') def encode\_edge\_tq\_plus(x, shift\_arr, scale\_arr, centroids, rademacher\_signs): \# 1\. Norm separation and Zero-handling norm \= np.linalg.norm(x) if norm \< 1e-7: return bytearray(len(x)//2), 0.0, 1.0 \# Return zero code u \= pad\_to\_pow2(x / norm) \# 2\. TQ+ Empirical Shift and Scale (Affine projection) u\_calib \= (u \- shift\_arr) / scale\_arr \# 3\. Multiplier-free Rademacher Phase Inversion & FWHT u\_phase \= u\_calib \* rademacher\_signs y \= fast\_walsh\_hadamard\_transform(u\_phase) / np.sqrt(len(u)) \# 4\. Lloyd-Max Quantization (Example: 4-bit) codes \= np.zeros(len(y), dtype=np.uint8) y\_hat \= np.zeros(len(y)) for i in range(len(y)): idx \= np.argmin(np.abs(centroids \- y\[i\])) codes\[i\] \= idx y\_hat\[i\] \= centroids\[idx\] \# 5\. Bit-packing (2 codes per byte for 4-bit) packed\_code \= pack\_4bit\_array(codes) \# 6\. Length-Renormalized Bias Correction correction \= norm / np.dot(y, y\_hat) return packed\_code, norm, correction def query\_lookup\_scoring(q, db\_packed\_codes, db\_corrections, shift\_arr, scale\_arr, rademacher\_signs, centroids): \# Prepare query q\_pad \= pad\_to\_pow2(q) q\_calib \= (q\_pad \- shift\_arr) / scale\_arr y\_q \= fast\_walsh\_hadamard\_transform(q\_calib \* rademacher\_signs) / np.sqrt(len(q\_pad)) \# Build SIMD-friendly LUT for the query against all possible centroids \# lut\[i\]\[k\] contains the precomputed dot product contribution of centroid k at dimension i lut \= np.outer(y\_q, centroids) scores \= \[\] for code, correction in zip(db\_packed\_codes, db\_corrections): raw\_score \= sum(lut\[i\]\[unpack(code, i)\] for i in range(len(y\_q))) scores.append(raw\_score \* correction) return scores **Golden Vector Validation Strategy**: Independent TinyRustLM implementations must cross-verify outputs against these edge-cases: * **Zero Vector (\[0, 0, ..., 0\])**: Ensures the ![][image15] norm division trap executes gracefully. * **Canonical Basis Vector (\[1, 0, ..., 0\])**: Confirms the FWHT evenly distributes the single active coordinate's magnitude (![][image91]) across all dimensions. * **Adversarial Outlier Vector (\[100.0, 0.01, 0.01, ...\])**: Stresses the TQ+ calibration arrays. Evaluates if the extreme shift requirement degrades the available resolution for the remaining 99% of the signal. ## **14\. TinyRustLM Applicability Matrix** (Agent's engineering analysis) | Domain | Applicability | Exact Reason | | :---- | :---- | :---- | | **1\. Static Model-Weight Quantization** | Not supported by current evidence | Neural network weights naturally form heavy-tailed ![][image92]\-stable distributions that benefit from outlier-aware quantization (e.g., AWQ). Applying a global data-oblivious rotation destroys localized parameter sparsity and disrupts standard linear algebra matrix-multiply optimizations required by WASM executors30. | | **2\. Online KV-Cache Storage** | Credible (with strict modification) | Dense orthogonal rotations ![][image1] will crush autoregressive decode latency. PolarQuant's trigonometry will similarly stall the CPU. To be viable, the KV-cache must rely *exclusively* on the Fast-TurboQuant (FWHT ![][image2]) pathway6. Additionally, the KL-divergence sensitivity must be verified locally. | | **3\. Persistent Vector Retrieval** | Credible | The TurboVec architecture (TQ+ calibration, pre-computed query LUTs, and bit-packed length-renormalized scoring) is explicitly designed for this constraint. It provides massive RAM reduction with offline persistence while enabling fast hybrid-filtering SIMD search4. | ## **15\. Performance and Accuracy Experiment Recommendations** To validate these theories within TinyRustLM without accumulating technical debt, the following minimal, isolated experiments are recommended: **Experiment A: Fast-TurboQuant KV-Cache Shadow Metric Evaluation** * **Baseline**: Uncompressed Float32 KV Cache (standard TinyRustLM decoder). * **Controlled Variable**: Introduce a parallel, shadow KV-cache using FWHT \+ 4-bit Lloyd-Max centroids. * **Quality Metric**: Compute the exact KL Divergence between the baseline Softmax attention output and the quantized attention output. (Do not rely on coordinate MSE). * **Performance Metric**: CPU cycles consumed by the FWHT rotation per token generation. * **Stop Rule**: If the median KL divergence exceeds 5%, or if FWHT calculation exceeds the memory bandwidth latency savings, abandon KV implementation7. **Experiment B: TurboVec Local SIMD Emulation** * **Baseline**: Existing local exact nearest neighbor (Flat L2/Cosine) implementation. * **Controlled Variable**: Implementation of TurboVec's length-renormalized scoring using WASM v128 SIMD pshufb equivalents. * **Quality Metric**: Recall@10 on a static dataset of ![][image93] embedded context chunks. * **Performance Metric**: Query latency (ms) and peak RSS memory utilization. * **Stop Rule**: If Recall@10 falls below 98% relative to the baseline, the dimension (![][image65]) is too small to support the asymptotic Gaussian assumption, requiring fallback to standard scalar quantization (SQ). ## **16\. Claims That Could Not Be Verified** 1. **"Zero Accuracy Loss"**: (Claim made by paper authors) Marketing abstracts assert "zero accuracy loss" at 3.5 bits28. Review of the LongBench task breakdown indicates this relies on aggregated averaging; individual sub-tasks fluctuate up or down significantly22. Mathematical lossless compression of floating-point vectors below the Shannon entropy bound is theoretically impossible17. 2. **TurboQuant Indexing Supremacy**: The TurboQuant authors' claims of dominating RaBitQ in latency and recall were systematically challenged by the RaBitQ authors, who proved the Google researchers disabled threading on the baseline tests11. 3. **QJL Superiority over EDEN**: The TurboQuant paper asserts its QJL residual step is optimal. The independent mathematical proof by Ben-Basat et al. strongly disputes this, demonstrating that reallocating that bit to an unbiased EDEN scalar reduces variance by a factor of 2.75x3. ## **17\. Unknowns Requiring Local Execution** (Facts that require local TinyRustLM verification) 1. **WASM SIMD Viability**: The TurboVec crate utilizes Rust's \#\!\[feature(stdsimd)\] intrinsics for AVX2/AVX-5125. The precise translation of these 8-bit lookup operations into WebAssembly's 128-bit SIMD instruction set, and their resulting cycle latency on typical browser engines (V8/SpiderMonkey), is entirely unknown. 2. **Low-Dimension Beta Collapse**: TinyRustLM utilizes extremely small 0.3B parameter models, suggesting embedding dimensions heavily truncated compared to datacenter models (![][image94]). The assumption that the Beta distribution sufficiently mimics a Gaussian at these low dimensions to support static Lloyd-Max centroids requires local validation on TinyRustLM's specific semantic topologies5. ## **18\. Annotated Primary-Source Bibliography** * **Zandieh, A., et al. (April 2025). *TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate.*** arXiv:2504.19874. ICLR 2026\. *Core definition of data-oblivious scalar quantization and the QJL residual correction. Asserts near-optimal MSE distortion.* * **Ben-Basat, R., et al. (April 2026). *A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work.*** arXiv:2604.18555. *Crucial mathematical critique proving TurboQuant is a suboptimal variant of EDEN (![][image5]). Proves that 1-bit QJL variance is mathematically inferior to 1-bit EDEN.* * **Gao, J., et al. (April 2026). *Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments.*** arXiv:2604.19528. *Documentation of severe reproducibility failures in the TurboQuant paper's comparative baselines against RaBitQ.* * **Pereira, P., et al. (June 2026). *Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach.*** arXiv:2606.21448. *Hardware-centric modification replacing the ![][image1] dense rotation with a multiplier-free Fast Walsh-Hadamard Transform and Rademacher phase inversion.* * **Han, I., et al. (February 2025). *PolarQuant: Quantizing KV Caches with Polar Transformation.*** arXiv:2502.02617. AISTATS 2026\. *Proposes recursive polar coordinate mapping to eliminate normalization overhead. Identified in this report as computationally hostile to edge CPU/WASM execution due to trigonometry.* * **Zandieh, A., et al. (June 2024). *QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead.*** arXiv:2406.03482. AAAI 2025\. *Establishes the foundational mathematics of the 1-bit asymmetric unbiased estimator.* * **Codrai, R. (Mid-2026). *turbovec*.** docs.rs/turbovec. GitHub repository. *The public Rust artifact that bridges theoretical TurboQuant into practical engineering by abandoning QJL, utilizing length-renormalized scoring, and adding TQ+ empirical calibration.* #### **Works cited** 1. TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate \- Hugging Face, [https://huggingface.co/papers/2504.19874](https://huggingface.co/papers/2504.19874) 2. TURBOQUANT: ONLINE VECTOR QUANTIZATION WITH NEAR-OPTIMAL DISTORTION RATE \- OpenReview, [https://openreview.net/pdf?id=tO3ASKZlok](https://openreview.net/pdf?id=tO3ASKZlok) 3. A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work \- arXiv, [https://arxiv.org/html/2604.18555v1](https://arxiv.org/html/2604.18555v1) 4. GitHub \- RyanCodrai/turbovec: A vector index built on TurboQuant, written in Rust with Python bindings, [https://github.com/RyanCodrai/turbovec](https://github.com/RyanCodrai/turbovec) 5. turbovec/CHANGELOG.md at main \- GitHub, [https://github.com/RyanCodrai/turbovec/blob/main/CHANGELOG.md](https://github.com/RyanCodrai/turbovec/blob/main/CHANGELOG.md) 6. Fast-TurboQuant A Multiplier-Free Online Vector Quantization Approach \- arXiv, [https://arxiv.org/html/2606.21448v1](https://arxiv.org/html/2606.21448v1) 7. Use scale factor for improvements · Issue \#87 · TheTom/turboquant\_plus \- GitHub, [https://github.com/TheTom/turboquant\_plus/issues/87](https://github.com/TheTom/turboquant_plus/issues/87) 8. QJL: 1-BIT QUANTIZED JL TRANSFORM FOR KV CACHE QUANTIZATION WITH ZERO OVERHEAD \- OpenReview, [https://openreview.net/pdf/b470267d0a4e09ab770de6b004939bc7c6114304.pdf](https://openreview.net/pdf/b470267d0a4e09ab770de6b004939bc7c6114304.pdf) 9. PolarQuant: Quantizing KV Caches with Polar Transformation \- arXiv, [https://arxiv.org/pdf/2502.02617](https://arxiv.org/pdf/2502.02617) 10. Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments \- arXiv, [https://arxiv.org/pdf/2604.19528](https://arxiv.org/pdf/2604.19528) 11. \[Literature Review\] Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments \- Moonlight, [https://www.themoonlight.io/en/review/revisiting-rabitq-and-turboquant-a-symmetric-comparison-of-methods-theory-and-experiments](https://www.themoonlight.io/en/review/revisiting-rabitq-and-turboquant-a-symmetric-comparison-of-methods-theory-and-experiments) 12. \[2606.21448\] Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach, [https://arxiv.org/abs/2606.21448](https://arxiv.org/abs/2606.21448) 13. PolarQuant: Quantizing KV Caches with Polar Transformation \- arXiv, [https://arxiv.org/html/2502.02617v1](https://arxiv.org/html/2502.02617v1) 14. TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate \- OpenReview, [https://openreview.net/forum?id=tO3ASKZlok](https://openreview.net/forum?id=tO3ASKZlok) 15. turbovec \- Rust \- Docs.rs, [https://docs.rs/turbovec](https://docs.rs/turbovec) 16. TurboQuant: A First-Principles Walkthrough \- Arkar Min Aung, [https://arkaung.github.io/interactive-turboquant/](https://arkaung.github.io/interactive-turboquant/) 17. skr3178/TurboQuant-explained: Implementation of TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate \- GitHub, [https://github.com/skr3178/TurboQuant-explained](https://github.com/skr3178/TurboQuant-explained) 18. TurboQuant \- Wikipedia, [https://en.wikipedia.org/wiki/TurboQuant](https://en.wikipedia.org/wiki/TurboQuant) 19. TurboQuant: From Paper to Triton Kernel in One Session \- Dejan SEO, [https://dejan.ai/blog/turboquant/](https://dejan.ai/blog/turboquant/) 20. Breaking Down TurboQuant \- Surya Sure, [https://www.suryasure.com/articles/breaking-down-turboquant](https://www.suryasure.com/articles/breaking-down-turboquant) 21. \[Literature Review\] TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate \- Moonlight, [https://www.themoonlight.io/en/review/turboquant-online-vector-quantization-with-near-optimal-distortion-rate](https://www.themoonlight.io/en/review/turboquant-online-vector-quantization-with-near-optimal-distortion-rate) 22. TurboQuant: Google Removed All Overhead From KV compression unlocking near unlimited upside | by Mandar Karhade, MD. PhD. | AI Advances, [https://ai.gopubby.com/turboquant-google-removed-all-overhead-from-kv-compression-unlocking-near-unlimited-upside-42a8f09495e7](https://ai.gopubby.com/turboquant-google-removed-all-overhead-from-kv-compression-unlocking-near-unlimited-upside-42a8f09495e7) 23. PolarQuant: Quantizing KV Caches with Polar Transformation \- DEV Community, [https://dev.to/patelchaitany/polarquant-quantizing-kv-caches-with-polar-transformation-255](https://dev.to/patelchaitany/polarquant-quantizing-kv-caches-with-polar-transformation-255) 24. GitHub \- AliesTaha/polar\_quant: PolarQuant: Fused KV cache decode kernel that ties cuBLAS on B200 with 4.1x compression, [https://github.com/AliesTaha/polar\_quant](https://github.com/AliesTaha/polar_quant) 25. Open Source Project of the Day (\#90): turbovec \- The Vector Index That Shrinks 10M Docs from 31 GB to 4 GB, [https://dev.to/wonderlab/open-source-project-of-the-day-90-turbovec-the-vector-index-that-shrinks-10m-docs-from-31-gb-120p](https://dev.to/wonderlab/open-source-project-of-the-day-90-turbovec-the-vector-index-that-shrinks-10m-docs-from-31-gb-120p) 26. How a 2021 Quantization Algorithm Quietly Outperforms Its 2026 Successor, [https://towardsdatascience.com/how-a-2021-quantization-algorithm-quietly-outperforms-its-2026-successor/](https://towardsdatascience.com/how-a-2021-quantization-algorithm-quietly-outperforms-its-2026-successor/) 27. Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments \- arXiv, [https://arxiv.org/html/2604.19528v1](https://arxiv.org/html/2604.19528v1) 28. TurboQuant: 31GB AI Memory Down to 4GB, Explained | TECHSY, [https://techsy.io/en/blog/google-turboquant-ai-memory-compression](https://techsy.io/en/blog/google-turboquant-ai-memory-compression) 29. TurboQuant in Qdrant, [https://qdrant.tech/articles/turboquant-quantization/](https://qdrant.tech/articles/turboquant-quantization/) 30. Embedding Compression via Spherical Coordinates \- arXiv, [https://arxiv.org/html/2602.00079v4](https://arxiv.org/html/2602.00079v4) 31. (PDF) Block-Sphere Vector Quantization \- ResearchGate, [https://www.researchgate.net/publication/405045609\_Block-Sphere\_Vector\_Quantization](https://www.researchgate.net/publication/405045609_Block-Sphere_Vector_Quantization) 32. What Is TurboQuant? DeepMind's Massive Context Algorithm \- ThoughtMinds, [https://thoughtminds.ai/blog/what-is-turboquant-deepmind-algorithm](https://thoughtminds.ai/blog/what-is-turboquant-deepmind-algorithm) [image1]: [image2]: [image3]: [image4]: [image5]: [image6]: [image7]: [image8]: [image9]: [image10]: [image11]: [image12]: [image13]: [image14]: [image15]: [image16]: [image17]: [image18]: [image19]: [image20]: [image21]: [image22]: [image23]: [image24]: [image25]: [image26]: [image27]: [image28]: [image29]: [image30]: [image31]: [image32]: [image33]: [image34]: [image35]: [image36]: [image37]: [image38]: [image39]: [image40]: [image41]: [image42]: [image43]: [image44]: [image45]: [image46]: [image47]: [image48]: [image49]: [image50]: [image51]: [image52]: [image53]: [image54]: [image55]: [image56]: [image57]: [image58]: [image59]: [image60]: [image61]: [image62]: [image63]: [image64]: [image65]: [image66]: [image67]: [image68]: [image69]: [image70]: [image71]: [image72]: [image73]: [image74]: [image75]: [image76]: [image77]: [image78]: [image79]: [image80]: [image81]: [image82]: [image83]: [image84]: [image85]: [image86]: [image87]: [image88]: [image89]: [image90]: [image91]: [image92]: [image93]: [image94]: