Publications

An INT4 deployment of a reasoning model is a choice of quantizer, repair, export form and decoding control, usually made by accuracy, generated tokens or a fixed-length benchmark, none of which sees how a serving engine charges for length. We serve Qwen3-4B in vLLM on an RTX 4090 and price each configuration by its measured GPU time per solved problem. Under KV-cache-limited load, INT4's extra tokens consume 41–47% of its per-token saving over BF16, against 6% at batch 1, and forcing output lengths and shrinking the KV pool trace this excess to KV-capacity preemption. A repair's time cost changes sign with its serving form: a distillation adapter served unmerged costs more than its INT4 base at batch 1, while merged and requantized it costs less at both loads. Configurations that match BF16's accuracy span a 1.61-fold range of GPU time per solved problem, ParoQuant's own kernel included; pass@1 has no rank agreement with this cost, and a fixed-length benchmark picks a configuration 21.4% costlier than the best. A scheduler replay fitted under load predicts five configurations' relative cost at unprofiled loads on the same GPU within 5.2%, and since on harder workloads and a second model the two configurations cheapest on GSM8K are never both accuracy-matched, each workload is screened and priced on its own.

Procedures whose attempts share state, such as search trees, shared-prefix sampling and self-debugging loops, can be scored by a pass@k computed from the total number of successes over all their attempts. We ask what that pooled total supports. In the model where each of R runs draws a latent success rate and makes n conditionally independent attempts, no function of the pooled total is unbiased for pass@k when R, n and k are all at least 2, whether pass@k is read as the procedure's chance of success within k attempts or as the success chance of k independent attempts. The distribution of the total still determines the independent-attempt target for every k and the procedure target for k ≤ n; we give the complete classification of when an unbiased function of the total exists. Per-run counts, by contrast, are known to admit unbiased estimators of the procedure target for k ≤ n and of the independent-attempt target for k ≤ R. Below that run threshold, at R = k − 1 runs, the delete-one jackknife over the runs' first attempts has exact worst-case bias 3.86 percentage points at k = 4 and 2.91 at k = 16, far above the minimax bias 2 · 4^−k, and the same values are lower bounds on the worst case of the jackknife over the runs' success fractions; yet in ten of eleven recorded benchmark cells from eight designs the jackknife's mean error against held-out runs is within 0.44 points, and −2.02 in the eleventh. An evaluation that logs only the pooled total, once R, n, k ≥ 2, gives up unbiased estimation that per-run counts would allow.

Tabular foundation models answer every query about a table by in-context inference, so whether their answers fit one joint distribution tests whether they can serve as probabilistic models of the table. A recent marginalization test compares a target's marginal predicted with another column deleted from the training table against a mixture of conditionals predicted from tables that keep it, and finds violations in every model evaluated. This comparison also changes the training evidence, and a coherent Bayes predictor violates it: its gap equals the information the deleted column carries, up to that of the other column, and vanishes when the two columns' parameters are a priori independent in a suitable factorization. On 20 synthetic families with exact posteriors, the test flags the exact Bayes predictor in 14, while its evidence-fixed residual (comparing the two conditionals a model returns for one table) stays at round-off; the same work's factorization test flags it in 16, and holding its evidence fixed reduces it to twice the residual. For TabICLv2 and TabPFNv2 at their default ensemble, this residual exceeds three times its seed floor, the change of the same predictions under a new ensemble seed, in 0 of 60 settings for either model, against 34 and 35 for the column-deletion statistic. Holding the evidence fixed also admits an exact certificate, a linear program over joint laws of the table that needs no prior. At an external ensemble of 64 single-estimator members, in at least 19 of 20 row blocks per model and table family, each formed by a three-variable table with its last row removed and every completion of that row, the answers on at least one table are not within 𝜏𝑊 = 0.01 of the conditionals of any joint law, by an exact rational certificate. Hence neither model's answers are those of a Bayesian model to within 𝜏𝑊 , and the group-mean incompatibilities are 5.9 to 1,900 times the models' seed noise. Coherence tests for in-context predictors should hold the training evidence fixed.

Self-consistency samples reasoning chains and returns the most frequent answer, and adaptive variants stop sampling early to save compute. Deployments act on the stopped answer, yet these rules check a posterior or window criterion as draws arrive, so the chance of stopping on an answer that is not the population mode has no time-uniform bound. In simulation, Adaptive Consistency (AC) at its published setting significantly exceeds the tolerance implied by its credible level in 27 of 70 cells. Existing anytime-valid mode certificates need the number of answers in advance, control nulls that must hold at every round rather than the error of the answer returned at stopping, certify a target fixed before sampling, or restart fixed-sample tests on fresh draws; a tuple-indexed repair for a data-selected leader is stated under a unique mode. We give a stopping rule for i.i.d. draws over an unknown, possibly countably infinite answer set that certifies at level δ that an answer it selects from the data is a population mode. Each discovered answer must beat every rival, seen or unseen, against a threshold set by the order in which it was first seen rather than by the size of the answer set. Under a unique mode, the expected stopping time of this complete rule attains the optimal leading constant as δ → 0 without knowledge of the answer set. The rule and its four variants never exceed δ in 140 i.i.d. simulation cells; where the rule and AC both certify at least 90% of replicates, the rule costs a median 3.2 times AC's draws. On GSM8K with Qwen2.5-1.5B at δ = 0.05, the rule, set by δ alone, lands within one wrong certificate and four certified questions in 200 of the coverage–error frontier that AC reaches with a threshold tuned in-sample; on Qwen3.5-2B and 9B it makes at most one more wrong certificate in 200 than AC at matched coverage.

Adaptive testing is entering language-model evaluation as an efficiency device, and its low item exposure is offered as incidental contamination protection (Li et al., 2026b). That argument assumes untargeted leakage, yet a maximum-information test asks predictable items that anyone observing administrations can learn. Here we fine-tune a 1.5B-parameter model on 1% of a self-calibrated MMLU bank, chosen by exposure in a clean adaptive run, and score adaptive, random and whole-bank protocols on the same response vectors. A 41-item adaptive test then rises by 1.47 to 1.59 logits over its own clean reading while the whole bank reads slightly lower. Against random-item training approximately matched for the loss of clean-item accuracy, the adaptive reading is 1.40 logits higher (95% CI 1.10 to 1.69). Two further models show the same sign pattern. In the released logs of two adaptive LLM evaluators, the most-administered 1% of each instrument's largest bank takes 30–60% of administrations (Fluid at 100 items). No change confined to a targeted set can move the expected reading of a reader that is design-unbiased for bank accuracy by more than the set's share of the bank. At 100 items, several such readers come within 10% of the released readers' error while a set read from their own logs moves them about one-for-one with bank accuracy, against 3.4-fold or more for the released readers; at 41 items the comparison is undetermined. Adaptive-test logs therefore need a threat model for their observers, and adaptive scores a whole-bank reading beside them.

Covariate-aware time-series foundation models forecast a target from the related series supplied with it, and which series they receive depends on how a pipeline was built: the same sensor exported twice, a temperature in two units, one weather station joined in for several zones. We ask whether these models count such representations as evidence. In synthetic worlds with a closed-form Bayes reference, we supply one source as m channels and measure by finite differences how strongly the forecast follows it relative to a second source, its implied log evidence ratio. Eight exact copies raise this ratio by 0.81, 0.71 and 2.28 nats in Chronos-2, t0-beta and TiRex-2, where the Bayes reference does not move: 0.33, 0.31 and 1.00 of the response to eight independent measurements. Measurements that share part of their error are over-counted as well, and in variate attention an exact copy is exactly a log-multiplicity weight on its key. The count reaches beyond synthetic data. On 18 fev-bench tasks, supplying every dynamic covariate eight times raises the error of three of four models with resolved intervals, and in a controlled pipeline on a public energy dataset whose own zone mapping duplicates a weather station, inputs constructed to repeat a zone's weather series change the day-ahead forecasts of all three models. Declaring the source removes the dependence without training: keeping one channel per declared alias, and adding to the keys of measurements that share an error the log of their effective number over their count, makes forecasts exactly invariant to copies. In a comparison fixed before its results and run on fresh worlds, this source bias has a smaller largest absolute mean excess over the Bayes derivative reference on such measurements than keeping as many of them as their effective number, a deletion heuristic, in all three models, but in point estimate a larger one than averaging the channels (percentile intervals resolve it in two models, simultaneous bands in none). With the error correlation correctly declared as zero it keeps the full response to independent measurements, as does any rule that leaves measurements declared independent unchanged, and on forecast loss it is not consistently better than averaging either. Copies sometimes help accuracy, and declaring a source gives that help up. For these models, how a covariate table is built is part of the model specification.

Language models are commonly served with 4-bit weights, and the format is chosen on perplexity and benchmark averages, which do not vary how strongly the context competes with the answer. A quantized model that fails to resolve such interference can answer fluently with the competing binding. The closest controlled study measured this failure with the competitor placed before the target only. Here we cross proactive and retroactive layouts with list length, interference load, and weight and KV-cache formats on recall trials paired within item in 1.5–8B open-weight models, and recompute attention in float32 on the same trials. Whether NF4 damage is layout-selective changes with the operating point within one model. On Qwen2.5-1.5B the NF4 penalty is proactive-selective on 16-pair lists (accuracy selectivity +0.231 [+0.181, +0.281]), and the format × layout interaction moves toward zero with list length (change +1.07 [+0.43, +1.71] from 16 to 64 pairs), where it is no longer resolved. On Granite-4.2-3B, at an operating point matched on full-precision accuracy alone, the layout contrast appears only under load (change −0.68 [−1.12, −0.24]). NF4 and FP4 at equal bit width leave different damage signatures, while INT8 weights cost little and 8- and 4-bit KV caches show no consistent layout-selective damage in the tested 1.5B setting. On Qwen2.5 an attention shift toward the competitor accompanies the damage and, read after quantization, adds to the full-precision margin in distinguishing intrusion flips from answers that stay correct (held-out AUC gain +0.193 at 1.5B), yet restoring the late layers where it appears does not recover detectably more accuracy than restoring early ones. A single-layout test at one operating point therefore cannot certify a 4-bit format for interference-heavy use: both layouts have to be measured at the workload's own list length and load.

Practitioners choosing between released 4-bit files of a model are guided by published comparisons such as "AWQ consistently outperforms ... GPTQ", although what they serve and how they measure it differ from what the comparison covered. We cross the measurement regime (perplexity against downstream accuracy) with the serving engine on released AWQ and GPTQ pairs on consumer GPUs, and report each verdict with its interval, margin and realized precision. On a 7B pair within the comparison's stated 7B–70B range, WikiText-2 perplexity places AWQ behind by 0.993% while LAMBADA accuracy, chosen after reading eight tasks, places it ahead by 3.136 points, both beyond their margins on both engines; the opposition persists when one dataset is scored both ways. In a verdict matrix over five claim cells, changing the measurement regime changed verdicts and changing the serving engine changed none; on that 7B pair and a smaller pair of its family, most of the eight tasks fixed before measurement gave intervals too wide to resolve the 1-point margin. The choice of which file to serve depends on the slice too: on seven of eight released pairs the published preference picks the file that solves fewer GSM8K problems, and on one pair the chat template the two files are served with sets the size of that loss. Scoring both files on paired development items, with the seconds a perplexity check would take, more than halves the mean loss of following the published preference on four pairs selected by a metadata rule before measurement. Each verdict therefore needs its slice and realized precision stated beside it.

Edge deployments of promptable segmenters are checked by the quality of the first mask or by simulated click counts, yet an interactive tool is used through its corrections. We measure deployment on the correction itself: from the same image, click history and starting mask, we compare the next mask of a TensorRT engine with that of its PyTorch reference, splitting the change into the part of the starting error removed and the new error introduced. Across 6 segmenters on a Jetson Orin Nano and two benchmarks, the explicit INT8 engines of SAM 2.1-T, SAM 2.1-S, EdgeTAM and MobileSAM and the implicit INT8 engines of EdgeTAM, MobileSAM and EdgeSAM change both parts of the next correction, each under Holm control within its test family, and EfficientViT-SAM-L0 does so already at FP16 (+0.028 of the starting error removed and +0.056 introduced); the changes largely persist on objects whose first masks score alike, and simulated users need more clicks for most of these engines. For EfficientViT-SAM-L0 the change traces to 46 convolutions in the last two backbone stages: constraining only those to FP32 in an FP16-enabled engine removes 95% and 100% of the change on held-out data and keeps both mean differences of the repaired engine from the reference within ±0.01 of the starting error, at 20.9 ms against 13.8 ms for FP16. In the explicit INT8 engines most of the change arises in the image-encoder trunk: keeping the trunk at FP16, a rule chosen on two models, brings the corrections of EdgeTAM, SAM 2.1-S and SAM 2.1-T within the same margin of the reference’s and removes most of MobileSAM’s change, with a faster encoder than explicit INT8. In video propagation with SAM 2.1-T and EdgeTAM, the explicit INT8 engines’ masks diverge from the reference’s over the 30 frames after a correction beyond an implementation-only floor, with no detected change in mean region quality (a test that simulation on development sequences had given low power), and the trunk rule removes at least three quarters of that divergence; only SAM 2.1-T’s repaired engine then agrees with the reference within one point. A deployment check for an interactive segmenter should replay corrections, not only score the first output.

When one key is updated again and again within a single context, language models answer with a value that has already been overwritten, long before the context window fills. Suppressing attention to the stale value reduces those answers, but no study has held the deletion fixed and varied only where the deleted attention goes, so it is unknown how much of a repair comes from the deletion and how much from the destination. Here we delete the answer position’s attention to a key’s superseded values by one rule at the same positions under every condition and send it to a different destination in each, in five models from four families (1.5B–8.79B parameters). Deleting it without a destination never restores clean accuracy: it recovers from 0.25 of the accuracy gap in Qwen2.5-1.5B to 0.88 in Granite-4.2-8B. Sending it to the queried key’s current value restores clean accuracy in every model and beats spreading it elsewhere, by +0.63 [+0.54, +0.72] in Qwen2.5-1.5B down to +0.18 [+0.11, +0.26] in Granite-4.2-8B (95% intervals). Sent instead to another key’s current value, it restores nothing and the model answers that value; sent to another key’s earlier value or to a key word, it answers that token. Without any deletion, swapping in the attention from a matched prompt in which the same old values are assigned to a new key raises accuracy more than swapping in that prompt’s values, by +0.558 to +0.835: the failure is in where the answer position reads. Removing the competitor is not enough: in these models, only sending the deleted attention to the current value restores clean recall.

Background and objective: Telemedicine assesses voice from recordings sent over telephone and video links, where lossy speech codecs commonly sit between the voice and the screening model, and clinicians act on each patient’s flag. Codec studies report aggregate accuracy, yet two pipelines with the same false-alarm rate can flag different patients. We ask which patients a codec changes at the operating false-alarm budget, and whether this exceeds re-measuring the same voice.

Methods: We audit clean-trained voice-pathology detectors (an acoustic-feature model, three frozen speech-encoder probes, a fine-tuned encoder) on sustained vowels from three public corpora, pairing each speaker’s score before and after each of 15 telephony and neural codec settings. At the uncompressed pipeline’s control-alarm budget, each setting’s lost detections are compared with a within-vowel yardstick, the loss from re-measuring another stretch of the same vowel without a codec, using a speaker bootstrap with Holm correction across settings. Band-limiting and spectral-shaping shams and level renormalization test alternative explanations, and recalibration on separate speakers tests a remedy.

Results: On the largest cohort (1026 speakers, within-corpus training), compression changes which patients are flagged even where sensitivity rises: with AMR-WB, the acoustic detector’s sensitivity rises from 35.6% to 40.5%, yet 29 previously flagged patients are lost (on vowel halves, 49 against 28 for re-measurement) and 52 others newly flagged. Of the 15 settings studied, 12 lose more flagged patients than re-measurement (9–12 over six cross-validation fold assignments), and three encoder probes agree (11–13 of 15); for the acoustic detector, band-limiting, spectral shaping and level do not explain the excess. For cross-corpus-trained detectors, recalibration keeps the median false-alarm rate over splits near its target, yet in 10–15 of 15 settings, depending on the detector, its median loss of previously flagged patients exceeds that of equally recalibrated re-measurement.

Conclusions: On the largest cohort, most settings studied lose more flagged patients than re-measurement at a matched false-alarm rate, so in telemedicine the codec is part of the screening protocol. Before adopting or changing a codec, a screening service should report which patients change at its operating budget against a re-measurement yardstick; stored paired scores suffice.

GPTQ-style quantization damps the calibration Hessian by a fixed fraction of its mean diagonal (one percent in the GPTQ authors' code, five percent in a maintained implementation), a choice that ignores how many calibration rows there are. Small tabular, time-series and text-embedding foundation models can be calibrated with about as many rows as their layers are wide or fewer, and there no fixed fraction fits: on the layers we measure, the best fixed multiplier moves from 0.0064 where rows and width are of the same order to 0.77 where calibration is scarcer. Building on the known damping–ridge identity, we show that the Cholesky factor GPTQ already builds holds the d nested ridge regressions that the damping regularizes. Under an idealized rounding model, this reading decomposes the layer's population error exactly and bounds, from the population activation spectrum alone, what GPTQ can gain over rounding to nearest; it also makes the penalty estimable, because the leave-one-out error summed over the regressions (LOOCV) comes out of the same factorization with no extra quantization pass. On 175 real-layer cells with at most 512 calibration rows from four foundation models, LOOCV, chosen among six cross-validation criteria, has a median excess cost (extra error over the best error among the selection sweep's arms, as a fraction of its gain over rounding to nearest) of 0.0048 against 0.0843 for the one-percent default and 0.0401 for the five-percent rule, lower than every fixed, searched or hindsight rule we compare; a confirmation with the rule fixed in advance, on re-split calibration pools and a model absent from the selection, repeats its advantage over the one-percent default where calibration rows and layer width are of the same order. In whole-model 4-bit quantization of the Qwen3.5-0.8B language model, LOOCV also gives lower WikiText-2 perplexity than the one-percent default in each of the three calibration-size and format settings we test, and lower than the five-percent rule with 512 calibration tokens. Where calibration rows and layer width are of the same order, damping can therefore be estimated from each layer's calibration set rather than fixed.

Search trees, planners and self-correcting agents make the attempts of one run share a random state, yet their results are often scored with a pass@k estimator built for independent draws, which public harnesses apply to whatever attempts they receive. Prior work separates the procedure's success probability from the independent-attempt one; we ask which of them such a number measures and how to report it validly, taking the complete run as the sampling unit. Whether a run succeeds within k attempts is an unbiased estimate of the procedure's success probability succ@k under any within-run dependence, and succ@k is the only quantity estimable from the first k attempts, with estimates in [0,1], that equals pass@k when attempts are independent. Pooling attempts across runs estimates a third quantity, which we give in closed form and which, for attempts conditionally i.i.d. within a run, lies between succ@k and the independent-attempt target pass@k^iid. Under a calibration condition and over a rich class of dependence structures, pass@k^iid has a uniformly unbiased estimator exactly when at least k complete runs are observed, however much is recorded inside each run, while per-question run-level confidence sequences are valid at any number of runs, for succ@k and, under the same condition, for pass@k^iid. In 8 designs spanning four dependence mechanisms, the pooled report exceeds the run-level estimate of succ@4 in every design, by +0.0205 to +0.1766, and on one design a run-level interval contains a fresh reference 0.952 of the time, against 0.104 for an attempt-level one. The fix is to name the target and count fresh runs.

Do music foundation models encode auditory-scene-analysis (ASA) cues, the grouping principles behind source perception? A high cue score alone cannot answer this: the ordering may be inherited from the model's input transform or, for timing, arise from onset positions alone. We introduce the Bregman Cue-Sensitivity Score (BCS), a training-free geometric read-out of four ASA cues, with a reference for each explanation: the matched front-end under the same operator, and index-only references for the temporal cue. Across sixteen checkpoints (thirteen in the main map, ten with a matched front-end; MusicGen-S/M/L as an extraction case study), tested fixed transforms match or exceed the highest main-map checkpoint score on three cues, and paired contrasts fall on both sides of the front-end. For temporal proximity, a model-free clock scores above every checkpoint; only 3 of thirteen checkpoints resolvably exceed their shared onset trajectory. Cue probes should therefore report each applicable reference.

This manuscript is a non-peer-reviewed preprint.

This manuscript is a non-peer-reviewed preprint.