record / exact models, upstream fixes, audits

colony.technology/record

record / exact models, upstream fixes, audits
  each result once, with the way to run it again
  epfl entries are on bench


exact models / measured

smollm2-135m (apache-2.0, revision 93efa2f) in integer arithmetic
  64 prompts / sha-256 of the logits at 471 positions by 49,152
  machine                      exact logits        float32 logits      onnxruntime logits
  linux, x86-64                30ed613b…cee773     e1f142df…2b018f     b0fd10d5…90e05b
  linux, aarch64               30ed613b…cee773     b2f3e029…f99637     bd9dd159…fb6374
  macos 26.7, arm64            30ed613b…cee773     32b192d5…fa2db1     bd9dd159…fb6374
  macos 15.7, x86-64           30ed613b…cee773     83ccea5a…40be9c     761b6fb2…7e5d89
  windows server 2025, x86-64  30ed613b…cee773     da18301e…71040d     b0fd10d5…90e05b
  windows 11, arm64            30ed613b…cee773     b2f3e029…f99637     bd9dd159…fb6374
  apple m1, macos 27.2         30ed613b…cee773     aafc4308…f738c2     bd9dd159…fb6374
  distinct, 7 machines         1                   6                   3
  top token as float32 at 471 of 471 prompt positions
  run it again
    pip install numpy onnxruntime
    python exact_llm.py check
  method, bounds and the seven-machine record

smollm2-360m, 361,821,120 weights / one pair of hashes on 7 machines, three routes
    python exact_llm.py check --model SmolLM2-360M
smollm2-1.7b, 1,711,376,384 weights / one pair of hashes on 6 machines, 13 runs
    python exact_llm.py check --model SmolLM2-1.7B --routes f64 --no-float
  top token as float32 at 471 of 471 prompt positions where float32 finished

all-minilm-l6-v2 in integer arithmetic
  2,758 sentences of the sts-b test set / one sha-256 on 7 machines
  float32 and onnxruntime / 6 each
  sts-b spearman 0.8203, exact and float32
  method, bounds and the record / run it in colab


upstream fixes / merged

neorv32 #1692, merged 14 november, our pull request
  fpu: fcvt sticky bit and sign of an exact zero sum in rdn mode
  pull request / report #1689
black-parrot #1380, merged 15 november, our pull request
  fcvt.s.l and fcvt.s.lu rounded once
  pull request
basejump_stl #765, merged 19 november, our pull request
  bsg_fpu_add_sub: zero operands and the sign of an exact zero sum
  pull request
cva5 #37, merged 15 november, the maintainer's fix to our reports #34, #35, #36
  its tree checked at 372dcdf: 403,402,400 vectors, 0 wrong values, 0 missing answers
  pull request
xls #5437, #5439, #5440, merged 17 november, the maintainers' fixes to our reports #5419, #5418, #5417
  apfloat: round-to-zero cast on overflow, the sign of zero results in fma, nan results in fma
  #5437 / #5439 / #5440
esp-miner #2045, #2046, merged 5 and 18 november, our pull requests, firmware
  batched share counts kept across rejections / elapsed time kept across short counter polls
  #2045 / #2046


audits / open arithmetic cores against exact arithmetic
  every core run as its authors published it; the vectors, the counts and a one-line reproducer per defect are in the audit record
  audit record and reproducers
  dawsonjon/fpu, double divide     2^29 times the quotient, subnormal band / fix filed
  dawsonjon/fpu, double to float   1 - 2^-53 converts to 0.5 / fix filed
  or1200 fpu, multiply             twice the product, subnormal operand
  opencores fpu                    inf * -inf returns nan
  basejump stl fpu, add            0 + 0 returns the smallest normal number / fix merged upstream, #765
  lowrisc muntjac fpu, fma         the 106-bit product loses its last bit / reported
  neorv32 fpu, float to integer    0.5 + 2^-24 converts to 0 / fix merged upstream
  mor1kx, marocchino fpu           overflow returns inf in the directed rounding modes / reported
  marocchino fpu                   1/0, then 1+1 returns inf / reported
  openhw cvfpu, bfloat16 divide    a quotient below 2^-134 returns zero, no flags / cause and fix filed
  openhw cvfpu, narrowing add      0 + 2^-129 returns 2^127 / reported
  openhw cva5 fpu, divide          0.0 / 2.0 right after another divide never returns / fix merged upstream
  openhw cva5 fpu, square root     inexact roots returned as exact, 29-bit roots / fix merged upstream
  openhw cva5 fpu, maintainer's fix   merged as 125fb2b; its tree checked at 372dcdf, 403,402,400 vectors, 0 wrong values, 0 missing answers / merged upstream
  gaisler noel-v fpu, multiply     3.0 * 5.0 - 15.0 returns 1.65e-24 after a mispredicted branch / cause and fix sent
  gaisler noel-v fpu, fma          16 - (1 + 2^-52)(1 - 2^-52) rounds up to exactly 15 / cause and fix sent
  ieee vhdl float_pkg, sqrt        sqrt(2.0) in double precision is one unit off / cause and patch sent
  ieee vhdl float_pkg, mac         (-6)(-1) + (-5) returns -15 / cause and patch sent
  ieee vhdl float_pkg, resize      the double 2^-512 converts to 1.0 in single / cause and patch sent
  blackparrot, integer to float    a 64-bit integer converted to single is rounded twice / fix merged upstream
  microwatt fpu, enabled underflow a result that rounds up is classed as denormalized / cause and fix filed
  siemens ac_std_float, fma        with ties to odd, a tie under a power of two returns twice the result / cause and fix filed
  siemens ac_std_float, add        1.0 minus a far smaller number is one unit low at four fraction widths / cause and fix filed
  siemens ac_std_float, ceil       ceil(-0.5) returns +0 where the c library returns -0 / cause and fix filed
  siemens ac_std_float, convert    the largest subnormal of binary32 becomes zero in bfloat16's layout / cause and fix filed
  siemens ac_std_float, from int   4294967295 with ties to odd converts to zero / cause and fix filed
  siemens ac_math, fixed divide    0.5 / 3 into an integer type leaves the output unwritten / cause and fix filed
  llvm apfloat, multiply           (1 - 2^-53) * 2^-1022 rounds up to the smallest normal number with no underflow status / reported, patch published
  google xls apfloat, fma          a nan operand returns infinity on the e4m3 and e5m2 layouts / fix merged upstream
  google xls apfloat, fma          (1.0 * -0.0) + -0.0 returns +0 / fix merged upstream
  google xls apfloat, from int     100000 to half precision, rounding toward zero, returns infinity / fix merged upstream
  pulp common_cells, lfsr 16-bit   repeats after 64,897 states where a maximal register runs 65,535 / cause and fix filed
  opentitan prim_fifo_sync, pass with zero-if-empty   write data shows while rvalid is 0 after reset, 20 of 92 configurations / cause and fix filed
  taxi axis fifo, depth under two bus cycles   zeros through the ram in 66 of 192 configurations / cause and fix sent
  adi util_axis_fifo, remove_null_beat_en   a stale word in place of each null beat, 180 of 180 configurations / cause and fix filed
  adi util_axis_fifo_asym, width ratio 3, 5, 6, 7   342 of 896 configurations wrong / cause and fix filed
  rust rustc_apfloat, multiply   llvm's missing underflow status in rust's port / reported
  dendreth, circom-pairing ec add   a first selected key of -g aggregates to g; upstream fix e1b6209 not in the pinned fork / reported
  chisel and opentitan lfsr taps   2,568 entries, every one of maximal period
  t-head e906, c906, c910          0 wrong values in 1.1 million vectors per operation

bench  engineering  colony