Skip to content

Releases: OldCrow/libstats

v2.0.4 — Unified DCL snapshot pattern

Choose a tag to compare

@OldCrow OldCrow released this 05 Jul 05:45
v2.0.4
74c63ee

What's changed

Refactoring

Extracted the double-checked locking + cache snapshot protocol used in every distribution into a single ThreadSafeCacheManager::withCacheSnapshot(SnapshotFn&&) helper (include/core/distribution_cache.h).

All Pattern A (scalar methods) and Pattern B (batch lambda) DCL blocks across 17 distribution source files are replaced with the unified helper — approximately 200 occurrences eliminated with no API or behavioural change.

No public API changes. Existing code compiles and runs identically.

Validation (M1 NEON + CI)

  • 69/69 tests pass (46 correctness + 22 timing + 1 benchmark)
  • toctou_validator: 0 violations across all 19 distributions (10.6M concurrent reads)
  • simd_verification: NEON speedups intact (PDF 6.5x, LogPDF 8.4x, CDF 3.2x)
  • parallel_correctness_verification: 100% deterministic, 100% thread-safe
  • CI green on macOS (AppleClang/NEON), Linux (GCC/AVX2), Windows (MSVC/AVX-512)

Files changed

  • include/core/distribution_cache.h — added withCacheSnapshot helper
  • 17 distribution source files converted: beta, binomial, cauchy, discrete, exponential, gamma, gaussian, laplace, lognormal, negative_binomial, pareto, poisson, rayleigh, student_t, uniform, von_mises, weibull

Full changelog: https://github.com/OldCrow/libstats/blob/main/CHANGELOG.md

v2.0.3 — Dispatch threshold recalibration

Choose a tag to compare

@OldCrow OldCrow released this 04 Jul 14:37

What's changed

Performance

  • Parallel dispatch threshold recalibration across all four SIMD tiers (kAvx512, kNeon, kAvx2, kAvx): fine-grained pylibstats/benchmarks/scipy_comparison.py --sizes sweeps identified profiler floor artefacts (64 thresholds) and premature dispatch entries. NEVER set where parallel never recovers to SIMD throughput within practical batch sizes.
    • kAvx512: Laplace PDF/LogPDF/CDF raised; Uniform PDF/LogPDF/CDF set to NEVER
    • kNeon: Laplace PDF, Rayleigh PDF, LogNormal PDF raised; Binomial/NegBinomial CDF set to NEVER
    • kAvx2: Uniform CDF NEVER; Gaussian PDF/LogPDF, Exponential PDF/LogPDF/CDF, Laplace LogPDF raised
    • kAvx: re-derived from kAvx2 via /2 or /1.5 scaling
  • AMD Precision Boost 2 / TDP interaction documented: Zen4 frequency steps down from boost clocks to TDP-limited frequency under sustained 100% CPU load (power constraint, not thermal — verified at <=50 C). Gaussian PDF/LogPDF and Exponential PDF thresholds confirmed correct; apparent N=90k cliffs in long benchmark runs are PB2 expiry, not dispatch miscalibration. Documented with cold-machine validation data.

Documentation

  • docs/SIMD_OPTIMIZATION_REFERENCE.md: new sections on trough-at-threshold diagnostic, dispatch vs cache boundary distinction, profiler floor artefacts, NEVER threshold criteria, Zen4 cache hierarchy (L2/L3/DRAM boundaries), cross-architecture accuracy differences (Bessel Tier 1 vs Tier 2 for VonMises), and known structural performance ceilings.
  • docs/SIMD_BENCHMARK_RESULTS.md: v2.0.2 cross-machine results added (Zen4 AVX-512 and Kaby Lake AVX2+FMA) with peak throughput, scipy speedup ratios, accuracy table, and benchmark commands.

Validated

  • 46/46 correctness tests on Zen4 AVX-512, Kaby Lake AVX2+FMA, Mac Mini M1 NEON
  • No accuracy changes

v2.0.2 — AMD Zen4 AVX-512 test threshold calibration

Choose a tag to compare

@OldCrow OldCrow released this 03 Jul 00:54
v2.0.2
3ab190d

Fix

  • ** ests/include/validators.h**: Split the complex-distribution SIMD multiplier in getSIMDValidationThreshold between AMD Zen4 (

v2.0.1 — Poisson entropy + Discrete degenerate distribution fixes

Choose a tag to compare

@OldCrow OldCrow released this 02 Jul 11:35
v2.0.1
2f8d579

What's fixed

PoissonDistribution::getEntropy() returned 0 for λ ∈ (20, 100]

The exact-summation loop exited immediately at k=0 when exp(−λ) < 1e-15 (λ ≳ 34.5), returning 0 instead of the correct value (≈ 3.38 nats for λ=50). The Stirling asymptotic branch threshold is lowered from λ > 100 to λ > 20; approximation error is < 0.001 nats at λ=20.

DiscreteDistribution rejected valid degenerate distributions (a == b)

validateDiscreteParameters and validateParameters used a >= b, incorrectly refusing create(a, a). A degenerate discrete distribution (one outcome with P=1, H=0) is mathematically valid. Guard relaxed to a > b. getKurtosis() returns NaN for n=1 to avoid division by zero in the formula −6(n²+1)/(5(n²−1)).

Validation

46/46 correctness + 22/22 timing tests on Mac Mini M1 NEON. Full CI matrix (macOS AppleClang, Ubuntu GCC-13/14, Ubuntu Clang-17, Windows MSVC) passes.

Full changelog

See CHANGELOG.md for details.

v2.0.0 — Architecture Overhaul

Choose a tag to compare

@OldCrow OldCrow released this 01 Jul 21:03
v2.0.0
bec10ae

v2.0.0 is a major architecture release. See CHANGELOG.md for the full entry and MIGRATION_GUIDE.md for the complete v1.x → v2.0.0 call mapping.

Highlights

  • 19 distributions across 7 families (Geometric, Laplace, Cauchy added 2026-06-28)
  • stats::analysis namespace replaces all in-class statistical methods
  • Result<T> redesigned as std::variantmakeError() never constructs T
  • Span-based SIMD batch API; Strategy-suffix methods removed
  • TOCTOU cache-locking fixed across all 19 distributions
  • Platform baseline raised to macOS 13 Ventura / AppleClang 15+ / GCC 13+ / Clang 17+

Validation: 46/46 correctness on Kaby Lake AVX2+FMA and Mac Mini M1 NEON.

v1.5.3_1 — Final v1.x - Linux/TBB Linkage Hotfix

Choose a tag to compare

@OldCrow OldCrow released this 01 Jul 01:55
v1.5.3_1
803119b

This is a hotfix for v1.5.3, the final v1.x feature release.

Fixed

• Fixed Linux/GCC consumer link failures when libstats detects Intel TBB.
• libstats_static and libstats_shared now link TBB publicly when LIBSTATS_HAS_TBB is enabled.
• Consumers using C++20 parallel STL algorithms through libstats now inherit the required TBB dependency transitively.
• Modern CMake TBB::tbb is preferred; pkg-config libraries remain available as a fallback.
• Removed the broken test-only ${TBB_LIBRARIES} link path. Tests now inherit TBB through libstats_static, matching downstream consumer behavior.

Validation

Validated on macOS with Homebrew TBB forced on:

• LIBSTATS_HAS_TBB=TRUE
• Static and shared library builds completed successfully.
• libstats_shared link command includes /usr/local/lib/libtbb.12.18.dylib.

Notes

No API changes. This release only fixes packaging/linkage behavior for Linux/TBB consumers.

v1.5.3 — Final v1.x Release

Choose a tag to compare

@OldCrow OldCrow released this 20 Jun 02:28
v1.5.3
6976469

v1.5.3 is the last release of the v1.x series. v2.0.0 is in active
development and will introduce breaking API changes. Pin to v1.5.3 if
you need a stable v1.x baseline; migration notes will accompany v2.0.0.

Validated on all four machines: Kaby Lake AVX2+FMA (39/39),
Ivy Bridge AVX (38/38), Mac Mini M1 NEON (39/39),
Asus TUF A16 AVX-512 (39/39).

Bug fixes

  • BF-1: Remove noexcept from UniformDistribution move constructor
    — the specifier contradicted its own @warning comment; a throwing
    mutex acquisition would have called std::terminate
  • BF-2: Fix version constants frozen at 1.2.0 — corrected to 1.5.3
    and tied to the CMakeLists project(VERSION)
  • BF-3: Move type aliases (Gaussian, Normal, …) inside the
    LIBSTATS_FULL_INTERFACE guard — using them without the full interface
    produced confusing incomplete-type errors on first use
  • BF-4: Fix libstats-config.cmake.in exported target names
    (libstats::staticlibstats::libstats_static) — find_package
    compatibility variables were never populated in any prior install

Deprecation sweep

Adds [[deprecated("...")]] compiler-warning attributes to every API
element scheduled for removal in v2.0.0, giving consumers a
compiler-warning cycle before the breaking release:

  • All 48 *WithStrategy explicit batch methods (16 distributions × 3)
  • DistributionBase::getBatchProbabilities/LogProbabilities/ CumulativeProbabilities/Quantiles
  • DistributionBase::getKLDivergence
  • MemoryPool, SmallVector, StackAllocator, simd_vector
    (entire distribution_memory.h)
  • PerformanceDispatcher::Thresholds::refineWithCapabilities

VonMises

  • Add getMedian() — the distribution is symmetric about μ; the method
    was absent from the core contract shared by all other distributions
  • Correct stale "VectorOps has no vector_cos" comments — the 4-step
    SIMD pipeline has been active since v1.4.0
  • Rename *BatchImpl*BatchUnsafeImpl throughout header and .cpp
    to match the naming convention used by all other distributions

CI fixes

  • avx512-compilation.yml: fix silently swallowed compilation failures
    (|| echo|| { …; exit 1; }); correct include path to
    build/include_shim (the previous -I include path never resolved the
    libstats/ prefix, so the check was a no-op on every prior run);
    update compiler matrix from gcc-12/clang-15 to gcc-13/clang-17
    (clang-15 is incompatible with the libstdc++14 headers now on
    ubuntu-latest)
  • ci.yml lint job: clang-format-15/clang-tidy-1517

Documentation

  • AGENTS.md: distribution count corrected (9 → 16); deprecated build
    types annotated; Homebrew LLVM section demoted from 'Recommended' to
    'Legacy'
  • README.md: prominent final-v1.x release banner; version badge updated

v1.5.2 — June 2026 audit remediation and structural refactors

Choose a tag to compare

@OldCrow OldCrow released this 19 Jun 04:24
v1.5.2
fbea3e4

What's Changed

Critical bug fixes

• gammaQ infinite recursion (gammaP/gammaQ mutual delegation, x >= a + 1): replaced 1 - gammaP(a, x) stub with a standalone Legendre continued-fraction expansion (Lentz's algorithm). Gamma, Chi-Squared, and Poisson CDF crashed via std::terminate for any moderate-to-large observation. Added gammaQuantile(a, p) (bisection on gammaP) as a protected static helper on DistributionBase.
• bayesianCredibleInterval ignores credibility_level: the parameter was unused; the function always returned bounds for a hardcoded z = 1.5. Now computes an equal-tailed interval from the exact Gamma posterior quantile via gammaQuantile / post_rate.
• safe_log(+inf) returns DBL_MAX: now returns +inf. Downstream std::isinf() checks were silently broken.
• safe_exp underflow returns MIN_PROBABILITY: now returns 0.0 (correct IEEE 754). Both the early guard and the post-exp check were silently inflating near-zero probabilities, corrupting mixture weights and likelihood sums.

Thread-safety fixes

• recordPerformance data race: min_time_ns/max_time_ns updated with non-atomic load-min-store. Replaced with compare-exchange loops.
• WorkStealingPool destructor drops tasks: set shutdown_ before draining. Abandoned tasks left futures blocking forever. Destructor now calls waitForAll() first.
• GaussianDistribution copy constructor: base-class copy ran without holding other.cache_mutex_. Reordered to default-construct base then copy all fields under lock.
• GammaDistribution copy constructor: unique_lock on source for a read-only operation. Changed to shared_lock.

Numerical quality

• betaI_continued_fraction: FPMIN guard on c fired after aa/c was already computed; near-zero c from a prior iteration could silently produce aa/1e-30 ≈ 1e+30. Guard now fires before the division.
• ConvergenceDetector: vector::erase(begin()) O(n) → deque::pop_front() O(1).
• Newton-Raphson derivative guard: break on near-zero derivative changed to return x. Hard stop; returns current best estimate rather than continuing useless iterations.

Structural / code quality

• LogSpaceOps::initialize() guarded with std::call_once (repeated TU-level calls now idempotent).
• result_of_t consolidated from three duplicate definitions to include/platform/internal/type_traits.h.
• parallelTransform: removed check_finite(static_cast(size)) — size_t is always finite, and the cast lost precision above 2⁵³.
• RecoveryStrategy: removed #undef STRICT/GRACEFUL/ROBUST/ADAPTIVE macro block; converted to enum class.
• WorkStealingPool startup timeout: THREAD_STARTUP_TIMEOUT_MS = 5000 replaces the misuse of MAX_DATA_POINTS_FOR_SW_TEST (a Shapiro-Wilk sample-size bound) as a millisecond duration.
• distribution_common.h: removed dead using std::shared_lock/shared_mutex/unique_lock — all .cpp files already use std:: qualification.
• PerformanceDispatcher::Thresholds distribution-specific fields marked [[deprecated]]; active thresholds are in dispatch_thresholds.h.
• LibDistributionType removed from forward_declarations.h — unused duplicate of detail::DistributionType.
• PerformanceDispatcher::SIMDArchitecture, detectSIMDArchitecture, and createForArchitecture marked [[deprecated]]; both already delegated to SIMDPolicy::Level.

Documentation

• CachedProperty: explicit @warning Not thread-safe Doxygen annotation.
• ThreadSafeCacheManager: dual-flag pattern (cache_valid_ + cacheValidAtomic_) documented with v2.0.0 consolidation rationale.

Validation

39/39 correctness tests pass on Kaby Lake AVX2+FMA. No SIMD performance changes; simd_verification results are identical to v1.5.1 on all machines.

Notes

This is the final v1.x cleanup before v2.0.0. The v2.0.0 plan (feat/v2-architecture) will raise the minimum macOS to 13 Ventura, drop Homebrew LLVM, introduce C++20 concepts, extract stats::analysis, and add noexcept move constructors.

v1.5.1 — Audit remediation, dispatch table expansion

Choose a tag to compare

@OldCrow OldCrow released this 16 Jun 05:56
v1.5.1
53341ff

What's changed

Correctness and maintainability fixes
• Fixed #include → #include in statistical_utilities.h (no standard header )
• shouldUseSIMDBatch() now delegates to SIMDPolicy::shouldUseSIMD() instead of a hardcoded threshold of 32
• Removed dead withCachedParameters template from dispatch_utils.h (no callers; contained an unsafe lock-free window)
• Removed stale commented-out cache access block from distribution_base.h
• Formalized arch::simd compatibility aliases in simd.h — replaced using namespace with explicit declarations; removed "temporary" labels from de-facto permanent API surface
• Added /utf-8 MSVC compile flag to silence C4566 for Unicode literals in tool sources

Dispatch table expanded to all 16 distributions
dispatch_thresholds.h previously had no entries for LogNormal, Pareto, Weibull, Rayleigh, VonMises, Binomial, and NegativeBinomial — those distributions fell through to NEVER. All four architecture tables (kNeon, kAvx, kAvx2, kAvx512) are now fully populated from two Release-mode profiling bundles per architecture.

Test fixes
• test_performance_dispatcher: strategy assertion is now threshold-aware; minimal_latency() thread-count expectation corrected (std::nullopt, not 1)
• test_von_mises_enhanced: tolerances relaxed to 1e-10 to match vector_cos AVX-512 absolute error floor (~6e-11)

simd_verification reporting fix
verifyVectorOp reported absolute diff for all primitive ops, producing a misleading max_diff=3.0e+197 for VectorExp (a sub-ULP result at exp(500)~5e+217). The tool now reports relative error for VectorExp and VectorLog (max_rel=) and absolute error for VectorErf and VectorCos (max_abs=). Pass/fail logic is unchanged.

Validation
All four machines/ISA were validated at v1.5.1. SIMD performance is identical to v1.5.0 — only dispatch table coverage and correctness fixes changed.

Deprecation: v1.5.x is the last series validated on macOS 10.15 Catalina and Ivy Bridge hardware. v2.0.0 will set the minimum to macOS 13 Ventura.

v1.5.0: Native SIMD Transcendentals

Choose a tag to compare

@OldCrow OldCrow released this 15 Jun 03:16
v1.5.0
f6fec44

Native implementations of exp, log, erf, and cos across all four target architectures, replacing scalar delegation stubs and lifting distribution-suite geometric-mean speedups significantly on every machine. All four machines validated at 39/39 correctness and 61/61 simd_verification.

What's new

Phase 1 — AVX2+FMA native exp/log/cos (Kaby Lake)
vector_exp_avx2 and vector_log_avx2 replace AVX-delegation stubs with SLEEF-inspired FMA Horner polynomials (< 1 ULP). vector_cos_avx2 likewise replaces AVX delegation. Primitive speedups: VectorExp 3.4×, VectorLog 1.7×, VectorCos 4.9×.

Phase 2 — High-accuracy vector_erf (all x86 backends)
Replaced Abramowitz & Stegun 7.1.26 (~1.5×10⁻⁷ max error) with a musl libc four-region rational polynomial (< 1 ULP; measured max error ~2.2×10⁻¹⁶). Gaussian CDF SIMD error: 6.97×10⁻⁸ → ~0. Kaby Lake VectorErf: 2.5×.

Phase 3 — NEON native transcendentals (Apple M1)
vector_exp_neon (SLEEF FMA Horner, < 1 ULP) and vector_log_neon (SLEEF atanh series, < 1 ULP) replace scalar loops. vector_erf_neon uses the ARM glibc erf_advsimd algorithm — a 769-entry lookup table (src/neon_erf_data.inc) with a 5-term Taylor correction (~2.29 ULP) — rather than the musl rational polynomial used by all x86 backends. The table eliminates the recursive exp call: 0.9× → 8.0×. Gaussian CDF: 2.2× → 13.9×. M1 distribution geomeans: PDF 5.9×, LogPDF 7.3×, CDF 3.1×.

Phase 4 — AVX-512 native transcendentals (Asus TUF A16, Zen 4)
vector_exp_avx512, vector_log_avx512, and vector_erf_avx512 replace 4-wide AVX delegation with native __m512d 8-wide polynomial implementations. No SVML or MKL dependency. Speedups: VectorExp 5.0×, VectorLog 3.9×, VectorErf 1.3×, VectorCos 8.5×.

Phase 5 — Dispatch threshold recalibration (all four architectures)
All ArchTable entries in dispatch_thresholds.h updated from v1.5.0 profiling bundles. Notable shifts: NEON Gaussian CDF → NEVER (table erf makes VECTORIZED unbeatable at all tested batch sizes); AVX-512 Exponential PDF → NEVER, StudentT PDF/LogPDF → NEVER (8-wide native exp eliminated the throughput bottleneck); AVX2 Gaussian PDF 50000 → 100000.

simd_verification expanded (54 → 61 tests)
VonMises distribution rows and primitive vector op rows (VectorExp/Log/Erf/Cos) added. The single composite wall-clock speedup is replaced by per-op-type geometric means (PDF/LogPDF/CDF) and per-primitive individual rows.

Validation

Machine SIMD Correctness simd_verification PDF geomean LogPDF geomean CDF geomean
Kaby Lake 2017 MBP AVX2+FMA 39/39 61/61 8.0× 9.6× 3.3×
Mac Mini M1 NEON 39/39 61/61 5.9× 7.3× 3.1×
Ivy Bridge 2012 MBP AVX 38/38 ¹ 61/61 5.6× 6.0× 2.6×
Asus TUF A16 (Windows) AVX-512 39/39 61/61 4.8× 5.1× 2.2×

¹ test_work_stealing_pool skipped on Catalina (compiler lacks concepts + ranges).

Deprecation

macOS Catalina (10.15) support is deprecated as of v1.5.0. This is the last release validated on macOS 10.15/Ivy Bridge hardware. v2.0.0 will require macOS 13 Ventura (Apple Clang 14+), consistent with libhmm.

Related

Issue #33: Cross-architecture experiment — glibc-inspired table+Taylor vs SLEEF polynomial for exp and log on NEON, and applicability of the approach for transcendentals on AVX2/AVX-512.