GitHub

Version 2.0

Version 2.2.1

  • fit_traces defaults to samplesteps=40_000 with warmup equal to samplesteps, which converged better in simulated tests. fit is unchanged.
  • fit_traces defaults to raw intensities (zeromedian=false) and estimates noisepriors from the data, because normalized traces with fixed default noise priors biased the fitted rates.

Version 2.2.0

Version 2.1.1

  • Capped EnzymeCore at 0.8.26 to avoid a DiffEqBase extension-loading error on fresh installs (see CHANGELOG.md). No functional changes.

Version 2.1.0

  • Added a quick-start API for steady-state RNA histograms: demo() runs a short fit of the bundled MYC data, fit_rna stages data and fits a G-state model with defaults (halflife in minutes), rna_histogram converts per-cell counts, and describe_fit/fit_summary report rates per minute. See Getting Started. fit is unchanged.

  • Integrated optional transient composition/multiplier solvers, higher-order splitting variants, state-dependent RNA decay, and a layout-checked steady-state block-Thomas solver from the earlier numerical-methods work. Existing solver defaults are unchanged. See Master-equation solvers for distinct boundary conventions and numerical convergence checks.

  • Integrated opt-in joint-residence regularization, non-hierarchical batched likelihoods, and centered exGaussian emissions with compact emission caching. Gaussian emissions and the unregularized objective remain the defaults.

  • Fixed warmup adaptation stopping after its first learned covariance update; warmup can also rescale proposals when covariance estimation is unavailable.

  • Correlation contexts infer built-in emission sizes (including five-parameter exGaussian emissions), while file writers respect saved per-unit noise priors. Mixed emission families and hidden units are supported. Coupled trace simulation wrappers infer the same counts when not supplied explicitly.

  • Consolidated theoretical correlation writers onto one cached observable engine, retaining the independent historical calculation for explicit validate=true checks. Validation is now opt-in rather than repeated by default.

  • Replaced dense per-lag matrix-power storage with propagated observable caches. Model and postprocessing folders use bounded threading through threaded and max_concurrent_jobs.

  • Made trace centering available directly in correlate_observables. Both CSV layouts preserve raw moments and replace only their _centered columns. Earlier generalized trace-centered files could contain centered-plus-mean hybrids in raw columns; regenerate those files before relying on raw moments.

  • Single-file trace-centered output no longer writes an unwanted global companion. File naming uses saved model dimensions, and frame intervals can be inferred from saved trace specifications.

  • Fixed the empirical correlation writer to pass traceinfo[1] to ON, reporter and intensity correlation calculations instead of assuming one minute per frame.

Version 2.0.7

Version 2.0.7 fixes a crash when a hierarchical model has no fitted individual parameters, such as a coupled fit retaining fixed per-trace noise. Summing the empty hierarchical conditional log density now returns zero. Existing rate files and model settings do not need to be changed for this fix.

This version also includes the opt-in likelihood_executor=:batched MH path for supported hierarchical trace models sharing a transition matrix. It batches the forward recursion across traces using two-frame buffers, without changing the statistical model, priors, or transition solver. The default remains :fast. See Parallel and batch processing for configuration and validation details.

Per-chain emission caching and automatic batch-size tuning are separate, unmerged performance work and are not available in 2.0.7.

Version 2.0.6

Version 2.0.6 is a correctness patch for the finite-trace theoretical correlation postprocessor introduced in 2.0.5:

  • write_correlation_functions_centered now raises the documented ArgumentError (with its accompanying @warn) when a correlation file's lag grid does not cover the requested trace window, instead of an unrelated MethodError caused by an invalid backslash line continuation inside the warning macro call.
  • write_correlation_functions_centered_folder, and the folder form of write_correlation_functions_centered, now include existing crosscorrelation-global_* files when scanning a folder, so global-centered inputs are trace-centered instead of silently skipped.
  • Fixed a MethodError in the legacy-schema trace-centering path caused by an incorrectly constructed mean-product lookup table.
  • Unified the legacy and generalized correlation CSV schemas around one rule: cc_ON/cc, ac1_ON/ac_x, etc. always mean the raw, uncentered moment, and a cc_ON_centered/cc_centered, ac1_ON_centered/ac_x_centered, etc. companion column is always present with the centered covariance (mean-product subtraction for base/global-centered files, the full finite-window operator for trace-centered files). The legacy schema previously conflated the two by storing a centered-plus-mean-product hybrid directly in cc_ON for trace-centered output; the generalized schema already followed this rule.

These were implementation bugs in the postprocessing/dispatch logic only; the underlying finite-record centering math and CSV schema are unchanged.

Version 2.0.5

Version 2.0.5 restores one coordinate convention across the legacy hierarchical=(2, fittedindividual, fixedeffects) trace stack. The scientific likelihood now receives only physical rates and noise parameters. Its first hierarchical block is the vector of population means; its second block is the vector of positive population coefficients of variation.

For a positive individual parameter, the hierarchy conditional is log-normal and is parameterized by its arithmetic mean and CV. For a signed parameter, such as a zero-median Gaussian noise mean, the conditional is normal. Every CV is positive even when its corresponding individual parameter is signed. The sampler still operates in unconstrained coordinates, and the individual inverse-transform Jacobian is included exactly once in the posterior density.

The HMM layer now calculates only observation likelihoods. The trace likelihood adds the hierarchy density above that layer, including for grid models, and continues to return data-only pointwise terms for WAIC.

This is a correctness change relative to versions 1.5.0 through 2.0.4, which could compare physical hyperparameters with transformed individual parameters. Old rate and info files remain readable, but continuing one targets the corrected posterior and may move away from the previous result.

The theoretical correlation writers now support trace_center=true with window_lengths; it applies the same per-trace sample-mean subtraction expected from finite empirical records. write_correlation_functions_centered exposes that transformation as standalone CSV/folder postprocessing, so existing theoretical outputs can be transformed without rebuilding the HMM when their lag grid covers the required trace window. Trace-centered CSVs preserve the source headings and are identified only by the filename. The explicit write_correlation_functions_centered_folder frontend processes independent CSV files concurrently when Julia is started with multiple threads.

Version 2.0.4

Version 2.0.4 exposes the ON/OFF dwell-time CDFs that were already calculated internally before deriving the normalized PDFs. write_ONOFFhistograms and write_ONOFFhistograms_key now write ON_CDF and OFF_CDF alongside the existing ON and OFF columns. Simulation output also includes SimON_CDF and SimOFF_CDF.

Key-based and legacy folder generators accept bin_size and maxtime in minutes. Use bin_size=100/60 to match data acquired every 100 seconds. An explicit bins vector remains supported and takes precedence.

Version 2.0.3

Version 2.0.3 brings coupled simulation into parity with the coupled transition matrix used by fitting and likelihood evaluation. The simulator now handles connections to different target transitions independently, reciprocal coupling, hidden three-unit models, and repeated entries in unit_model. Stationary-state indexing works for any number of units, and coupled trace simulation preserves a_grid.

The release also makes the two reporter-state conventions explicit:

  • Rany uses one sentinel source state and multiplies the target transition by 1 + gamma whenever at least one reporter position is occupied.
  • Rsum uses one connection per reporter position. If n positions are occupied, a tied strength gives the factor 1 + n*gamma.

The historical shorthand make_coupling("R5", G, R) expands to Rsum, not Rany. In direct simulator calls, the rate vector contains one strength per expanded connection; repeat a tied fitted gamma for each R-position connection.

Malformed unit-model maps, rate/noise layouts, and coupling sign metadata now raise ArgumentError rather than being silently misinterpreted. These changes can alter coupled simulated trajectories but do not change fitted rate-file formats.

Version 2.0.2

Version 2.0.2 corrects the parameter domain for Rsum coupled models. An Rsum coupling shared by m R positions contributes through 1 + γR₁ + ... + γRₘ. Because those R positions can be occupied simultaneously, the physical lower bound is γ > -1/m, not γ > -1 for each expanded connection independently.

The real-line parameter transform now detects tied R-position connections and uses (-1/m, 0) for inhibitory Rsum couplings and (-1/m, Inf) for free Rsum couplings. This applies to ordinary and hierarchical fits. Default CSV priors are also kept strictly inside the appropriate interval. Rany models and tied couplings between mutually exclusive source states retain the usual lower bound of -1.

Historical Rsum fits are not modified on disk. Before using an older rate file as an initial condition, check every tied group and migrate values for which 1 + mγ <= 0 into the open valid interval. For new CSV workflows, use explicit Rsumk or Ranyk tokens when the distinction matters.

Version 2.0.1

Version 2.0.1 is a correctness patch for coupled-model simulation and correlation analysis.

  • Centered theoretical correlations now subtract the correct stationary means and approach zero at long lag.
  • Coupled simulation now handles multiple active coupling connections to the same target transition correctly.
  • Physical-time equilibration follows any requested event-count warmup and ends at a time cutoff rather than a reaction epoch.
  • Sparse simulated traces retain the requested recording endpoint.
  • simulate_trials(...; nexperiments=N) runs independent experiments in parallel and reports ON-ON correlation intervals across experiments.

These corrections can change coupled simulated traces and theoretical correlation outputs relative to version 2.0.0. Fitted rate-file formats and the public fitting interface are unchanged.

StochasticGene 2.0 is the first stable release of the API tested on the 1.11 beta line. It requires Julia 1.11.

Highlights

  • fit supports Metropolis-Hastings, NUTS, and ADVI through a common keyword interface and run-spec representation.
  • Recursive/shared-parameter fits keep fitted parameters separate from the complete rate vectors consumed by likelihoods, with transition and emission cache groups for repeated HMM computations.
  • Key-based info_<key>.jld2 workflows support reproducible continuation, analysis, and scheduler command generation.
  • RNA workflows support explicit legacy and non-truncated likelihood modes, metadata-aware decay and allele handling, genome-scale makeswarm_genes, and dataframe assembly.
  • Trace analysis includes key-aware writers, generalized state-observable correlations with reusable HMM contexts, and posterior burst prediction.
  • Coupled and shared analyses preserve complete per-unit or per-group rate outputs while fitting only the selected parameter representation.

Migration Notes

  • Use stage_write_run_specs plus makeswarm, or the CSV command-file helpers, for key-based batch jobs. makeswarm_genes remains the high-level RNA gene batch entry point.
  • Prefer trace_specs and dwell_specs over legacy traceinfo and dttype metadata in new scripts.
  • Prefer tuple/vector datatype and modality-keyed datapath for multimodal fits; legacy combined datatype strings remain available for compatibility.
  • The terminal summary now labels the likelihood evaluated at coordinate-wise median rates explicitly. It is not the median of sampled likelihood values.
  • Development-only notebook and documentation packages are no longer runtime dependencies. Documentation dependencies remain isolated in docs/.

Release Checks

The release workflow checks that the package precompiles on Julia 1.11, all exports resolve, the standard test suite passes, public exports have static docstring coverage, and the Documenter manual builds without warnings.