Reliability

Validation.

A generated sampler is only useful if it is correct. AI4BayesCode validates in two stages. Before any code is written it confirms the model is the one you meant. After, it checks that the sampler compiles, reads correctly, and recovers the right posterior.

No automated check catches every bug. Validation raises the floor, and because AI4BayesCode runs inside a coding agent, you can keep digging after the sampler is generated.

Stage 1

Before generation: get the model right.

Most errors start in the model, not the code. Before generating anything, AI4BayesCode makes the parsed model explicit and asks you to confirm it. The review is always shown, never offered as an optional extra, and nothing is generated until you sign off. Every check that comes later verifies the code against what the system understood your model to be, so this is the one point where a misread model can still be caught.

Prior completion.

Priors you already wrote are used exactly as you wrote them, and the system does not ask about them again. If your description already gives every prior, it asks nothing and goes straight to code. For the priors that are missing it asks one parameter at a time, and it offers clear options. You can take a default weakly-informative prior, ask for a prior built from the published literature, fix the parameter to a known value, or supply your own. For the literature option it searches the literature, shows you the sources it used, and offers a tighter version and a wider one, defaulting to the wider one. If you named a distribution but not its numbers, you can also leave those numbers adjustable so you can change them later without regenerating the code. Nothing is left implicit.

You confirm the model.

You see the model twice. The moment you describe it, the system writes it back as formulas and asks whether it read you correctly, before it asks you anything else. Once the priors are settled it shows the full picture again for a final sign-off, in three views: the model formulas, which are the likelihood and priors it parsed, a diagram of the model showing how each quantity produces the next, and a summary table of every parameter with its prior and the sampler that will handle it. The table is where the sampling plan is proposed, and you can ask for a different sampler on any parameter right there. Whenever you ask for a change, the whole review is redrawn, so you are always confirming something you can actually see. You can also ask to see it again at any point before generation starts. When something is ambiguous, such as two parameterizations of the same distribution, it asks before continuing.

Papers work as input.

You can hand over a paper instead of typing out a model. The system reads it, writes back the model it found, and asks you to confirm before anything is generated. Extraction can misread a paper, so this check matters even more here than it does for a model you typed yourself. If the paper defines several models, it lists them first and you choose which ones to build, and each one becomes its own sampler.

Stage 2

After generation: check the sampler.

Once the sampler is written, it runs a three-layer pipeline, cheapest first. If a layer fails, the system revises the code from the accumulated feedback and tries again. You set the maximum number of attempts before generation starts, and the default is five. When the only problem is that the chains have not settled yet, they are re-run five times longer first, and that longer run does not use up an attempt.

1 · Syntax

The generated C++ has to compile. The compiler does this for free. If it does not build, the system fixes the code and retries.

2 · Semantic

Twenty-six checks for bugs that compile and run but still give a wrong posterior: distribution parameterization, dimension mismatch, Jacobian handling, dependency consistency, update logic, gradient correctness, and more. Most of them are done by reading the generated code. A few go further and compile and run a small test program while the sampler is being generated, including the one that checks hand-written gradients against automatic differentiation.

3 · Runtime

The sampler is run on data simulated under the target model. All three runtime checks run on the same pair of chains, and every draw is kept so the later checks can reuse them. They run in order, and a failure stops the ones after it. First a short smoke test of about ten steps, which catches crashes, non-finite values, prediction that quietly changes the chain state, and prediction that breaks once part of the model is held fixed. Then the real run, 4000 burn-in draws and 4000 kept draws in each of the two chains, with R-hat and ESS to ask whether the sampler converges. Then a posterior check on those same chains, where the Bayesian p-values decide whether it converged to the right posterior. PSIS-LOO is reported next to them as a diagnostic, and it never fails the sampler on its own.

Some models are fitted with variational blocks instead of MCMC chains, in whole or in part. R-hat does not apply to those blocks, so the runtime layer measures how close the fitted approximation is to the true posterior and grades it as pass, caution, or fail. When a model uses both kinds of block, both checks run and both have to pass.

Show the 26 semantic checks
  1. Distribution parameterization. Every distribution is annotated with its exact parameterization and mean.
  2. Parallel vs sequential update. Vector Gibbs updates write each component back before the next one is drawn. Drawing them all at once is allowed only when the components do not depend on each other.
  3. Dead parameters. Every parameter has a block that actually samples it, and nothing is sampled and then left unused.
  4. Missing intercept or offset. The regression mean includes the intercept and any offset.
  5. Jacobian handling. Constrained NUTS parameters get their change-of-variable term from the library wrapper, never from a hand-written line in the model code.
  6. DAG consistency. The prediction graph matches how the code actually computes things, and every value the prediction step reads can be reached from that graph.
  7. Dependency declaration. Each block declares the inputs it reads.
  8. Binding correctness. The R interface is used correctly, covering matrix layout, list handling, and how errors are raised. Two of those rules exist because the same file is also compiled for Python, so one wrapper serves both languages.
  9. Numerical stability. Log-sum-exp, clamping, and overflow guards are in place.
  10. State mutation in prediction. Posterior prediction does not alter the chain’s state.
  11. Joint-NUTS audit. Extra checks when a joint NUTS block is used.
  12. Gradient verification. Hand-written gradients are checked against automatic differentiation.
  13. RNG separation. Sampling and prediction draw from separate random streams.
  14. Bijection sanity. Custom trans-dimensional maps are probed at a set of test points. They have to come back to the value they started from, never go flat or fold over, and have forward and reverse steps that agree with each other.
  15. Library parity. Gibbs and other specialized blocks are tested by drawing many samples and confirming the mean and variance match the distribution the block is supposed to be drawing from.
  16. Plain-language sampling note. Every closed-form Gibbs step carries a short comment saying in plain words why that step is exact for this model. The note is written for whoever reads the file, so it never cites check numbers or file paths.
  17. No ad-hoc Gibbs. No hand-written Gibbs outside the approved set.
  18. Dense-metric justification. A dense NUTS metric is justified and pilot-scaled.
  19. Vectorized gradients. Gradients use BLAS-friendly matrix operations.
  20. No per-step warmup. The sampler never re-warms inside a recording step.
  21. VI block contract. Variational blocks conform to the required interface.
  22. VI optimizer. Variational blocks use the RAABBVI optimizer.
  23. Re-adaptation safety. Every model with a NUTS kernel offers re-tuning. Re-tuning leaves the chain exactly as it found it and draws from its own random stream.
  24. Joint-NUTS pre-flight. Funnel models are written in the non-centered form, which is the version that does not get stuck. Each parameter’s constraint matches the range the model gave it, and the joint density actually reads every parameter it declares.
  25. Trans-dimensional models use RJMCMC. Models where a coefficient can be exactly zero, and models where the number of parameters can change, are sampled with reversible jump instead of a fixed-size sampler. One way of getting this wrong leaves every runtime check looking clean while the exact zeros quietly disappear, which is why it is caught by reading the code.
  26. Freeze controls. The controls that hold part of a model fixed while the rest keeps sampling are present and behave safely. Freezing a name that does not exist, or a block that cannot be frozen, is refused outright, and you are warned if you re-freeze something already frozen or leave a stale derived value behind.

These twenty-six are the checks that come with the library itself. A contributed block brings its own checks as well, and those run in addition to these whenever your sampler uses that block.

The two stages work together: the first makes sure the right model is being fit, and the second makes sure the sampler fits it correctly.

Even so, no validation catches every bug. That is why AI4BayesCode runs inside a coding agent: when the sampler is generated, the conversation does not end. You can keep going and ask the agent to probe edge cases, read through the code with you, and hunt down anything the automated checks missed.