Report failed computations in positive characteristic

[wegank]

In positive characteristic, msolve can fail, print "Computation failed", and still exit with status 0, leaving an empty output file. For example, without genericity handling (-c 0):

msolve -c 0 -f input_files/nonradical-radicalshape-no-square-31.ms -o out

There are two reasons:

1. In positive characteristic, core_msolve always returned 0, even after printing "Computation failed". This is what happens in the example above.
2. If the Gröbner basis has the wrong dimension or degree, msolve_trace_qq returns 4. This can happen when probabilistic linear algebra is unlucky. In characteristic 0, core_msolve restarts the computation in that case. In positive characteristic this status was ignored: nothing was written and the computation was considered done.

Changes

• Fix memory leaks when restarting after a wrong dimension or degree:
  • msolve_trace_qq returned 4 without freeing its data. It now calls free_msolve_trace_qq_initial_data first, as its other return paths do.
  • In characteristic 0, core_msolve restarted without freeing the arrays allocated at the start of each run (bld, blen, bexp, bcf, param). They are now freed before goto restart.
• Return an error when the computation fails in positive characteristic:
  • Status 4 now restarts the computation with the same linear form, as in characteristic 0. With probabilistic linear algebra each restart draws new random multipliers. There is no limit on the number of restarts, the same as in characteristic 0.
  • Any other failure makes core_msolve return 1, so msolve exits with status 1. When the status is positive it also sets mpz_param->dim = -1, so the Julia interface reports the failure instead of exporting a parametrization.
  • The arrays allocated for the run and the lucky primes are freed before returning.

Behaviour change

A computation that fails in positive characteristic now exits with status 1 instead of 0. In the example above the output file is still empty, but msolve now exits with status 1.

The test test/diff/diff_nonradical-radicalshape-no-square-31.sh runs this input with -c 0 and relied on the old exit status. It now expects a nonzero exit status, and still checks that the output file is empty.

Tests

• The example above exits with status 1 instead of 0. Without -c 0, it succeeds and exits with status 0 as before.
• make check passes for this branch on top of master, and also with the linear algebra PR applied first.
• make distcheck passes, also with SEED=1617753600.
• With the probabilistic linear algebra bug fixed in a separate PR still present, -l 42 on input_files/nonradical-radicalshape-31.ms for seeds 1..40 no longer gives an empty output file with exit status 0: each failing run either recovers by restarting or exits with status 1.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com