MPI execution

Load an existing path and slicing set, distribute work across MPI processes, and combine the results.

Distributing work by slice

tnmpi reads a saved path and slicing set and distributes slices among MPI processes. Every slice follows the same path. Processes first accumulate their local results, then MPI Allreduce sums these elementwise so that every process receives the complete result.

Path optimization and slice selection take place before execution, using Light, Heavy, or standalone algorithms. Each MPI process executes its assigned slices along the saved path, performs each slice's matrix computations locally, and retains the complete input tensors and output.

Saving a path and slicing set

Generate a plan file with arctn_plan:

from arctn import arctn_plan

plan = arctn_plan(
    inputs, output, size_dict,
    preset="heavy", target_size=1048576,
)
plan.save("plan.json")

The file stores network structure, SSA path, and slicing indices, not array values. MPI uses the final saved path and slices regardless of whether fixed or dynamic mode generated them. An empty slicing set means a single whole-network task.

Alternatively, save standalone-algorithm results with tnpath and --save-path. Version 2 files contain the network and do not need --net; version 1 files require a matching external network JSON.

Building and running

Install system MPI, then build the executable; see installation.

cargo build --release --locked --features mpi --bin tnmpi

mpirun -n 4 target/release/tnmpi \
  --load-path plan.json --data tensors.bin

--dtype selects the input and contraction data type, defaulting to f64:

tensors.bin stores arrays consecutively in input-tensor order, each in row-major little-endian format without a file header. Complex elements store the real part before the imaginary part. Every input must match --dtype. complex128 is a 128-bit complex value in total, not a 128-bit real. For example:

mpirun -n 4 target/release/tnmpi \
  --load-path plan.json --data tensors-complex128.bin --dtype complex128

Every process must access the same data. Without --data, identical demonstration arrays of the selected type are generated using --seed; this seed does not affect path optimization. The final Allreduce performs elementwise summation using the MPI data type corresponding to the selected type.

After execution, MPI rank 0 prints a JSON summary with process count, slice count, per-slice path metrics, and timing:

• load_wall_seconds_max: maximum across processes of the time to read, validate, and broadcast the plan and prepare inputs.
• execution_wall_seconds_max: maximum process execution time, including local slice computation and final Allreduce, but excluding input preparation and numerical cross-checking.

Reuse the same plan file and inputs to compare process counts. When there are fewer slices than processes, the extra processes have no slices to compute but still participate in result reduction.

Checking results

mpirun -n 2 target/release/tnmpi \
  --load-path plan.json --data tensors.bin --check

--check compares the result on rank 0 against a single-process sliced contraction. It skips the check when the output exceeds $2^{22}$ elements or the slice count exceeds $2^{12}$, and records the skip in the summary. Floating-point summation order may vary with process count and MPI implementation, so bitwise-identical results are not guaranteed across runs.

target_size bounds the element count of an individual intermediate tensor within a slice, not process RSS. Every process must still accommodate inputs, final output, and temporary execution space; adding processes does not automatically reduce these memory requirements proportionally.

Threads and job settings

Each MPI process runs its slice loop sequentially. Rayon defaults to one thread per process, configurable with RAYON_NUM_THREADS. With the mt feature enabled, MATMUL_NUM_THREADS controls matrix-multiplication threads within each process.

The MPI launcher or job scheduler manages CPU binding and job time limits.