---
title: "MPI execution"
description: "Load an existing path and slicing set, distribute work across MPI processes, and combine the results."
eyebrow: "Parallel and distributed execution"
---

## Distributing work by slice {#main-path}

`tnmpi` reads a saved path and slicing set and distributes slices among MPI processes. Every slice follows the same path. Processes first accumulate their local results, then MPI Allreduce sums these elementwise so that every process receives the complete result.

Path optimization and slice selection take place before execution, using Light, Heavy, or standalone algorithms. Each MPI process executes its assigned slices along the saved path, performs each slice's matrix computations locally, and retains the complete input tensors and output.

## Saving a path and slicing set {#modes}

Generate a plan file with [arctn_plan](/docs/python-api#plan):

```python
from arctn import arctn_plan

plan = arctn_plan(
    inputs, output, size_dict,
    preset="heavy", target_size=1048576,
)
plan.save("plan.json")
```

The file stores network structure, SSA path, and slicing indices, not array values. MPI uses the final saved path and slices regardless of whether fixed or dynamic mode generated them. An empty slicing set means a single whole-network task.

Alternatively, save standalone-algorithm results with [tnpath](/docs/cli#tnpath) and `--save-path`. Version 2 files contain the network and do not need `--net`; version 1 files require a matching external network JSON.

## Building and running {#run}

Install system MPI, then build the executable; see [installation](/docs/installation#mpi).

```bash
cargo build --release --locked --features mpi --bin tnmpi

mpirun -n 4 target/release/tnmpi \
  --load-path plan.json --data tensors.bin
```

`--dtype` selects the input and contraction data type, defaulting to `f64`:

| `--dtype` | Element representation | Bytes per element |
| --- | --- | --- |
| `f32` | 32-bit real | 4 |
| `f64` | 64-bit real | 8 |
| `complex64` | One `f32` each for real and imaginary parts | 8 |
| `complex128` | One `f64` each for real and imaginary parts | 16 |

`tensors.bin` stores arrays consecutively in input-tensor order, each in row-major little-endian format without a file header. Complex elements store the real part before the imaginary part. Every input must match `--dtype`. `complex128` is a 128-bit complex value in total, not a 128-bit real. For example:

```bash
mpirun -n 4 target/release/tnmpi \
  --load-path plan.json --data tensors-complex128.bin --dtype complex128
```

Every process must access the same data. Without `--data`, identical demonstration arrays of the selected type are generated using `--seed`; this seed does not affect path optimization. The final Allreduce performs elementwise summation using the MPI data type corresponding to the selected type.

After execution, MPI rank 0 prints a JSON summary with process count, slice count, per-slice path metrics, and timing:

- `load_wall_seconds_max`: maximum across processes of the time to read, validate, and broadcast the plan and prepare inputs.
- `execution_wall_seconds_max`: maximum process execution time, including local slice computation and final Allreduce, but excluding input preparation and numerical cross-checking.

Reuse the same plan file and inputs to compare process counts. When there are fewer slices than processes, the extra processes have no slices to compute but still participate in result reduction.

## Checking results {#correctness}

```bash
mpirun -n 2 target/release/tnmpi \
  --load-path plan.json --data tensors.bin --check
```

`--check` compares the result on rank 0 against a single-process sliced contraction. It skips the check when the output exceeds $2^{22}$ elements or the slice count exceeds $2^{12}$, and records the skip in the summary. Floating-point summation order may vary with process count and MPI implementation, so bitwise-identical results are not guaranteed across runs.

`target_size` bounds the element count of an individual intermediate tensor within a slice, not process RSS. Every process must still accommodate inputs, final output, and temporary execution space; adding processes does not automatically reduce these memory requirements proportionally.

## Threads and job settings {#threads-and-time}

Each MPI process runs its slice loop sequentially. Rayon defaults to one thread per process, configurable with `RAYON_NUM_THREADS`. With the `mt` feature enabled, `MATMUL_NUM_THREADS` controls matrix-multiplication threads within each process.

The MPI launcher or job scheduler manages CPU binding and job time limits.