Benchmark suite
Einsum Benchmark datasets and the purpose of path-planning, slicing, and numerical-execution tests.
On this page
Dataset composition
ArcTN is tested with the public Einsum Benchmark. It contains 168 instances in seven categories: graphical models, tensor-network language models, model counting, quantum computing, random problems, structured problems, and weighted model counting.
| Test set | Instances | Scope |
|---|---|---|
| Full Einsum Benchmark | 168 | Tensor-network contraction problems from different applications |
| Quantum-computing subset (QC32) | 32 | Instances in the quantum-computing category of the full set |
The 32 in QC32 is the number of instances, not the number of qubits. Each complete instance supplies an einsum expression, numerical tensors, reference paths, and other data; see the official field descriptions. The expression and array shapes determine the inputs, output, and size_dict used by ArcTN; see tensor-network input.
Download the data from Zenodo, or load it through the einsum_benchmark Python package as described in the official getting-started guide.
What the tests measure
| Test | Measurements |
|---|---|
| Path planning | Search a path from network structure; record search time and path metrics |
| Slicing | Check slicing results under an intermediate-size limit; record slice count and total work |
| Numerical execution | Load arrays and execute the path; record time and memory usage and verify the result |
Path planning requires only network structure; numerical execution also requires matching arrays. Planning, slicing, and execution can be run consecutively to measure total tensor-network contraction time.
Test conditions
- Record the dataset version, instance names, and selected subset.
- Record parameters and thread count for search tests. For execution tests, also record data type, backend, hardware, and exactly which stages are timed.
target_sizebounds the number of intermediate-tensor elements per slice; it is not an actual process-memory limit.
See path metrics for metric definitions and timing and CPU usage for measurement guidance.