---
title: "Benchmark suite"
description: "Einsum Benchmark datasets and the purpose of path-planning, slicing, and numerical-execution tests."
eyebrow: "Benchmark"
---

## Dataset composition {#datasets}

ArcTN is tested with the public [Einsum Benchmark](https://benchmark.einsum.org/). It contains **168 instances in seven categories**: graphical models, tensor-network language models, model counting, quantum computing, random problems, structured problems, and weighted model counting.

| Test set | Instances | Scope |
| --- | --- | --- |
| Full Einsum Benchmark | 168 | Tensor-network contraction problems from different applications |
| Quantum-computing subset (QC32) | 32 | Instances in the quantum-computing category of the full set |

The 32 in QC32 is the number of instances, not the number of qubits. Each complete instance supplies an einsum expression, numerical tensors, reference paths, and other data; see the [official field descriptions](https://benchmark.einsum.org/documentation/guides/instance_data/). The expression and array shapes determine the `inputs`, `output`, and `size_dict` used by ArcTN; see [tensor-network input](/docs/tensor-network).

Download the data from [Zenodo](https://zenodo.org/records/11477304), or load it through the `einsum_benchmark` Python package as described in the [official getting-started guide](https://benchmark.einsum.org/documentation/guides/gettingstarted/).

## What the tests measure {#planning-only}

| Test | Measurements |
| --- | --- |
| Path planning | Search a path from network structure; record search time and path metrics |
| Slicing | Check slicing results under an intermediate-size limit; record slice count and total work |
| Numerical execution | Load arrays and execute the path; record time and memory usage and verify the result |

Path planning requires only network structure; numerical execution also requires matching arrays. Planning, slicing, and execution can be run consecutively to measure total tensor-network contraction time.

## Test conditions {#protocol}

- Record the dataset version, instance names, and selected subset.
- Record parameters and thread count for search tests. For execution tests, also record data type, backend, hardware, and exactly which stages are timed.
- `target_size` bounds the number of intermediate-tensor elements per slice; it is not an actual process-memory limit.

See [path metrics](/docs/path-metrics) for metric definitions and [timing and CPU usage](/docs/performance-boundaries) for measurement guidance.