Tensor and autodiff

Applicable version · ArcQML 0.1.0

Tensor data and metadata

Tensor is the numerical object users manipulate directly. It combines Storage, a version counter, TensorMeta, and AutogradMeta in a shared handle. TensorMeta contains shape, dtype, device, layout, strides, and offset. Public queries include shape, ndim, numel, dtype, device, layout, is_contiguous, strides, and offset.

Tensor::clone() copies only the handle, sharing the underlying Storage, version counter, and autograd metadata. deep_clone() copies values, Tensor metadata, and existing gradients and preserves requires_grad, but creates a new leaf Tensor with independent storage and version counter, without inheriting the original graph. It is not a differentiable copy operator. storage_mut() acquires a write lock and increments the shared version counter when its guard is released. This counter detects in-place changes to forward data before backpropagation.

Autograd graphs and gradient lifetimes

Only leaf Tensor objects can enable requires_grad; differentiable leaf dtype values are limited to F32, F64, and C64. Differentiable operations record their graph when gradient recording is enabled in the current thread and at least one parent requires gradients. Only leaf gradients persist by default; call retain_grad() explicitly to keep gradients on intermediate Tensor objects.

backward() automatically supplies upstream gradient 1 only for scalar outputs with shape=[]. Nonscalar outputs require backward_with_grad(gradient), with a gradient compatible in shape, dtype, and other requirements. Default backward frees the traversed graph nodes. To reuse the graph, use backward_with_grad_retain_graph from the first backward pass; this cannot be repaired after freeing it. retain_grad() retains intermediate gradients, not the graph. Gradients accumulate, so training iterations must explicitly call zero_grad. Prefer combining losses sharing a forward pass into one scalar before backpropagation.

Parameter semantics

Parameter is a semantic wrapper around Tensor and the main numerical object for trainable parameters. Parameter::new requires a leaf Tensor with dtype F32, F64, or C64, sets requires_grad to true, and defaults to trainable to true. Invalid inputs panic; use Parameter::try_new for recoverable errors. Circuit parameter registration requires numel=1 and F32/F64; use scalar parameters with shape=[] for training. Quantum gate-angle training does not accept C64 parameters.

freeze() sets trainable to false, causing optimizers to skip the parameter. It does not disable requires_grad on the underlying Tensor, which may still receive gradients. unfreeze() restores trainable=true and enables gradient recording on the underlying Tensor. Thus requires_grad controls backpropagation participation, while trainable controls optimizer updates.