Memory layout

PyTorch Developer Podcast

Memory layout specifies how the logical multi-dimensional tensor maps its elements onto physical linear memory. Some layouts admit more efficient implementations, e.g., NCHW versus NHWC. Memory layout makes use of striding to allow users to conveniently represent their tensors with different physical layouts without having to explicitly tell every operator what to do.

Further reading.

• Tutorial https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html
• Memory format RFC https://github.com/pytorch/pytorch/issues/19092
• Layout permutation proposal (not implemented) https://github.com/pytorch/pytorch/issues/32078