GPU
GPU
Also known as: graphics processing unit
A processor built to run many identical calculations at once, used for deep learning training and inference.
Neural network maths is a huge number of multiplications and additions. A GPU runs them in parallel, far faster than a CPU running them in order.
Memory size caps the model size. If the model does not fit, it has to be split, and speed drops sharply.
Renting costs by the hour; owning costs up front. Uneven workloads usually rent first and reconsider once demand is steady.
- MemoryIt caps model size and concurrency.
- CostRent or own turns on how steady demand is.