The gradient is stored, not the points on the gradient