It is even wrong about things

  > A validation set is one you consult repeatedly while building the model — to compare candidates, tune hyperparameters, and decide what to try next.
The validation can be done like that but it is better to do a cross-fold validation or selecting a random subset. These are equally valid. Keeping the validation like a hold-out set can end up creating biases if the statistics don't match. Honestly, this is true about the test set too. Though I've found that most people really don't think about this much. It's mundane, boring, hard to do, but deceptively important.