I don't know about that particular video, but they're not always made up. There are videos and photos of objects that we know were colored in a certain way (from contemporary descriptions) - flags are very handy in that regard, for example. These can be used to calibrate the algorithm. Of course, a B&W image will still be ambiguous, and the algorithm has to make guesses. But those guesses aren't completely random.
It's the computer trying to copy the color from the training image set onto the video. My point about a guy in a bright red or hot pink suit in the B&W original being represented as sepia/navy in the colorization is still the same. It's impossible to turn B&W image into a color image without guessing what the colors are. The average color of a guy's suit in the colorized copy might be right if it's trained on a realistic dataset, but ultimately it's still guessing and making up information for each individual suit.
