They basically stitch together a set of images of the same place such that the result does not contain occlusions. This is done by always selecting the input image with the most frequent color, similar to OP's idea, assuming that occlusions will cause 'outlier' color values. They use a Markov Random Field to cleverly control where the stitching seams should be and use Poisson Matting to create a smoothly stitched result.
They basically stitch together a set of images of the same place such that the result does not contain occlusions. This is done by always selecting the input image with the most frequent color, similar to OP's idea, assuming that occlusions will cause 'outlier' color values. They use a Markov Random Field to cleverly control where the stitching seams should be and use Poisson Matting to create a smoothly stitched result.