I'm not suggesting that 20 random characters is easier to remember, but for average Joe, it might as well be the same. Not only do they have to remember the words, the sequence, and how to spell them. Unfortunately we cannot expect this from users in general - the worst offenders write down a password like "rocket", so there is no hope that they'll try to remember a sequence of random words.
We shouldn't have remember passwords at all IMO. It's creating entropy by remembering things, but the human brain is inheritly bad at remembering exact things. Things like a yubikey is a better idea, plug it in, enter your pincode, and use a key pair to authentication. All the user have to do is keep track of the physical thing and the pincode.
But you can't suggest that remembering 5 random words is harder than 20 random characters?
The goal is "simple" if possible, but "simpler" is still a lot better than "practically impossible".