It's not a singular source of gravity at the center though, it's the collection of all mass in the galaxy interacting with each other as well. Like a daisy chain of gravity, which explains why it looks like a spiral instead of an evenly distributed circle.

(I think anyway, I just made it up, I'm not learned in this area, just a HN shitposter)