I 100% agree with you. However, I think the fundamental reason that user needs are being ignored is because product managers are given way too much authority, and design does not get any respect at most companies. This 'respect' is gained seemingly through providing short-term gains for investors.

The inherent structure of product teams is dysfunctional, and even at more design-mature companies the PM still has the final say. I am not sure how this can be fixed to be honest. If any of you have any ideas I'd love to hear them.