This idiom or expression means:

Arrogant.
Inflexible.
Condescending.
Looking down on others.

A. Why don’t you like my friend?
B. He always gets on his high horse, no matter what subject we're talking about.
A. Well, that’s easy. Just tell him to get off his high horse!!!

Also, related:
Off one’s high horse.