Why could ethical egoism lead to a form of bigotry?

If we pursue only our self interest it leaves us open to judge other egos with different self interests than our own. Ethical egoism divides the world into two categories; ourselves and everyone else, with our own self interests being superior to others