Color is not an absolute, it is always experienced in relative terms. Your eyes adjust to make sense of what you are seeing through a process called simultaneous contrast.
This is why after a few minutes on the slopes in your 80s themed ski gear, your eyes adjust to your amber Vuarnets and colors look more or less normal… until you take them off for a wee apres ski hot drink and suddenly the whole world seems purple-ish ultramarine.
Josef Albers, your favorite painter of squares has you covered with a full explanation.
Listen to Brainpickings talk you through it