The pH of stomach secretion is low. This means it is acidic. Too much production can corrode the stomach wall. Therefore in such a situation drinking a basic solution with a higher pH neutralizes some of the stomach acidic secretions hence relieving the effect;
Acid + Base - Salt + Water
