For all my friends in the greater Boston area who are also under a strict boil water order, I recommend adding a couple dashes of table salt to your pot of boiling water. I read this tip somewhere, unfortunately I've lost the source, but it really helps the water taste better.
In addition, this is a great time to develop a love of seltzer water. To make it taste like a beverage you'd want to drink, add a drink mix flavor packet. My low-cal favorites without weird chemical additives are Nature's Splash and Crystal Light Pure Fitness. Be advised that you should add the drink mix to the seltzer water slowly as it has a tendency to fizz a bit. So glad a friend taught me this tip!