The internal plumbing of your house may be the culprit if discolored water appears only for a minute or two after you turn on the tap. Many houses have galvanized iron pipe, and when the zinc coating on the inside of galvanized iron pipe wears thin, the water becomes discolored as it comes in contact with bare iron. The longer the water sits in the pipes, the worse the discoloration will be. That is why this problem is most noticeable the first time you turn on the tap in the morning. If only a few taps are affected, only a portion of your internal plumbing has galvanized pipe.
After running your tap for a few minutes, clean water from your water heater or the water main will replace the discolored water. Since iron is an essential nutrient, this condition poses no health hazard. If the discoloration bothers you, however, flush the tap until the water becomes clear and save the water for iron-loving plants.