I think everyone has trouble with their mask when wearing a hood. It's hard to get a good seal around your face and water constantly seeps in.
Next time, try putting your mask on first, then slipping your hood over it. With the mask under the hood, you can get a proper seal without your hood messing it up.
Thanks to the awesome Gabe Scotti, of Kaimana Divers for the tip. If you're ever in Oahu, Hawai'i, be sure to give Gabe a call.