What can I do about the yellow film around my eyes,around my contact lense?

I,as well as quite a few friends I know,unfortunately sleep everynight in our permeable gas (squishy) contact lenses..I replace them every 2 weeks as directed,but I sleep in them.I(we) have developed "yellow" eyes around the contact lense ring.Any idea on whether taking them out everynight from now on will clear it up?