Fun, nerdy fact: Soap doesn't actually clean, in the traditional sense. Soap molecules combine with water molecules, and this allows the water to get into small pores and places that it cannot get to on its own, but the soap isn't what's actually doing the cleaning. So you're really only running a risk of having more oils on your skin than dirt. If you think of a soap molecule as polar, then one end would attach to water, and the other would attach to oil, but without water, the soap is useless.
And I guess my confession is that I'm a nerd.