It's finger grease and stuff - the cloudy bits is what you need to clean off, the darker areas are clean areas of wood.
Go to a health food type of store, and ask for a bottle of essential lemon oil. You'll get just a small bottle, but you only need a few drops for the entire fretboard. If you mention you're applying this to wood, they probably think you're a nutter (aren't we all).
So just a few drops to wipe away the cloudy bits. Perhaps a little extra to replenish the wood. Follow Mark's link for some more discussions.