Soft, non-abrasive cloth or a Q-tip and a very small amount of mild non-abrasive cleaner (possibly SimpleGreen). For less involved cleaning, I'd try a little plain tap water mixed with a small amount of liquid soap (like Dawn).
Whatever you do, don't soak the fretboard - make sure your cloth is just damp enough to remove the dirt.
And, as you probably have figured out, the critical areas are right around the frets because that's where most of the crud collects. I use a damp Q-tip for that and sometimes have to resort to very careful use of a sharp X-acto blade for the really crusty stuff.
If you do a light cleaning with each string change and routinely clean your hands before you play, you should have little or no build-up.
Do let us know how those Optima strings perform - I've been curious enough to want to try them, but the price is a little scary.