I'd suppose once a year, I'll mask off the fingerboard between the frets (and down the sides of the fingerboard where, essentially you only see the frets poking out of all this blue tape) and I'll go at the frets with fine steelwool. They go right back to really bright really fast. I typically don't develop 'ruts' in the frets, but the ever-so-slight amounts of metal I'm removing would head it off, anyway.
Then I'll pull all the tape, and since the fingerboard is string-less at this point, then, it's lemon oil time.
I've used the Nev'r Dull on the brass bits, never tried the frets. Great stuff, but you have to keep wiping the black residue off the brass bits for a while to get it all off.
Would not be inclined to use Flitz for frets, as I have rotten luck with paste-y compounds finding all kinds of grain pores that until then I never knew were there . . .
I'd bring up something else. Usually as part of the lemon oil routine, I'll run a toothpick under each side of each fret to remove the residue of God-Knows-What that tends to build up there. Usually can't find toothpicks that stand up to ebony and steel, so I use those bamboo skewers they sell at the grocery stores. Much stronger, holds their point way better, I've used them to 'refill' the screw holes on guitars with them and Gorilla Glue to re-start a new screw. Like I said, much stronger.
J o e y