Okay, look at this somewhat stylized image. Take a look at the frets, and at the oblong shape of darker wood that seems to lie between them (look CLOSE; the distinction is somewhat subtle). These darker circles actually represent CLEAN wood. This is the wood that your constant fingering and playing has managed to wipe clean. The lighter rings closer to the frets actually represent the grungy, oily, and dirty areas on your fretboard. These are the areas that are somewhat "protected" by the strings, and are unable to be wiped clean by your fingers as you play (and, most likely, the areas that are protected by the strings when you try to wipe off your strings, frets, or guitar body).
Does this surprise you?? Take a quick look at your own instrument. If you don't have a rosewood or dark-stained fretboard handy, then these suggestions may be less obvious to you (at least as described HERE: I've NEVER had to identify an accumulation of grime on a light-colored neck). But even on the OTHER necks, there SHOULD be SOME indication when the neck is dirty (Any input from the WWW world beyond???). If you don't see the discoloration I'm talking about right away, then you are either a collector who doesn't actually PLAY his/her axe, or your fingers don't actually excrete fretboard-clogging oils that tend to clog the frets. If this is the case, then you are extraordinarily lucky (from a strictly guitar point of view)(
).
Don't believe me? Ask a friend to show you the fretboard of another guitar that has been subjected to intermittent or consistent playing. I think you'll find similar patterns of grime and wear on THEM.