I'm so glad you asked. It's BONE!
There sure are lots of nut materials out there: including regular plastic, Micarta, and Tusq. It gets confusing, so my tech support phone starts ringing: “Hey Stewmac! What nut material do you guys prefer?!?”
For guitar, bass, mandolin or banjo, I recommend bone.
That's because a well made bone nut gives bright, resonant tone that really sounds alive.
Don’t take my word for it, you can prove it yourself.
Try your own comparison: play a guitar with a plastic nut long enough to get to know its tone and sustain. (If you’re feeling scientific, record the guitar’s sound to compare later.) Now install a bone nut and listen again. The bone brings your guitar to life with a better attack and sustain.
Shooz Matthews says it's so.
Lots of top luthiers won’t use anything but bone for nuts and saddles. Shooz Matthews, whose clients include Steve Howe and the tone-finicky Dwight Yoakam, has tried every material out there, and now uses bone on all the high end instruments that come across his bench.
Stealing from the dog!?
Years ago, Dan Erlewine would search his back yard for old steak bones he’d thrown out for his pal Kurt. After bleaching in the sun for a season or two, these pieces were ready to be shaped and installed. The nut on this guitar was made from one of Kurt’s old dog bones!
White bone is bleached for appearance, and looks like what you’d expect to see on a new guitar. But the bleaching process does more than lighten the bone: it slightly softens it, too. This means that bleached bone isn't quite as dense as vintage, which gives vintage the edge on tone.
Vintage bone hasn’t been bleached, so it has a nice old-looking vibe to it. It also has a slightly brighter tone than the bleached, because it's denser. This is something to think about when determining which type of bone to use. I prefer the unbleached.
“What about nuts for use with tremolos?”
The answer’s still bone. Even on tremolo-equipped guitars, a properly slotted bone nut outperforms the specialty materials. After final polishing with abrasive cord, the slots have a nice hard surface for the strings to ride on. The result is a great sounding, hang-up free nut with no glitches to cause tuning issues.
Here I’m playing surf style with a whammy bar and a bone nut. It works great, even though my Ohio surf came from a magic marker.
By now it’s clear that I can’t say enough about good ‘ol bone. Here’s an online guide to making a bone nut: stewmac.com/nuts. And a great source of in-depth information is Dan Erlewine’s DVD, Nut Making, Step By Step.