Songwriting comes from a variety of instruments, the basis of songs never being dependent on one set medium (in this case, medium being the instrument the song derives from). For me, that is an acoustic guitar.
A natural born drummer, yes, Folds found his percussive ear to be easier to train than his melodic one. But as time passed, and his babysitter at the tender age of 9 taught him some choice chords and melodies, he began to explore the various musical vineyards of his youth.
Growing up is as hard as any, but the deft songwriter found time in his youth to sit down and learn, though ignorant to the theory behind it, the piano and its various uses when starting to write rock music.