OK, here's how I fixed it: There's this screw-on cap that presses the collet into a cone shaped void in the spindle. The cap was screwing all the way and bottoming out on the spindle, and the the collet showed there was plenty more space between it's kerfs to go. So what I did was just insert a small washer between the cap and the collet so that the cap can push the collet further into the cone. Works great, but I'll have the to put that little washer in a prescription bottle and in the case, otherwise I'd never remember it or find it.
Also, I bought a new collet part, just to see if the problem isn't wear on the ID of the collet.
A thanks to Jeff for trying.