I've been posting more about this in the C Programming help topic than in this topic.

Anyway, I've decided that I'm going to make the list of tokens be stored in an external text file on the calculator. Therefore, I could also make the dialog box prompt for choice of token file- which would mean this program could easily tokenize in 82, 83, 85, and 86 Basic as well (with some changes on the PC end of things in how the file is created).
That might be even
less useful, since I don't think there are many 82/83/85/86 Basic developers out there anymore, but I still think it's a cool thing to add.