I took Dilantin for 15 years, and my understanding of the possible Cerebellar Syndrome was from taking elevated doses for a long period of time. I thought that if you were at a relatively low dose and weren't taking other drugs that might boost levels in your system, then the risk for damage to the cerebellum was low.
Of course, like all of these drugs, individual results vary, and anything can happen for any given individual, so its difficult to say. I know when I took Dilantin, I was aware of the risk and brought it up to my doc, and made it a point to look for any apparent damage whenever I had an MRI.
I take Keppra now, and for me, I felt much less dizzy and more stable and clear minded on Dilantin than i do on Keppra. Also, I never had any issues with cerebellum issues show up in any way as result of my Dilantin use. But, that was my experience. So, it really does depend on the individual.
Wish you the best, and let us know what your doc says, but at a minimum it is worth sharing your concerns with your doc and getting him/her to focus on it and give you a well thought out opinion.