Some questions that might be worth considering:
1. Suppose you get in an accident, or need surgery for something. Your history says "anxiety" when it's really epilepsy, and that knowledge might have helped the first responders or might affect how the surgeon approaches your care and treatment.
2. What if a family member has a similar problem? Say your cousin begins to experience seizures, and she says "well, my cousin had something like that but it's just anxiety so I'll just get some stress relief and massages and I'll be better" when in fact she has a serious disease with genetic links. If your cousin knew there was a history of epilepsy in the family, perhaps your cousin would seek treatment.
3. What if the symptoms change or increase? You won't know if you're experiencing anxiety or epilepsy, and you won't be able to discuss things truthfully with a doctor.
I do believe that sometimes the words aren't that important. For example, one of my daughter's diagnoses is called one thing by many doctors, and another thing by many others. However, the treatment is the same, and her doctors have explained that the two names are pretty much recognizable and interchangeable. A specialist will realize, no matter what you call it, what the symptoms are, what the treatment is. And all her doctors say "yes, it's often referred to as ___________, or some doctors refer to it as _________, but diagnostically, clinically, and in terms of treatment, they're the exact same condition."
However, anxiety and epilepsy are two different things. Perhaps they have treatments in common, but they aren't the same thing. No doctor would say "well, call it anxiety, call it epilepsy, same thing". And that's where the line should be drawn.