If an Hero is at max level (now LV100) has a sense to chose the skill "Increase ability level" in the Academy?

Yes. There is actually no limit of 100 on skills. The only limit is that you can only manually assign 100 levels to it. But the actual level can be above 100.

But are there enemy has level over LV100?

You should go for "increase ability level" on those heroes who's abilities have multi-target affect or stacking effects, I try explaining my point:
- AoE damage on a spell hits more targets ==> X increased damage applies on 5 targets ==> X*5
- Lumos's shield gives X resistance to your allies ==> every point is considered *5.
- Serafine's awakening skill grants X magic damage bonus on every spell cast. Giving bonus points there, you basically grant way more bonus as long as the fight goes.

Many heroes benefit on ability level, while others prefer penetration, flat damage or cc-resistance(e.g. Ezio)