I was also highly annoyed in the elite 4, I beat everyone in the elite 4 using just Yveltal who was only around level 55, and the fact that they had only 3 legends bothered me.
There was a few other things about the game that i found annoying, which was the lack of legends. Every other game has a decent amount of legends for you to find and catch, then in X and Y there are only 4, and those 4 were the easiest legends i have ever caught. The other reason is the lack of challengers you can fight after you beat the elite 4, you can re-fight the elite 4 again, but they are only around lv 63, so they are not that useful for training, and I fought almost every trainer in the region, making training a lot more difficult.
Every other game as challenges after you beat the game like having the islands in firered/leafgreen, gold and silver gave you kanto to beat after johto, ruby saphire and emerald had the battle frontier to beat, diamond pearl and platinum had the area in the north with all the new strong trainers you could beat, and black and white had the whole right side of the map that you could explore and fight people in, but X and Y seemed to lack anything that you could do after you beat the game.