Well, I'll analyze by the games I've played, and decide from there.
FE4: Yurius is a demon... when attacking with anyone BUT Yuria with Narga. The fact that a lv. 19 Yuria with Narga can do tons of damage, and a high level promoted Yuria utterly rapes the Loputsu incarnate. This makes him fairly low, personally. I've found Alvis to be much harder than Yurius, and yes, not having the broken-ness that is Narga is one reason.

Alvis has maxed stats, Fala Flame, and tons of avoid. Celice with Tyrfing with most maxed stats will take a long time to bring him down. He would even have better stats than Aless, because the Tyrfing now makes Aless's +10 MDEF from the Mistoltin look weak, considering the other bonuses the Tyrfing gives, and because Alvis has Awareness, making Critical useless. Shanan and whoever is using Holsety; be it Corpul, Arthur, or Sety; are able to double attack; but are unable to do significant damage without insane luck. Faval does have an advantage because of the Ichival's strength boost, as does Althenna, but Alvis still does ungodly amounts of damage. Overall, I found Alvis much harder than Yurius.
Ishtar is worth a mention, but because she doesn't have Awareness, a number of things can happen: Arthur/Sety Criticals with Holsety (Plus the Triangle advantage and insane speed boost), Shanan pulls off Meteor sword, Aless crits with the Mistoltin. That's only the holy weapon users too... Lackhe could pull of Meteor sword with the Hero's Sword, or if Holyn is the father, Moonlight sword. Both are almost instant death, and the same applies for Shakasher. Too many things could go wrong for the poor sage to be a major concern, IMO.
FE5: ... Ledirick died in one hit to Leaf. ... Berdo is a pain to get to, but when he's reached... that thing would die to Marty. next.
FE6: Zephidel is tricky. He has decent stats, and the Exxacus is an awesome weapon. But, he can be taken down easily. Fazilla, Forblaze, Roy, Mulgre, or the Malte can defeat Zephy. Granted it will be a pain, but it will happen.
Idoun is one rounded by Roy and Fa. Aside from that she is difficult, but that is akin to the utter Yuria rapeage.
FE7: Nergal... -__-;
Aureola or Armads destroys this prick. When Renault can OHKO you... as a boss, that's sad.
Fire dragon? Armads + Forblaze/Aureola kills.
I don't mention Luna because that spell destroys everything. See SDA's FE7 speedrun, Athos Luna-crits both Nergal and the Fire Dragon. One rounds, both of them. Next.
FE8: Not any debate. Fodeath dies so quickly, it's not even funny. Myrrh critical, Anyone with a S-rank weapon really, he puts up more of a fight than Idoun, but it's still just sad.
FE9: Now, a challenge. The Black Knight is so luck reliant it's scary. Luna owns Ike, and you need to pray for Aether to activate. Multiple times. Maybe Nihil/Wrath works, but I haven't tried it.
Ashnard is also tough. Seeing as he can 2-round almost everyone makes Fortify a godsend, and because only Ike, the Dragons, and the Laguz Kings/Giffca can hurt him makes Ashnard a tough nut to crack. It is less luck relaint, but Ashnard will still be deadly.
Plus, he revives on HM. And moves. Meaning, you'd better hope your healers/other units are protected, and that Ike has good enough stats. This can be very difficult.
FE10: VERY, VERY tough final bosses. Exculding Sephrain, of course. Ashera is very difficult... using the +10 magic defense tiles seems to be the best plan of action for anyone she targets. (STOP ONE-SHOTTING MY HERON!!!) Besides, the auras are very annoying to take down. Fortify is again god. Ike has to deal the last blow, but then again... you'd be using him anyway.
As for the ...
Very tough. VERY, VERY, tough. The fact that he doesn't go after Kurthnaga or Ena is moot really, because they won't do much damage to him anyway unless you heron abuse. A marksman and anyone with Awareness seems to be the best option here. Marksman attacks from range, while awareness people attack without worrying about the mastery skill. Plus, if you decided to bless a long-range Magic tome, you can rain thunderstorm, meteor, Blizzard, Fenrir, or Purge on the fool. Then again, you can do that with anyone. either way, he's still a very difficult boss, and one of the few FE bosses that needs some planning or strategy to take down without loosing someone.
Overall, I'd say the hardest boss (aside from the Luck-reliant BK duel in FE9) is either Alvis, Ashera, or . I've explained why I think so, and since they are the few bosses that can put up a good fight, it makes the games more interesting.