X-Men 1 was a solid film for what it was at the time; a rare attempt at a serious comic book film. It did it's job well I thought.
X-Men 2 was great again, although there were some questions about Magneto's attempted murder of millions of humans, given his history. He might be a guy about to do everything in his power to protect mutants, no matter what, but he's not the kind of guy to shamelessly murder millions of humans.
X3 was the travesty (I think I've heard Singer slating DOFP as "rewriting some of the former X-Men mistakes" - or something along those lines, and heavily insinuating X3).
http://blogs.coventrytelegraph.net/thegeekfiles/2008/11/x3lit.html shows some of Singer, Dougherty & Harris had planned. Sadly, I think X3 was overweiged by expectation from the fan masses, and couldn't deliver on all the heavy expectations. Not least with Fox's demanding schedule. X3 might have crumbled with Singer at the helm.....or it might have soared. Who knows...
If there's no Superman Returns, then Singer et al stay on with X-Men 3, and with James Marsden not hopping over to Superman either, probably Cyclops in a full role (without dying hopefully, as a massive Cyclops fan).
Bonus: an early script for X1 early in development, before Singer came on board:
http://www.dailyscript.com/scripts/x-men_walker3.html