By this logic one should expect the Latin Empire to be recognized as roman successor state, and I don't think I saw even once such a claim. Latin Empire is considered to be a foreign conquering force, in spite of being christian, claiming the titles, ruling the same lands (well, very limited...