I have to agree that crossovers probably deter readers. For one, they are mostly only enjoyable if you know all the fics involved. Secondly, the fandoms all have to get the same amount of attention, which is often where crossovers fail. It's also important to portray each fandom as accurately as possible and at the same time ensure that the story makes sense for someone who is unfamiliar with one or more of said fandoms. Last but not least, the crossover has to make sense. There has to be a logical and satisfying explanation as to why these fandoms overlap.
The more fandoms you use the more difficult you make it for yourself; people have to know them all, or at least find them enjoyable/be willing to read it and you have to juggle a lot of characters.