Well, I'm sure Disney did like Second Empire because as he was growing up, it was still very common. During President Grant's administration, Second Empire became the almost default style for federal buildings (not to mention that many commercial buildings were built in the style once it caught on for residential and Govt..), and I'm sure plenty of towns still had them.

As for the Mansion itself, I'm sure the prevalence of Second Empire as "haunted houses" had a lot to do with the decision to build the manor in that style. It's just a no-brainer. As for being based on any particular building, I have my doubts. As I said, many of the features that define Phantom Manor architecturally are just commonalities in the Second Empre/Italianate styles.

Here's a brief sampling of some various homes built in similar fashions to PM:



















And an interesting little article I found regarding the Second Empire's recent popularity with home-owners: [LINK]