(SE5 1.20) Game hangs on "Processing after all players turns..."

I'm playing a game on the hardest difficulty level (my game never had this problem on the average difficulty level), and after I've spent about 10 minutes doing a turn, and I "end turn" almost 100% of the time my game will hang (I've let it sit for 20 min just to be sure) on the "Processing after all players turns...", requiring me to kill SE5 via the task manager.

Useful information:
* SE5 1.20, no mods
* I'm playing against 9 other computer players, turn based
* Auto save is on for every turn
* Computer players are on the hardest difficulty level
* One interesting clue; I've now taken to saving my game before I end my turn, and almost always I have this issue. BUT when I load my save and I immediatly "end turn" I NEVER have this problem. Which makes me believe it's something that I do during the course of a normal turn which causes this behavior, as loading the game seems to "fix" the problem long enough for me to end turn.
* This issue never happend when I played on medium difficulty (I'm playing the same map as the same race so those two variables are constant)

Windows XP
Pentium D 3.2ghz
2gb of ram
ATI X800 pro
All of the latest windows updates, DX9.0c, and recent gfx drivers

I would really like for this issue to be fixed, it makes playing SE5 a labor of love. This is honestly the best game in the genre since Stars!2.0 and Ascendancy.