Spaces on external monitors are swapped

It sounds very annoying indeed.

OS X does not always remember the spaces setups very well. There are a number of threads about this in the forum here.

You may want to check this post Loosing layout after restart (Yosemite) - it might help.

I can also recommend you try my Display Spaces Manager app - you can find it here:

http://downloads.binaryage.com/DisplaySpacesManager-1.0.3.zip

It can record a spaces configuration (meaning which monitor spaces are on), and will try to move the spaces back to that configuration when you re-activate a configuration.

Let me know if it helps at all.