Change the crossed over of the neural pathways at the optic chiasm the other way round. Instead cross the pathways coming from the nasal hemiretins, cross the ones from the parietal hemiretins!

See also: http://murzieuro.blogspot.com/2010/03/why-most-neural-pathways-are-crossed_24.html