Only restore mode when we actually rotated something.
