When you rotate the layer camera, you’re doing just that - rotating the layer camera - and not rotating the scene. It just rotates the view of the layers. In this case, you’re rotating what gets sent to the screen by 90 degrees in a clockwise direction. So when you’re moving the red square from right to left on the screen, it’s movement is really rotated by 90 degrees anti-clockwise, and it’s really moving it down the scene.
To understand this better, add a third object above the red one, and see it’s position relative to the red one on the screen. It’ll be to the right. You know it’s really above it, but it appears to the right because of the camera rotation. So when you move right to left on the screen, it’s moving away from the third object. And in the actual unrotated scene, this is down the scene, towards the blue square.