If you look at this sheet you'll see that the image in cell B3 shows a duck with its head to the left, but the actual image — click on "enlarge" in cell C3 to see it — shows the duck with its head upward. Documentation on the IMAGE function is minimal, with no sign of a parameter to control rotation.
If you look at the image details you'll find that width is the longer side and not the height. That makes me believe the actual orientation of the image itself is what you get inside your cell.
I recommend you create a new image with the desired orientation and try it again.
IMAGE() function does not have the ability to rotate.