I set the format in the cell to DD/MM/YY but when I click on that cell, I get a popup calendar with the week starting with Monday. But I want the calendar to start on Sunday. My Google Calendar is set to start on Sunday so I'm not sure why this is happening. See image.
The pop-up calendar marks the first day of the week according to the locale set in your account (not just in an individual spreadsheet).
Try choosing a different account locale at https://myaccount.google.com/language. You will first need to choose a language and then a country/locale.
For instance, the United States has a default calendar that starts on Sunday.
This is not ideal for some. But to my knowledge, it is the only way to set the calendar start day differently.
I've got the opposite problem: In Google calendar I can set the week to start on Monday as I would like. But in the popup calendar in google sheets the week always starts on Sunday. I've tried changing language between and locale but no matter what I do I can't get the popup week to start on Monday.