I was assuming that Google spreadsheet exports UTF-8 encoded CSVs. However, this does not seem to be the case — my German Umlauts, etc. (äüöß, etc.) are not recognized, i.e., when opening the file in XLS.
Here is an example file: Link
I question that Google is exporting CSVs as UTF-8. I exported a sheet as csv and ran the following bash command:
file -I "file_name.csv"
and it returns...
file_name.csv: text/plain; charset=us-ascii
Not sure if it matters, but used a Chrome browser for this.