Is this possible?
Nope. The Excel Online App currently does not support importing/opening CSV files.
You'll have to convert it to XLSX first, which you could do using Libre/OpenOffice, or one of many online file converter services.
The following steps worked for me:
open csv file in notepad++
replace all commas with tabs (you may have to enable search mode extended with \t)
Select all and copy to the clipboard.
open a blank excel file online
copy contents of the clipboard.
Excel should properly separate the tabs into separate cells