Ran into the same issue a few moments ago and was somewhat surprised to find Excel Online doesn't support this. Was equally surprised to find this StackExchange post returned pretty high atop search engine results for keywords "excel for web import csv" and no mention of Google Sheets here.
Google Sheets (in-browser) supports importing CSV via File → Import and subsequently exporting via File → Download → Microsoft Excel (*.xlsx) once you've finished setting it up the way you want: which you can then open it in Excel Online from there.
Alternatively, you could look over the Add-Ins via Insert → Add-Ins within Excel Online directly and search for something that might work for you. There seems to be one in there called "CSV Import+Export" but I prefer not to install third-party software or plugins from an unrecognized source if I can get the same thing done with software I do recognize and trust (as outlined above).