The item isn't taking up storage space, as the storage quota comes off the organization that owns the Shared Drive. It is however displayed in Shared with Me, and that is your concern. So, it is more a problem of organization/clutter. Under usual circumstances, one would be able to remove the item from Shared with Me, but that option is disabled. I confirm that I'm able to reproduce the problem.
The best way forward is to leave feedback to the Google Drive developers about how you don't like this problem. You can do that by clicking on the ? icon on the top right of drive.google.com, then clicking "Send feedback". Alternatively, in the mobile app, tap the menu, then tap "Help and feedback", then "Send feedback".