Somebody has to (I believe) "own" the folder even if it was shared.
So if you are the original owner of the folder and then shared it, unfortunately I don't think you can
One solution I had to a similar issue is I signed up for a new Google Apps account (back when they were free - I am sure a regular GMail account will do just fine) and then shared the folders with that user.
I then created a new folder with my dummy GMail account, and created a similar folder structure in that account, and copied all the files there. It was fast and I think it basically didn't actually copy the files.
I then (held my breath....) and deleted the original folder from my normal GMail account and shared the folder with myself from the dummy account.
It's a bit of an odd solution, but I think the reason is Google wants to make sure there is someone whose account has the files and folders associated with it