In my Shared With Me folder in Google Drive I have several files that have been shared with me along with others. If I "remove" that file from my Shared With Me folder will those files still be visible to others that the file may have been shared with? I do know that within Shared With Me, in Folders that have been organized and shared by others, if I remove a file in that folder it is removed from others viewing it as well, other than the owner of the doc. But what about just the files that have not been placed into a Folder organized and shared by the owner, files that have simply been shared to me as well as others and show just as Files withing Shared With Me and not within Folders?
It won't be deleted.
When you remove an entire shared folder in one go (by clicking on the folder, and removing it), you won't remove anything inside it. You can even add an entire shared folder to your My Drive, then later on remove the entire folder with one action, and the collaborators won't lose anything. It simply just won't be in your Drive anymore.
The same occurs with items in Shared with Me. When you remove an item from Shared with Me, you are simply taking it off the list you see yourself, and the item remains the same for everybody else.
You are correct in saying that items removed from within a shared folder do get removed, so please don't do that!
A final point: one used to "add" shared items to one's Drive; this is changing now to a shortcut being created by default, which won't sync with your hard drive, or show the owner or last modified details in the My Drive section. You may get around this by pressing Shift + Z while the shared item is selected, if you want to add the original item to your Drive.