Google allows me to make the file private but I am not sure that making it private has effect.
I created a public folder. When I attempt to create a new document in that folder, Google Drive tells me:
Do you want to create the element in a shared folder? The created element will have the same sharing settings as the selected folder.
This suggests that I can't have a private document in a shared folder.
But what if I created the document first, then made the folder public?
So I made the folder private again, and created a document in it.
Then I reverted the folder to public. Inspecting the document's sharing settings, it is now
Anyone with the link. Then I changed the document's sharing setting to
Opening an incognito window in Chrome, pasting in the link to the document, gives me the Google login form.
I also created a new document in the folder, without altering its sharing settings. This is viewable in the incognito window.
Visiting the folder in the incognito window gives me a list containing only the public document. When logged in, I see both documents.
So yes, it does seem you can have a private document in a public folder. The folder's setting merely acts as a default for documents in that folder with no explicit setting.