Yesterday I got an alert from Google (New sign-in alert) that someone used my password. It says a Galaxy Note 2 with an IP 126.96.36.199 signed in.
What can I do about this other than change my password (which I have done)?
Once you have logged back into your account and changed your password, go to this page.
Go through each of the steps, remove any unknown devices, or apps that have access to your account.
You can also check a log of Security Events here.
I would also recommend enabling two factor security on all of your accounts; this will require a login confirmation to be confirmed by a text message token.
You can force the other devices to log out. You can also change your authentication to be a 2 step process. Forcing all other devices to log out will ensure that whoever is using your account will no longer remain logged in (even if the device says keep me logged in). Then changing the login to be a 2 step process will make your account that much more secure. Read through this conversation. The person on this link had the same issue as you. I think the responder to the question in this link says it pretty clearly.
Hope this helps