Does Google Drive allows to hotlink the images? For example, I can upload images there and link to them on my website? Will it violate their terms and condition?
You can do it and it will not violate the terms and conditions at all.
Just make sure that the "Share Options" of the image is "Anyone who has a link" and "Does not require sign-in".
It will be a violation only if you do not have copyright of the particular image(s) in question.