I noticed the other answers describe how to do this to Google Drive, but you said you wanted them downloaded to your (I presume local) file system.
Check the description here for a detailed guide to using Thunderbird for this purpose.
With a simple modification, you can download only the attachments you want (in Step 3, just choose the appropriate folder/label or other subset).
EDIT:
Since writing the above, I have discovered a much simpler & easier to use solution. The tool Mail Attachment Downloader from GearMage does exactly what you requested and also provides a great deal of flexibility. I've started using it myself and I am very pleased with the results. It also works with Yahoo, Hotmail and almost any other mail server that supports POP3 or IMAP.
I am adding the following in case you decide to go with my pre-edit answer, so that all parts of this answer can "stand on their own":
- Download and install Thunderbird
- To Thunderbird add the AttachmentExtractor extension from here
- Once it's been added you can right-click and choose "Extract Selected Attachments to..." from the menu
Want more detailed instructions? You can try that link above.
Personally, I now recommend using the standalone, dedicated tool by GearMage. Good luck.