WordPress sure seem to know who I am as I only have to enter my email address & name for my little avatar to appear when commenting on others' blogs, but I was wondering if there is any way that I can locate all comments I've made on all WordPress blogs?
Wordpress uses gravatars just like StackExchange sites source. Gravatar uses a hash of your email to identify you and return your avatar. By providing your email (same one you use here?), that's all Gravatar needs.
Other than trying to do a search on Google or your preferred search engine, I don't think this would be possible.