Unfortunately, the Tumblr Dashboard is rather selective as far as what types of raw code it parses. Generally, it will only parse basic HTML, such as text-formatting tags and hyperlinks. Anything else is ignored.
However, if visitors actually read the post on your blog rather than their Dashboards, they will find that far more—if not all—code is parsed. As such, if you wish to embed a Gist, you might wish to include a note directing readers to use the post's permalink to view the post on your blog.
As far as highlighting source code, the <pre>
HTML tag should serve you well. Again, however, this may not parse on users' Dashboards.
And as a reminder: whenever you are inserting any code into a Tumblr post or page that you want parsed, make sure that the text editor is set to "edit source" or "HMTL" rather than "Rich Text," "Plain Text," or "Markdown."