I am trying to create a header where the underline completely extends horizontally to the end of the document, but I cannot find out how to do this.
If you're trying to do the same thing I was - extend the underline on the same line after some text - here's the trick:
The fix is to insert a special character. Go to Insert → Special Characters → Format & Whitespace → Whitespace, and select the ninth from the left on the top row: PUNCTUATION SPACE or U+2008. It presents a uniform space for Docs to underline. Then copy it (ctrl+c) and paste it (ctrl-v) where you need it.
Word to the wise, SAVE these instructions. I had been doing this to create headings in my resume for months, but suddenly Docs started messing up the formatting of the text. I removed the extra underlined parts and re-inserted the special characters and now it's working again.