You are correct.
There is NO option to play by album when on the ARTISTS
screen.
The only option one has is to click on Shuffle all
The solution
To play the songs album by album in YouTube Music
While in the artist screen (or any other screen), pick and right click one of the songs that you like and from the pop-up menu choose Go to album
When in the album click on the 3 dots (kebab menu) next to the REMOVE FROM LIBRARY
and from the pop-up make a choice between Play next
and Add to queue
OR
Go to Library --> ALBUMS
Once in the ALBUMS
screen, right click on the album you want.
From the options on the pop menu you can once more choose between Play next
and Add to queue
.
Difference between Play next
and Add to queue