I know how to label shapes in draw.io by simply clicking on the shape and then typing, but how can I go back and easily edit These Labels without first deleting and recreating the entire text?
Usually double-clicking the text portion of the shape lets you edit it. Some of the shapes don't work this well and tend to just give you a handle to reposition the text instead (e.g. the UML Diagram Generalization arrow). If that's the case, right click on the shape and select edit. It should select the text portion for editing.