There is a chrome extension called Tweaks for YouTube that allows this:
n to next chapter and
p to previous chapter
There doesn't appear to be a native keyboard shortcut for chapter navigation as of the writing of this answer. Details:
- The Keyboard shortcuts for YouTube help center article does not list chapter navigation in the list of shortcuts
?while on youtube.com brings up the list of shortcuts, and chapters are not in that list either.