Personally, I don't generally see those when I am logged into Facebook via browser; but I know what you are talking about and it is definitely not in regard to how long a person was active at any point in time, ever.
The number next to the name will be the amount of time since the person last made an action regarding the Chat \ Messenger. With that being said, though, sometimes it depends on a device and what exactly the person is doing (e.g; which activities they are participating in via FB).
Sometimes I browser - for whatever reason - will reload the chat time if you get a message and reload the page simultaneously...which becomes really inconvenient when you really are not on chat, but it tell people you are...of course they assume they're being ignored.
More times than not, though, it will be in relation to someone who is mobile, and it will display the last time they made any type of interaction with it! Furthermore, you can also look closely at the area it is included, and you will either see a little globe (indicating a web browser is being used to access it) or a little cell phone (obviously indicating mobile use). But it does have a tendency to be incorrect or update in the background, so if your plan on using this feature to attempt to prove that someone is lying about something or ignoring you at a certain period in time - you may end up looking a bit foolish if you put all your faith into it.
If you'd like, you can attempt to verify that answer by logging in on a phone and onto the web or another phone with a different account (both would have to be friends on FB first to see it) and then stop using it and time it - then go to messenger on the other account and see if the time it shows is accurate in comparison to the time you discontinued use.
Of course, that you be slightly complicated and likely not have any positive pay-off in the end, so it'd be your choice.
Hopefully that answers your question as detailed as possible!!