How is chat replay synchronized on twitch videos?

Question

Recently, twitch.tv videos have added a "chat replay" feature, where you can see the messages that were sent on chat at the time the video was being streamed live.

The stream delay on twitch makes chat quite interesting. Typically most viewers see each other's chat messages in sync with the stream, because most viewers have the same delay. That is, if another viewer sends a chat message about the stream, it's about the same event that you can see on the stream. On the other hand, viewers see the streamer's chat messages some time before the event shows up in the video, while the streamer sees viewers' messages some time after the event really happened.

When you're watching the video playback, are you getting the streamer's point of view (i.e. the streamer's messages are on-time but viewers' messages are about what happened 30 s ago), or a viewer's point of view (the streamer's messages are too early but viewers' messages are on-time)?

Answer 1

Chat replay works from the viewer's point of view, as if the Stream was being replayed live. That's because the "stream delay" is really Twitch's servers processing the video before broadcasting it back out on the streamer's channel page.

You can most clearly see this when watching archives of streamers that include chat in their stream video, such as ProtonJon. Twitch's chat replay is typically 30 seconds ahead of the in-stream chat window.