Bug - Fixed "modern" chat in Relay Browser doesn't play nice with KoLmafia chat GUI

If you have "modern" (tabbed) native chat open in the Relay Browser at the same time as you have KoLmafia's chat GUI open, at least one thing does not work correctly.

- If you type something in the Relay Browser, KoL echoes it and we see it in bothe the Relay Browser and in the chat GUI - GOOD
- If you type something in the chat GUI, we see it there and then, when KoL serves it to the Relay Browser, we see it there - GOOD - but then we see it again in the chat GUI - BAD.

So, you want to solve that by having mafia cease the first output into the chat GUI when it is typed there? That way when KoL serves it, we will only see it that once. That seems to be the way which would best fit with the way it is supposed to work.

I don't know how I want to solve it yet.

The chat GUI interacts with KoL using the "older" chat protocol. Which is to say, what we we had before "modern" chat came along. (I'm ignoring the "ancient" chat protocol which we do not support at all.) You can have "older" chat open in the Relay Browser at the same time as you have the chat GUI open and nothing is repeated. I see code - which I don't understand yet - to process what "older" chat in the Relay Browser gets back from KoL and feed it over to the chat GUI. We have similar code for feeding back the "modern" chat results - but messages get doubled.

I need to understand what is different.

A very complicated problem.

Another incompatibility:

In the chat GUI, do "/msg PLAYER msg". We submit the sent message, and open up a tab in the GUI with it, but when the Relay Browser asks for new messages, there is no indication that we sent the message. KoL does not send any indication that you have sent a private message.

If the Relay Browser is running older chat, when it asks for new messages, KoLmafia asks the browser and also looks at its chat history and includes anything it has sent since the last time the browser asked for messages.

Revision 14681 attempts to send private messages send via the chat GUI down to the Relay Browser when "modern" chat is open. It's not quite right; something is going on with timing such that it thinks that some such messages have already been send on down. But, this is the large bulk of the code that will be required, so I thought I'd get it in, bugs and all.

I presume this is why I'm getting two entries for each chat comment in relay browser older chat, without chat GUI open. I only see one for my own comments, and the second one turns up after approx. 4 seconds, so they are not always together. Chat logging only includes one entry.

I know you know there are bugs present, but wanted to know you know this one!

I was hoping not to break older chat, but it seems that hope was dashed.
Thanks for the report!

Try revision 14684

That's sorted it.

I'm still pondering the "type a message in the chat GUI and it is echoed a second time when the relay browser shows it" problem. Here are the request sequences.

Message typed in relay browser "modern" chat. It does not show there until the browser asks for new messages and KoL echoes it. When that happens, we pass it off to the chat GUI and down to the browser. It shows once in each place.

Type a message in the chat GUI. KoL echoes it in the response. The chat GUI displays it. When the relay browser asks for new messages, KoL responds with the message - and we pass it off to the chat GUI (where it is echoed for the second time) and down to the browser.

I don't know how to deal with this, yet. Presumably we could look at the message, see that it was a public message sent by us, and match it with the saved queue of sent messages, somehow, and not pass it to the chat GUI. Unfortunately, the "older" chat protocol does not associate any sort of message ID or timestamp with the message, even though the "modern" chat includes both with each message.

I'll have to look at that timestamp. Presumably, we could remember when we sent each message - but, even though the timestamp is in UTC, our system clock is unlikely to be synchronized with KoL's.

Revision 14605 deals with that situation. When "modern" chat in the Relay Browser gets new messages from KoL, we pass them back to the Chat Manager to be recorded and put into the chat GUI. If the message is "public" and was sent by this player, we look at the Sent Messages in the list of History Entries and compare recipient and contents. If a message matches both, skip it.

Given this, I think this is Fixed. If somebody comes up with another issue, please open a new bug report. Thanks!

Bale said:

So, you want to solve that by having mafia cease the first output into the chat GUI when it is typed there? That way when KoL serves it, we will only see it that once. That seems to be the way which would best fit with the way it is supposed to work.

Click to expand...

If you type in the chat GUI, we have to display your message, since we have no way of knowing if you are currently running chat in the relay browser. Yes, we can see chat polls coming from the browser every few seconds, but if you type /exit, that is local to the browser and we have no way of knowing that chat is no longer running.

Hum. That implementation... well, I do understand why you did it like this, since we seemingly have to support two different chat protocols. Devil's advocate (if I'm reading the diff right): if someone says something terse like "yeah" in browser chat, then says "yeah" again some time in the next 25 messages they send, that message wouldn't be echoed in the chat GUI.

I'm not meaning this as being critical since honestly this fix is Good Enough for everyone but the most OCD of developers. I only ask out of Developer Curiosity, since it seems we're throwing away a lot of useful metadata in that "modern" JSON response like timestamps. Do we not retain that metadata because of some backwards-compatibility issues? I haven't thought through the permutations of the cross product of (old/modern) chat and (simultaneous/not-simultaneous) chat GUI usage. Honestly it's all this backwards-compatibility complexity that's kept me from ever touching chat code in the first place.

I assume that KoL normalizes messages internally and every message has a message id and a timestamp. You don't get to specify those things when you submit a message via either lchat or mchat, but when you receive a message via mchat, you can see them. When you receive a message via lchat, you don't.

With the chat GUI and the relay browser running simultaneously, each receives messages in whatever protocol it is communicating in. If the mchat protocol, you see KoL's timestamps. If the the lchat protocol, no.

Now, regarding your comment, when trying to match a message received in the browser via mchat protocol, our concern is to not give it to the GUI if it was originated (and echoed) there. Therefore, we compare only against SentMessage history entries. So, if someone types "yeah" in the chat GUI and then, within 25 messages, types "yeah" in the relay browser which is running mchat, we will think that we originated the message.

I actually bumped the rolling buffer up from 5 messages to 25 messages, which will exacerbate that possibility. Had I left it at 5 and you typed a lot of short messages into the chat GUI before the browser refreshes, it is possible that it will echo a message which has rolled off the list. Is that more or less likely than interspersing short messages in the two chat clients?

I also considered whether it would be useful to switch the chat GUI to use mchat rather than lchat. It would allow us to be completely compatible with the browser running mchat, but, aside from it being a vastly larger effort, we'd have the equivalent can of worms (ball of worms? can of wax? ball of wax?) if the user decided to run lchat in the browser.

The compatibility issue is a huge mess and I thought about this for days before concluding that although mchat provides all that nice metadata, if you run one protocol in the browser and one in the GUI - and we cannot stop users from doing that - we can't make it work perfectly. Either it is impossible, or it is too hard for me - at least in the amount of time I am willing to put into it.

If somebody else wants to work on it, be my guest!

I may shrink the rolling buffer back down to 5 again. Would I prefer to see a bug report because the buffer is too small, or because it is too large?

One other thing to consider: when matching messages, perhaps we could look at only messages that were sent after the browser last polled - except determining that is not exactly easy, either. Right off hand, I don't know how to do it, with the current structure.

Edit: actually, maybe I do. We already deliver private message to the browser (since otherwise it wouldn't know you'd sent them) that were done since it last polled. Let me try that out.

OK, Revision 14686 does that. It also synchronizes on ChatPoller.chatHistoryEntries, since both the chat poller that updates the GUI and RelayRequests form the browser manipulate it, and it was not difficult to get a ConcurrentModificationException during my testing.

Nice! I pride myself on being one of the best rubber ducks in the industry

Veracity said:

The compatibility issue is a huge mess and I thought about this for days before concluding that although mchat provides all that nice metadata, if you run one protocol in the browser and one in the GUI - and we cannot stop users from doing that - we can't make it work perfectly.

Click to expand...

Yes, it's unfortunate we have to support two protocols at once. If my time were free and I had the motivation to do it, I could see hacking the javascript of lchat to submit/request things in JSON format, then rewriting our chat system to properly use all of the metadata. Boy would that be a ̶w̶a̶s̶t̶e̶ ̶o̶f̶ ̶t̶i̶m̶e̶ ̶ fun project.