Send a message "User A left the conversation" (no notification needed) when one of the participants leave. This will inform the remaining participant, that the leaving participant won't receive further messages from that conversation.
