Problem
Admins have no way to manually review conversations and rate them themselves. The existing thumbs up/down feedback can only be given by the user who owns the chat, so it doesn't cover an admin QA workflow where a data team reviews other people's conversations after the fact to measure agent accuracy.
Today the only signal available in the admin views is end-user feedback, which is sparse and self-reported. There is no way to build an accuracy score over time from a systematic admin review pass.
Related: #293 (user-initiated "ask data team to review" - this request is the reverse: admin-initiated review of any conversation), #832 (chart of end-user feedback distribution), #963 (surface feedback comments inline in chat replay), #653 (data team review system, closed).
Expected behavior
- From the admin chat replay view, an admin can open any conversation and tag it good / bad, independently of the end user's own thumbs feedback.
- The tag can carry a free-text comment (why it was wrong, what the answer should have been).
- Admin ratings are stored separately from user feedback so the two are not conflated.
- Reviewed / not-reviewed state is visible in the conversation list so admins can work through a review queue.
- Admin ratings aggregate into an accuracy score trackable over time (e.g. % good over reviewed conversations, filterable by time range).
Customer ID
c3625ba9-36e3-43a9-9689-b4670e59ab94
Problem
Admins have no way to manually review conversations and rate them themselves. The existing thumbs up/down feedback can only be given by the user who owns the chat, so it doesn't cover an admin QA workflow where a data team reviews other people's conversations after the fact to measure agent accuracy.
Today the only signal available in the admin views is end-user feedback, which is sparse and self-reported. There is no way to build an accuracy score over time from a systematic admin review pass.
Related: #293 (user-initiated "ask data team to review" - this request is the reverse: admin-initiated review of any conversation), #832 (chart of end-user feedback distribution), #963 (surface feedback comments inline in chat replay), #653 (data team review system, closed).
Expected behavior
Customer ID
c3625ba9-36e3-43a9-9689-b4670e59ab94