Skip to content

Channel-delivered Portal append failures: emit metric + compensating retry #349

Description

@LikiosSedo

Context

Follow-up from the review of #347. In src/gateway/internal-api.ts, the
delegation.append_message handler swallows Portal append failures when the
message was already delivered to (or registered on) a channel
:

const deliveredToChannel = await deliverBackgroundChannelMessage(event.message);
const channelRegistered = hasBackgroundChannelDelivery(event.message.sessionId);
try {
  response = { ok: true, id: await appendDelegationMessage(frontendClient, event.message) };
} catch (err) {
  if (!deliveredToChannel && !channelRegistered) throw err;
  console.warn(`[internal-api] Portal append failed for channel background ...`, err);
  response = { ok: true };
}

Non-blocking — this fail-soft is deliberate and correct in priority (channel
users care about the reply, not the historical record; surfacing the reply beats
"reply shows up then disappears on Portal re-render"). Deferred from #347.

Problem

When appendDelegationMessage throws but deliverBackgroundChannelMessage
succeeded, the message lands in Feishu/Lark but never enters Portal's
chat_messages table. We return { ok: true } with only a console.warn. This
is a chat-history divergence / audit gap: a later chat_messages audit can
contradict what a Feishu group remembers, with no signal that it happened.

Proposed

  • Metrics hook: emit a counter (e.g. portal_append_failed_but_channel_delivered)
    so operators can see when the divergence occurs (it should be rare).
  • Compensating write: a single retry-with-jitter in the catch block to
    close the gap for transient DB hiccups before giving up.

Severity

Non-blocking. Pre-existing fail-soft pattern in the codebase. If a
chat_messages audit ever disagrees with a Feishu group's history, this handler
is the first place to look.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions