Skip to content

ticdc lag reached more than 10min and ticdc crash when inject pdleader data io hang #9054

Description

@Lily2025

What did you do?

1、run tpcc with threads 10 and warehouse 1000
2、After 10 minutes, simulates the io of pd leader is hang but the pod is still active
fault start time:2023-05-15 12:05:38
3、After 10 minutes, recovery the fault
fault recover time:2023-05-15 12:15:38

What did you expect to see?

changefeed lag is less than 30s

What did you see instead?

1 、changefeed lag reached more than 10min after inject fault
2、 ticdc crash

image

image

image

Versions of the cluster

git hash:1335f98cdbaf77239bbcbc6b61561e4254449ffe

current status of DM cluster (execute query-status <task-name> in dmctl)

No response

Metadata

Metadata

Assignees

Labels

affects-7.5This bug affects the 7.5.x(LTS) versions.area/ticdcIssues or PRs related to TiCDC.severity/majortype/bugThe issue is confirmed as a bug.

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions