Skip to content

[Fix] join_table_handler break insert select into two queries to avoid deadlock#11

Open
maxshine wants to merge 4 commits into
jinzhu:masterfrom
maxshine:maxshine-FixJoinTableHandlerDeadlock
Open

[Fix] join_table_handler break insert select into two queries to avoid deadlock#11
maxshine wants to merge 4 commits into
jinzhu:masterfrom
maxshine:maxshine-FixJoinTableHandlerDeadlock

Conversation

@maxshine

Copy link
Copy Markdown

Make sure these boxes checked before submitting your pull request.

  • [*] Do only one thing
  • [*] No API-breaking changes
  • [*] New code/logic commented & tested

For significant changes like big bug fixes, new features, please open an issue to make an agreement on an implementation design/plan first before starting it.

What did this pull request do?

The join_table_handler uses INSERT INTO TABLE ... NO EXIST (SELECT FROM TABLE) query to do the mapping table checking and insert record in one statement. This query would try to acquire IX (insert intent exclusive lock by insert part) and S (share lock by select part). At concurrent scenario, this would lead to deadlock between transactions.

This PR changes the join_table_handler logic by breaking the statement into two. The first part do the SELECT FOR UPDATE or SELECT for sqlite3 check whether there is the mapping record, as it will hold necessary lock first and avoid another transaction contending.

Below is an example to reproduce deadlock manually using join_table_handler generated SQL:

CREATE TABLE `precede_step_builds_map` (
  `succeed_step_build_id` bigint(20) unsigned NOT NULL COMMENT '后继step_build主键',
  `precede_step_build_id` bigint(20) unsigned NOT NULL COMMENT '前置step_build主键',
  PRIMARY KEY (`succeed_step_build_id`,`precede_step_build_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
Transaction 1
begin;
INSERT INTO `precede_step_builds_map` (`precede_step_build_id`,`succeed_step_build_id`) SELECT 3593195,3593194 FROM DUAL WHERE NOT EXISTS (SELECT * FROM `precede_step_builds_map` WHERE `succeed_step_build_id` = 3593195 AND `precede_step_build_id` = 3593194);

Transaction 2
begin;
INSERT INTO `precede_step_builds_map` (`precede_step_build_id`,`succeed_step_build_id`) SELECT 3593197,3593196 FROM DUAL WHERE NOT EXISTS (SELECT * FROM `precede_step_builds_map` WHERE `succeed_step_build_id` = 3593197 AND `precede_step_build_id` = 3593196);

Transaction 3
begin;
INSERT INTO `precede_step_builds_map` (`precede_step_build_id`,`succeed_step_build_id`) SELECT 3593199,3593198  FROM DUAL WHERE NOT EXISTS (SELECT * FROM `precede_step_builds_map` WHERE `succeed_step_build_id` = 3593199 AND `precede_step_build_id` = 3593198);


A deadlock emerges while transaction 1 commit 

Please kindly help review this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant