Skip to content

Worker Line Execution Counter#4159

Draft
MicahMaphet wants to merge 7 commits intobitpay:masterfrom
MicahMaphet:worker-recorder
Draft

Worker Line Execution Counter#4159
MicahMaphet wants to merge 7 commits intobitpay:masterfrom
MicahMaphet:worker-recorder

Conversation

@MicahMaphet
Copy link
Copy Markdown
Contributor

Description

A tool for finding where code gets stuck by counting executions per a line. I looked at other tools, but they didn't seem to quite do what I wanted. I made this to find out why bitcore-node takes a while to stop, perhaps it could be added to the code base.

Changelog

  • Script that counts executions on the lines for the workers
  • Script that analyzes the log files to find problematic code

Testing Notes

Record the code execution for any of the workers.

$ ./scripts/workerRecorder.js p2p

Analyze the log file:

$ scripts/workerRecordAnalyzer.js --include `pwd`/build/src 
[MODULE_TYPELESS_PACKAGE_JSON] Warning: Module type of file:///home/micah/dev/bitcore/packages/bitcore-node/scripts/workerRecordAnalyzer.js is not specified and it doesn't parse as CommonJS.
Reparsing as ES module because module syntax was detected. This incurs a performance overhead.
To eliminate this warning, add "type": "module" to /home/micah/dev/bitcore/packages/bitcore-node/package.json.
(Use `node --trace-warnings ...` to show where the warning was created)
│
◆  Select a log file to analyze
│  ○ .dockerignore
│  ○ .gitignore
│  ○ .vscode
│  ○ Dockerfile
│  ○ LICENSE
│  ○ README.md
│  ○ bin
│  ○ bitcore-node.service
│  ○ build
│  ○ docs
│  ● lineHits-2026-05-06 15:48:37.923 EDT.log
│  ○ node_modules
│  ○ package-lock.json
│  ○ package.json
│  ○ repair-sample.json
│  ○ scripts
│  ○ src
│  ○ test
│  ○ tsconfig.json
│  ○ tsconfig.prod.json
└

After selecting the log file:

◇  Select a log file to analyze
│  lineHits-2026-05-06 15:48:37.923 EDT.log
│
◇   ──────────────────────────────────────────────────────╮
│                                                         │
│  Analyzing lineHits-2026-05-06 15:48:37.923 EDT.log...  │
│                                                         │
├─────────────────────────────────────────────────────────╯
│
◆
│  ○   12511965 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/elliptic/node_modules/bn.js/lib/bn.js
│  ○    2960493 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/bson/lib/bson/parser/deserializer.js

...

│  ○     144509 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/mongodb/lib/core/topologies/read_preference.js
│  ○     122034 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/mongodb/lib/core/connection/msg.js
│  ○     103266 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/mongodb/lib/db.js
│  ●      93283 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/mongodb/lib/core/wireprotocol/command.js
│  ○      86005 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/mongodb/lib/core/wireprotocol/query.js
│  ○      83819 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/mongodb/lib/operations/execute_operation.js
│  ○      79720 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/safe-stable-stringify/index.js

...

│  ○      16144 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/path-to-regexp/index.js
│  ○      15397 /home/micah/dev/bitcore/packages/bitcore-node/node_modules/mime/mime.js
│  ...
└

Once selected the file will be displayed

◇   ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                                             │
│                                                                                                                                             │
│  ================================================================================                                                           │
│  /home/micah/dev/bitcore/packages/bitcore-node/build/src/models/events.js                                                                   │
│  ================================================================================                                                           │
│     1     1 | "use strict";                                                                                                                 │
│     2     1 | Object.defineProperty(exports, "__esModule", { value: true });                                                                │
│     3     1 | exports.EventStorage = exports.EventModel = void 0;                                                                           │
│     4     1 | const base_1 = require("./base");                                                                                             │
│     5     1 | class EventModel extends base_1.BaseModel {                                                                                   │
│     6     1 |     constructor(storage) {                                                                                                    │
│     7     1 |         super('events', storage);                                                                                             │
│     8     1 |         this.allowedPaging = [];                                                                                              │
│     9     1 |     }                                                                                                                         │
│    10     1 |     async onConnect() {                                                                                                       │
│    11     1 |         await this.collection.createIndex({ type: 1, emitTime: 1 }, { background: true });                                    │
│    12     1 |         await this.collection.createIndex({ emitTime: 1 }, { background: true, expireAfterSeconds: 60 * 5 });                 │
│    13     1 |     }                                                                                                                         │
│    14     1 |     signalBlock(block) {                                                                                                      │
│    15     1 |         return this.collection.insertOne({ payload: block, emitTime: new Date(), type: 'block' });                            │
│    16     1 |     }                                                                                                                         │
│    17     1 |     signalTx(tx) {                                                                                                            │
│    18     1 |         return this.collection.insertOne({ payload: tx, emitTime: new Date(), type: 'tx' });                                  │
│    19     1 |     }                                                                                                                         │
│    20     1 |     signalTxs(txs) {                                                                                                          │
│    21     1 |         this.collection.insertMany(txs.map(tx => ({ payload: tx, emitTime: new Date(), type: 'tx' })));                       │
│    22     1 |     }                                                                                                                         │
│    23     1 |     signalAddressCoin(payload) {                                                                                              │
│    24     1 |         return this.collection.insertOne({ payload, emitTime: new Date(), type: 'coin' });                                    │
│    25     1 |     }                                                                                                                         │
│    26     1 |     signalAddressCoins(coins) {                                                                                               │
│    27     1 |         return this.collection.insertMany(coins.map(coin => ({ payload: coin, emitTime: new Date(), type: 'coin' })));        │
│    28     1 |     }                                                                                                                         │
│    29   201 |     getBlockTail(lastSeen) {                                                                                                  │
│    30   201 |         return this.collection.find({ type: 'block', emitTime: { $gte: lastSeen } }).addCursorFlag('noCursorTimeout', true);  │
│    31   201 |     }                                                                                                                         │
│    32   201 |     getTxTail(lastSeen) {                                                                                                     │
│    33   201 |         return this.collection.find({ type: 'tx', emitTime: { $gte: lastSeen } }).addCursorFlag('noCursorTimeout', true);     │
│    34   201 |     }                                                                                                                         │
│    35   201 |     getCoinTail(lastSeen) {                                                                                                   │
│    36   201 |         return this.collection.find({ type: 'coin', emitTime: { $gte: lastSeen } }).addCursorFlag('noCursorTimeout', true);   │
│    37   201 |     }                                                                                                                         │
│    38     1 | }                                                                                                                             │
│    39     1 | exports.EventModel = EventModel;                                                                                              │
│    40     1 | exports.EventStorage = new EventModel();                                                                                      │
│    41     1 | //# sourceMappingURL=events.js.map                                                                                            │
│                                                                                                                                             │
│                                                                                                                                             │
│                                                                                                                                             │
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
│
◆
│  ● Continue
│  ○ Exit
└

Checklist

  • I have read CONTRIBUTING.md and verified that this PR follows the guidelines and requirements outlined in it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant