LinkedIn Sales Navigator connector via PhantomBuster#211
Conversation
…raper via PhantomBuster Two-layer implementation: generic PhantomBuster API client (launch/poll/fetch) in pkg/enum/phantombuster/ and Sales Nav-specific CSV parsing in linkedin.go. CLI wired under `enum passive linkedin` following the Apollo/Hunter pattern. Result type includes verification-ready fields (Sources, VerificationStatus, Confidence) per 10T-373 so confirmation oracles bolt on with no migration. Implements 10T-436. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CSV columns (department, industry, seniority) are mapped first; when absent, keyword matching against the title and headline provides best-effort inference for downstream enrichment workflows. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tment mapping
The v1 output endpoint wraps responses in {"status":"success","data":{...}}.
Parsing the top-level JSON directly left containerStatus empty, causing the
poll loop to exit immediately on the first poll. Now unwraps the data envelope
before parsing OutputStatus.
Also removes the industry CSV column from the department mapping — it contains
the company's industry (e.g. "Production et distribution de médias"), not the
person's department. Department inference from title/headline handles this
correctly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Codex Review
Critical Issues
pkg/enum/phantombuster/client.go:223silently truncates S3 result files at 10 MB withio.LimitReaderand never checks whether more data remained. Large Sales Navigator exports can produce incomplete rosters, and if truncation lands on a row boundaryParseSalesNavCSVcan return success with missing profiles instead of surfacing data loss.cmd/brutus/cmd_enum_linkedin.go:62advertises--result-file result.json, butrunEnumLinkedinalways sends downloaded data toParseSalesNavCSVatcmd/brutus/cmd_enum_linkedin.go:149. Any JSON result file will fail or be misparsed, so the documented path is broken.
Security
No security concerns flagged.
Suggestions
pkg/enum/phantombuster/linkedin.go:66leavescsv.Reader.FieldsPerRecordat the default, so ragged rows with omitted trailing empty fields error out beforegetFieldcan apply the intended “missing columns map to empty strings” behavior. ConsiderFieldsPerRecord = -1plus a test for rows shorter than the header.
Targeted tests were not run because the sandbox is read-only and Go could not create its module cache.
Reviewed by Codex (gpt-5.5)
There was a problem hiding this comment.
Claude Review
Critical issues
PollUntilDone(pkg/enum/phantombuster/client.go:726) treats anycontainerStatus != "running"as finished. Right afterLaunch, PhantomBuster's output endpoint commonly reports a transient/empty status (e.g.starting, queued, or missingcontainerStatus) before the container flips torunning. The first poll can then exit immediately withexitCode == 0, andRunAndFetchproceeds to download a stale/absentresult.csv. Recommend terminating only onstatus.ContainerStatus == StatusNotRunningand continuing to poll on any other value.Profile.Confidenceis populated as a verification-ready field but is dropped from the JSONL output structlinkedinJSON(cmd_enum_linkedin_output.go:274). If downstream (10T-373) expects it on the wire, add it; otherwise it is silently lost.
Security
- API key handling looks correct: sent only via the
X-Phantombuster-Key-1header through thedoJSONchoke point, never logged, and scrubbed from classified errors.--api-keyeven warns about process-list exposure. - S3 download URL is built from a fixed
s3BaseURLconst plus API-returned folder paths (not user input) and bounded bymaxResponseBytes(10MB) — no SSRF/unbounded-read concern.
Test coverage
DownloadResultandRunAndFetchare never actually exercised —TestDownloadResult_Successbypasses them becauses3BaseURLis a const it cannot override, so it hits the test server directly. Makings3BaseURLaClientfield (likebaseURL) would let the real S3 path (URL construction, non-200 handling, byte limit) be tested.
There was a problem hiding this comment.
Gemini Review
Critical Issues
- Logic Error in
inferDeptSeniorityRole Inference:strings.Containsis used without word boundaries to match short acronyms inseniorityKeywords. Consequently, any title containing "Director" will match the"cto"keyword (dire**cto**r) and be misclassified asc-suite. Similarly, "Coordinator" matches"coo", and "Production" matches"cto"or"cro". A word-boundary-aware search (like regex) or space-padding is required to prevent widespread misclassification. - Premature Polling Exit on Unexpected API Responses: In
PollUntilDone, if the PhantomBuster API returns a 200 OK with an unexpected payload structure (such as a missingcontainerStatusfield or an unhandled{ "status": "error" }envelope),status.ContainerStatusevaluates to"". The loop checksstatus.ContainerStatus != StatusRunning("" != "running"), which is true, causing it to falsely return success and attempt to download results before the job is finished. - Silent Truncation of Large Result CSVs: In
DownloadResult,io.ReadAll(io.LimitReader(resp.Body, maxResponseBytes))is used. If an S3 result file exceeds the 10MB limit,LimitReaderreturns anEOFwhichio.ReadAlltreats as a successful read. The function will silently return a truncated byte slice with anilerror, leading to malformed CSV parsing and silently dropped data. You must explicitly verify if the limit was reached (e.g., checking byte length) and return an error.
Security
No security concerns flagged.
Suggestions
- Clear Terminal Line on Progress: When logging polling progress updates with
\rinrunEnumLinkedin, trailing characters from longer, previous status labels will remain visible on the terminal. Consider appending the ANSI escape code\033[K(clear to end of line) to the format string to ensure the line is fully overwritten. - Handle UTF-8 BOM in CSVs: If PhantomBuster exports CSVs with a UTF-8 Byte Order Mark (
\xef\xbb\xbf), the first header column will silently fail to map incolIndexbecausestrings.TrimSpaceignores it. Consider explicitly stripping the BOM from thedataslice before passing it tocsv.NewReader. - Safely Encode URLs with Spaces: In
DownloadResult,fmt.Sprintfis used to construct the S3 URL. If the user-provided--result-filecontains spaces,http.NewRequestWithContextwill fail. Consider usingurl.JoinPath(s3BaseURL, info.OrgS3Folder, info.S3Folder, filename)which automatically handles URL encoding for the path segments.
Reviewed by Gemini (gemini-3.1-pro-preview)
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 85ab44c9a3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| FullName: getField(record, colIndex, "fullname"), | ||
| Title: coalesce(getField(record, colIndex, "jobtitle"), getField(record, colIndex, "job"), getField(record, colIndex, "title")), | ||
| Company: coalesce(getField(record, colIndex, "companyname"), getField(record, colIndex, "company")), | ||
| CompanyURL: coalesce(getField(record, colIndex, "companylinkedinurl"), getField(record, colIndex, "companyurl")), |
There was a problem hiding this comment.
When this command is pointed at the documented Sales Navigator Profile Scraper output, the main fields come back blank: PhantomBuster lists name, currentJob, currentCompanyUrl, and currentCompanyName as output columns (docs), but this mapping only accepts fullName, jobTitle/job/title, and companyName/company. The default result.csv from that scraper therefore loses the name/title/company fields that the human/JSON roster is supposed to emit.
Useful? React with 👍 / 👎.
| } | ||
| } | ||
|
|
||
| result, err := pb.ParseSalesNavCSV(resultData) |
There was a problem hiding this comment.
The help explicitly suggests --result-file result.json, and PhantomBuster documents both CSV and JSON result files (docs), but after downloading any file this line always runs the CSV parser. For a JSON result file, the data is parsed as CSV headers/rows (often yielding zero profiles or a parse error), so users selecting the documented JSON export won't get their profiles.
Useful? React with 👍 / 👎.
| return nil, fmt.Errorf("S3 download failed (HTTP %d)", resp.StatusCode) | ||
| } | ||
|
|
||
| data, err := io.ReadAll(io.LimitReader(resp.Body, maxResponseBytes)) |
There was a problem hiding this comment.
When the S3 result is larger than 10 MiB, io.LimitReader just stops at the cap and io.ReadAll still returns nil error. If the cut happens at a CSV row boundary, the parser will emit a plausible but incomplete roster, silently dropping all later profiles; read maxResponseBytes+1 or stream with an explicit overflow error instead of returning truncated data.
Useful? React with 👍 / 👎.
UNC1739
force-pushed
the
feat/enum-passive-linkedin
branch
from
85ab44c to
501fc47
Compare
1 participant
Summary
enum passive linkedincommand — launches a pre-configured PhantomBuster Sales Navigator scraper, polls until completion, downloads the CSV from S3, and parses profiles into structured outputpkg/enum/phantombuster/package: generic PB API client (launch, poll with exponential backoff, fetch from S3) + Sales Nav CSV parser with verification-ready fields (10T-373 compatible)type: "linkedin"for downstream pipeline consumptionTest plan
--skip-launchfetches results from a previous run without re-launching the phantom--jsonoutput pipes cleanly into downstream tooling (enum generate)🤖 Generated with Claude Code