Enhance deep username research and real-time scan timing by DkHrR · Pull Request #30 · s41r4j/phomber

DkHrR · 2026-03-30T16:40:36Z

Summary

add deep username research pipeline with public profile enrichment fields
add live scan timing updates with elapsed and estimated remaining time
keep final timing summary with variance at scan completion
improve scanner reliability and defaults for username identity depth/timeouts

Included changes

phomber.py
- deep username identity extraction and aggregation
- cross-platform profile/source probing
- real-time timer ticker (elapsed + est remaining) for all scanners
- final timing summary line for each scanner
README.md
- document timer behavior and username deep research tuning env vars
.gitignore and requirements.txt
- add missing project hygiene/dependency files
pyproject.toml
- dependency/script cleanup aligned with runtime usage

Validation

syntax check passes for updated files
live terminal verification confirms real-time countdown updates for username and ip scans

Copilot

Pull request overview

This PR extends the PH0MBER CLI scanners by adding a deep username research/enrichment pipeline and introducing real-time scan timing output (elapsed + estimated remaining), alongside packaging/docs hygiene updates.

Changes:

Added a deep username research pipeline using a Sherlock-style site manifest, profile probing, and best-effort public identity extraction.
Introduced a live scan timing ticker for scanners plus a final timing/variance summary on completion.
Added/updated project packaging and documentation artifacts (requirements.txt, .gitignore, pyproject.toml, README updates).

Reviewed changes

Copilot reviewed 3 out of 5 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
`phomber.py`	Adds live timing ticker, environment-tunable username enrichment, improved validations, and some reliability tweaks.
`README.md`	Updates install instructions and documents new timer/username behavior (partially).
`requirements.txt`	Adds a pip requirements list for running from source.
`pyproject.toml`	Adjusts runtime dependencies and console script entry point.
`.gitignore`	Adds standard Python/project ignores and history/config exclusions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-30T16:46:20Z

 [project.scripts]
-phomber = "phomber.phomber:main"
-
-[project.entry-points."phomber"]
-phomber = "phomber.phomber:main"
+phomber = "phomber:main"


The console script entry point references phomber:main, but the project does not declare any packages/py-modules for setuptools to include. Add explicit setuptools configuration (e.g., py-modules = ["phomber"] or a package directory) so pip install . actually installs an importable phomber module for this entry point.

Copilot · 2026-03-30T16:46:21Z

+- all scanners now display estimated and real-time research duration
+- `username` scan now performs deep profile research across direct platform profiles and best-effort public identity extraction
+- tune depth via `PHOMBER_USERNAME_IDENTITY_SOURCES` (default: `20`)
+- tune per-profile fetch timeout via `PHOMBER_PROFILE_FETCH_TIMEOUT` (default: `8` seconds)


README mentions only PHOMBER_USERNAME_IDENTITY_SOURCES and PHOMBER_PROFILE_FETCH_TIMEOUT, but the code also introduces other tuning env vars (PHOMBER_USERNAME_SITE_LIMIT, PHOMBER_SHERLOCK_DATA_URL, PHOMBER_IPINFO_TOKEN). Document these or remove/support defaults consistently so users can understand/override the new behavior.

Suggested change

- tune per-profile fetch timeout via `PHOMBER_PROFILE_FETCH_TIMEOUT` (default: `8` seconds)

- tune per-profile fetch timeout via `PHOMBER_PROFILE_FETCH_TIMEOUT` (default: `8` seconds)

- tune maximum number of sites checked in `username` scan via `PHOMBER_USERNAME_SITE_LIMIT`

- override the Sherlock sites data source used by `username` scan via `PHOMBER_SHERLOCK_DATA_URL`

- provide an IP geolocation token via `PHOMBER_IPINFO_TOKEN` to enrich IP-related lookup results

Copilot · 2026-03-30T16:46:21Z

+        return complete_scan_timing('number', started_at, result)

    # Information gathering about the number
    printit(f'    [+] Grathering information about \33[1;49;96m{phone_number}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)


Typo in user-facing output: “Grathering” should be “Gathering”.

Suggested change

printit(f' [+] Grathering information about \33[1;49;96m{phone_number}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)

printit(f' [+] Gathering information about \33[1;49;96m{phone_number}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)

Copilot · 2026-03-30T16:46:21Z

+        return finalize_scan(False)

    # Information gathering about the ip address
    printit(f'    [+] Grathering information about \33[1;49;96m{ip_address}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)


Typo in user-facing output: “Grathering” should be “Gathering”.

Copilot · 2026-03-30T16:46:21Z

+        return complete_scan_timing('mac', started_at, result)

    # Information gathering about the mac address
    printit(f'    [+] Grathering information about \33[1;49;96m{mac_address}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)


Typo in user-facing output: “Grathering” should be “Gathering”.

Suggested change

printit(f' [+] Grathering information about \33[1;49;96m{mac_address}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)

printit(f' [+] Gathering information about \33[1;49;96m{mac_address}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)

Copilot · 2026-03-30T16:46:22Z

            prv_op += final_output+'\n'

-    return True
+    return finalize_scan(True)


ip_lookup always returns success (True) at the end even when all external lookups fail and the function prints a failure table. Track whether any provider actually returned usable data and return False when all providers fail so the UI expression/output state is accurate.

Copilot · 2026-03-30T16:46:22Z

+    def _worker():
+        while not stop_event.wait(1.0):
+            elapsed_seconds = max(0.0, time.perf_counter() - started_at)
+            remaining_seconds = max(0.0, estimate_seconds - elapsed_seconds)
+            status_line = (
+                f'    [+] {scanner_name} scan running | '
+                f'elapsed: {elapsed_seconds:6.1f}s | '
+                f'est remaining: {remaining_seconds:6.1f}s'
+            )
+            sys.stdout.write('\r\33[1;49;93m' + status_line.ljust(140) + '\033[0m')
+            sys.stdout.flush()


The live scan progress ticker writes to stdout from a background thread while the main scan also prints output, which can interleave and corrupt terminal formatting. Consider synchronizing writes (e.g., a shared lock) and clearing the live line before any normal print/printit output while the ticker is active.

Copilot · 2026-03-30T16:46:23Z

    | \33[1;49;97mType\033[0m          | \33[1;49;96mOSINT Framework\033[0m                    |
-    | \33[1;49;97mDeveloper\033[0m     | \33[1;49;96ms41r4j\033[0m                             |
+    | \33[1;49;97mDeveloper\033[0m     | \33[1;49;96ms41r4j, DkHrR\033[0m                      |
    | \33[1;49;97mGithub\033[0m        | \33[1;49;96mhttps://github.com/s41r4j/phomber\033[0m  |


Brand spelling: use “GitHub” instead of “Github” in the displayed info table.

Suggested change

| \33[1;49;97mGithub\033[0m | \33[1;49;96mhttps://github.com/s41r4j/phomber\033[0m |

| \33[1;49;97mGitHub\033[0m | \33[1;49;96mhttps://github.com/s41r4j/phomber\033[0m |

Copilot · 2026-03-30T16:46:23Z

+        return finalize_scan(False)

    # Information gathering about the domain
    printit(f'    [+] Grathering information about \33[1;49;96m{domain}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)


Typo in user-facing output: “Grathering” should be “Gathering”.

Suggested change

printit(f' [+] Grathering information about \33[1;49;96m{domain}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)

printit(f' [+] Gathering information about \33[1;49;96m{domain}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)

Copilot · 2026-03-30T16:46:23Z

+try:
+    PHOMBER_USERNAME_SITE_LIMIT = int(os.environ.get('PHOMBER_USERNAME_SITE_LIMIT', '80'))
+except (TypeError, ValueError):
+    PHOMBER_USERNAME_SITE_LIMIT = 80
+try:
+    PHOMBER_USERNAME_IDENTITY_SOURCES = int(os.environ.get('PHOMBER_USERNAME_IDENTITY_SOURCES', '20'))
+except (TypeError, ValueError):
+    PHOMBER_USERNAME_IDENTITY_SOURCES = 20
+if PHOMBER_USERNAME_IDENTITY_SOURCES < 1:
+    PHOMBER_USERNAME_IDENTITY_SOURCES = 1


PHOMBER_USERNAME_SITE_LIMIT is parsed from the environment but not clamped/sanitized (unlike PHOMBER_USERNAME_IDENTITY_SOURCES). If it’s set to 0 or negative, the Sherlock loop will skip all sources; consider enforcing a minimum of 1 (or explicitly documenting that 0 disables checks).

DkHrR

Nice

DkHrR

Nice

Enhance deep username research and real-time scan timers

59ff9a2

Copilot AI review requested due to automatic review settings March 30, 2026 16:40

Copilot started reviewing on behalf of DkHrR March 30, 2026 16:41 View session

Add remaining archive updates and smoke script

a8f80f1

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Include remaining archived README changes

56cdef3

DkHrR commented Mar 31, 2026

View reviewed changes

Harden startup compatibility and expand scanner coverage

678c8d7

DkHrR commented Apr 5, 2026

View reviewed changes

-- tune per-profile fetch timeout via `PHOMBER_PROFILE_FETCH_TIMEOUT` (default: `8` seconds)
+- tune per-profile fetch timeout via `PHOMBER_PROFILE_FETCH_TIMEOUT` (default: `8` seconds)
+- tune maximum number of sites checked in `username` scan via `PHOMBER_USERNAME_SITE_LIMIT`
+- override the Sherlock sites data source used by `username` scan via `PHOMBER_SHERLOCK_DATA_URL`
+- provide an IP geolocation token via `PHOMBER_IPINFO_TOKEN` to enrich IP-related lookup results

	printit(f' [+] Grathering information about \33[1;49;96m{phone_number}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)
	printit(f' [+] Gathering information about \33[1;49;96m{phone_number}\33[1;49;93m...', coledt=[1, 49, 93], space_down=True)

	\| \33[1;49;97mGithub\033[0m \| \33[1;49;96mhttps://github.com/s41r4j/phomber\033[0m \|
	\| \33[1;49;97mGitHub\033[0m \| \33[1;49;96mhttps://github.com/s41r4j/phomber\033[0m \|

Uh oh!

Conversation

DkHrR commented Mar 30, 2026

Summary

Included changes

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

DkHrR left a comment

Choose a reason for hiding this comment

Uh oh!

DkHrR left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants