diff --git a/README.md b/README.md index 04d24c7..081cd3d 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,30 @@ print(f"Verdict: {completed.level}") print(f"MITRE techniques: {completed.mitre_techniques}") ``` +### Filename canonicalisation (`dynamic_mimetype_check`) + +By default the server appends the real (Magika-detected) extension to the +persisted filename when it disagrees with what the file was named. Opt out per +submission — sandbox / open-in-browser take it as a `metafields` key, static +/ CDR take it as a top-level kwarg: + +```python +# sandbox / open-in-browser → inside metafields +client.create_sandbox_submission( + "./malware.txt", + environment="w10_x64", + metafields={"dynamic_mimetype_check": False, "timeout": 120}, +) + +# static / CDR → top-level kwarg +client.create_static_submission("./malware.txt", dynamic_mimetype_check=False) +client.create_cdr_submission("./report.doc", dynamic_mimetype_check=False) +``` + +See [Recipe 13](docs/RECIPES.md#13-control-dynamic_mimetype_check) for the full +breakdown including reading back `FileInfo.is_mimetype_checked` after the +submission completes. + ## Core concepts - **Submissions** — one analysis target (file or URL) with a stable `uuid`, a workspace diff --git a/docs/RECIPES.md b/docs/RECIPES.md index 9bfeabe..ca6e8f8 100644 --- a/docs/RECIPES.md +++ b/docs/RECIPES.md @@ -22,6 +22,7 @@ client = ThreatZone(api_key="") 10. [Use the async client](#10-use-the-async-client) 11. [Build a daily report summary](#11-build-a-daily-report-summary) 12. [Cross-reference MITRE techniques with indicators](#12-cross-reference-mitre-techniques-with-indicators) +13. [Control Magika filename canonicalisation per submission (`dynamic_mimetype_check`)](#13-control-dynamic_mimetype_check) --- @@ -631,3 +632,110 @@ for technique_id in sorted(layer): - Both endpoints are `ReportStatusGuard('dynamic')`-gated — wrap them with the retry loop from [recipe 7](#7-handle-the-reportunavailableerror-exception) if the submission may still be in analysis. + +--- + +### 13. Control `dynamic_mimetype_check` + +When Threat.Zone receives a file, Magika (a content-based mimetype detector) inspects +the bytes and may decide that the real extension differs from the filename extension +(for example, a PowerShell script saved as `malware.txt` is detected as `ps1`). By +**default** the persisted `submission.file.name` is canonicalised by appending the +real extension — `malware.txt` becomes `malware.txt.ps1`. The same canonical +name flows into every downstream artifact, queue message, sandbox `task.yaml`, and UI +surface so the true content type is everywhere consistent. + +This canonicalisation is **enabled by default** for every submission endpoint, and +can be disabled per request. The SDK shape mirrors the API wire shape: + +| Endpoint | SDK kwarg | Wire field | +|---|---|---| +| `create_sandbox_submission` | `metafields={"dynamic_mimetype_check": ...}` | `metafields.dynamic_mimetype_check` | +| `create_open_in_browser_submission` | `metafields={"dynamic_mimetype_check": ...}` | `metafields.dynamic_mimetype_check` | +| `create_static_submission` | `dynamic_mimetype_check=...` | top-level form field | +| `create_cdr_submission` | `dynamic_mimetype_check=...` | top-level form field | + +For archive and email submissions, only the outer container filename is +canonicalised — the in-archive `entrypoint` string is forwarded verbatim. + +**a) Default behaviour (recommended).** Omit the field entirely. The server's +default (`true`) applies, and the persisted name is rewritten when Magika disagrees +with the declared extension: + +```python +client.create_sandbox_submission("./malware.txt", environment="w10_x64") +client.create_static_submission("./suspicious.bin") +client.create_cdr_submission("./report.doc") +``` + +**b) Keep filename verbatim — sandbox / open-in-browser.** Send the flag +inside the `metafields` dict that the SDK forwards as a JSON object: + +```python +client.create_sandbox_submission( + "./malware.txt", + environment="w10_x64", + metafields={ + "dynamic_mimetype_check": False, + "timeout": 120, + "mouse_simulation": False, + }, + private=True, +) + +client.create_open_in_browser_submission( + "https://example.test/page", + metafields={"dynamic_mimetype_check": False}, +) +``` + +The dict-style `metafields` is normalised to the JSON object shape the API expects; +all other keys you pass through (timeout, mouse_simulation, work_path, etc.) are +validated server-side against your plan. + +**c) Keep filename verbatim — static / CDR.** Pass the top-level kwarg +directly: + +```python +client.create_static_submission( + "./malware.txt", + private=True, + dynamic_mimetype_check=False, +) + +client.create_cdr_submission( + "./report.doc", + dynamic_mimetype_check=False, +) +``` + +`dynamic_mimetype_check=None` (or omitting it) sends nothing on the wire, letting +the server apply its default of `true`. Use `False` only when you have a specific +reason to disable canonicalisation — e.g. you intentionally want to feed a +mis-named sample through unchanged for a regression test. + +**d) Read back the result.** After the submission completes you can confirm +whether canonicalisation actually took effect via the `FileInfo.is_mimetype_checked` +flag on the submission detail: + +```python +detail = client.get_submission(uuid) +print(detail.file.name) # may be "malware.txt.ps1" +print(detail.file.is_mimetype_checked) # True when canonicalisation ran +``` + +**Async client.** All four create methods accept the same kwargs on +`AsyncThreatZone`: + +```python +async with AsyncThreatZone(api_key="...") as client: + await client.create_static_submission( + "./malware.txt", + dynamic_mimetype_check=False, + ) + await client.create_sandbox_submission( + "./malware.txt", + environment="w10_x64", + metafields={"dynamic_mimetype_check": False, "timeout": 120}, + ) +``` diff --git a/src/threatzone/_async_client.py b/src/threatzone/_async_client.py index 4a2c7ba..072487e 100644 --- a/src/threatzone/_async_client.py +++ b/src/threatzone/_async_client.py @@ -233,6 +233,7 @@ async def create_static_submission( private: bool = False, entrypoint: str | None = None, password: str | None = None, + dynamic_mimetype_check: bool | None = None, ) -> SubmissionCreated: """ Create a new static analysis submission. @@ -242,6 +243,10 @@ async def create_static_submission( private: If True, submission is private to your workspace. entrypoint: Entry point for archive files. password: Password for encrypted archives. + dynamic_mimetype_check: When True (default on the server), + Magika-detected real extension is appended to the filename if + it differs from the declared extension. Send False to keep the + original filename verbatim. Omit to use the server default. Returns: SubmissionCreated with the new submission UUID. @@ -251,6 +256,7 @@ async def create_static_submission( private=private, entrypoint=entrypoint, password=password, + dynamic_mimetype_check=dynamic_mimetype_check, ) response = await self._http.post("/submissions/static", **multipart) return SubmissionCreated.model_validate(response.json()) @@ -262,6 +268,7 @@ async def create_cdr_submission( private: bool = False, entrypoint: str | None = None, password: str | None = None, + dynamic_mimetype_check: bool | None = None, ) -> SubmissionCreated: """ Create a new CDR (Content Disarm & Reconstruction) submission. @@ -271,6 +278,10 @@ async def create_cdr_submission( private: If True, submission is private to your workspace. entrypoint: Entry point for archive files. password: Password for encrypted archives. + dynamic_mimetype_check: When True (default on the server), + Magika-detected real extension is appended to the filename if + it differs from the declared extension. Send False to keep the + original filename verbatim. Omit to use the server default. Returns: SubmissionCreated with the new submission UUID. @@ -280,6 +291,7 @@ async def create_cdr_submission( private=private, entrypoint=entrypoint, password=password, + dynamic_mimetype_check=dynamic_mimetype_check, ) response = await self._http.post("/submissions/cdr", **multipart) return SubmissionCreated.model_validate(response.json()) diff --git a/src/threatzone/_sync_client.py b/src/threatzone/_sync_client.py index 23f2328..70f0f47 100644 --- a/src/threatzone/_sync_client.py +++ b/src/threatzone/_sync_client.py @@ -230,6 +230,7 @@ def create_static_submission( private: bool = False, entrypoint: str | None = None, password: str | None = None, + dynamic_mimetype_check: bool | None = None, ) -> SubmissionCreated: """ Create a new static analysis submission. @@ -239,6 +240,10 @@ def create_static_submission( private: If True, submission is private to your workspace. entrypoint: Entry point for archive files. password: Password for encrypted archives. + dynamic_mimetype_check: When True (default on the server), + Magika-detected real extension is appended to the filename if + it differs from the declared extension. Send False to keep the + original filename verbatim. Omit to use the server default. Returns: SubmissionCreated with the new submission UUID. @@ -248,6 +253,7 @@ def create_static_submission( private=private, entrypoint=entrypoint, password=password, + dynamic_mimetype_check=dynamic_mimetype_check, ) response = self._http.post("/submissions/static", **multipart) return SubmissionCreated.model_validate(response.json()) @@ -259,6 +265,7 @@ def create_cdr_submission( private: bool = False, entrypoint: str | None = None, password: str | None = None, + dynamic_mimetype_check: bool | None = None, ) -> SubmissionCreated: """ Create a new CDR (Content Disarm & Reconstruction) submission. @@ -268,6 +275,10 @@ def create_cdr_submission( private: If True, submission is private to your workspace. entrypoint: Entry point for archive files. password: Password for encrypted archives. + dynamic_mimetype_check: When True (default on the server), + Magika-detected real extension is appended to the filename if + it differs from the declared extension. Send False to keep the + original filename verbatim. Omit to use the server default. Returns: SubmissionCreated with the new submission UUID. @@ -277,6 +288,7 @@ def create_cdr_submission( private=private, entrypoint=entrypoint, password=password, + dynamic_mimetype_check=dynamic_mimetype_check, ) response = self._http.post("/submissions/cdr", **multipart) return SubmissionCreated.model_validate(response.json()) diff --git a/tests/test_client.py b/tests/test_client.py index 8048304..61e3a1d 100644 --- a/tests/test_client.py +++ b/tests/test_client.py @@ -493,25 +493,25 @@ class TestConfigBuildUrl: def test_build_url_without_leading_slash(self): config = ClientConfig( api_key="test", - base_url="https://api.threat.zone", + base_url="https://app.threat.zone/public-api", timeout=30.0, max_retries=2, verify_ssl=False, ) - url = config.build_url("v1/submissions") + url = config.build_url("submissions") - assert url == "https://api.threat.zone/v1/submissions" + assert url == "https://app.threat.zone/public-api/submissions" def test_build_url_with_leading_slash(self): config = ClientConfig( api_key="test", - base_url="https://api.threat.zone", + base_url="https://app.threat.zone/public-api", timeout=30.0, max_retries=2, verify_ssl=False, ) - url = config.build_url("/v1/submissions") + url = config.build_url("/submissions") - assert url == "https://api.threat.zone/v1/submissions" + assert url == "https://app.threat.zone/public-api/submissions" diff --git a/uv.lock b/uv.lock index 46fe8e3..bd6cadc 100644 --- a/uv.lock +++ b/uv.lock @@ -1,5 +1,5 @@ version = 1 -revision = 2 +revision = 3 requires-python = ">=3.10" [[package]] @@ -607,7 +607,7 @@ wheels = [ [[package]] name = "threatzone" -version = "1.0.0" +version = "1.0.1" source = { editable = "." } dependencies = [ { name = "httpx" },