Skip to content

refactor(core): remove ext: modules from the module map#19040

Merged
mmastrac merged 28 commits into
denoland:mainfrom
nayeemrmn:ext-imports-check-in-cli
May 28, 2023
Merged

refactor(core): remove ext: modules from the module map#19040
mmastrac merged 28 commits into
denoland:mainfrom
nayeemrmn:ext-imports-check-in-cli

Conversation

@nayeemrmn

@nayeemrmn nayeemrmn commented May 8, 2023

Copy link
Copy Markdown
Contributor

Rather than disallowing ext: resolution, clear the module map after initializing extensions so extension modules are anonymized. This operation is explicitly called in deno_runtime. Re-inject node: specifiers into the module map after doing this.

Fixes #17717.

Comment thread cli/resolver.rs Outdated
Comment thread cli/resolver.rs Outdated
Comment thread cli/tests/testdata/run/extension_import.ts.out Outdated
Comment thread core/modules.rs Outdated
Comment thread core/modules.rs Outdated
Comment thread core/runtime.rs
Comment thread runtime/lib.rs Outdated
Comment thread core/extensions.rs Outdated
@nayeemrmn nayeemrmn changed the title refactor(core): move ext: import check to cli refactor(core): remove ext: modules from the module map May 23, 2023
Comment thread runtime/worker.rs Outdated
Comment thread core/extensions.rs Outdated
Comment thread ext/node/polyfill.rs Outdated

@bartlomieju bartlomieju left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for a slow response @nayeemrmn, I've been busy preparing for v1.34 release. I really like the direction of this PR and I believe it cleans up a lot of concepts.

That said I need to take another look at core/modules.rs with fresh eyes before approving and landing it.

I also left some comments with questions and suggestions; but overall it's great.

Comment thread ext/node/polyfill.rs Outdated

@mmastrac mmastrac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love these changes and love to see so many special cases deleted.

I think there's only one nit and one change I'd like to see -- moving this new retain/rename operation so that it's attached to the extension macro itself and called after JS init, which helps decouple the code.

Comment thread runtime/worker.rs
/// modules as `node:` specifiers.
pub fn init_runtime_module_map(js_runtime: &mut JsRuntime) {
js_runtime.clear_module_map(
deno_node::SUPPORTED_BUILTIN_NODE_MODULES

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this rename and retain operation callback should live in the extension definition macro (deno_core::extension!) itself.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this rename and retain operation callback should live in the extension definition macro (deno_core::extension!) itself.

That was my first idea. The problem is we don't want this step to be taken when we're creating the intermediary RUNTIME_SNAPSHOT since the CLI extensions will want internal access. While thinking of how on earth I'd document or option-ise this very specific behaviour, it became clear to me that it's not a property of the extension but rather a bespoke operation called by the host only at the final step. So I added a procedural binding for it.

IMO It turns out that this solution is the less coupled one and is easier to delete if e.g. we figure out a better cross-crate JS extension interface.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense to me. I think we can take another look at the runtime/realm setup process after this lands (especially if we can get modules working correctly in realms soon).

Comment thread ext/node/polyfill.rs
impl NodeModulePolyfill {
pub fn module_name(&self) -> &'static str {
debug_assert!(self.specifier.starts_with("node:"));
&self.specifier[5..]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strip_prefix and unwrap

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was previously that #19040 (comment)

I think either way is fine because all of these should start with "node:"

Comment thread core/runtime.rs
/// Clear the module map, meant to be used after initializing extensions.
/// Optionally pass a list of exceptions `(old_name, new_name)` representing
/// specifiers which will be renamed and preserved in the module map.
pub fn clear_module_map(

@mmastrac mmastrac May 28, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See runtime/worker comments - this should be called internally based on extension macro config

Comment thread core/runtime.rs
// 2. Iterate through all extensions:
// a. If an extension has a `esm_entry_point`, execute it.

// TODO(nayeemrmn): Module maps should be per-realm.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this code is unfortunately quite broken with realms.

@mmastrac

Copy link
Copy Markdown
Contributor

After reviewing the code and feedback I'm good with this as-is. I'll approve this as-is. Great PR.

@mmastrac mmastrac merged commit b6a3f8f into denoland:main May 28, 2023
@nayeemrmn nayeemrmn deleted the ext-imports-check-in-cli branch May 28, 2023 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

prevent importmap to acess internal specifiers

5 participants