Skip to content

Webpack 5 upgrade#7351

Merged
na9da merged 49 commits into
mainfrom
webpack5
Feb 12, 2025
Merged

Webpack 5 upgrade#7351
na9da merged 49 commits into
mainfrom
webpack5

Conversation

@na9da

@na9da na9da commented Dec 5, 2024

Copy link
Copy Markdown
Collaborator

What this PR does

Fixes #6998
Depends on TerriaJS/TerriaMap#718 and #7311

Upgrades webpack and other build dependencies to the latest version.

Note: bulk of the changes are from running the saas-migrator that upgrades .scss files to modern api.

Changes summary

  • Packages upgraded
    webpack, ...
  • Babel transforms
    • Removed transformation to CJS for the default build. We still use it for node builds.
    • Other transforms removed plugin-proposal-class-properties, proposal-object-rest-spread, plugin-syntax-dynamic-import. These must now be well supported by browsers.
  • Cleaned up configureWebpack.js
    • Removed rules that are no longer necessary
    • Replaced raw-loader file-loader url-loader etc with builtin asset modules
    • Removed use of webpack specific qualifiers for imports like raw-loader!file-loader etc to make it more ESM compliant. Added extra config in webpack to correctly handle imports of static files (like csv and xml).
    • Added rules to polyfill nodejs modules that are referenced from some of our dependencies.
    • Removed hot reload config. AFAICT this was not being used and required maintaining two different code paths in webpack configuration. If required we can add it back later after testing that it works.
    • Replaced multiple configureWebpack parameters with a single options parameter.
    • Removed string-replace-webpack-plugin as it was outdated and has a few vulnerabilities.
      • TODO: Move rewrite of default Cesium API key warning to the app
  • Upgraded SASS
    • Migrated SASS files to use modern API (by running the sass-migrator script)
    • Replaced webpack aliases ~terriajs-variables and ~terriajs-mixins with respective relative path. This was necessary to run the migrator. Also it means simpler webpack config.
    • css-modules-typescript-loader that generates TS types for SASS modules is outdated and no longer works with webpack 5. I have made a repo from a fork of another similar package that can generate types. The other alternative would be to use this TS plugin - but not sure if it'll slow down compilation/editing.
  • Modified TIFFImageryProvider to work with webpack5 build.
  • TODO: Find a replacement for svg-sprite-loader. It appears to be not maintained and has a bunch of audit warnings.
  • Rewrote a couple of specs that depended on some CJS feature
  • TODO: Test in Windows.

I'll annotate the changes to make review easier.

Test me

Checklist

  • There are unit tests to verify my changes are correct or unit tests aren't applicable (if so, write quick reason why unit tests don't exist)
  • I've updated relevant documentation in doc/.
  • I've updated CHANGES.md with what I changed.
  • I've provided instructions in the PR description on how to test this PR.

na9da added 15 commits December 4, 2024 15:28
- Rewrite webpack aliased paths like ~terriajs-variables and ~terriajs-mixins to
  plain relative paths so that we can run the auto migrator.
  This was done using the following shell command:
  ```
     find lib/ -name '*.scss' -exec bash -c 'sed -i "s|~terriajs-variables|$(realpath -s --relative-to=$(dirname "$1") lib/Sass/common/_variables)|" $1' bash  {} \;
  ```
- Run auto migrator
  ```
     find lib/ -name '*.scss' -exec npx sass-migrator module {} \;
  ```
Instead of using 'raw-loader!' etc in the import path, we now
handle these import in webpack config. See webpack.config.make.js.
Removes use of 'raw-loader!' etc in import paths.
Instead handle the imports in webpack config.
Webpack literally works only for 'process.env.NODE_ENV'.
@na9da na9da changed the base branch from main to cjs-to-esm December 5, 2024 01:28
Comment thread package.json
Comment thread package.json
@pjonsson

pjonsson commented Dec 5, 2024

Copy link
Copy Markdown
Contributor

I looked through this and besides the things I commented on, the things that are there looks good to me. Would love to get the (approved) CJS to ESM PR merged soon so this PR doesn't end up in limbo.

@na9da na9da mentioned this pull request Dec 8, 2024
5 tasks
Base automatically changed from cjs-to-esm to main December 8, 2024 23:36
@na9da na9da marked this pull request as ready for review December 8, 2024 23:53
Comment thread package.json Outdated
@zoran995 zoran995 mentioned this pull request Feb 10, 2025
4 tasks

@zoran995 zoran995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have cherry-picked changes from #7450, it should be easier to review now as code-diff overthere got pretty large and we can now handle this in single review cycle

  • Fixed review comments
  • Removed unused weback plugins
  • upgraded webpack plugins
  • I didn't see any reason why we would need pmtiles alias in webpack config - marked it with review comment for double check.
  • I had to restore worker-loader as webpack native web worker support tends to get pretty complex, I will keep looking for a fix but it might be good to keep worker-loader and handle that as a follow-up. Important note: webpack worker support behaves differently in development and production builds. That's why it works on CI but fails locally. I have run into this issue in the past but still don't have a solution that always works.
  • side-effects are removed from package.json we can look into it later

Comment thread lib/Sass/common/_buttons.scss
Comment thread lib/ReactViews/ExplorerWindow/Tabs/MyDataTab/my-data-tab.scss
Comment thread buildprocess/configureWebpack.js
Adds a custom property `Ion.defaultTokenMessage` for changing the credit message shown on the map when using default Ion token.
@na9da na9da self-assigned this Feb 11, 2025
@na9da

na9da commented Feb 11, 2025

Copy link
Copy Markdown
Collaborator Author

Regarding the web worker change, it appears to work if we set publicPath to auto instead of build/.

I think webpack will use publicPath + directory-containing-parent-script which evaluates to build/build/<downloadWorkerChunkName>.js.

We could test it out in a different PR.

Comment thread lib/Core/polyfill.js Outdated
Comment thread package.json
Comment thread package.json
This was mostly required back when using polyfill.io to perform polyfills.
@pjonsson

Copy link
Copy Markdown
Contributor

Regarding the web worker change, [..]

We could test it out in a different PR.

If this works in the current state I suggest merging it, this PR is already a significant step forward.

I'll rebase the ESLint fixes after this gets merged so the TypeScript upgrade can get merged too.

@na9da

na9da commented Feb 12, 2025

Copy link
Copy Markdown
Collaborator Author

Merging! Thanks everyone, for the contributions.

PS: will merge the terriamap PR after a new terriajs release, but i'll see if I can include the Typescript upgrade PR into this release.

@pjonsson

Copy link
Copy Markdown
Contributor

@na9da since you wrote "Merging!" I was going to rebase the ESLint fixes for TS 5.7 before I go to bed so those could be merged, but I'm guessing you got interrupted by something else. Will this PR get merged soon, or should I go to bed now and rebase things tomorrow?

@ljowen ljowen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested building on OSX and Windows10 (arm64)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrade to webpack 5

4 participants