Add concurrency support for more than one thread

[vicary]

What did you implement:

Adding concurrency support for more than one thread.

Conditionally supersedes #680 if release/5.4.0 is more preferable than master.

How did you implement it:

Please see #517 (comment), basically I did two changes:

1. Replacing serizliedCompile: Boolean with concurrency: Number
2. Translates serializedCompile: true to concurrency: 1 for backward compatibility

How can we verify it:

Use the config below when packaging:

package:
  individually: true

custom:
  webpack:
    concurrency: 5

Todos:

• Write tests
• Write documentation
• Fix linting errors
• Make sure code coverage hasn't dropped
• Provide verification config / commands / resources
• Enable "Allow edits from maintainers" for this PR
• Update the messages below

Is this ready for review?: Yes
Is it a breaking change?: No

[coyoteecd]

@miguel-a-calles-mba @j0k3r can you have a look and merge this? I am using vicary's fork for half a year and would love to switch back to the official package.

[j0k3r]

@coyoteecd we are in the process of cleaning up stuff to release 5.4.0 and this will be part of it

[coyoteecd]

@j0k3r I don't understand what's going on here. Was this PR closed accidentally? Because I don't see it merged anywhere, and #680 is closed without being merged as well.

[j0k3r]

Sorry I made a mistake while merging release/5.4.0 into master and removed it. The removal automatically closed that PR. It should now be rebased against the master and fix conflicts.

[vicary]

Not sure if I'm doing it wrong, but I found myself learning the backstory every conflict or two in the process of rebasing. This takes a day or two, could only do this on weekends.

[vicary]

@j0k3r During my rebase I found that the current master has these patterns.

// tests/packageModules.test.js
        return BbPromise.each([ '1.18.1', '2.17.0', '10.15.3' ], version => {
          getVersionStub.returns(version);
          return expect(module.packageModules()).to.be.fulfilled.then(() => BbPromise.all([]));
        }).then(() =>
          BbPromise.each([ '1.17.0', '1.16.0-alpha', '1.15.3' ], version => {
            getVersionStub.returns(version);
            return expect(module.packageModules()).to.be.fulfilled.then(() => BbPromise.all([]));
          })
        );

It has a strong code-smell here .then(() => BbPromise.all([])); which should either not there, or some tests are missed during rebase.

Not meant to make things more complicated than it should, please feel free to ignore this if it's intended.

I'll be rebasing to master as-is, just a friendly heads-up.

[vicary]

The rebase is finished, this should be ready for merging.

[j0k3r]

Awesome job for the rebase @vicary 👏

[j0k3r]

Also, I think some tests should be updated as well about { concurrency: Infinity }. Don't you think?

[vicary]

Also, I think some tests should be updated as well about { concurrency: Infinity }. Don't you think?

I agree that using Infinity may causes confusions. In fact compile.test.js is largely testing if the compiler has even run once, concurrency config there is mostly for cosmetic reasons.

To reduce ambiguity, how about we change the tests to use 1 instead?

[j0k3r]

Good job!