Project Management Automation: Avoid gracefully handling error

[aduth]

This pull request seeks to change the behavior of our GitHub automation tasks to ensure that they fail loudly, rather than be silently absorbed.

For example, the following action was considered a success, despite the fact that it failed to assign the milestone:

https://github.com/WordPress/gutenberg/runs/423269588?check_suite_focus=true

main: Task addMilestone failed with error: HttpError: Validation Failed: {"resource":"Milestone","code":"already_exists","field":"title"}

We should want these tasks to fail loudly to ensure that they are working properly. This specific tasks has been dysfunctional for quite some time, but was only recently noticed to not be assigning the milestone.

If a task expects to possibly be throwing errors, it should do its own try / catch and deal with the error accordingly. In this case, a duplicate milestone may be "fine" (it was previously anticipated), so the specific step of creating the milestone could be allowed to fail with the duplicate, as long as it still proceeds to add the milestone to the pull request. Currently, this last part was never reached, because it never recovers from the thrown error.

Testing Instructions:

It will likely not be possible to test this, assuming there are no errors in current automation.

That said, there is an error with the milestone assignment automation, which occurs when the next milestone has already been created. This could either be used as verification that the failures are logged loudly, or could be considered a blocker to resolve first before this is merged.

[aduth]

If a task expects to possibly be throwing errors, it should do its own try / catch and deal with the error accordingly. In this case, a duplicate milestone may be "fine" (it was previously anticipated), so the specific step of creating the milestone could be allowed to fail with the duplicate, as long as it still proceeds to add the milestone to the pull request. Currently, this last part was never reached, because it never recovers from the thrown error.

See also: #20011

[aduth]

I'm merging this first, before #20011, anticipating that the action will fail with an error, but doing so intentionally to verify the expected behavior that the task error is not silently absorbed.

[aduth]

Interesting! The process failed, but the task itself was still reported as a success:

https://github.com/WordPress/gutenberg/runs/423870820?check_suite_focus=true

(node:2810) UnhandledPromiseRejectionWarning: HttpError: Validation Failed: {"resource":"Milestone","code":"already_exists","field":"title"}
at /home/runner/work/gutenberg/gutenberg/packages/project-management-automation/node_modules/@octokit/request/dist-node/index.js:66:23
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async addMilestone (/home/runner/work/gutenberg/gutenberg/packages/project-management-automation/lib/add-milestone.js:88:2)
at async main (/home/runner/work/gutenberg/gutenberg/packages/project-management-automation/lib/index.js:52:4)

I'll follow-up to investigate whether there are documented options for signalling that the the action failed.

Also because at some point in the future, we'd need to handle the failure explicitly anyways:

(node:2810) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

[aduth]

There appear to be two options, based on the documentation.

Using the setFailed message from the @actions/core package:

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/creating-a-javascript-action#write-the-action-code

Or setting the exit code of the process:

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/development-tools-for-github-actions#setting-an-actions-status-using-exit-codes

I might have thought the uncaught error would already be causing a non-zero status code, but apparently not?

[epiqueras]

Unhandled promise rejections

Don't do that. We need to do it explicitly.

[aduth]

Follow-up at #20012