Enhancement Proposal: Adapt plugin system to utilize entry_points for identification

[ronpandolfi]

The current plugin system employs 'marker files' to identify plugins.

Pros:

• Additional metadata may be attributed to each plugin module without loading the module.

Cons:

• Metadata at the granularity of individual plugins may not be interesting/useful/practical.
• This is a non-standard approach
• Locating marker files requires recursively traversing file trees. On some systems, this can be impractical, or even cause infinite recursion.
• The scope of directories to scan for marker files must be carefully defined.
• The marker file must also indicate the plugin module's location in package namespace for relative imports to work

I propose to utilize the entry_points definition in setup.py of each plugin-containing package as a manifest of the package contents (see simple examples below).

Pros:

• Standard, vanilla python implementation of a plugin system
• No searching required to locate plugins
• Relative imports should work as normal (verification needed)
• This enables mirroring the local environment to a remote resource more selectively (by reverse-lookup from the entry_point manifests)
• Can catch fresh eggs by subscribing to callback (see bonus example comments)

Side effects:

• The EZPlugin design will have to self-register with the plugin manager

Simple examples:
https://amir.rachum.com/blog/2017/07/28/python-entry-points/

Bonus examples (useful tips in comment by PJE):
https://aroberge.blogspot.com/2008/12/plugins-part-6-setuptools-based.html

Reference:
https://setuptools.readthedocs.io/en/latest/pkg_resources.html

[ronpandolfi]

@danielballan Here's some quick notes on the proposed design.

[danielballan]

See also https://entrypoints.readthedocs.io/en/latest/api.html#high-level-api a super-lightweigh library for avoiding some pitfalls involved in using pkg_resources directly.