Create documentation and schedule for project (healthcheck) presentations

[TheJuanAndOnly99]

Goals

• TBC by @eddie-knight and @harkeganesh

Existing material (potentially to be made consistent with each other)

• TOC Health Check Responsibility
• Current proposed review process
• Incubating and Graduated referencing health checks

Proposed Steps discussed at April 1st meeting

• List of active projects
• Compile into logical groupings
• Sort into order of presentations
• Template for each project to be refined (this was initially started by Ganesh)
• Potentially including project ratings in checkins (this is to be discussed/confirmed)

[mindthegab]

Hey @eddie-knight @psmulovics @harkeganesh @maria-mcparland (FYI @opoupeney @TheJuanAndOnly99 @karlmoll @lucaborella89 @robmoffat @jgavronsky), first off, thank you so much for getting this going. Apologies for not joining today.

With the goal of unblocking / unpausing this quickly, I took some time to edit the description of this task (which was empty) hopefully it helps and please let me know if there's existing resources I left out.

Before we start scheduling project presentations, I wanted to:

1. make sure I am aligned with your goals, so I can align the FINOS team (i left the goals section on the issue empty as this is ultimately your call)
2. share some considerations, based on how we tried (and failed) in the past to do healthchecks as FINOS team, and how other foundations I participated in do that
3. share some suggestions (take it or leave it, again this is your call) to make this process as efficient (high value, low overhead) for everyone (TOC, maintainers, FINOS team)

I'll add three separate comments below so that it might be easier to engage inline. Let me know your thoughts!

[mindthegab]

1 - Goals of this activity

Based on the fact we just transitioned lifecycle, I would consider the goals (in decreasing order of priority) to be:

1. Expedited archival process for projects that had no activity for a long time (see https://landscape.finos.org/ for an easy set of first priorities)
2. Assess all the other projects are in compliance with the new definition of the lifecycle stages they are in
3. Recommend projects for a different lifecycle stage (forward or backward) depending on findings
4. Establish an ongoing review process every 6 months and relative responsibilities between FINOS team (support) and TOC (responsible)

Am I missing something here? Do you think there's a different order of priorities?

[mindthegab]

2 - Considerations based on previous experiences, other communities, this specific community

• FINOS tried to implement healtchecks, asking for active maintainers engagement, with very little success. Very few projects responded, even less so produced any reports. I think the TOC had a similar experience with your initial attempt at a liaison program.
• While that was earlier in our history, the feedback we got was that it was too onerous (differently from other communities, this is not always their day job) and that we should first look at automated reports (quantitative) and then ask for qualitative reports (either in writing or presentations)
• At that time, we were not part of the linux foundation, therefore we didn't have access to LFX insights which constitutes a very good initial source of health for each project - e.g. see the difference between Calm (very healthy) and Security Reference Data. That's why the Health Check Responsibility explicitly references Insights.
• Heck, in those pages, I just noticed there also a Insight Badge with Health we can simply add in the readme of projects to quickly show their health status (@TheJuanAndOnly99 @opoupeney @robmoffat @lucaborella89 @jgavronsky something to consider for your projects)
• As currently designed (unless I misunderstood), the process requires every 6 months to have every incubating or graduated project to have a 15m presentation at the TOC. With 43 currently incubating or graduated projects, assuming 4 per TOC meeting, this would take 10/11 TOC meetings - i.e about 5 months (considering 2x TOC meetings a month). Unless I am misunderstanding something, this would consume the full schedule, considering health checks are scheduled every 6 months. Seems to onerous.
• As we deal with fairly senior people on the board, we have moved to a "reporting by exception" model, where something is actually put on the agenda only if it requires steering, while relying on written updates if everything is a-okay. I wonder if the TOC, given the seniority and busy schedule of everyone involved, should consider a similar approach. See my next comment on a recommendation along these lines.

[mindthegab]

A draft / potential recommended process based on existing tools, processes, lists, responsibilites (please do shoot it down / edit / just an idea)

Pre-healthcheck

1. All Incubating / Graduated projects should have a maintainers.md so that maintainers are clearly identified in Insights, e.g. Calm (status: in Progress, responsible: @TheJuanAndOnly99)
2. All Incubating / Graduated projects should display an Insights Health Badge in their README (e.g. )
3. If projects can't be even bothered in a month to accept these 2 PRs, FINOS team should raise an archival issue to the TOC for expedited archival (responsible: @TheJuanAndOnly99)
4. The TOC can definitely still try to engage with the maintainers of those projects, but then take a vote

The above should weed out a lot of the "dormant" projects which should really be archived, and allow the TOC to focus on real projects.

Healthcheck

1. TOC to revise the report template for completeness. Particularly:

• Make sure it references recently introduced Incubating and graduate maintainance requirements so they know what to ask
• From my Apache past, I found this template useful as well (as you can see it asks for qualitative information the project might need help from the TOC)

2. @TheJuanAndOnly99 to confirm finos-project-maintainers@finos.org is maintained, both when new projects are added and when maintainers are added to existing projects
3. TOC can then use finos-project-maintainers@finos.org to announce project health checks and request individual reports to be completed per report template (give at least a month notice)
4. @TheJuanAndOnly99 can send a reminder one week before the deadline
5. Once deadline is passed, TOC members review reports and decide (by exception) which projects they need more information from, because:
   • They haven't submitted a report -> put them on notice for archival at the next health check
   • The provided information was incomplete / not satisfactory / doesn't fulfill the maintenance requirements -> require presentation at TOC
   • They have asked for TOC help -> presentation or follow up on TOC list / private list
   • At TOC discretion (because they want to hear cool stuff they did, etc)
6. FINOS team can provide maintainer contacts or appropriate maintainer list to communicate (e.g. fluxnova-maintainers@lists.finos.org) or even better update this page to have also maintainers list (vs only users list)
7. FINOS can provide help adding projects to the TOC agenda - NOTE: I strongly discourage to set up separate meetings - just invite them to standing TOC meetings, we've already been told multiple times about meeting fatigue
8. Folllowing presentation, if needed, TOC confirms with maintainers if a liaison is desirable to work directly with project until the next check-in
9. 6 months later, start from 1 :)

[awaiken]

Gab, very nice and logical approach. For the next TOC meeting, let's pick a random or less active project and go through the process together. Also, in your Goals section: Expedited archival process for projects that had no activity for a long time - If we don't have a definition of "long time" lets agree on one and what defines "activity"? Is that 1 commit a month?

[mindthegab]

Hey @psmulovics @eddie-knight @maria-mcparland @harkeganesh and TOC: I hear you @TheJuanAndOnly99 is being asked to start setting up 10 project healthchecks.

Just so I can align the team with OSFF in two weeks from now, was there any decision / comment on the process I outlined above? I only saw @awaiken commenting here. Particularly, would be very helpful if you guys chimed in the issue re:

1. Are you going to make any update the reporting template before we starting reporting?
2. Do you want the health badge as a signal on each project health?
3. Do you want to send a note to all maintainers letting them know healtchecks are starting and what their reporting requirements are? I feel if you just start inviting to present they will be caught off guard.
4. Shouldn't we start from the projects that look less healthy vs strategic initiatives? That would be very helpful - because the strategic initiatives already enjoy a lot of attention.

Reason I'm asking is that, with 4 TOC members being re-elected, it might be more important to have fully buttoned down approach that starting presentations per se.

[awaiken]

Gab, I want to get clear on one point: after one month, they enter the expedited archival process? That seems a little quick. I know you're not asking much, but for example, if a project has a preponderance of EMEA maintainers and it's during the summer months, then it would not be too surprising if they didn't accept the PRs you mentioned.