Make the linting consistent across our repos

[tombrooks248]

Particularly across the Data Enginering focused repos:

digital-land-python
pyspark-jobs
airflow-dags
collection-task
digitial-land-postgres

It would be really great to bring the linting all into line so its consistent.

I feel like sometimes I can use make format, sometimes make lint, sometimes end up running black . and then it turns out I am linting a bunch of files I have not made changes to, implying that linting is not compulsory in that repo, which I think it should be.

This ticket is about doing an audit of the above repos (or any others if think they should be added) figuring out how they work. Deciding which one is the best way of working and then applying them to the other repos.

I think my personal ideal would be that make format should apply a consistent version of flake8 and black across the repo, and all files in a given repo are brought up to date with that.
That linting should be applied as part of a pre-commit hook and again in the CI in Github.

But I leave it up to whoever picks up this ticket to decide what would work best in consultation with the other Data Engineers and @eveleighoj.