Skip to content

AddressForAll/pg_govLite

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

26 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

pg_govLite is designed to perform basic data governance tasks in metadata management, complementing System Catalogs and acting as a unified metadata store. It manages the Medallion architecture and tags for all data objects (columns, tables, schemas, functions, etc.), with RDF semantic tags, and governed tags.

Objectives

Adopt Convention over Configuration (CoC) principle for the basic metadata definition and maintenance tasks:

  • Semantic tags and control tags;
  • Medallion control of data objects (datasets and its part-whole hierarchy), like input/output and intermediary datasets;
  • Data tier controls the identity-service and activation of data-quality services. See also "relevance driver" (business, regulatory, analytics, or operational)
  • To generate human-readable structures System Catalog as standard structured content, for documentation.

All modules are external implementations, for example data quality is DQX, Secure control is external ABAC, etc. The pg_govLite is a central orchestration of modules by tags and triggers.

Documentation

See and, please, contribute commenting or correcting:

Below a fast presentation. UML class:

Documentation helper scripts are also included in src:

  • src/doc01-UDF-mediawiki.sql generates MediaWiki documentation for PostgreSQL UDFs, including summary rows, page sections, full pages, and XML dumps. Usage examples are available in src/doc01-UDF-mediawiki.md.
  • src/inst04-documentation_examples.sql installs executable documentation examples as views in gvlt_doc_examples, exposes function-example dependencies, and supports secondary example marking. Usage examples are available in src/inst04-documentation_examples.md.

User manual and executable tutorial

The user manual is available as a Word document:

  • manual/pg_govLite_User_Manual_and_Practical_Governance_Tutorial.docx

The manual explains the project resources and includes a practical governance tutorial using a public-data scenario inspired by IBGE municipality data and ANS supplementary-health data.

The executable SQL version of the tutorial is:

  • assets/data/manual01-public_health_tutorial.sql

It creates a deterministic local sample, so it does not download external data. The script creates the tutorial schemas and tables, applies governed tags, publishes a Gold view, registers an executable documentation example, and runs final checks with RAISE EXCEPTION if an expected result is missing.

Install or test

This project foresees installation on Linux, using make command.

  • make show all options.
  • make all will install all in your database (edit Makefile to set your configurations). Standard installation.
  • make manualExamples will execute the SQL examples from the user manual tutorial. Run make core first when the database has not been initialized yet.
  • make dropDbTest will DROP the test-database and install all in it. Important for developers, to avoid Software regression.

About

pg_govLite is designed to perform basic data governance tasks in metadata management. It implements Medallion architecture and governed tags.

Topics

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors