Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
415 changes: 415 additions & 0 deletions docs/PRD-4.x.md

Large diffs are not rendered by default.

37 changes: 32 additions & 5 deletions packages/metaed-console/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,37 @@
# metaed-console

This package can be used for running the MetaEd build process from the command line.
Command-line interface for running the MetaEd build pipeline.

1. Build the project from the parent directory with `npm run build`.
2. To confirm it is functional, try `node ./dist/index.js -h`.
## Input Configuration

CLI arguments via `yargs`:

- `--config / -c` — Path to a JSON configuration file (relative paths are resolved relative to the console module directory; use absolute paths to avoid ambiguity)
- `--defaultPluginTechVersion / -x` — Default plugin technology version
- `--accept-license / -a` — Required flag to accept the license agreement
- `--suppressPrereleaseVersion` — Suppress pre-release version in output paths (default: `true`)

The config file supplies a `metaEdConfiguration` object with project paths, artifact
directories, and plugin settings. See `metaed-edfi-5.2.json` for a fully-worked sample.

## Output

Executes the full MetaEd pipeline (load → parse → build → validate → enhance →
generate → write) and writes all generated artifacts to the configured artifact
directory. Exits with code 0 on success, 1 on failure.

## Business Logic

Assembles the default plugin set, resolves the data standard version from project
metadata, builds the pipeline state, and runs all registered validators, enhancers,
and generators in sequence. Logs timing information for each phase.

## Usage

1. Build the project from the repo root with `npm run build`.
2. To confirm it is functional, try `node packages/metaed-console/dist/index.js -h`.
3. The easiest way to run this is with a config file. See `metaed-edfi-5.2.json` for a fully-worked
sample config file. Note that this shows Alliance Mode _on_, which is only appropriate in the Ed-Fi Alliance's build
processes. External users should set this to `false`. To run with a config file:
`node ./dist/index.js -a -c ./metaed-edfi-5.2.json`.
processes. External users should set this to `false`. From the repo root, run with a config file using an absolute path
or a path that resolves from the console module directory, for example:
`node packages/metaed-console/dist/index.js -a -c packages/metaed-console/metaed-edfi-5.2.json`.
34 changes: 34 additions & 0 deletions packages/metaed-core/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# metaed-core

Core engine for the MetaEd DSL pipeline. Provides the shared model, plugin
abstractions, and pipeline orchestration used by all MetaEd packages.

## Input Configuration

`MetaEdConfiguration` type with key fields:

- `artifactDirectory` — Where generated artifacts are written
- `deployDirectory` — Target for deployment operations
- `pluginTechVersion` — Technology version field (present for compatibility; current plugin setup assigns every plugin `defaultPluginTechVersion`)
- `defaultPluginTechVersion` — Fallback technology version
- `projects` — Array of project definitions (name, version, namespace, paths)
- `projectPaths` — File system paths for MetaEd source files
- `pluginConfigDirectories` — Directories containing plugin configuration
- `allianceMode` — Whether running in Ed-Fi Alliance mode
- `suppressPrereleaseVersion` — Controls version formatting in output paths

## Output

- Parsed MetaEd model (domain entities, associations, descriptors, etc.)
- Pipeline execution results (validation failures, generated output)
- `GeneratedOutput` objects containing a human-readable name, namespace, file name,
folder name, and content as either a string (`resultString`) or a binary `Buffer`
(`resultStream`; takes precedence over `resultString` when set)

## Business Logic

Orchestrates the sequential pipeline: initialize → load → parse → build → namespace
init → plugin config load → then for each plugin in dependency order: validate →
enhance → generate → write output. Defines the plugin contract (validators, enhancers,
generators), the domain model types, and the shared infrastructure for file I/O,
logging, and configuration resolution.
9 changes: 7 additions & 2 deletions packages/metaed-core/docs/jsonnet-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ MetaEd now supports [Jsonnet](https://jsonnet.org/) configuration files in addit
## File Naming

Configuration files can use either extension:

- `.config.json` - Standard JSON configuration
- `.config.jsonnet` - Jsonnet configuration with advanced features

Expand All @@ -26,6 +27,7 @@ Since JSON is a valid subset of Jsonnet, you can rename any `.config.json` file
## Basic Example

### Original JSON

```json
{
"config": [
Expand Down Expand Up @@ -56,6 +58,7 @@ Since JSON is a valid subset of Jsonnet, you can rename any `.config.json` file
```

### Improved Jsonnet

```jsonnet
// Define common values as variables
local namespace = "EdFi";
Expand Down Expand Up @@ -109,6 +112,7 @@ local enableFeature = std.extVar("ENABLE_FEATURE") == "true";
```

Pass external variables via MetaEd configuration:

```javascript
{
externalVariables: {
Expand All @@ -120,7 +124,8 @@ Pass external variables via MetaEd configuration:

### Complex Configuration Example

See [examples/edfiApiSchema.config.jsonnet](../examples/edfiApiSchema.config.jsonnet) for a comprehensive example showing:
See [examples/edfiApiSchema.config.jsonnet](../../metaed-plugin-edfi-api-schema/examples/edfiApiSchema.config.jsonnet) for a comprehensive example showing:

- Helper functions for creating rules
- Version range constants
- Conditional configuration
Expand All @@ -140,4 +145,4 @@ All existing JSON configuration files continue to work without any changes. Json

### Validation

Configuration validation using Joi schemas works the same for both JSON and Jsonnet files. The Jsonnet is evaluated to JSON first, then validated against the plugin's configuration schema.
Configuration validation using Joi schemas works the same for both JSON and Jsonnet files. The Jsonnet is evaluated to JSON first, then validated against the plugin's configuration schema.
19 changes: 19 additions & 0 deletions packages/metaed-default-plugins/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# metaed-default-plugins

Convenience package that assembles the standard MetaEd plugin stack in the correct
dependency order.

## Input Configuration

None. This package has no runtime configuration of its own.

## Output

Exports `defaultPlugins(): MetaEdPlugin[]` which returns the canonical ordered list of
all standard MetaEd plugins, cached after first initialization.

## Business Logic

Imports and initializes all standard Ed-Fi plugins (unified, ODS relational, SQL Server,
PostgreSQL, change query, record ownership, XSD, API schema, handbook, dictionaries,
etc.) and returns them in the dependency order required by the pipeline.
37 changes: 37 additions & 0 deletions packages/metaed-odsapi-deploy-console/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# metaed-odsapi-deploy-console

Command-line interface for deploying MetaEd artifacts into an ODS/API repository.

## Input Configuration

CLI arguments via `yargs`:

- `--config / -c` — Path to JSON configuration file
- `--source / -s` — Source project directories (array)
- `--target / -t` — Parent directory containing the `Ed-Fi-ODS` and `Ed-Fi-ODS-Implementation` repositories
- `--projectNames / -p` — Project name overrides applied to discovered projects in discovery order (array)
- `--defaultPluginTechVersion / -x` — Default plugin technology version
- `--core` — Deploy core artifacts
- `--suppressDelete` — Skip removal of existing extension artifacts
- `--accept-license / -a` — Required license acceptance flag
- `--suppressPrereleaseVersion` — Suppress pre-release version in paths
- `--additionalMssqlScriptsDirectory` — Extra SQL Server scripts directory
- `--additionalPostgresScriptsDirectory` — Extra PostgreSQL scripts directory

## Output

Deploys MetaEd artifacts into the target ODS/API repository structure. In source-scan
mode, also runs the MetaEd build pipeline first. Exits with code 0 on success, 1 on
failure, and logs duration.

## Business Logic

Two operating modes:

- **Source-scan mode** (`--source`/`--target`): scans source directories for MetaEd
projects, builds `MetaEdConfiguration`, runs the full generation pipeline, then
delegates to `metaed-odsapi-deploy` to copy artifacts into the destination repository
structure.
- **Config-based mode** (`--config`): uses the supplied `metaEdConfiguration` with a
pre-built `artifactDirectory` and runs only the deploy tasks — the build pipeline is
not executed.
42 changes: 42 additions & 0 deletions packages/metaed-odsapi-deploy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# metaed-odsapi-deploy

Library for deploying MetaEd-generated artifacts into the ODS/API repository layout.

## Input Configuration

Accepts `MetaEdConfiguration` plus deploy-specific options:

- `dataStandardVersion` — Determines output path structure
- `deployCore` — Whether to deploy core (standard) artifacts
- `suppressDelete` — Skip removal of existing extension artifact directories
- `additionalMssqlScriptsDirectory` — Extra SQL Server scripts to copy
- `additionalPostgresScriptsDirectory` — Extra PostgreSQL scripts to copy

## Output

Copies generated artifacts into the ODS/API file system structure:

- Core artifacts → `Ed-Fi-ODS/Application/EdFi.Ods.Standard/Standard/{version}/Artifacts/`
- Extension artifacts → `Ed-Fi-ODS-Implementation/Application/EdFi.Ods.Extensions.{name}/Versions/{version}/Standard/{dsVersion}/Artifacts/`

Sub-path mappings:

| Source Directory | Deploy Destination |
|---|---|
| ApiMetadata | Metadata |
| Database/SQLServer/ODS/Data | MsSql/Data/Ods |
| Database/SQLServer/ODS/Structure | MsSql/Structure/Ods |
| Database/PostgreSQL/ODS/Data | PgSql/Data/Ods |
| Database/PostgreSQL/ODS/Structure | PgSql/Structure/Ods |
| Interchange | Schemas |
| XSD | Schemas |

## Business Logic

Runs a sequence of deployment tasks in order: first verifies that required extension
projects exist, then removes old extension artifacts (unless suppressed), deploys core
artifacts (ODS/API ≥ 7.0 only), and deploys extension artifacts. Additional MSSQL and
PostgreSQL script directories are passed into the core/extension deploy steps rather than
copied as a separate standalone task. After deployment it refreshes csproj timestamps,
runs the legacy-directory check, and stops on the first failure.

24 changes: 13 additions & 11 deletions packages/metaed-plugin-edfi-api-catalog/README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,26 @@
# MetaEd Plugin: Ed-Fi API Catalog
# metaed-plugin-edfi-api-catalog

This plugin generates an Excel spreadsheet catalog of API resources and their properties from the MetaEd API schema.

## Overview

The API Catalog plugin reads the enhanced MetaEd model from the `metaed-plugin-edfi-api-schema` plugin and generates a comprehensive Excel spreadsheet listing all API resources, their properties, and associated metadata.

## Input Configuration

No plugin-specific configuration. Depends on API schema metadata produced by
`metaed-plugin-edfi-api-schema`.

## Dependencies

This plugin depends on:

- `@edfi/metaed-core`
- `@edfi/metaed-plugin-edfi-api-schema`

The plugin must be loaded after the `metaed-plugin-edfi-api-schema` plugin to access the enhanced API schema data.

## Output

The plugin generates a single Excel file (`Ed-Fi-API-Catalog.xlsx`) with two worksheets:
The plugin generates a single Excel file (`Documentation/Ed-Fi-API-Catalog/Ed-Fi-API-Catalog.xlsx`) with two worksheets:

### Resources Worksheet

Expand Down Expand Up @@ -45,14 +49,12 @@ Contains one row per property within each resource with the following columns:
- **Is Nullable**: Boolean indicating if the property can be null
- **Is Required**: Boolean indicating if the property is required

## Usage

This plugin is included in the default plugins list and will automatically generate the API catalog when MetaEd processes a data model.
## Business Logic

The generated Excel file can be found in the output directory under:
```
Documentation/Ed-Fi-API-Catalog/Ed-Fi-API-Catalog.xlsx
```
Walks the API schema resource definitions built by the upstream API schema plugin,
extracts resource and property metadata rows, and writes a multi-sheet XLSX workbook
suitable for documentation and review purposes. Both resources and descriptors are
included; reference properties use `dataType = 'reference'`.

## Implementation Details

Expand Down
26 changes: 26 additions & 0 deletions packages/metaed-plugin-edfi-api-schema/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# metaed-plugin-edfi-api-schema

MetaEd plugin that generates the DMS API schema JSON used by the Ed-Fi API.

## Input Configuration

Registers two optional configuration schemas:

- `educationOrganizationSecurableElements` — Configures security elements for
education organizations
- `educationOrganizationIdentitySecurableElements` — Configures identity security
elements for education organizations

## Output

Generates one JSON file per namespace:

- `ApiSchema/ApiSchema.json` (for core)
- `ApiSchema/ApiSchema-{projectExtension}.json` (for extensions)

## Business Logic

Builds the DMS API schema from MetaEd-enhanced namespace data through a series of
enhancers that derive resource schemas, document paths, and reference structures.
Serializes the result as pretty-printed JSON. Also exports many API-schema types and
enhancers consumed by downstream plugins (api-catalog, odsapi).
22 changes: 22 additions & 0 deletions packages/metaed-plugin-edfi-handbook/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# metaed-plugin-edfi-handbook

MetaEd plugin that generates the Ed-Fi Data Handbook in HTML and Excel formats.

## Input Configuration

No plugin-specific configuration. Relies on model data enriched by upstream enhancers.

## Output

Generates two artifacts:

- `Documentation/Ed-Fi-Handbook/Ed-Fi-Data-Handbook-Index.html` — Interactive HTML
single-page application for browsing handbook entries
- `Documentation/Ed-Fi-Handbook/Ed-Fi-Handbook.xlsx` — Excel workbook with the same
content in spreadsheet form

## Business Logic

Enhances model metadata into handbook entries (entity descriptions, properties,
relationships), then emits both an HTML SPA index and an Excel workbook. Entries are
sorted and formatted for documentation and stakeholder review.
25 changes: 25 additions & 0 deletions packages/metaed-plugin-edfi-ods-changequery-postgresql/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# metaed-plugin-edfi-ods-changequery-postgresql

MetaEd plugin that generates PostgreSQL-specific SQL for ODS change-query support.

## Input Configuration

No plugin-specific configuration. Depends on the shared `metaed-plugin-edfi-ods-changequery`
plugin for model enrichment.

## Output

Generates PostgreSQL-flavored change-query SQL scripts under
`Database/PostgreSQL/ODS/Structure/Changes/`, including:

- Tracked-delete table and schema creation
- Change-version sequence
- Change-tracking triggers
- Change-version indexes
- Indirect update cascade triggers

## Business Logic

Applies PostgreSQL-specific enhancers and generators to the change-query model built by
the common change-query plugin. Emits DDL using PostgreSQL syntax for sequences,
triggers, and indexes.
25 changes: 25 additions & 0 deletions packages/metaed-plugin-edfi-ods-changequery-sqlserver/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# metaed-plugin-edfi-ods-changequery-sqlserver

MetaEd plugin that generates SQL Server-specific SQL for ODS change-query support.

## Input Configuration

No plugin-specific configuration. Depends on the shared `metaed-plugin-edfi-ods-changequery`
plugin for model enrichment.

## Output

Generates SQL Server-flavored change-query SQL scripts under
`Database/SQLServer/ODS/Structure/Changes/`, including:

- Tracked-delete table and schema creation
- Change-version sequence
- Change-tracking triggers
- Change-version indexes
- Indirect update cascade triggers

## Business Logic

Applies SQL Server-specific enhancers and generators to the change-query model built by
the common change-query plugin. Emits DDL using SQL Server syntax for sequences,
triggers, and indexes.
Loading
Loading