Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ DOMAIN=platform.example.com
# GitHub
GIT_TOKEN=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

# Platform Admin
ADMIN_USERNAME=ibl_admin
# Platform Admin.
# NOTE: `ibl_admin` is reserved for system use — pick a different name.
ADMIN_USERNAME=platform_admin
ADMIN_EMAIL=admin@example.com
ADMIN_PASSWORD=change-me-min-8-chars

Expand Down
4 changes: 3 additions & 1 deletion .env.provision.example
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,10 @@ SSH_KEY_METHOD=generate # generate | existing_file | aws_keypair
# SSH_KEY_NAME=my-existing-keypair-name

# --- Compute (optional, defaults shown) ---
# NOTE: t3.2xlarge has 32 GB RAM. If you enable AI features in the setup
# step, 64 GB (e.g. m5.4xlarge or r5.2xlarge) is strongly recommended.
INSTANCE_TYPE=t3.2xlarge
VOLUME_SIZE=50 # min 20 GB
VOLUME_SIZE=100 # min 100 GB
VOLUME_TYPE=gp3 # gp2 | gp3 | io1

# --- Network (optional) ---
Expand Down
47 changes: 42 additions & 5 deletions .env.setup.example
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,45 @@
# REQUIRED FOR BOTH MODES
# ============================================================================

# AWS credentials (passed to ansible as extra_vars; embedded in /ibl/config.yml)
# ============================================================================
# AWS credentials — TWO distinct sets serve two distinct purposes
# ============================================================================

# --- S3 access (customer-created, post-provision) ---
# Read / write on the dm-media, dm-static, and backups buckets Terraform
# created in YOUR OWN AWS account. Written to the root of /ibl/config.yml
# by the `ibl_platform` role; consumed by DM / edX containers at runtime.
#
# After `iblai infra provision-env` finishes it prints the exact S3-only
# IAM policy JSON + three `aws iam` commands to mint a minimum-privilege
# user. Paste that user's AccessKeyId + SecretAccessKey below.
#
# Do NOT reuse your provisioning admin keys here.
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# --- ECR pulls (provided by ibl.ai) ---
# `docker login` against IBL's container registry. Written to
# ~/.aws/credentials [default] on the host by the `awscli` role so
# `aws ecr get-login-password` picks them up without env-var overrides.
#
# For ECR images, use AWS credentials provided by ibl.ai — or contact us
# at https://ibl.ai/contact
#
# If you leave these blank, the S3 keys above fall through to ECR (the
# old single-key-set behavior — works only when one key has both scopes).
ECR_AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
ECR_AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
# ECR_AWS_DEFAULT_REGION=us-east-1 # defaults to AWS_DEFAULT_REGION

# GitHub PAT — needs read on iblai/iblai-cli-ops + iblai/iblai-prod-images
# (or your overrides via GITHUB_ORG / CLI_OPS_REPO / PROD_IMAGES_REPO below)
GIT_TOKEN=<your github personal access token — ghp_… or github_pat_…>

# Platform admin (created on the LMS + DM)
ADMIN_USERNAME=ibl_admin
# Platform admin (created on the LMS + DM).
# NOTE: `ibl_admin` is reserved for system use (owns SPA OAuth records).
# Pick any other username — e.g. `platform_admin`, your name, etc.
ADMIN_USERNAME=platform_admin
ADMIN_EMAIL=admin@example.com
ADMIN_PASSWORD=change-me-min-8-chars

Expand All @@ -43,8 +72,16 @@ ADMIN_PASSWORD=change-me-min-8-chars
# CLI_OPS_RELEASE_TAG=3.19.0
# PROD_IMAGES_TAG=main

# Platform identity (used by SSO roles to derive backend_name + platform_key)
# PLATFORM_NAME=main
# Platform identity. Used by SSO roles to derive backend_name + platform_key,
# AND by ibl_tenant_platform to launch a tenant Platform via run_launch_steps
# (Platform + admin User + UserPlatformLink) when set to a non-default value.
# - Leave unset / blank → defaults to 'main' (system default tenant
# created by the platform itself, no extra launch)
# - PLATFORM_NAME=main → REJECTED ('main' is reserved as an explicit input)
# - PLATFORM_NAME=acme → launches an 'acme' tenant with admin user
# 'acmeadmin@<base_domain>' (password printed
# one-time at the end of setup)
# PLATFORM_NAME=acme

# Feature toggles
# ENABLE_AI=true
Expand Down
42 changes: 41 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,45 @@
# Changelog

## [1.11.0] — 2026-05-20

### Added
- **Post-provision S3 IAM helper** (`src/iblai_infra/runtime_iam.py`). After `provision` / `provision-env` succeeds, the CLI prints the exact **S3-only** minimum-privilege IAM policy JSON the operator needs to attach to a scoped runtime user in their own AWS account — and writes the same JSON to `<workspace>/runtime-iam-policy.json` so it can be piped into `aws iam put-user-policy --policy-document file://...`. The policy scopes S3 to the literal bucket ARNs Terraform just created (no wildcards, no `s3:*`, no bucket-policy / lifecycle / encryption mutation). Skipped automatically for `DeploymentType.CALL` (no S3 buckets).
- **Three copy-paste `aws iam` commands** in the post-provision output (`create-user`, `put-user-policy`, `create-access-key`) using `<project>-<env>-s3-runtime` as the user name — operator pastes the resulting `AccessKeyId` + `SecretAccessKey` directly into `.env.setup`.
- **README sub-section** under "Provision infrastructure" documenting the S3 IAM step + the scope table, plus a credential-set table clarifying that **ECR pull credentials are a separate IBL-provided handoff**, not part of this flow.

### Changed
- **Two-credential split end-to-end.** Previously a single `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` from `.env.setup` had to serve both ECR auth (IBL's account) and S3 access (customer's account) — works only when one key happens to have both scopes. Now:
- `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` carry the **S3** keys (customer-created post-provision) and are written to the root of `/ibl/config.yml` by a new task in the `ibl_platform` role; consumed by DM / edX at runtime via iblai-cli-ops templating.
- New `ECR_AWS_ACCESS_KEY_ID` / `ECR_AWS_SECRET_ACCESS_KEY` (optional `ECR_AWS_DEFAULT_REGION`) carry the **ECR** keys (ibl.ai-provided). The `awscli` role writes these to `~/.aws/credentials` `[default]` profile on the host so `aws ecr get-login-password` finds them without env-var overrides anywhere.
- The four `Login to ECR` tasks across `ibl_spa`, `ibl_launch_services`, `ibl_platform`, `ibl_service_update` no longer set `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` env-vars at command time — they rely on the default profile populated by `awscli`.
- `SetupConfig` gains `ecr_aws_access_key_id`, `ecr_aws_secret_access_key`, `ecr_aws_default_region` (all optional). Secret is `Field(exclude=True)`.
- `runner.py::_build_extra_vars` passes both sets as separate ansible extra-vars. When `ECR_AWS_*` is empty, the S3 keys fall through to the ECR slot — backwards-compatible with single-key-set deployments.
- **`.env.setup.example`** now shows two clearly-labeled `AWS_*` blocks (S3 + ECR) with usage / destination spelled out inline.
- **Section 4 of the README** (non-interactive `.env` flow) renumbered as a 3-step sequence (provision → mint S3 user → setup) so the IAM step isn't missed.
- **README credential-set table** under "Provision infrastructure" gains a "Lives in" column documenting `/ibl/config.yml` root vs `~/.aws/credentials [default]` so the operator knows exactly where each set lands on the server.

## [1.10.0] — 2026-05-20

### Added
- **`ibl_tenant_platform` ansible role** — launches a tenant `Platform` (Platform + admin User + UserPlatformLink) via `run_launch_steps` when `PLATFORM_NAME` is set to anything other than `main`. NOT a raw `Platform.objects.create()` — the state machine fires every after_launch signal (default apps, edX hooks, UserPlatformLink flags). Wired into both `playbook.yml` (setup / setup-env) and `launch_playbook.yml` (launch / launch-env). Skips + logs on re-runs when the tenant already exists. Also writes `PLATFORM_NAME=<KEY>` (uppercase) at the root of `/ibl/config.yml` and enforces `Platform.show_paywall=False` + `Platform.is_advertising=False` as defense in depth. Surfaces the generated admin password via the `IBLAI_FIXTURE_OUTPUT` pipeline — printed once after the Rich Live display tears down, never persisted to disk.
- **Microsoft SSO writes `IBL_SPA.AUTH`** — `microsoft_sso_config` now also patches `EXTERNAL_IDP_LOGOUT_URL` and `IBL_DIRECT_SSO_URL` (using `microsoft_sso_tenant_id`, falling back to `common`), then restarts the Auth + Mentor SPAs so the new auth flow takes effect.
- **`INSTANCE_RAM_GB` helper + 32 GB memory warning** — non-blocking heads-up suggesting 64 GB (e.g. `m5.4xlarge` / `r5.2xlarge`) when the operator picks a 32 GB instance. Always shown in the interactive provision wizard and `provision-env`; conditional in `launch` / `launch-env` (only when AI is enabled).
- **Final `ibl global-proxy reload`** added as `post_tasks` in both `playbook.yml` and `launch_playbook.yml`, so any nginx state touched by SSO roles (edX restarts in `google_sso_config` / `microsoft_sso_config`) is reloaded before the playbook exits.
- **`RESERVED_ADMIN_USERNAMES` + `RESERVED_PLATFORM_NAMES`** — `models.py` constants, surfaced via `is_reserved_admin_username()` and `is_reserved_platform_name()` helpers and an `InfraConfig` model_validator.

### Changed
- **Stripe billing UI off by default** — `IBL_SPA.MENTOR.STRIPE_ENABLED=false` and `IBL_SPA.MENTOR.ENABLE_ADVERTISING=false` are now written unconditionally by `ibl_spa` (fresh installs) and `ibl_launch_services` (AMI launches). **Behavior change:** Stripe-using deployments must explicitly flip `IBL_SPA.MENTOR.STRIPE_ENABLED` back to `'true'` post-setup. The previous "always on" SPA flag surfaced billing UI even when Stripe wasn't actually configured.
- **100 GB minimum root volume for single / multi server** — enforced by Pydantic (`InfraConfig` model_validator gated on `DeploymentType.SINGLE`, plus `MultiServerConfig.validate_volume_sizes`) and matching interactive + CLI + .env input checks. **Behavior change:** values below 100 GB are now rejected upfront. Default `ComputeConfig.volume_size` bumped 50 → 100. Call-server unchanged (LiveKit only needs ~40 GB).
- **`ADMIN_USERNAME=ibl_admin` rejected at every input layer** — reserved for the SPA OAuth Application owner the platform itself maintains. New default suggestion is `platform_admin`. Interactive prompts, `.env` parsers, and `--admin-username` flag all reject `ibl_admin` with a clear reserved-name error. **Behavior change:** scripted deploys passing `ADMIN_USERNAME=ibl_admin` must rename.
- **`PLATFORM_NAME=main` rejected as an explicit input** — unset / blank silently resolves to `main` (preserving SSO `backend_name=main-oauth2` and skipping the tenant launcher). **Behavior change:** scripted deploys passing `PLATFORM_NAME=main` should drop the line.
- **README** — refreshed against current playbook (16 roles, phase-grouped table), three deployment topologies, sizing guidance, tenant launcher, reserved-name rules. -50 lines net.

### Removed
- **All references to a specific canonical-client name** from comments, docstrings, prompt instructions, error hints, and example .env files. Placeholders: `<client>` for monorepo org names, `acme` for tenant-key examples.

### Fixed
- **Slow `_test_ssh()` retry-path tests** — five tests in `tests/ansible/test_runner.py` exercise the SSH-retry exhaust path (10 retries × 15 s sleep). They now mock `time.sleep` alongside the existing `subprocess.run` mock, cutting ~11 minutes off the full suite. Test count: 562 passing in ~1.3 s.

## [1.7.0] — 2026-05-06

### Added
Expand Down Expand Up @@ -38,7 +78,7 @@
## [1.5.0] — 2026-04-30

### Added
- **Monorepo subdirectory installs** — `--cli-ops-repo` / `--prod-images-repo` (and the matching setup prompts) now accept a `repo/subdir` path, e.g. `kaplan-iblai-infra-ops/kaplan-iblai-prod-images`. The ansible role appends `&subdirectory=<subdir>` to the install URL so a single client monorepo can host both `iblai-cli-ops` and the prod-images package
- **Monorepo subdirectory installs** — `--cli-ops-repo` / `--prod-images-repo` (and the matching setup prompts) now accept a `repo/subdir` path, e.g. `<client>-iblai-infra-ops/<client>-iblai-prod-images`. The ansible role appends `&subdirectory=<subdir>` to the install URL so a single client monorepo can host both `iblai-cli-ops` and the prod-images package
- **`parse_repo_path()` helper** in `models.py` — splits operator input into `(repo, subdir)`. Bare `iblai-cli-ops` keeps the canonical behavior; subdir-form unlocks per-client monorepo deployments
- **`cli_ops_subdir` / `prod_images_subdir` extra-vars** passed through `AnsibleRunner` to the `ibl_cli_ops` role (single-server + call-server templates)

Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ Sets `state.provider = "launch"` to distinguish from interactive provisioning.

`iblai infra setup-env [<name>] -f .env` — non-interactive Ansible bootstrap from a `.env` file. Single-server only (multi/call rejected upstream). Two modes:

- **Provisioned-name:** `setup-env kapsix -f .env` — loads `ProjectState`, derives `target_host` / `ssh_private_key_path` / `base_domain` / `aws_default_region` from it. `.env` only carries credentials, image tags, admin user, optional integrations.
- **Provisioned-name:** `setup-env <name> -f .env` — loads `ProjectState`, derives `target_host` / `ssh_private_key_path` / `base_domain` / `aws_default_region` from it. `.env` only carries credentials, image tags, admin user, optional integrations.
- **Free-standing:** `setup-env -f .env` (no name) — builds a synthetic `ProjectState` with `provider="bootstrap"` (matching `_run_setup_interactive`). `.env` must include `PROJECT_NAME`, `TARGET_HOST`, `SSH_PRIVATE_KEY_PATH`, `BASE_DOMAIN`.

**Schema** (`.env.setup.example` is the source of truth). Always required: AWS keys, `GIT_TOKEN` (or `GIT_ACCESS_TOKEN`), `ADMIN_USERNAME`/`ADMIN_EMAIL`/`ADMIN_PASSWORD`. Free-standing additionally needs the four "where to deploy" fields. Optional integrations follow the same trigger pattern as `iblai infra launch` — SMTP enabled when `SMTP_HOST` set, Stripe when `STRIPE_SECRET_KEY` set, Google SSO when `GOOGLE_SSO_CLIENT_ID` set, Microsoft SSO when `MICROSOFT_SSO_CLIENT_ID` set.
Expand Down
Loading