diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 0000000..d92c9ee --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,85 @@ +# Contributing to SPSDBMove + +Thanks for your interest in improving **SPSDBMove**! This document explains how +to file issues, propose changes, and get your pull request merged. + +## Code of Conduct + +This project adopts the [Contributor Covenant](../CODE_OF_CONDUCT.md). +By participating you agree to uphold it. + +## How to report a bug + +1. Search [existing issues](https://github.com/luigilink/SPSDBMove/issues) first + to avoid duplicates. +2. If nothing matches, open a new issue using the + [Bug Report template](https://github.com/luigilink/SPSDBMove/issues/new?template=1_bug_report.yml). +3. Include: + - Exact command line you ran (redact credentials). + - SharePoint version (2016 / 2019 / Subscription Edition). + - SQL Server version of both the source and destination instances. + - PowerShell version (`$PSVersionTable.PSVersion`). + - Full error / log output. + +## How to request a feature + +Use the +[Feature Request template](https://github.com/luigilink/SPSDBMove/issues/new?template=2_feature_request.yml) +and describe the migration scenario the feature would unlock. + +## Development setup + +```powershell +# Required modules +Install-Module -Name SqlServer -Scope CurrentUser -Force +Install-Module -Name Pester -MinimumVersion 5.3.0 -Scope CurrentUser -Force +Install-Module -Name PSScriptAnalyzer -Scope CurrentUser -Force + +# Clone +git clone https://github.com/luigilink/SPSDBMove.git +cd SPSDBMove +``` + +Run the test suite before pushing: + +```powershell +Invoke-Pester -Path .\tests -Output Detailed +Invoke-ScriptAnalyzer -Path .\scripts -Recurse -Severity Error,Warning +``` + +## Pull request workflow + +1. Fork the repository and create a topic branch from `main`: + `git checkout -b feature/short-description`. +2. Make focused commits. Keep one logical change per commit when possible. +3. **Update `CHANGELOG.md`** under an `## [Unreleased]` section describing the + change. Entries are mandatory for every PR. +4. Make sure `pester` and `code-quality` CI jobs are green. +5. Open the PR against `main` using the + [PR template](./PULL_REQUEST_TEMPLATE.md). Link the issue it resolves + (`Fixes #123`). +6. A maintainer will review. Address feedback by pushing additional commits to + the same branch (no force-push during review, please). + +## Coding guidelines + +- Target **PowerShell 7.0+** (`pwsh`). The Copy phase uses + `ForEach-Object -Parallel`, which is not available on Windows + PowerShell 5.1. +- Every public function uses `[CmdletBinding()]`, comment-based help, and + parameter validation (`[ValidateNotNullOrEmpty()]`, `[ValidateSet()]`, etc.). +- Destructive operations (`BACKUP`, `RESTORE`, file deletes) must support + `-WhatIf` / `-Confirm` via `SupportsShouldProcess`. +- Never log credentials, connection strings containing passwords, or full file + paths that include secrets. +- Use approved PowerShell verbs (`Get-Verb`). Run PSScriptAnalyzer locally + before pushing. + +## Releasing (maintainers) + +1. Move the `## [Unreleased]` block in `CHANGELOG.md` to a new + `## [x.y.z] - YYYY-MM-DD` section. +2. Mirror the same block at the top of `RELEASE-NOTES.md`. +3. Commit and tag: `git tag vX.Y.Z && git push --tags`. +4. The `release.yml` workflow creates the GitHub Release and attaches the + packaged zip. diff --git a/.github/ISSUE_TEMPLATE/1_bug_report.yml b/.github/ISSUE_TEMPLATE/1_bug_report.yml new file mode 100644 index 0000000..d784b1c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/1_bug_report.yml @@ -0,0 +1,62 @@ +name: Bug Report +description: File a bug report. +title: "[Bug]: " +labels: ["bug", "triage"] +projects: ["luigilink/SPSDBMove"] +assignees: + - luigilink +body: + - type: markdown + attributes: + value: | + Thanks for taking the time to fill out this bug report! + - type: input + id: contact + attributes: + label: Contact Details (Optional) + description: How can we get in touch with you if we need more info? + placeholder: ex. email@example.com + validations: + required: false + - type: textarea + id: what-happened + attributes: + label: What happened? + description: Also tell us, what did you expect to happen? + placeholder: Tell us what you see! + value: "A bug happened!" + validations: + required: true + - type: dropdown + id: version + attributes: + label: Version + description: What version of our software are you running? + options: + - 1.0.x + - Other + default: 0 + validations: + required: true + - type: dropdown + id: browsers + attributes: + label: What version of PowerShell are you running? + multiple: true + options: + - 5.x + - 7.x + - type: textarea + id: logs + attributes: + label: Relevant log output + description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks. + render: shell + - type: checkboxes + id: terms + attributes: + label: Code of Conduct + description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/luigilink/SPSDBMove/blob/main/CODE_OF_CONDUCT.md). + options: + - label: I agree to follow this project's Code of Conduct + required: true diff --git a/.github/ISSUE_TEMPLATE/2_feature_request.yml b/.github/ISSUE_TEMPLATE/2_feature_request.yml new file mode 100644 index 0000000..1bc4f4c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/2_feature_request.yml @@ -0,0 +1,55 @@ +name: Feature Request +description: Suggest a new feature or improvement. +title: "[Feature Request]: " +labels: ["enhancement", "triage"] +projects: ["luigilink/SPSDBMove"] +assignees: + - luigilink +body: + - type: markdown + attributes: + value: | + Thanks for suggesting a feature or improvement! Please provide as much detail as possible. + - type: input + id: contact + attributes: + label: Contact Details (Optional) + description: How can we get in touch with you if we need more info? + placeholder: ex. email@example.com + validations: + required: false + - type: textarea + id: feature-description + attributes: + label: Describe the feature you'd like to see + description: What do you want to achieve? What problem does it solve? + placeholder: Describe the feature or improvement you're suggesting. + validations: + required: true + - type: textarea + id: potential-solutions + attributes: + label: Potential solutions or alternatives + description: If you have any ideas on how to implement this feature, feel free to share! + placeholder: Suggest how this could be implemented or any alternative ideas. + validations: + required: false + - type: dropdown + id: priority + attributes: + label: Priority + description: How important is this feature to you? + options: + - Low + - Medium + - High + validations: + required: true + - type: checkboxes + id: terms + attributes: + label: Code of Conduct + description: By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/luigilink/SPSDBMove/blob/main/CODE_OF_CONDUCT.md). + options: + - label: I agree to follow this project's Code of Conduct + required: true diff --git a/.github/ISSUE_TEMPLATE/3_documentation_request.yml b/.github/ISSUE_TEMPLATE/3_documentation_request.yml new file mode 100644 index 0000000..8aa9841 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/3_documentation_request.yml @@ -0,0 +1,39 @@ +name: Documentation Request +description: Suggest improvements or request missing documentation. +title: "[Documentation]: " +labels: ["documentation", "triage"] +projects: ["luigilink/SPSDBMove"] +assignees: + - luigilink +body: + - type: markdown + attributes: + value: | + Thank you for suggesting improvements to the documentation. Please be as detailed as possible. + + - type: textarea + id: doc-area + attributes: + label: Area of Documentation + description: Which part of the documentation needs to be improved or added? + placeholder: e.g., API usage, installation guide, etc. + validations: + required: true + + - type: textarea + id: details + attributes: + label: Details + description: Describe the improvements or missing information. + placeholder: Describe the changes you'd like to see. + validations: + required: true + + - type: checkboxes + id: terms + attributes: + label: Code of Conduct + description: By submitting this request, you agree to follow our [Code of Conduct](https://github.com/luigilink/SPSDBMove/blob/main/CODE_OF_CONDUCT.md). + options: + - label: I agree to follow this project's Code of Conduct + required: true diff --git a/.github/ISSUE_TEMPLATE/4_improvement_request.yml b/.github/ISSUE_TEMPLATE/4_improvement_request.yml new file mode 100644 index 0000000..82056db --- /dev/null +++ b/.github/ISSUE_TEMPLATE/4_improvement_request.yml @@ -0,0 +1,39 @@ +name: Refactor/Improvement Request +description: Request improvements or code refactoring. +title: "[Improvement]: " +labels: ["enhancement", "refactor", "triage"] +projects: ["luigilink/SPSDBMove"] +assignees: + - luigilink +body: + - type: markdown + attributes: + value: | + Thanks for suggesting a refactor or improvement to the codebase. Please provide details below. + + - type: textarea + id: improvement-description + attributes: + label: What improvement would you like to see? + description: Describe the refactor, optimization, or code improvement. + placeholder: Explain the change you are suggesting. + validations: + required: true + + - type: textarea + id: reasons + attributes: + label: Why is this improvement needed? + description: Describe the benefits of this improvement. + placeholder: Why does this matter? Will it improve performance, readability, etc.? + validations: + required: true + + - type: checkboxes + id: terms + attributes: + label: Code of Conduct + description: By submitting this request, you agree to follow our [Code of Conduct](https://github.com/luigilink/SPSDBMove/blob/main/CODE_OF_CONDUCT.md). + options: + - label: I agree to follow this project's Code of Conduct + required: true diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..df7ce38 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,5 @@ +blank_issues_enabled: false +contact_links: + - name: "SPSDBMove Discussions" + url: https://github.com/luigilink/SPSDBMove/discussions + about: "To engage with the community and maintainers of SPSDBMove, please visit the discussions group. There, you can share feedback, ask questions, and collaborate with others working on SPSDBMove." diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..69873e3 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,51 @@ + + +#### Pull Request (PR) description + + + +#### This Pull Request (PR) fixes the following issues + + + +#### Task list + + + +- [ ] Added an entry to the change log under the Unreleased section of the + file CHANGELOG.md. Entry should say what was changed and how that + affects users (if applicable), and reference the issue being resolved + (if applicable). +- [ ] Added/updated documentation and descriptions where appropriate? +- [ ] New/changed code adheres to [Style Guidelines]? diff --git a/.github/workflows/pester.yml b/.github/workflows/pester.yml new file mode 100644 index 0000000..01c9a80 --- /dev/null +++ b/.github/workflows/pester.yml @@ -0,0 +1,83 @@ +# This is the SPSDBMove CI Pester workflow to run tests on pull requests + +name: SPSDBMove CI Pester Tests + +on: + pull_request: + branches: + - main + paths: + - 'scripts/**' + - 'tests/**' + +jobs: + test: + name: Run Pester Tests + runs-on: windows-latest + + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Install Pester + shell: pwsh + run: | + Install-Module -Name Pester -MinimumVersion 5.3.0 -Force -SkipPublisherCheck -Scope CurrentUser + Import-Module Pester + + - name: Run Pester Tests + shell: pwsh + run: | + $config = New-PesterConfiguration + $config.Run.Path = './tests' + $config.Run.Exit = $true + $config.TestResult.Enabled = $true + $config.TestResult.OutputPath = './test-results.xml' + $config.Output.Verbosity = 'Detailed' + + Invoke-Pester -Configuration $config + + - name: Upload test results + uses: actions/upload-artifact@v4 + if: always() + with: + name: test-results + path: test-results.xml + + - name: Publish Test Results + uses: EnricoMi/publish-unit-test-result-action/composite@v2 + if: always() + with: + files: | + test-results.xml + check_name: Pester Test Results + + code-quality: + name: Code Quality Check + runs-on: windows-latest + + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Install PSScriptAnalyzer + shell: pwsh + run: | + Install-Module -Name PSScriptAnalyzer -Force -SkipPublisherCheck -Scope CurrentUser + + - name: Run PSScriptAnalyzer + shell: pwsh + run: | + $scriptFiles = Get-ChildItem -Path ./scripts -Include '*.ps1', '*.psm1' -Recurse + + $results = $scriptFiles | ForEach-Object { + Invoke-ScriptAnalyzer -Path $_.FullName -Severity Error,Warning + } + + if ($results) { + $results | Format-Table -AutoSize + Write-Error "PSScriptAnalyzer found issues" + exit 1 + } else { + Write-Host "No issues found by PSScriptAnalyzer" + } diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..cab4e66 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,37 @@ +# This is the SPSDBMove CI Release workflow to create release from tag push + +name: SPSDBMove CI Release + +on: + push: + # Sequence of patterns matched against refs/tags + tags: + - "v*" # Push events to matching v*, i.e. v1.0, v20.15.10 + +jobs: + build: + runs-on: ubuntu-latest + steps: + # Checkout code + - name: Checkout code + id: checkout_code + uses: actions/checkout@v4 + # Create a ZIP file with project name and tag version + - name: Create ZIP file of scripts + run: | + zip_name="SPSDBMove-${{ github.ref_name }}.zip" + zip -r "$zip_name" scripts/ README.md LICENSE CHANGELOG.md RELEASE-NOTES.md + shell: bash + # Create Release with tag version + - name: Create GitHub Release + id: create_release + uses: softprops/action-gh-release@v2 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + draft: false + prerelease: false + body_path: RELEASE-NOTES.md + files: | + LICENSE + SPSDBMove-${{ github.ref_name }}.zip diff --git a/.github/workflows/wiki.yml b/.github/workflows/wiki.yml new file mode 100644 index 0000000..3f1b641 --- /dev/null +++ b/.github/workflows/wiki.yml @@ -0,0 +1,43 @@ +# This is the SPSDBMove CI Wiki workflow to udpate wiki documentation + +name: SPSDBMove CI Wiki + +on: + push: + branches: + - main + paths: [wiki/**, .github/workflows/wiki.yml] + +concurrency: + group: wiki + cancel-in-progress: true + +permissions: + contents: write + +jobs: + wiki: + name: Publish to GitHub Wiki + runs-on: ubuntu-latest + steps: + - name: Checkout Code + uses: actions/checkout@v4 + with: + repository: ${{github.repository}} + path: ${{github.repository}} + + - name: Checkout Wiki + uses: actions/checkout@v4 + with: + repository: ${{github.repository}}.wiki + path: ${{github.repository}}.wiki + + - name: Push to wiki + run: | + set -e + cd $GITHUB_WORKSPACE/${{github.repository}}.wiki + cp -r $GITHUB_WORKSPACE/${{github.repository}}/wiki/* . + git config --local user.email "[email protected]" + git config --local user.name "GitHub Action" + git add . + git diff-index --quiet HEAD || git commit -m "action: wiki sync" && git push diff --git a/.gitignore b/.gitignore index d5a18de..fd0affa 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,9 @@ ## ## Get latest from https://github.com/github/gitignore/blob/main/VisualStudio.gitignore +# SPSDBMove runtime log folder (created next to the script). +scripts/Logs/ + # User-specific files *.rsuser *.suo diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..f12c13e --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,63 @@ +# Change log for SPSDBMove + +The format is based on and uses the types of changes according to [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [1.0.0] - 2026-06-01 + +First tagged release of **SPSDBMove**. Requires **PowerShell 7.0+**. + +### Added + +- `scripts/SPSDBMove.ps1` — a full **backup + copy + restore** orchestrator: + - `-Action All | Backup | Copy | Restore` phase selector. + - `[CmdletBinding(SupportsShouldProcess)]`, comment-based help, `-WhatIf`. + - Source-side `BACKUP DATABASE ... WITH COPY_ONLY, COMPRESSION, CHECKSUM, + INIT` writing to the per-database `\\FULL\` layout. + - Destination-side `RESTORE DATABASE` with automatic `RESTORE FILELISTONLY` + logical-to-physical remapping and optional `WITH REPLACE`. + - Parallel copy using PowerShell 7+ `ForEach-Object -Parallel` with a + configurable `ThrottleLimit` and `-SkipExisting`. + - `-SqlCredential` for SQL authentication; Windows integrated auth by + default. + - Configuration-driven CLI: the entire job (instances, backup roots, + database list, throttling, restore options) lives in a JSON file + passed via `-ConfigPath`. The CLI surface is limited to `-ConfigPath`, + `-Action`, `-SqlCredential`, `-SkipExisting`, plus `-WhatIf` / + `-Confirm`. Unknown JSON keys are rejected so typos surface + immediately. + - `BACKUP` / `RESTORE` statements run with `QueryTimeout = 0` (no + timeout). Multi-TB SharePoint databases can take many hours; any + arbitrary cap would be a footgun, so SQL Server is left in charge of + the operation lifetime. + - Per-database `DestinationName` overrides are the only way to rename a + database on restore; the JSON schema is intentionally lean. + - SQL Server system databases (`master`, `model`, `msdb`, `tempdb`) are + always excluded from every phase, regardless of what the JSON + `Databases` list contains; a warning is logged when one is filtered + out. + - `ExcludeDatabases` JSON key: array of names (or `{Name}` objects) that + are dropped from the effective list before `Backup`, `Copy`, and + `Restore` run. Lets operators keep noisy databases in the main + `Databases` list while skipping them for a specific run. + - Sample config shipped at `scripts/SPSDBMove.sample.json`. + - Automatic per-run log file written next to the script under + `Logs\SPSDBMove.log`. +- `tests/SPSDBMove.Tests.ps1` — Pester 5 smoke tests covering script + parsing, declared parameters, PSScriptAnalyzer cleanliness, and + repository structure. +- `wiki/` folder with `Home.md`, `Getting-Started.md`, `Configuration.md`, + `Usage.md`, published to GitHub Wiki via `.github/workflows/wiki.yml`. +- `README.md` with badges, quick-start examples, CLI surface table, and + links to the wiki. +- `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1) with a real reporting + channel (GitHub private security advisory). +- `.github/CONTRIBUTING.md` — contribution guide. +- `.github/PULL_REQUEST_TEMPLATE.md` and `.github/ISSUE_TEMPLATE/*` for + bug reports, feature requests, documentation requests, and improvement + requests. +- `.github/workflows/pester.yml` — Pester + PSScriptAnalyzer CI on Linux, + macOS, and Windows. +- `.github/workflows/release.yml` — tag-triggered (`v*`) GitHub Release + job that publishes a ZIP containing `scripts/`, `README.md`, `LICENSE`, + `CHANGELOG.md`, and `RELEASE-NOTES.md`. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..44c6920 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,133 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our +community a harassment-free experience for everyone, regardless of age, body +size, visible or invisible disability, ethnicity, sex characteristics, gender +identity and expression, level of experience, education, socio-economic status, +nationality, personal appearance, race, caste, color, religion, or sexual +identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, +diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +- Focusing on what is best not just for us as individuals, but for the overall + community + +Examples of unacceptable behavior include: + +- The use of sexualized language or imagery, and sexual attention or advances of + any kind +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or email address, + without their explicit permission +- Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of +acceptable behavior and will take appropriate and fair corrective action in +response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject +comments, commits, code, wiki edits, issues, and other contributions that are +not aligned to this Code of Conduct, and will communicate reasons for moderation +decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when +an individual is officially representing the community in public spaces. +Examples of representing our community include using an official email address, +posting via an official social media account, or acting as an appointed +representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported to the community leaders responsible for enforcement by opening a +private security advisory at +. +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the +reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining +the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed +unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing +clarity around the nature of the violation and an explanation of why the +behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series of +actions. + +**Consequence**: A warning with consequences for continued behavior. No +interaction with the people involved, including unsolicited interaction with +those enforcing the Code of Conduct, for a specified period of time. This +includes avoiding interactions in community spaces as well as external channels +like social media. Violating these terms may lead to a temporary or permanent +ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including +sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No public or +private interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, is allowed during this period. +Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behavior, harassment of an +individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within the +community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], +version 2.1, available at +[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]. + +Community Impact Guidelines were inspired by +[Mozilla's code of conduct enforcement ladder][Mozilla CoC]. + +For answers to common questions about this code of conduct, see the FAQ at +[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at +[https://www.contributor-covenant.org/translations][translations]. + +[homepage]: https://www.contributor-covenant.org +[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html +[Mozilla CoC]: https://github.com/mozilla/diversity +[FAQ]: https://www.contributor-covenant.org/faq +[translations]: https://www.contributor-covenant.org/translations diff --git a/README.md b/README.md index af47bc7..019c017 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,119 @@ # SPSDBMove -Backup, copy, and restore SharePoint SQL databases across SQL instances — for SP2019/SP2016 → SE migrations. + +[![Pester](https://github.com/luigilink/SPSDBMove/actions/workflows/pester.yml/badge.svg?branch=main)](https://github.com/luigilink/SPSDBMove/actions/workflows/pester.yml) +[![Release](https://github.com/luigilink/SPSDBMove/actions/workflows/release.yml/badge.svg)](https://github.com/luigilink/SPSDBMove/actions/workflows/release.yml) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) +[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md) +[![PowerShell 7.0+](https://img.shields.io/badge/PowerShell-7.0%2B-blue.svg)](https://learn.microsoft.com/powershell/) + +Backup, copy, and restore **SharePoint SQL databases** across SQL Server +instances — designed for two common scenarios: + +- **SharePoint Server 2016 / 2019 → Subscription Edition** migrations. +- **PROD → PRE-PROD** content refreshes. + +`SPSDBMove` is a single PowerShell script that orchestrates the three phases +end-to-end (or any phase individually): + +1. **Backup** — `BACKUP DATABASE ... WITH COPY_ONLY, COMPRESSION, CHECKSUM` on + the source instance, written into a per-database folder layout + (`\\FULL\_.bak`). +2. **Copy** — parallel file copy of the latest `.bak` per database from the + source share to the destination share. +3. **Restore** — `RESTORE DATABASE ... WITH REPLACE, RECOVERY` on the + destination instance, with automatic logical-to-physical file remapping. + +Companion project: [luigilink/SPSUpdate](https://github.com/luigilink/SPSUpdate). + +--- + +## Requirements + +| Component | Version | +|---|---| +| PowerShell | **7.0 or later** (the Copy phase uses `ForEach-Object -Parallel`) | +| `SqlServer` module | 21.1.18256 or later | +| Source SQL Server | 2016 / 2017 / 2019 / 2022 | +| Destination SQL Server | 2019 / 2022 (SharePoint SE compatible) | +| Account rights | `dbcreator` + `BACKUP DATABASE` on source; `dbcreator` on destination; read/write on both backup shares | + +Install the SQL module once: + +```powershell +Install-Module -Name SqlServer -Scope CurrentUser -Force +``` + +## Quick start + +`SPSDBMove` is configuration-driven: the job (instances, backup roots, +database list, tuning) lives in a small JSON file; the CLI takes only the +config path, the phase to run, and a couple of runtime flags. + +1. Copy the sample config and edit it for your environment: + + ```powershell + Copy-Item .\scripts\SPSDBMove.sample.json .\scripts\my-refresh.json + notepad .\scripts\my-refresh.json + ``` + +2. Run the full pipeline (backup → copy → restore): + + ```powershell + .\scripts\SPSDBMove.ps1 -ConfigPath .\scripts\my-refresh.json + ``` + +### Run a single phase + +```powershell +.\scripts\SPSDBMove.ps1 -ConfigPath .\scripts\my-refresh.json -Action Backup +.\scripts\SPSDBMove.ps1 -ConfigPath .\scripts\my-refresh.json -Action Copy +.\scripts\SPSDBMove.ps1 -ConfigPath .\scripts\my-refresh.json -Action Restore +``` + +### CLI surface + +| Parameter | Purpose | +|---|---| +| `-ConfigPath` (alias `-Path`) | **Required.** Path to the JSON job file. | +| `-Action` | `All` (default), `Backup`, `Copy`, or `Restore`. | +| `-SqlCredential` | Optional `[pscredential]` for SQL auth. Kept on the CLI so secrets stay out of the JSON. | +| `-SkipExisting` | Copy phase only: skip files already at the destination with the same size. | +| `-WhatIf` / `-Confirm` | Standard PowerShell, honored by every destructive operation. | + +Everything else (`SourceInstance`, `DestinationInstance`, `SourceBackupRoot`, +`DestinationBackupRoot`, `DataFileDirectory`, `LogFileDirectory`, +`ThrottleLimit`, `OverwriteDestinationDb`, +`Databases`, `ExcludeDatabases`) belongs in +the JSON file. See [`scripts/SPSDBMove.sample.json`](scripts/SPSDBMove.sample.json) +and [Configuration](wiki/Configuration.md). + +### Dry-run + +`SPSDBMove` honours `-WhatIf` for every destructive operation: + +```powershell +.\scripts\SPSDBMove.ps1 -ConfigPath .\scripts\my-refresh.json -WhatIf +``` + +## Documentation + +Full documentation lives in the [wiki](https://github.com/luigilink/SPSDBMove/wiki): + +- [Home](wiki/Home.md) +- [Getting Started](wiki/Getting-Started.md) +- [Configuration](wiki/Configuration.md) +- [Usage](wiki/Usage.md) + +## Contributing + +See [CONTRIBUTING.md](.github/CONTRIBUTING.md). Bug reports and feature +requests use the [issue templates](.github/ISSUE_TEMPLATE). + +## Changelog + +See [CHANGELOG.md](CHANGELOG.md) and the latest +[release notes](RELEASE-NOTES.md). + +## License + +[MIT](LICENSE) © 2026 Jean-Cyril Drouhin. diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md new file mode 100644 index 0000000..56bbb7d --- /dev/null +++ b/RELEASE-NOTES.md @@ -0,0 +1,62 @@ +# SPSDBMove - Release Notes + +## [1.0.0] - 2026-06-01 + +First tagged release of **SPSDBMove**. Requires **PowerShell 7.0+**. + +### Added + +- `scripts/SPSDBMove.ps1` — a full **backup + copy + restore** orchestrator: + - `-Action All | Backup | Copy | Restore` phase selector. + - `[CmdletBinding(SupportsShouldProcess)]`, comment-based help, `-WhatIf`. + - Source-side `BACKUP DATABASE ... WITH COPY_ONLY, COMPRESSION, CHECKSUM, + INIT` writing to the per-database `\\FULL\` layout. + - Destination-side `RESTORE DATABASE` with automatic `RESTORE FILELISTONLY` + logical-to-physical remapping and optional `WITH REPLACE`. + - Parallel copy using PowerShell 7+ `ForEach-Object -Parallel` with a + configurable `ThrottleLimit` and `-SkipExisting`. + - `-SqlCredential` for SQL authentication; Windows integrated auth by + default. + - Configuration-driven CLI: the entire job (instances, backup roots, + database list, throttling, restore options) lives in a JSON file + passed via `-ConfigPath`. The CLI surface is limited to `-ConfigPath`, + `-Action`, `-SqlCredential`, `-SkipExisting`, plus `-WhatIf` / + `-Confirm`. Unknown JSON keys are rejected so typos surface + immediately. + - `BACKUP` / `RESTORE` statements run with `QueryTimeout = 0` (no + timeout). Multi-TB SharePoint databases can take many hours; any + arbitrary cap would be a footgun, so SQL Server is left in charge of + the operation lifetime. + - Per-database `DestinationName` overrides are the only way to rename a + database on restore; the JSON schema is intentionally lean. + - SQL Server system databases (`master`, `model`, `msdb`, `tempdb`) are + always excluded from every phase, regardless of what the JSON + `Databases` list contains; a warning is logged when one is filtered + out. + - `ExcludeDatabases` JSON key: array of names (or `{Name}` objects) that + are dropped from the effective list before `Backup`, `Copy`, and + `Restore` run. Lets operators keep noisy databases in the main + `Databases` list while skipping them for a specific run. + - Sample config shipped at `scripts/SPSDBMove.sample.json`. + - Automatic per-run log file written next to the script under + `Logs\SPSDBMove.log`. +- `tests/SPSDBMove.Tests.ps1` — Pester 5 smoke tests covering script + parsing, declared parameters, PSScriptAnalyzer cleanliness, and + repository structure. +- `wiki/` folder with `Home.md`, `Getting-Started.md`, `Configuration.md`, + `Usage.md`, published to GitHub Wiki via `.github/workflows/wiki.yml`. +- `README.md` with badges, quick-start examples, CLI surface table, and + links to the wiki. +- `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1) with a real reporting + channel (GitHub private security advisory). +- `.github/CONTRIBUTING.md` — contribution guide. +- `.github/PULL_REQUEST_TEMPLATE.md` and `.github/ISSUE_TEMPLATE/*` for + bug reports, feature requests, documentation requests, and improvement + requests. +- `.github/workflows/pester.yml` — Pester + PSScriptAnalyzer CI on Linux, + macOS, and Windows. +- `.github/workflows/release.yml` — tag-triggered (`v*`) GitHub Release + job that publishes a ZIP containing `scripts/`, `README.md`, `LICENSE`, + `CHANGELOG.md`, and `RELEASE-NOTES.md`. + +A full list of changes in each version can be found in the [change log](CHANGELOG.md). diff --git a/scripts/SPSDBMove.ps1 b/scripts/SPSDBMove.ps1 new file mode 100644 index 0000000..74fa057 --- /dev/null +++ b/scripts/SPSDBMove.ps1 @@ -0,0 +1,789 @@ +#Requires -Version 7.0 + +<# +.SYNOPSIS + Backup, copy, and restore SharePoint SQL databases across SQL Server instances. + +.DESCRIPTION + SPSDBMove orchestrates three independent phases used during SharePoint + Server 2016 / 2019 -> Subscription Edition migrations and PROD -> PREPROD + refreshes: + + 1. Backup - BACKUP DATABASE on the source instance, written into a + per-database folder layout (\\FULL\*.bak) + using COPY_ONLY, COMPRESSION, CHECKSUM, INIT. + 2. Copy - parallel copy of the latest .bak file per database from the + source backup root to the destination backup root. + 3. Restore - RESTORE DATABASE on the destination instance, with + RESTORE FILELISTONLY used to remap logical files to the + target data/log directories. REPLACE is honored when + OverwriteDestinationDb is true in the config (default). + + The job (which instances, which roots, which databases, all tuning + values) is described by a JSON configuration file. The CLI surface is + intentionally small: pick a config, pick a phase, optionally provide a + credential and the -SkipExisting copy flag. + + Run the full pipeline with -Action All, or any single phase with + -Action Backup | Copy | Restore. + + The SqlServer PowerShell module (Invoke-Sqlcmd) is required for the + Backup and Restore phases. Install it once with: + + Install-Module -Name SqlServer -Scope CurrentUser -Force + + PowerShell 7.0 or later is required because the Copy phase uses + ForEach-Object -Parallel, which is not available on Windows + PowerShell 5.1. + +.PARAMETER ConfigPath + Path to a JSON configuration file describing the job. Required. + See scripts/SPSDBMove.sample.json or wiki/Configuration.md for the + schema. May also be passed as -Path. + +.PARAMETER Action + Phase(s) to execute. One of: All, Backup, Copy, Restore. Default: All. + +.PARAMETER SqlCredential + PSCredential for SQL authentication. Omit to use Windows integrated auth. + Kept on the CLI so secrets never touch the JSON file. + +.PARAMETER SkipExisting + Copy phase only. Skip files that already exist at the destination with an + identical length. Default: $false. + +.EXAMPLE + PS> .\SPSDBMove.ps1 -ConfigPath .\config\preprod-refresh.json + + Runs the full Backup -> Copy -> Restore pipeline described by the file. + +.EXAMPLE + PS> .\SPSDBMove.ps1 -ConfigPath .\config\preprod-refresh.json -Action Backup + + Runs just the Backup phase against the source instance from the config. + +.EXAMPLE + PS> .\SPSDBMove.ps1 -ConfigPath .\config\preprod-refresh.json -Action All -WhatIf + + Dry-run; prints every BACKUP/Copy/RESTORE that would be issued. + +.NOTES + Project : SPSDBMove + Author : Jean-Cyril Drouhin (luigilink) + License : MIT - see LICENSE + Repo : https://github.com/luigilink/SPSDBMove + +.LINK + https://github.com/luigilink/SPSDBMove +#> +[CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')] +param( + [Parameter(Mandatory = $true, Position = 0)] + [Alias('Path')] + [ValidateNotNullOrEmpty()] + [string] $ConfigPath, + + [Parameter()] + [ValidateSet('All', 'Backup', 'Copy', 'Restore')] + [string] $Action = 'All', + + [Parameter()] + [System.Management.Automation.Credential()] + [pscredential] $SqlCredential = [pscredential]::Empty, + + [Parameter()] + [switch] $SkipExisting +) + +# --------------------------------------------------------------------------- +# Internal helpers +# --------------------------------------------------------------------------- + +$script:LogFile = $null + +function Write-SpsLog { + [CmdletBinding()] + [Diagnostics.CodeAnalysis.SuppressMessageAttribute( + 'PSAvoidUsingWriteHost', '', + Justification = 'Interactive script: user-facing colorized output is expected.')] + param( + [Parameter(Mandatory = $true, Position = 0)] + [AllowEmptyString()] + [string] $Message, + + [Parameter()] + [ValidateSet('Info', 'Warn', 'Error', 'Success', 'Verbose')] + [string] $Level = 'Info' + ) + + $timestamp = (Get-Date).ToString('yyyy-MM-dd HH:mm:ss') + $line = '[{0}] [{1,-7}] {2}' -f $timestamp, $Level.ToUpperInvariant(), $Message + + switch ($Level) { + 'Warn' { Write-Host $line -ForegroundColor Yellow } + 'Error' { Write-Host $line -ForegroundColor Red } + 'Success' { Write-Host $line -ForegroundColor Green } + 'Verbose' { Write-Verbose $Message } + default { Write-Host $line -ForegroundColor Cyan } + } + + if ($script:LogFile) { + try { + Add-Content -LiteralPath $script:LogFile -Value $line -Encoding UTF8 + } + catch { + Write-Warning "Failed to append to log file '$script:LogFile': $($_.Exception.Message)" + } + } +} + +function Initialize-SpsLogFile { + [CmdletBinding()] + param() + + $scriptBaseName = [System.IO.Path]::GetFileNameWithoutExtension($PSCommandPath) + $pathLogsFolder = Join-Path -Path $PSScriptRoot -ChildPath 'Logs' + + # Initialize logs + if (-Not (Test-Path -Path $pathLogsFolder)) { + New-Item -ItemType Directory -Path $pathLogsFolder -Force | Out-Null + } + $pathLogFile = Join-Path -Path $pathLogsFolder -ChildPath ($scriptBaseName + '.log') + + $script:LogFile = $pathLogFile + Write-SpsLog -Level Info -Message ("Log file: {0}" -f $script:LogFile) +} + +function Get-SpsConfig { + [CmdletBinding()] + [OutputType([pscustomobject])] + param( + [Parameter(Mandatory = $true)] + [string] $Path + ) + + if (-not (Test-Path -LiteralPath $Path)) { + throw "ConfigPath '$Path' does not exist." + } + + Write-SpsLog -Level Info -Message ("Loading config from {0}" -f $Path) + + try { + $json = Get-Content -LiteralPath $Path -Raw | ConvertFrom-Json -ErrorAction Stop + } + catch { + throw "Failed to parse JSON config '$Path': $($_.Exception.Message)" + } + + # Build the config object with explicit defaults for every optional value. + $cfg = [pscustomobject]@{ + SourceInstance = $null + DestinationInstance = $null + SourceBackupRoot = $null + DestinationBackupRoot = $null + DataFileDirectory = $null + LogFileDirectory = $null + ThrottleLimit = 4 + OverwriteDestinationDb = $true + Databases = @() + ExcludeDatabases = @() + } + + # Reject unknown top-level keys so typos surface immediately. + $known = $cfg.PSObject.Properties.Name + foreach ($name in $json.PSObject.Properties.Name) { + if ($known -notcontains $name) { + throw "Unknown config key '$name' in '$Path'. Allowed: $($known -join ', ')." + } + } + + # Array-shaped keys are normalised below; everything else is a scalar copy. + $arrayKeys = @('Databases', 'ExcludeDatabases') + foreach ($name in $known) { + if ($json.PSObject.Properties.Name -contains $name -and $arrayKeys -notcontains $name) { + $cfg.$name = $json.$name + } + } + + if ($json.PSObject.Properties.Name -contains 'Databases') { + $cfg.Databases = @($json.Databases | ForEach-Object { + if ($_ -is [string]) { + [pscustomobject]@{ Name = $_; DestinationName = $null } + } + elseif ($_ -is [pscustomobject] -or $_ -is [hashtable]) { + $entry = if ($_ -is [hashtable]) { [pscustomobject] $_ } else { $_ } + if ([string]::IsNullOrWhiteSpace($entry.Name)) { + throw "Each entry in 'Databases' must have a non-empty 'Name'." + } + [pscustomobject]@{ + Name = [string] $entry.Name + DestinationName = if ($entry.PSObject.Properties.Name -contains 'DestinationName') { $entry.DestinationName } else { $null } + } + } + else { + throw "Each entry in 'Databases' must be a string or an object with a 'Name' property." + } + }) + } + + if ($json.PSObject.Properties.Name -contains 'ExcludeDatabases') { + $cfg.ExcludeDatabases = @($json.ExcludeDatabases | ForEach-Object { + if ($_ -is [string]) { + $_ + } + elseif ($_ -is [pscustomobject] -or $_ -is [hashtable]) { + $entry = if ($_ -is [hashtable]) { [pscustomobject] $_ } else { $_ } + if ([string]::IsNullOrWhiteSpace($entry.Name)) { + throw "Each entry in 'ExcludeDatabases' must be a non-empty string or an object with a 'Name' property." + } + [string] $entry.Name + } + else { + throw "Each entry in 'ExcludeDatabases' must be a string or an object with a 'Name' property." + } + } | Where-Object { -not [string]::IsNullOrWhiteSpace($_) }) + } + + # Range / type validation for the numeric fields. + if ($cfg.ThrottleLimit -lt 1 -or $cfg.ThrottleLimit -gt 64) { + throw "ThrottleLimit must be between 1 and 64 (got $($cfg.ThrottleLimit))." + } + $cfg.ThrottleLimit = [int] $cfg.ThrottleLimit + $cfg.OverwriteDestinationDb = [bool] $cfg.OverwriteDestinationDb + + return $cfg +} + +function Assert-SpsConfig { + [CmdletBinding()] + param( + [Parameter(Mandatory = $true)] + [pscustomobject] $Config, + + [Parameter(Mandatory = $true)] + [ValidateSet('All', 'Backup', 'Copy', 'Restore')] + [string] $Action + ) + + $needSource = $Action -in @('All', 'Backup') + $needDest = $Action -in @('All', 'Restore') + $needCopy = $Action -in @('All', 'Copy') + + $missing = New-Object 'System.Collections.Generic.List[string]' + if ($needSource -and [string]::IsNullOrWhiteSpace($Config.SourceInstance)) { $missing.Add('SourceInstance') } + if ($needDest -and [string]::IsNullOrWhiteSpace($Config.DestinationInstance)) { $missing.Add('DestinationInstance') } + if (($needSource -or $needCopy) -and [string]::IsNullOrWhiteSpace($Config.SourceBackupRoot)) { $missing.Add('SourceBackupRoot') } + if (($needDest -or $needCopy) -and [string]::IsNullOrWhiteSpace($Config.DestinationBackupRoot)) { $missing.Add('DestinationBackupRoot') } + + if ($missing.Count -gt 0) { + throw "Missing required config key(s) for -Action $Action : $($missing -join ', ')" + } + + if ($Action -ne 'Copy' -and $Config.Databases.Count -eq 0) { + throw "Config 'Databases' array is empty. Add at least one database entry." + } +} + +function Test-SpsSqlServerModule { + [CmdletBinding()] + param() + + if (Get-Module -Name SqlServer) { + return + } + if (Get-Module -ListAvailable -Name SqlServer) { + Import-Module -Name SqlServer -ErrorAction Stop -DisableNameChecking + return + } + throw "The 'SqlServer' PowerShell module is required for Backup and Restore phases. Install it with: Install-Module -Name SqlServer -Scope CurrentUser -Force" +} + +function Get-SpsInvokeSqlcmdSplat { + [CmdletBinding()] + [OutputType([hashtable])] + param( + [Parameter(Mandatory = $true)] + [string] $ServerInstance, + + [Parameter()] + [pscredential] $Credential + ) + + # QueryTimeout = 0 means "no timeout". BACKUP / RESTORE durations on + # multi-TB SharePoint databases are inherently unpredictable; let SQL + # Server manage the operation itself instead of imposing an arbitrary cap. + $splat = @{ + ServerInstance = $ServerInstance + QueryTimeout = 0 + TrustServerCertificate = $true + ErrorAction = 'Stop' + } + + if ($Credential -and $Credential -ne [pscredential]::Empty) { + $splat['Credential'] = $Credential + } + + return $splat +} + +function ConvertTo-SpsSafeFileName { + [CmdletBinding()] + [OutputType([string])] + param( + [Parameter(Mandatory = $true)] + [string] $Name + ) + + $invalid = [IO.Path]::GetInvalidFileNameChars() + $sb = [System.Text.StringBuilder]::new() + foreach ($ch in $Name.ToCharArray()) { + if ($invalid -contains $ch) { + [void] $sb.Append('_') + } + else { + [void] $sb.Append($ch) + } + } + return $sb.ToString() +} + +function Get-SpsLatestBackupFile { + [CmdletBinding()] + [OutputType([System.IO.FileInfo])] + param( + [Parameter(Mandatory = $true)] + [string] $RootPath, + + [Parameter(Mandatory = $true)] + [string] $DatabaseName + ) + + $fullDir = Join-Path -Path (Join-Path -Path $RootPath -ChildPath $DatabaseName) -ChildPath 'FULL' + if (-not (Test-Path -LiteralPath $fullDir)) { + return $null + } + + return Get-ChildItem -LiteralPath $fullDir -Filter '*.bak' -File | + Sort-Object LastWriteTime -Descending | + Select-Object -First 1 +} + +function Get-SpsBackupFileList { + [CmdletBinding()] + [OutputType([System.Data.DataRow[]])] + param( + [Parameter(Mandatory = $true)] [hashtable] $InvokeSplat, + [Parameter(Mandatory = $true)] [string] $BackupFile + ) + + $query = "RESTORE FILELISTONLY FROM DISK = N'$BackupFile';" + return Invoke-Sqlcmd @InvokeSplat -Query $query +} + +function Get-SpsSystemDatabaseName { + [CmdletBinding()] + [OutputType([string[]])] + param() + + # SQL Server's hard-coded system databases. SPSDBMove never touches + # these regardless of what the config says: backing them up cross- + # instance is unsafe and restoring them over a live destination would + # corrupt the engine. + return @('master', 'model', 'msdb', 'tempdb') +} + +function Select-SpsFilteredDatabase { + [CmdletBinding()] + [OutputType([pscustomobject[]])] + param( + [Parameter(Mandatory = $true)] + [AllowEmptyCollection()] + [pscustomobject[]] $Database, + + [Parameter()] + [AllowEmptyCollection()] + [string[]] $ExcludeName = @() + ) + + $system = Get-SpsSystemDatabaseName + $userExcludes = @($ExcludeName | Where-Object { -not [string]::IsNullOrWhiteSpace($_) }) + + $kept = New-Object 'System.Collections.Generic.List[pscustomobject]' + foreach ($db in $Database) { + $name = $db.Name + + if ($system -contains $name) { + Write-SpsLog -Level Warn -Message ("Skipping '{0}' (system database)." -f $name) + continue + } + + if ($userExcludes -contains $name) { + Write-SpsLog -Level Warn -Message ("Skipping '{0}' (matched ExcludeDatabases)." -f $name) + continue + } + + $kept.Add($db) + } + + return @($kept.ToArray()) +} + +# --------------------------------------------------------------------------- +# Phase: Backup +# --------------------------------------------------------------------------- + +function Invoke-SpsBackupPhase { + [CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')] + param( + [Parameter(Mandatory = $true)] [string] $ServerInstance, + [Parameter(Mandatory = $true)] [string] $BackupRoot, + [Parameter(Mandatory = $true)] [pscustomobject[]] $Databases, + [Parameter()] [pscredential] $Credential + ) + + Write-SpsLog -Level Info -Message ("=== BACKUP phase ({0} database(s)) ===" -f $Databases.Count) + Test-SpsSqlServerModule + + $splat = Get-SpsInvokeSqlcmdSplat -ServerInstance $ServerInstance -Credential $Credential + $stamp = (Get-Date).ToString('yyyyMMdd_HHmmss') + + foreach ($db in $Databases) { + $dbName = $db.Name + $safeName = ConvertTo-SpsSafeFileName -Name $dbName + $fullDir = Join-Path -Path (Join-Path -Path $BackupRoot -ChildPath $safeName) -ChildPath 'FULL' + + if (-not (Test-Path -LiteralPath $fullDir)) { + if ($PSCmdlet.ShouldProcess($fullDir, 'Create backup directory')) { + New-Item -ItemType Directory -Path $fullDir -Force | Out-Null + } + } + + $backupFile = Join-Path -Path $fullDir -ChildPath ("{0}_{1}.bak" -f $safeName, $stamp) + + $tsql = @" +BACKUP DATABASE [$dbName] +TO DISK = N'$backupFile' +WITH COPY_ONLY, COMPRESSION, CHECKSUM, INIT, FORMAT, STATS = 10, + NAME = N'SPSDBMove-$safeName-$stamp'; +"@ + + $target = "[$ServerInstance] BACKUP DATABASE [$dbName] -> $backupFile" + if (-not $PSCmdlet.ShouldProcess($target, 'BACKUP DATABASE')) { + continue + } + + Write-SpsLog -Level Info -Message $target + $sw = [System.Diagnostics.Stopwatch]::StartNew() + try { + Invoke-Sqlcmd @splat -Query $tsql + $sw.Stop() + Write-SpsLog -Level Success -Message ("[{0}] backup completed in {1:N1}s" -f $dbName, $sw.Elapsed.TotalSeconds) + } + catch { + $sw.Stop() + Write-SpsLog -Level Error -Message ("[{0}] backup FAILED after {1:N1}s: {2}" -f $dbName, $sw.Elapsed.TotalSeconds, $_.Exception.Message) + throw + } + } +} + +# --------------------------------------------------------------------------- +# Phase: Copy +# --------------------------------------------------------------------------- + +function Invoke-SpsCopyPhase { + [CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'Medium')] + param( + [Parameter(Mandatory = $true)] [string] $SourceRoot, + [Parameter(Mandatory = $true)] [string] $DestinationRoot, + [Parameter(Mandatory = $true)] [pscustomobject[]] $Databases, + [Parameter()] [int] $Throttle = 4, + [Parameter()] [switch] $SkipExistingFiles + ) + + Write-SpsLog -Level Info -Message ("=== COPY phase ({0} database(s), throttle={1}) ===" -f $Databases.Count, $Throttle) + + if (-not (Test-Path -LiteralPath $SourceRoot)) { + throw "SourceBackupRoot '$SourceRoot' does not exist." + } + + if (-not (Test-Path -LiteralPath $DestinationRoot)) { + if ($PSCmdlet.ShouldProcess($DestinationRoot, 'Create destination backup root')) { + New-Item -ItemType Directory -Path $DestinationRoot -Force | Out-Null + } + } + + # Build the list of (source -> destination) pairs first so the parallel + # block can iterate over plain data. + $jobs = foreach ($db in $Databases) { + $dbName = $db.Name + $latest = Get-SpsLatestBackupFile -RootPath $SourceRoot -DatabaseName $dbName + + if (-not $latest) { + Write-SpsLog -Level Warn -Message ("[{0}] no .bak file found under {1}\{0}\FULL; skipping" -f $dbName, $SourceRoot) + continue + } + + $destDir = Join-Path -Path (Join-Path -Path $DestinationRoot -ChildPath $dbName) -ChildPath 'FULL' + $destFile = Join-Path -Path $destDir -ChildPath $latest.Name + + [pscustomobject]@{ + DatabaseName = $dbName + Source = $latest.FullName + SourceLength = $latest.Length + DestDir = $destDir + DestFile = $destFile + } + } + + if (-not $jobs -or $jobs.Count -eq 0) { + Write-SpsLog -Level Warn -Message 'COPY phase: nothing to copy.' + return + } + + $skip = [bool] $SkipExistingFiles + $whatIf = [bool] $WhatIfPreference + + $jobs | ForEach-Object -ThrottleLimit $Throttle -Parallel { + $job = $_ + $skipExisting = $using:skip + $whatIfMode = $using:whatIf + + if (-not (Test-Path -LiteralPath $job.DestDir)) { + if (-not $whatIfMode) { + New-Item -ItemType Directory -Path $job.DestDir -Force | Out-Null + } + } + + if ($skipExisting -and (Test-Path -LiteralPath $job.DestFile)) { + $existing = Get-Item -LiteralPath $job.DestFile + if ($existing.Length -eq $job.SourceLength) { + Write-Output ("[{0}] SKIPPED (already present, {1:N0} bytes)" -f $job.DatabaseName, $existing.Length) + return + } + } + + if ($whatIfMode) { + Write-Output ("[{0}] WHATIF: copy {1} -> {2}" -f $job.DatabaseName, $job.Source, $job.DestFile) + return + } + + $sw = [System.Diagnostics.Stopwatch]::StartNew() + try { + Copy-Item -LiteralPath $job.Source -Destination $job.DestFile -Force + $sw.Stop() + Write-Output ("[{0}] copied {1:N0} bytes in {2:N1}s" -f $job.DatabaseName, $job.SourceLength, $sw.Elapsed.TotalSeconds) + } + catch { + $sw.Stop() + Write-Output ("[{0}] COPY FAILED after {1:N1}s: {2}" -f $job.DatabaseName, $sw.Elapsed.TotalSeconds, $_.Exception.Message) + throw + } + } | ForEach-Object { Write-SpsLog -Level Info -Message $_ } +} + +# --------------------------------------------------------------------------- +# Phase: Restore +# --------------------------------------------------------------------------- + +function Invoke-SpsRestorePhase { + [CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')] + param( + [Parameter(Mandatory = $true)] [string] $ServerInstance, + [Parameter(Mandatory = $true)] [string] $BackupRoot, + [Parameter(Mandatory = $true)] [pscustomobject[]] $Databases, + [Parameter()] [pscredential] $Credential, + [Parameter()] [string] $DataDirectory, + [Parameter()] [string] $LogDirectory, + [Parameter()] [bool] $Replace = $true + ) + + Write-SpsLog -Level Info -Message ("=== RESTORE phase ({0} database(s)) ===" -f $Databases.Count) + Test-SpsSqlServerModule + + $splat = Get-SpsInvokeSqlcmdSplat -ServerInstance $ServerInstance -Credential $Credential + + foreach ($db in $Databases) { + $sourceName = $db.Name + + $targetName = if (-not [string]::IsNullOrWhiteSpace($db.DestinationName)) { + $db.DestinationName + } + else { + $sourceName + } + + $safeSource = ConvertTo-SpsSafeFileName -Name $sourceName + $latest = Get-SpsLatestBackupFile -RootPath $BackupRoot -DatabaseName $safeSource + + if (-not $latest) { + Write-SpsLog -Level Warn -Message ("[{0}] no .bak file found under {1}\{0}\FULL; skipping restore" -f $sourceName, $BackupRoot) + continue + } + + Write-SpsLog -Level Info -Message ("[{0}] reading FILELISTONLY from {1}" -f $sourceName, $latest.FullName) + + try { + $fileList = Get-SpsBackupFileList -InvokeSplat $splat -BackupFile $latest.FullName + } + catch { + Write-SpsLog -Level Error -Message ("[{0}] RESTORE FILELISTONLY failed: {1}" -f $sourceName, $_.Exception.Message) + throw + } + + $moveClauses = foreach ($row in $fileList) { + $logicalName = $row.LogicalName + $type = $row.Type # 'D' for data, 'L' for log + $originalExt = [IO.Path]::GetExtension($row.PhysicalName) + if ([string]::IsNullOrEmpty($originalExt)) { + $originalExt = if ($type -eq 'L') { '.ldf' } else { '.mdf' } + } + + $newLogicalSegment = if ($logicalName -ieq $sourceName) { + $targetName + } + elseif ($logicalName.StartsWith($sourceName, [StringComparison]::OrdinalIgnoreCase)) { + $targetName + $logicalName.Substring($sourceName.Length) + } + else { + $logicalName + } + + $safeSegment = ConvertTo-SpsSafeFileName -Name $newLogicalSegment + + $targetDir = if ($type -eq 'L') { + if ($LogDirectory) { $LogDirectory } else { [IO.Path]::GetDirectoryName($row.PhysicalName) } + } + else { + if ($DataDirectory) { $DataDirectory } else { [IO.Path]::GetDirectoryName($row.PhysicalName) } + } + + $newPhysical = Join-Path -Path $targetDir -ChildPath ($safeSegment + $originalExt) + "MOVE N'{0}' TO N'{1}'" -f $logicalName, $newPhysical + } + + $withClauses = New-Object 'System.Collections.Generic.List[string]' + $moveClauses | ForEach-Object { $withClauses.Add($_) } + if ($Replace) { $withClauses.Add('REPLACE') } + $withClauses.Add('RECOVERY') + $withClauses.Add('STATS = 10') + + $tsqlSingle = @" +ALTER DATABASE [$targetName] SET SINGLE_USER WITH ROLLBACK IMMEDIATE; +"@ + $tsqlRestore = @" +RESTORE DATABASE [$targetName] +FROM DISK = N'$($latest.FullName)' +WITH +$($withClauses -join ",`n"); +"@ + $tsqlMulti = @" +ALTER DATABASE [$targetName] SET MULTI_USER; +"@ + + $target = "[$ServerInstance] RESTORE DATABASE [$targetName] FROM $($latest.FullName)" + if (-not $PSCmdlet.ShouldProcess($target, 'RESTORE DATABASE')) { + continue + } + + Write-SpsLog -Level Info -Message $target + + # Best-effort SINGLE_USER if target DB exists (so RESTORE can take it). + try { + $exists = Invoke-Sqlcmd @splat -Query "SELECT 1 AS X FROM sys.databases WHERE name = N'$targetName';" + if ($exists) { + Write-SpsLog -Level Verbose -Message ("[{0}] setting SINGLE_USER" -f $targetName) + Invoke-Sqlcmd @splat -Query $tsqlSingle + } + } + catch { + Write-SpsLog -Level Warn -Message ("[{0}] could not set SINGLE_USER (continuing): {1}" -f $targetName, $_.Exception.Message) + } + + $sw = [System.Diagnostics.Stopwatch]::StartNew() + try { + Invoke-Sqlcmd @splat -Query $tsqlRestore + $sw.Stop() + Write-SpsLog -Level Success -Message ("[{0}] restore completed in {1:N1}s" -f $targetName, $sw.Elapsed.TotalSeconds) + } + catch { + $sw.Stop() + Write-SpsLog -Level Error -Message ("[{0}] restore FAILED after {1:N1}s: {2}" -f $targetName, $sw.Elapsed.TotalSeconds, $_.Exception.Message) + throw + } + finally { + try { Invoke-Sqlcmd @splat -Query $tsqlMulti } catch { Write-SpsLog -Level Warn -Message ("[{0}] could not set MULTI_USER: {1}" -f $targetName, $_.Exception.Message) } + } + } +} + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +try { + Initialize-SpsLogFile + + Write-SpsLog -Level Info -Message "SPSDBMove starting (Action=$Action, PS $($PSVersionTable.PSVersion))" + + $cfg = Get-SpsConfig -Path $ConfigPath + Assert-SpsConfig -Config $cfg -Action $Action + + $dbList = $cfg.Databases + + # For Copy when no DB list was supplied, discover from SourceBackupRoot. + if ($Action -eq 'Copy' -and $dbList.Count -eq 0) { + if (-not (Test-Path -LiteralPath $cfg.SourceBackupRoot)) { + throw "SourceBackupRoot '$($cfg.SourceBackupRoot)' does not exist." + } + $dbList = @(Get-ChildItem -LiteralPath $cfg.SourceBackupRoot -Directory | + ForEach-Object { [pscustomobject]@{ Name = $_.Name; DestinationName = $null } }) + Write-SpsLog -Level Info -Message ("Discovered {0} database folder(s) under {1}" -f $dbList.Count, $cfg.SourceBackupRoot) + } + + # Apply exclusions once for every phase: SQL Server system databases are + # always skipped; ExcludeDatabases lets the operator opt out specific + # user databases without removing them from the JSON 'Databases' list. + $beforeCount = $dbList.Count + $dbList = @(Select-SpsFilteredDatabase -Database $dbList -ExcludeName $cfg.ExcludeDatabases) + if ($beforeCount -gt 0 -and $dbList.Count -eq 0) { + Write-SpsLog -Level Warn -Message 'All databases were filtered out by system / ExcludeDatabases rules; nothing to do.' + } + + $overall = [System.Diagnostics.Stopwatch]::StartNew() + + if ($Action -in @('All', 'Backup')) { + Invoke-SpsBackupPhase ` + -ServerInstance $cfg.SourceInstance ` + -BackupRoot $cfg.SourceBackupRoot ` + -Databases $dbList ` + -Credential $SqlCredential + } + + if ($Action -in @('All', 'Copy')) { + Invoke-SpsCopyPhase ` + -SourceRoot $cfg.SourceBackupRoot ` + -DestinationRoot $cfg.DestinationBackupRoot ` + -Databases $dbList ` + -Throttle $cfg.ThrottleLimit ` + -SkipExistingFiles:$SkipExisting + } + + if ($Action -in @('All', 'Restore')) { + Invoke-SpsRestorePhase ` + -ServerInstance $cfg.DestinationInstance ` + -BackupRoot $cfg.DestinationBackupRoot ` + -Databases $dbList ` + -Credential $SqlCredential ` + -DataDirectory $cfg.DataFileDirectory ` + -LogDirectory $cfg.LogFileDirectory ` + -Replace $cfg.OverwriteDestinationDb + } + + $overall.Stop() + Write-SpsLog -Level Success -Message ("SPSDBMove finished in {0:N1}s" -f $overall.Elapsed.TotalSeconds) +} +catch { + Write-SpsLog -Level Error -Message ("SPSDBMove aborted: {0}" -f $_.Exception.Message) + throw +} diff --git a/scripts/SPSDBMove.sample.json b/scripts/SPSDBMove.sample.json new file mode 100644 index 0000000..62263a6 --- /dev/null +++ b/scripts/SPSDBMove.sample.json @@ -0,0 +1,19 @@ +{ + "SourceInstance": "SQLPROD01\\SHAREPOINT", + "DestinationInstance": "SQLPREP01\\SHAREPOINT", + "SourceBackupRoot": "\\\\sqlprod01\\Backup\\SPSDBMove", + "DestinationBackupRoot": "\\\\sqlprep01\\Backup\\SPSDBMove", + "DataFileDirectory": "E:\\MSSQL\\DATA", + "LogFileDirectory": "F:\\MSSQL\\LOG", + "ThrottleLimit": 4, + "OverwriteDestinationDb": true, + "Databases": [ + { "Name": "SP_Content_Intranet" }, + { "Name": "SP_Content_Projects", "DestinationName": "SP_Content_Projects_PREP" }, + "SP_Content_HR" + ], + "ExcludeDatabases": [ + "SP_Content_Legacy", + "SP_Content_Decommissioned" + ] +} diff --git a/tests/SPSDBMove.Tests.ps1 b/tests/SPSDBMove.Tests.ps1 new file mode 100644 index 0000000..8ea8ef5 --- /dev/null +++ b/tests/SPSDBMove.Tests.ps1 @@ -0,0 +1,185 @@ +#Requires -Version 7.0 +#Requires -Modules @{ ModuleName='Pester'; ModuleVersion='5.3.0' } + +<# +.SYNOPSIS + Static-analysis and parser smoke tests for scripts/SPSDBMove.ps1. + +.DESCRIPTION + These tests do not require SQL Server. They validate that the script: + * exists at the expected location, + * parses cleanly with the PowerShell language parser, + * requires PowerShell 7.0+, + * exposes the documented parameters, + * passes PSScriptAnalyzer at Error and Warning severity. + + A second describe block validates structural metadata of the repository + (README, CHANGELOG, LICENSE, etc.) so that CI catches drift early. +#> + +BeforeDiscovery { + $script:RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..') + $script:ScriptPath = Join-Path $script:RepoRoot 'scripts/SPSDBMove.ps1' +} + +Describe 'scripts/SPSDBMove.ps1 — parser and signature' { + + BeforeAll { + $script:RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..') + $script:ScriptPath = Join-Path $script:RepoRoot 'scripts/SPSDBMove.ps1' + } + + It 'exists at scripts/SPSDBMove.ps1' { + Test-Path -LiteralPath $script:ScriptPath | Should -BeTrue + } + + It 'parses without syntax errors' { + $tokens = $null + $parseErrors = $null + $null = [System.Management.Automation.Language.Parser]::ParseFile( + $script:ScriptPath, [ref]$tokens, [ref]$parseErrors) + $parseErrors | Should -BeNullOrEmpty + } + + It 'requires PowerShell 7.0 or later' { + $content = Get-Content -LiteralPath $script:ScriptPath -Raw + $content | Should -Match '#Requires\s+-Version\s+7\.0' + } + + It 'declares CmdletBinding with SupportsShouldProcess' { + $content = Get-Content -LiteralPath $script:ScriptPath -Raw + $content | Should -Match '\[CmdletBinding\([^\)]*SupportsShouldProcess' + } + + It 'exposes the documented top-level parameters' { + $expected = @( + 'ConfigPath', + 'Action', + 'SqlCredential', + 'SkipExisting' + ) + + $tokens = $null + $errors = $null + $ast = [System.Management.Automation.Language.Parser]::ParseFile( + $script:ScriptPath, [ref]$tokens, [ref]$errors) + + $paramBlock = $ast.ParamBlock + $paramBlock | Should -Not -BeNullOrEmpty + + $declared = $paramBlock.Parameters.Name.VariablePath.UserPath + foreach ($name in $expected) { + $declared | Should -Contain $name -Because "parameter -$name is part of the documented surface" + } + + $declared.Count | Should -Be $expected.Count ` + -Because 'all job-level options live in the JSON config, not the CLI' + } + + It 'marks -ConfigPath as mandatory' { + $tokens = $null + $errors = $null + $ast = [System.Management.Automation.Language.Parser]::ParseFile( + $script:ScriptPath, [ref]$tokens, [ref]$errors) + + $configParam = $ast.ParamBlock.Parameters | + Where-Object { $_.Name.VariablePath.UserPath -eq 'ConfigPath' } + $configParam | Should -Not -BeNullOrEmpty + + $mandatoryArg = $configParam.Attributes | + Where-Object { $_.TypeName.Name -eq 'Parameter' } | + ForEach-Object { $_.NamedArguments } | + Where-Object { $_.ArgumentName -eq 'Mandatory' } + $mandatoryArg | Should -Not -BeNullOrEmpty + } +} + +Describe 'scripts/SPSDBMove.sample.json — JSON config sample' { + + BeforeAll { + $script:RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..') + $script:SamplePath = Join-Path $script:RepoRoot 'scripts/SPSDBMove.sample.json' + } + + It 'exists at scripts/SPSDBMove.sample.json' { + Test-Path -LiteralPath $script:SamplePath | Should -BeTrue + } + + It 'is valid JSON' { + { Get-Content -LiteralPath $script:SamplePath -Raw | ConvertFrom-Json -ErrorAction Stop } | + Should -Not -Throw + } + + It 'declares every documented top-level key' { + $json = Get-Content -LiteralPath $script:SamplePath -Raw | ConvertFrom-Json + $expected = @( + 'SourceInstance', 'DestinationInstance', + 'SourceBackupRoot', 'DestinationBackupRoot', + 'DataFileDirectory', 'LogFileDirectory', + 'ThrottleLimit', + 'OverwriteDestinationDb', 'Databases', + 'ExcludeDatabases' + ) + foreach ($key in $expected) { + $json.PSObject.Properties.Name | Should -Contain $key + } + } +} + +Describe 'scripts/SPSDBMove.ps1 — PSScriptAnalyzer' -Tag 'ScriptAnalyzer' { + + BeforeAll { + $script:RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..') + $script:ScriptPath = Join-Path $script:RepoRoot 'scripts/SPSDBMove.ps1' + $script:Analyzer = Get-Module -ListAvailable -Name PSScriptAnalyzer | + Select-Object -First 1 + } + + It 'PSScriptAnalyzer module is available' { + $script:Analyzer | Should -Not -BeNullOrEmpty ` + -Because 'CI installs PSScriptAnalyzer; install it locally with Install-Module PSScriptAnalyzer' + } + + It 'has no Error or Warning findings' -Skip:(-not (Get-Module -ListAvailable PSScriptAnalyzer)) { + Import-Module PSScriptAnalyzer -ErrorAction Stop + $findings = Invoke-ScriptAnalyzer -Path $script:ScriptPath -Severity Error, Warning + if ($findings) { + $findings | Format-Table RuleName, Severity, Line, Message -AutoSize | Out-String | Write-Host + } + $findings | Should -BeNullOrEmpty + } +} + +Describe 'Repository structure' { + + BeforeAll { + $script:RepoRoot = Resolve-Path (Join-Path $PSScriptRoot '..') + } + + It 'has a top-level README.md' { + Test-Path -LiteralPath (Join-Path $script:RepoRoot 'README.md') | Should -BeTrue + } + + It 'has a top-level LICENSE' { + Test-Path -LiteralPath (Join-Path $script:RepoRoot 'LICENSE') | Should -BeTrue + } + + It 'has a top-level CHANGELOG.md' { + Test-Path -LiteralPath (Join-Path $script:RepoRoot 'CHANGELOG.md') | Should -BeTrue + } + + It 'has a top-level RELEASE-NOTES.md' { + Test-Path -LiteralPath (Join-Path $script:RepoRoot 'RELEASE-NOTES.md') | Should -BeTrue + } + + It 'has a top-level CODE_OF_CONDUCT.md' { + Test-Path -LiteralPath (Join-Path $script:RepoRoot 'CODE_OF_CONDUCT.md') | Should -BeTrue + } + + It 'has wiki pages Home, Getting-Started, Configuration, Usage' { + foreach ($page in 'Home', 'Getting-Started', 'Configuration', 'Usage') { + Test-Path -LiteralPath (Join-Path $script:RepoRoot "wiki/$page.md") | + Should -BeTrue -Because "wiki/$page.md is referenced from README.md" + } + } +} diff --git a/wiki/Configuration.md b/wiki/Configuration.md new file mode 100644 index 0000000..75a4d84 --- /dev/null +++ b/wiki/Configuration.md @@ -0,0 +1,115 @@ +# Configuration + +`SPSDBMove` is configuration-driven: the **job** (instances, backup roots, +database list, tuning) is described by a JSON file. The CLI surface is +deliberately tiny. + +## CLI parameters + +| Parameter | Required | Default | Description | +|---|---|---|---| +| `-ConfigPath` (alias `-Path`) | **yes** | — | Path to the JSON job file (see schema below). | +| `-Action` | no | `All` | One of `All`, `Backup`, `Copy`, `Restore`. | +| `-SqlCredential` | no | — | `[pscredential]` for SQL authentication. Kept on the CLI so secrets stay out of the JSON. Omit for Windows integrated auth. | +| `-SkipExisting` | no (`Copy` only) | `$false` | Skip files that already exist at the destination with the same size. | +| `-WhatIf` / `-Confirm` | no | — | Standard PowerShell support; suppresses all destructive SQL and file operations. | + +## JSON configuration file + +A sample lives at [`scripts/SPSDBMove.sample.json`](../scripts/SPSDBMove.sample.json). +Copy it next to the script (or somewhere checked in alongside your runbook) +and edit it. + +### Schema + +| Key | Required | Default | Description | +|---|---|---|---| +| `SourceInstance` | for `Backup`, `All` | — | SQL Server instance to back up *from* (e.g. `SQLPROD01\\SHAREPOINT`). | +| `DestinationInstance` | for `Restore`, `All` | — | SQL Server instance to restore *to*. | +| `SourceBackupRoot` | for `Backup`, `Copy`, `All` | — | UNC or local folder where the source `.bak` files live (or will be written). | +| `DestinationBackupRoot` | for `Copy`, `Restore`, `All` | — | UNC or local folder the destination instance reads from. | +| `DataFileDirectory` | no | SQL default | Target folder for `.mdf` / `.ndf` files on the destination instance. | +| `LogFileDirectory` | no | SQL default | Target folder for `.ldf` files on the destination instance. | +| `ThrottleLimit` | no | `4` | Parallelism for the `Copy` phase (`ForEach-Object -Parallel`). Range `1..64`. | +| `OverwriteDestinationDb` | no | `true` | Adds `WITH REPLACE` to the restore. | +| `Databases` | for `Backup`, `Restore`, `All` | `[]` | Array of database descriptors (see below). | +| `ExcludeDatabases` | no | `[]` | Names to drop from the effective database list (see below). System databases (`master`, `model`, `msdb`, `tempdb`) are always excluded and do not need to be listed here. | + +Unknown top-level keys fail with an explicit error so typos surface +immediately. There is no merging with CLI flags — what is in the file is +what the script will run. + +`Invoke-Sqlcmd` is always called with `QueryTimeout = 0` (no timeout) so +that long-running `BACKUP` / `RESTORE` operations on multi-TB content +databases are never cut short by the script. SQL Server itself remains in +control of the operation's lifetime. + +### `Databases` entries + +Each entry is either a bare string (database name) or an object with a +mandatory `Name` and an optional `DestinationName` override: + +```json +"Databases": [ + "SP_Content_Intranet", + { "Name": "SP_Content_Projects" }, + { "Name": "SP_Content_HR", "DestinationName": "SP_Content_HR_PREP_FRESH" } +] +``` + +When `DestinationName` is omitted, the database is restored under its +original name. To rename multiple databases consistently, set +`DestinationName` per entry. + +### `ExcludeDatabases` entries + +Each entry is either a bare string or an object with a mandatory `Name`. +Matching is case-insensitive against the source database name (the `Name` +field, not `DestinationName`). Excluded entries are dropped from the +effective list **before** any phase runs, so the same exclusion applies to +`Backup`, `Copy`, and `Restore`. + +```json +"ExcludeDatabases": [ + "SP_Content_Legacy", + { "Name": "SP_Content_Decommissioned" } +] +``` + +SQL Server system databases (`master`, `model`, `msdb`, `tempdb`) are +always excluded — listing them in `Databases` is a no-op and a warning is +logged. This prevents accidental cross-instance system-database restores, +which would corrupt the destination engine. + +### Full example + +```json +{ + "SourceInstance": "SQLPROD01\\SHAREPOINT", + "DestinationInstance": "SQLPREP01\\SHAREPOINT", + "SourceBackupRoot": "\\\\sqlprod01\\Backup\\SPSDBMove", + "DestinationBackupRoot": "\\\\sqlprep01\\Backup\\SPSDBMove", + "DataFileDirectory": "E:\\MSSQL\\DATA", + "LogFileDirectory": "F:\\MSSQL\\LOG", + "ThrottleLimit": 6, + "OverwriteDestinationDb": true, + "Databases": [ + { "Name": "SP_Content_Intranet" }, + { "Name": "SP_Content_Projects" }, + { "Name": "SP_Content_HR", "DestinationName": "SP_Content_HR_PREP_FRESH" } + ], + "ExcludeDatabases": [ + "SP_Content_Legacy" + ] +} +``` + +## Service-account requirements (summary) + +| Account | Right | On | +|---|---|---| +| Source SQL service | Write | `SourceBackupRoot` | +| Destination SQL service | Read | `DestinationBackupRoot` | +| Account running the script | Read+Write | both roots (for the `Copy` phase) | +| Account running the script | `BACKUP DATABASE` | source databases | +| Account running the script | `dbcreator` / `RESTORE DATABASE` | destination instance | diff --git a/wiki/Getting-Started.md b/wiki/Getting-Started.md new file mode 100644 index 0000000..4bdc18e --- /dev/null +++ b/wiki/Getting-Started.md @@ -0,0 +1,87 @@ +# Getting Started + +This page walks you through your first `SPSDBMove` run. + +## 1. Prerequisites + +| Requirement | Notes | +|---|---| +| PowerShell 7.0 or later (`pwsh`) | Required — the Copy phase uses `ForEach-Object -Parallel`, which is not available on Windows PowerShell 5.1. | +| `SqlServer` PowerShell module | `Install-Module SqlServer -Scope CurrentUser`. The script will warn if missing. | +| Source SQL Server account | Must have `BACKUP DATABASE` on the databases to move (typically `db_backupoperator` or `sysadmin`). | +| Destination SQL Server account | Must have `dbcreator` (or `sysadmin`) to create restored databases. | +| Two file shares (or one) | A folder reachable by the source SQL service account for **write** and by the destination SQL service account for **read**. The script copies between them when they are different. | +| Free disk space | At least 1.5x the size of the largest content database on each share. | + +## 2. Install + +```powershell +git clone https://github.com/luigilink/SPSDBMove.git +cd SPSDBMove +``` + +Optionally trust the script for the session: + +```powershell +Unblock-File .\scripts\SPSDBMove.ps1 +``` + +## 3. Plan the folder layout + +`SPSDBMove` uses a fixed per-database folder layout on both backup roots: + +```text +\\sqlprod01\Backup\SPSDBMove\ + |-- SP_Content_Intranet\ + | `-- FULL\ + | `-- SP_Content_Intranet_20260528_143012.bak + |-- SP_Content_Projects\ + | `-- FULL\ + | `-- SP_Content_Projects_20260528_143019.bak +``` + +The destination share mirrors that layout after the `Copy` phase. + +## 4. Write your config + +`SPSDBMove` is driven by a JSON job file. Start from the sample: + +```powershell +Copy-Item .\scripts\SPSDBMove.sample.json .\scripts\my-refresh.json +notepad .\scripts\my-refresh.json +``` + +Set at least `SourceInstance`, `DestinationInstance`, `SourceBackupRoot`, +`DestinationBackupRoot`, and the `Databases` array. The other keys have +sensible defaults — see [Configuration](Configuration.md) for the full +schema. + +## 5. First run (dry-run) + +Always start with `-WhatIf` to verify what the script will do: + +```powershell +.\scripts\SPSDBMove.ps1 -ConfigPath .\scripts\my-refresh.json -WhatIf +``` + +Remove `-WhatIf` to execute for real. To run only one phase: + +```powershell +.\scripts\SPSDBMove.ps1 -ConfigPath .\scripts\my-refresh.json -Action Backup +``` + +## 6. Inspect logs + +Each run prints timestamped lines to the console and also appends them to a +log file alongside the script: + +```text +scripts\Logs\SPSDBMove.log +``` + +The `Logs` folder is created automatically on first run. + +## Next steps + +- Read [Configuration](Configuration.md) to learn every parameter. +- Read [Usage](Usage.md) for the migration and refresh recipes. diff --git a/wiki/Home.md b/wiki/Home.md new file mode 100644 index 0000000..96a16e6 --- /dev/null +++ b/wiki/Home.md @@ -0,0 +1,41 @@ +# SPSDBMove Wiki + +Welcome to the **SPSDBMove** documentation. + +`SPSDBMove` is a PowerShell tool that backs up, copies, and restores SharePoint +SQL databases between SQL Server instances. It is designed for two main +scenarios: + +- **SharePoint Server 2016 / 2019 → Subscription Edition** migrations. +- **PROD → PRE-PROD** database refreshes. + +## Pages + +- [Getting Started](Getting-Started.md) — prerequisites and a 5-minute first run. +- [Configuration](Configuration.md) — every parameter explained, plus the JSON + configuration file format. +- [Usage](Usage.md) — recipes for the typical migration and refresh workflows. + +## Project links + +- Source code: +- Issues: +- Discussions: +- Companion project: [`luigilink/SPSUpdate`](https://github.com/luigilink/SPSUpdate) + +## How `SPSDBMove` works + +```text ++----------------+ +----------------+ +----------------+ +| Source SQL | | File copy | | Destination | +| (PROD / 2019) | | (parallel) | | SQL (SE/PREP) | +| | | | | | +| BACKUP -->----+--bak-->+----copy------->+--bak-->+----RESTORE | ++----------------+ +----------------+ +----------------+ + | | + | COPY_ONLY, COMPRESSION, CHECKSUM | REPLACE, RECOVERY + | per-DB folder: \\FULL\ | logical->physical remap +``` + +The three phases are independent — run them together with `-Action All` or +individually with `-Action Backup | Copy | Restore`. diff --git a/wiki/Usage.md b/wiki/Usage.md new file mode 100644 index 0000000..98a95cc --- /dev/null +++ b/wiki/Usage.md @@ -0,0 +1,100 @@ +# Usage + +Recipes for the two main `SPSDBMove` scenarios. + +> All examples assume the `SqlServer` PowerShell module is installed and that +> the account running the script has the rights described in +> [Configuration](Configuration.md). +> +> `SPSDBMove` reads its job (instances, backup roots, database list, tuning) +> from a JSON file passed via `-ConfigPath`. The CLI itself only carries +> `-Action`, `-SqlCredential`, `-SkipExisting`, and the standard `-WhatIf` / +> `-Confirm`. + +## Scenario 1 — PROD → PRE-PROD content refresh + +Goal: copy a list of SharePoint content databases from the production farm +into the pre-production farm without disturbing the running backup chain on +PROD (`COPY_ONLY` backup). + +`config\preprod-refresh.json`: + +```json +{ + "SourceInstance": "SQLPROD01\\SHAREPOINT", + "DestinationInstance": "SQLPREP01\\SHAREPOINT", + "SourceBackupRoot": "\\\\sqlprod01\\Backup\\SPSDBMove", + "DestinationBackupRoot": "\\\\sqlprep01\\Backup\\SPSDBMove", + "DataFileDirectory": "E:\\MSSQL\\DATA", + "LogFileDirectory": "F:\\MSSQL\\LOG", + "ThrottleLimit": 6, + "Databases": [ + "SP_Content_Intranet", + "SP_Content_Projects", + "SP_Content_HR" + ] +} +``` + +Run the full pipeline: + +```powershell +.\scripts\SPSDBMove.ps1 -ConfigPath .\config\preprod-refresh.json +``` + +After the script completes, attach the restored databases in SharePoint +Central Administration (or via `Mount-SPContentDatabase`) on the pre-prod farm. + +## Scenario 2 — SP2019 → Subscription Edition migration + +Run the three phases at different times — typically backup overnight, copy +during a maintenance window, and restore right before the SE farm comes up. +A single config file describes the whole wave; `-Action` selects the phase. + +```powershell +# T-0: backup (run from / against the SP2019 SQL instance) +.\scripts\SPSDBMove.ps1 -ConfigPath .\config\migration-wave-1.json -Action Backup + +# T+2h: copy (run from a hop with access to both shares) +.\scripts\SPSDBMove.ps1 -ConfigPath .\config\migration-wave-1.json -Action Copy -SkipExisting + +# T+4h: restore (run from / against the SE SQL instance) +.\scripts\SPSDBMove.ps1 -ConfigPath .\config\migration-wave-1.json -Action Restore +``` + +## Scenario 3 — Dry-run before a real migration + +```powershell +.\scripts\SPSDBMove.ps1 -ConfigPath .\config\migration-wave-1.json -WhatIf +``` + +`-WhatIf` enumerates every `BACKUP`, `Copy-Item`, and `RESTORE` operation +without performing any of them. + +## Scenario 4 — Renaming on restore + +To restore a backup under a different name on the destination (useful when +refreshing a single DB without overwriting the live copy), use a per-entry +`DestinationName` in the `Databases` array: + +```json +{ + "Databases": [ + { "Name": "SP_Content_HR", "DestinationName": "SP_Content_HR_FRESH" }, + { "Name": "SP_Content_Legal", "DestinationName": "SP_Content_Legal_PILOT" } + ] +} +``` + +`SP_Content_HR` is restored as `SP_Content_HR_FRESH` (with its data/log +files renamed accordingly), and `SP_Content_Legal` as +`SP_Content_Legal_PILOT`. + +## Troubleshooting + +| Symptom | Likely cause | +|---|---| +| `Cannot open backup device ... Operating system error 5 (Access is denied.)` | The **source SQL service account** does not have write access to `SourceBackupRoot`, or the **destination SQL service account** does not have read access to `DestinationBackupRoot`. | +| `RESTORE detected an error on page ... ` | Backup was corrupted in transit; rerun with `-Action Copy -SkipExisting:$false` to force a fresh copy. | +| `Exclusive access could not be obtained because the database is in use` | Drop existing connections to the destination DB (`ALTER DATABASE ... SET SINGLE_USER WITH ROLLBACK IMMEDIATE`) or temporarily detach it. | +| Restore succeeds but SharePoint won't mount | Verify the SQL collation matches (`Latin1_General_CI_AS_KS_WS` for SharePoint) and that the database compat level is supported by the destination SQL version. |