-
Notifications
You must be signed in to change notification settings - Fork 53
FEAT: Add conda packaging recipes and publish tooling for mssql-python #720
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Jahnvi Thakkar (jahnvi480)
wants to merge
4
commits into
main
Choose a base branch
from
jahnvi/conda-onboarding
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
dcafa6d
FEAT: Add conda packaging recipes and publish tooling for mssql-python
jahnvi480 a1ff766
FIX: Address PR review on conda recipes and harness
jahnvi480 0013f70
FIX: address round-2 conda recipe review (#563)
jahnvi480 ca0be45
Merge branch 'main' into jahnvi/conda-onboarding
jahnvi480 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # ============================================================================= | ||
| # Local "test-before-live" gate for the conda packages. | ||
| # | ||
| # Builds BOTH conda packages (driver + binding) by repackaging the currently-live | ||
| # PyPI wheels, assembles a LOCAL conda channel, installs into a clean env, and runs | ||
| # an import + (optional) live-connect smoke test. Requires NO channel onboarding/ | ||
| # permission (but uses -c microsoft / -c conda-forge to resolve dependencies). | ||
| # | ||
| # Prereq: conda + conda-build + anaconda-client on PATH (Miniforge/Miniconda). | ||
| # | ||
| # Usage (PowerShell): | ||
| # $env:DB_CONNECTION_STRING = "<conn string>" # optional; type directly, never commit | ||
| # ./conda/build_and_test_local.ps1 | ||
| # | ||
| # When the Aug 20 release ships, bump `version` in the two meta.yaml files to | ||
| # 1.14.0 and re-run; in CI the recipes consume the signed artifacts instead of PyPI. | ||
| # ============================================================================= | ||
|
|
||
| param( | ||
| [string]$PyVer = "3.12", | ||
| [string]$BuildEnv = "mssql-condabuild", | ||
| [string]$TestEnv = "mssql-conda-test" | ||
| ) | ||
|
|
||
| $ErrorActionPreference = "Stop" | ||
| function Assert-LastExit($msg) { if ($LASTEXITCODE -ne 0) { throw "FAILED: $msg (exit $LASTEXITCODE)" } } | ||
|
|
||
| $RepoRoot = Split-Path -Parent $PSScriptRoot | ||
| # Build in a space-free path — conda-build dislikes spaces (repo lives under OneDrive). | ||
| $BldDir = Join-Path $env:TEMP "mssql-conda-bld" | ||
| $env:CONDA_BLD_PATH = $BldDir | ||
| New-Item -ItemType Directory -Force -Path $BldDir | Out-Null | ||
|
|
||
| Write-Host "== 1. Create build env ($BuildEnv, python=$PyVer) ==" -ForegroundColor Cyan | ||
| conda create -y -n $BuildEnv "python=$PyVer" conda-build anaconda-client | ||
| Assert-LastExit "create build env" | ||
|
|
||
| Write-Host "== 2. Build driver package FIRST (it is a dependency) ==" -ForegroundColor Cyan | ||
| conda run -n $BuildEnv conda build "$RepoRoot\conda\mssql-python-odbc" --no-test --output-folder $BldDir | ||
| Assert-LastExit "conda build mssql-python-odbc" | ||
|
|
||
| Write-Host "== 3. Build the binding package ==" -ForegroundColor Cyan | ||
| conda run -n $BuildEnv conda build "$RepoRoot\conda\mssql-python" --no-test --output-folder $BldDir | ||
| Assert-LastExit "conda build mssql-python" | ||
|
|
||
| Write-Host "== 4. Index the local channel ==" -ForegroundColor Cyan | ||
| conda run -n $BuildEnv conda index $BldDir | ||
| Assert-LastExit "conda index" | ||
|
|
||
| Write-Host "== 5. Clean-env install from LOCAL + microsoft + conda-forge ==" -ForegroundColor Cyan | ||
| # Mirror the real user install path (-c microsoft). The driver companion resolves from | ||
| # our LOCAL channel; azure-core/azure-identity/msal resolve from the lean `microsoft` | ||
| # channel. We deliberately do NOT let conda-forge own azure-core: its recipe over-declares | ||
| # flask/six as runtime deps, dragging in celery/boto3/botocore (~9 MB) -- see | ||
| # conda-forge/azure-core-feedstock#71. --strict-channel-priority keeps the locally built | ||
| # packages authoritative and lets `microsoft` own only the azure-* SDK packages. | ||
| conda create -y -n $TestEnv "python=$PyVer" -c "file:///$BldDir" -c microsoft -c conda-forge --strict-channel-priority --override-channels mssql-python | ||
| Assert-LastExit "install mssql-python from local channel" | ||
|
|
||
| Write-Host "== 6a. Functional: import + version ==" -ForegroundColor Cyan | ||
| conda run -n $TestEnv python -c "import mssql_python; print('import OK, version =', mssql_python.__version__)" | ||
| Assert-LastExit "import mssql_python" | ||
|
|
||
| Write-Host "== 6b. Assert the driver companion is present (came from conda, not system) ==" -ForegroundColor Cyan | ||
| conda run -n $TestEnv python -c "import mssql_python_odbc, os; print('driver companion OK:', mssql_python_odbc.__version__); print('located at:', os.path.dirname(mssql_python_odbc.__file__))" | ||
| Assert-LastExit "import mssql_python_odbc companion" | ||
|
|
||
| Write-Host "== 6c. DB-less driver-load proof (real ODBC driver must load, not just the shim) ==" -ForegroundColor Cyan | ||
| conda run -n $TestEnv python "$RepoRoot\conda\driver_load_probe.py" | ||
| Assert-LastExit "driver-load proof" | ||
|
|
||
| if ($env:DB_CONNECTION_STRING) { | ||
| Write-Host "== 6d. Live connect smoke (SELECT 1) ==" -ForegroundColor Cyan | ||
| conda run -n $TestEnv python -c "import os,mssql_python; c=mssql_python.connect(os.environ['DB_CONNECTION_STRING']); print('SELECT 1 =>', c.cursor().execute('SELECT 1').fetchone())" | ||
| Assert-LastExit "live connect smoke" | ||
| } | ||
| else { | ||
| Write-Host "== 6d. SKIPPED live connect (set DB_CONNECTION_STRING to enable) ==" -ForegroundColor Yellow | ||
| } | ||
|
|
||
| Write-Host "" | ||
| Write-Host "RESULT: PASS — conda packages build, resolve together, and import cleanly." -ForegroundColor Green | ||
| Write-Host "Local channel: $BldDir" -ForegroundColor Green | ||
| Write-Host "When permission lands, publish is one step: anaconda upload --user microsoft <pkgs>" -ForegroundColor Green |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| """DB-less ODBC driver-load proof for the conda test-before-live gate. | ||
|
|
||
| Importing ``mssql_python`` triggers the one-time native ODBC driver load | ||
| (``std::call_once`` in the C++ binding). To prove the driver payload is present | ||
| AND architecture-correct WITHOUT a live SQL Server, we additionally attempt a | ||
| connection to an unreachable local port and classify the failure: | ||
|
|
||
| * a connection / network failure -> the native driver loaded and attempted TCP (PASS) | ||
| * a "driver not found" style error -> the repackaged wheel is missing / mis-arch (FAIL) | ||
|
|
||
| This gates on the actual DRIVER, not just the tiny ``mssql_python_odbc`` Python | ||
| shim, and needs no ``DB_CONNECTION_STRING`` secret. A real live ``SELECT 1`` still | ||
| runs separately whenever a server is wired. | ||
|
|
||
| Exit code 0 = driver loaded; non-zero = driver did not load (blocks publish). | ||
| """ | ||
|
|
||
| import sys | ||
|
|
||
| import mssql_python | ||
|
|
||
| # Substrings that only appear when the native ODBC driver could NOT be loaded / | ||
| # resolved (missing companion, wrong architecture, dangling shared object). A | ||
| # genuine connection failure (host unreachable / refused / login timeout) proves | ||
| # the opposite -> the driver loaded fine. | ||
| _DRIVER_MISSING_MARKERS = ( | ||
| "mssql-python-odbc", # our own "install the driver package" DriverError | ||
| "libmsodbcsql", # posix driver .so failed to load | ||
| "image not found", # macOS dlopen failure | ||
| "cannot open shared object", # linux dlopen failure | ||
| "can't open lib", # unixODBC could not open the driver | ||
| "no such file or directory", # driver binary absent | ||
| ) | ||
|
|
||
|
|
||
| def main() -> None: | ||
| # Unreachable endpoint (nothing listens on TCP port 1) -> fast connection | ||
| # refusal AFTER the driver has loaded and attempted the socket. | ||
| conn_str = ( | ||
| "Server=127.0.0.1,1;Database=x;Uid=x;Pwd=x;" | ||
| "Encrypt=no;TrustServerCertificate=yes;" | ||
| ) | ||
| try: | ||
| mssql_python.connect(conn_str) | ||
| except Exception as exc: # noqa: BLE001 - classified by message below, on purpose | ||
| msg = str(exc).lower() | ||
| if any(marker in msg for marker in _DRIVER_MISSING_MARKERS): | ||
| sys.exit("DRIVER DID NOT LOAD / wrong arch: " + str(exc)) | ||
| print("DRIVER_LOADED (expected connection failure):", str(exc)[:200]) | ||
| return | ||
| # Reaching a real server on 127.0.0.1:1 is not expected, but a successful | ||
| # connect still proves the driver loaded. | ||
| print("DRIVER_LOADED (unexpected connect success)") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| {% set name = "mssql-python-odbc" %} | ||
| # Single source of truth: the publish pipeline exports MSSQL_ODBC_VERSION derived | ||
| # from the signed wheel filename, shared with the binding recipe's version-lock pin. | ||
| {% set version = environ.get('MSSQL_ODBC_VERSION', '18.6.2.1') %} | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Companion driver package: the Microsoft ODBC Driver 18 payload. | ||
| # | ||
| # This recipe REPACKAGES the already-signed, platform-specific | ||
| # `mssql_python_odbc-<ver>-py3-none-<plat>.whl` — it does NOT compile anything. | ||
| # | ||
| # PROPRIETARY: the driver binaries ship under the Microsoft ODBC Driver EULA, | ||
| # which is exactly why this package can never go to conda-forge and must live on | ||
| # the Microsoft-owned `microsoft` Anaconda channel. | ||
| # | ||
| # LOCAL prototype (this file): the wheel is pulled from PyPI so we can build and | ||
| # validate the mechanics with no channel/permission. In CI, swap the build | ||
| # script to consume the signed release artifacts instead of PyPI: | ||
| # %PYTHON% -m pip install --no-deps --no-index --find-links %ARTIFACTS% \ | ||
| # mssql-python-odbc==%PKG_VERSION% -vv | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| package: | ||
| name: {{ name|lower }} | ||
| version: {{ version }} | ||
|
|
||
| build: | ||
| number: 0 | ||
| script: {{ PYTHON }} -m pip install --no-deps{% if environ.get('ARTIFACTS_ODBC') %} --no-index --find-links "{{ environ.get('ARTIFACTS_ODBC') }}"{% endif %} mssql-python-odbc=={{ version }} -vv | ||
|
|
||
| requirements: | ||
| host: | ||
| - python | ||
| - pip | ||
| run: | ||
| - python | ||
| # conda drops the wheel's platform tag (manylinux_2_28 / macosx_15_0); re-assert | ||
| # the floor so the proprietary driver payload can't be installed on an | ||
| # unsupported host. The wheel install in this recipe's build already requires | ||
| # these, so the constraint is never stricter than what shipped. | ||
| - __glibc >=2.28 # [linux] | ||
| - __osx >=15.0 # [osx] | ||
|
|
||
| test: | ||
| imports: | ||
| - mssql_python_odbc | ||
|
|
||
| about: | ||
| home: https://github.com/microsoft/mssql-python | ||
| # Proprietary payload: governed by the Microsoft ODBC Driver 18 EULA (+ the bundled | ||
| # VC++ runtime license), NOT the repo's MIT LICENSE. Ship the actual EULA text. | ||
| license: LicenseRef-Microsoft-Proprietary | ||
| license_file: | ||
| - ../../mssql_python_odbc/licenses/MICROSOFT_ODBC_DRIVER_FOR_SQL_SERVER_LICENSE.txt | ||
| - ../../mssql_python_odbc/licenses/MICROSOFT_VISUAL_STUDIO_LICENSE.txt | ||
| summary: Microsoft ODBC Driver 18 payload for mssql-python (internal companion package) | ||
| description: | | ||
| Internal implementation package. Do not install or upgrade directly — install | ||
| `mssql-python`, which depends on the exact matching version of this package. | ||
| dev_url: https://github.com/microsoft/mssql-python | ||
|
|
||
| extra: | ||
| recipe-maintainers: | ||
| - jahnvithakkar |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| {% set name = "mssql-python" %} | ||
| # Single source of truth: the publish pipeline exports MSSQL_PYTHON_VERSION / | ||
| # MSSQL_ODBC_VERSION derived from the signed wheel filenames, so the package | ||
| # version AND the companion pin below can never drift. Defaults are the LOCAL | ||
| # prototype target; bump after the Aug 20 release. | ||
| {% set version = environ.get('MSSQL_PYTHON_VERSION', '1.13.0') %} | ||
| {% set odbc_version = environ.get('MSSQL_ODBC_VERSION', '18.6.2.1') %} | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # The Python binding (native C++ `ddbc_bindings` + Rust `mssql_py_core`). | ||
| # | ||
| # This recipe REPACKAGES the signed, platform/Python-specific | ||
| # `mssql_python-<ver>-cp3xx-cp3xx-<plat>.whl` — no compilation. Because it carries | ||
| # a native extension, the package is NOT `noarch`; one build per OS/arch/CPython. | ||
| # | ||
| # The proprietary driver is NOT in this package — it is pulled in via the | ||
| # version-locked `mssql-python-odbc` run dependency (model 7B). Both packages are | ||
| # published together and version-locked (see issue #706 sequencing). | ||
| # | ||
| # LOCAL prototype (this file): the wheel is pulled from PyPI. In CI, swap the | ||
| # build script to consume the signed release artifacts: | ||
| # %PYTHON% -m pip install --no-deps --no-index --find-links %ARTIFACTS% \ | ||
| # mssql-python==%PKG_VERSION% -vv | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| package: | ||
| name: {{ name|lower }} | ||
| version: {{ version }} | ||
|
|
||
| build: | ||
| number: 0 | ||
| script: {{ PYTHON }} -m pip install --no-deps{% if environ.get('ARTIFACTS_PY') %} --no-index --find-links "{{ environ.get('ARTIFACTS_PY') }}"{% endif %} mssql-python=={{ version }} -vv | ||
|
|
||
| requirements: | ||
| host: | ||
| - python | ||
| - pip | ||
| run: | ||
| - python | ||
| # No version floor: on the `microsoft` channel azure-identity / azure-core / msal | ||
| # ship as CalVer (e.g. 2026.06.01), so a semver floor like `>=1.12.0` is a | ||
| # misleading no-op there (every published build already satisfies it). | ||
| - azure-identity | ||
| - mssql-python-odbc =={{ odbc_version }} | ||
| # conda drops the wheel's platform tag (manylinux_2_28 / macosx_15_0); re-assert | ||
| # that floor as a virtual-package run constraint so it can't install on an | ||
| # unsupported host. The wheel install in the companion's build already requires | ||
| # these, so this is never stricter than what actually shipped. | ||
| - __glibc >=2.28 # [linux] | ||
| - __osx >=15.0 # [osx] | ||
|
|
||
| test: | ||
| imports: | ||
| - mssql_python | ||
|
|
||
| about: | ||
| home: https://github.com/microsoft/mssql-python | ||
| license: MIT | ||
| license_file: ../../LICENSE | ||
| summary: Microsoft driver for Python to interact with SQL Server / Azure SQL | ||
| description: | | ||
| mssql-python is Microsoft's DB API 2.0 (PEP 249) driver for SQL Server, | ||
| Azure SQL, and Azure Synapse. Installs the Microsoft ODBC Driver 18 via the | ||
| companion `mssql-python-odbc` package. | ||
| dev_url: https://github.com/microsoft/mssql-python | ||
| doc_url: https://github.com/microsoft/mssql-python/wiki | ||
|
|
||
| extra: | ||
| recipe-maintainers: | ||
| - jahnvithakkar |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.