diff --git a/README.md b/README.md index 31cf0226..a39bbefb 100644 --- a/README.md +++ b/README.md @@ -22,11 +22,32 @@ If you want to see what a semantic model looks like, skip to [Examples](#example ## Streamlit in Snowflake Deployment -> **Note**: Deploying this app in Streamlit in Snowflake requires support for python 3.9+ in Streamlit in Snowflake, which is in Private Preview. Please contact your account representative to enable if not already enabled. We also require streamlit 1.35.0, which is generally available. +> **Note**: Deploying this app in Streamlit in Snowflake requires support for python 3.9+ in Streamlit in Snowflake. This feature is part of the 2024-08 BCR (Behavior Change Bundle) which is currently **disabled by default**. A user with ACCOUNTADMIN privileges can enable this feature in your account by running the following code in Snowsight: +> +```sql +SELECT SYSTEM$ENABLE_BEHAVIOR_CHANGE_BUNDLE('2024_08'); +``` +> You can check to see if this BCR is already enabled by running: +```sql +SELECT SYSTEM$BEHAVIOR_CHANGE_BUNDLE_STATUS('2024_08'); +``` +> A typical error you might see if you do not have this feature enabled looks like this: + + +> For more information on this bundle please see our [BCR Documentation](https://docs.snowflake.com/en/release-notes/behavior-changes). + [Snowflake CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli-v2/index) is recommended for deploying the app in Streamlit in Snowflake. Please see Snowflake CLI [installation instructions](https://docs.snowflake.com/en/developer-guide/snowflake-cli-v2/installation/installation) to install. **Snowflake CLI version 3.0+ is required**. Follow the below instructions to install the Semantic Model Generator in Streamlit in Snowflake. -If you do not have Snowflake CLI installed, Steps #2 and #3 below can be replicated manually with the [VS Code Snowflake extension](https://docs.snowflake.com/en/user-guide/vscode-ext) or Snowsight. Please note that parameters passed (with flag `-D`) will be need to be hard-coded in the `.sql` files directly. If using Snowsight, you may use the files upload wizard to upload files. Please pay close attention to maintain the directory structure referenced in `setup_sis/app_setup.sql`. +If you do not have Snowflake CLI installed, the setup can be replicated manually with the [VS Code Snowflake extension](https://docs.snowflake.com/en/user-guide/vscode-ext), Snowsight or [Snowflake Native Git Integration](https://docs.snowflake.com/en/developer-guide/git/git-overview): + - For VS Code extention, please note that parameters passed (with flag `-D`) will be need to be hard-coded in the `.sql` files directly. + - If using Snowsight, you may use the files upload wizard to upload files. Please pay close attention to maintain the directory structure referenced in `setup_sis/app_setup.sql`. + - If using the Native Git Integration, copy and paste the code from this [setup file](https://github.com/Snowflake-Labs/semantic-model-generator/blob/main/sis_setup/sissetup_snowsightgit.sql) and run in Snowsight. Please note that you will have to set the Streamlit Warehouse as a variable in line 2: +```sql +SET my_streamlit_warehouse=; +``` + +### Snowflake CLI Installation 1. Configure Snowflake CLI @@ -49,20 +70,23 @@ The app, titled Semantic Model Generator can be opened directly in Snowsight. Al snow streamlit get-url SEMANTIC_MODEL_GENERATOR --open --database cortex_analyst_semantics --schema semantic_model_generator ``` -3. **OPTIONAL**: Setup Looker Integration +### **OPTIONAL**: Setup Looker Integration The Semantic Model Generator supports translating metadata from a Looker Explore. To add this functionality to the Streamlit in Snowflake app, we need to create an external access integration to allow Snowflake to reach your Looker instance. -Run the below command to create the external access integration. Before running, replace the following parameters: -- `` with your [Looker Base URL](https://cloud.google.com/looker/docs/admin-panel-platform-api#api_host_url). Pass the domain excluding https:// such as snowflakedemo.looker.com. -- `` with your [Looker Client Secret](https://cloud.google.com/looker/docs/api-auth#authentication_with_an_sdk) -- `` with the owning role used in step #2 above +Have the following information available: +- ``: [Looker Base URL](https://cloud.google.com/looker/docs/admin-panel-platform-api#api_host_url). Pass the domain excluding https:// such as snowflakedemo.looker.com. +- ``: [Looker Client Secret](https://cloud.google.com/looker/docs/api-auth#authentication_with_an_sdk) +- ``: the owning role used in step #2 above + +If using the CLI method, run the below command to create the external access integration. Before running, replace the parameters. -**Hint**: Running the below, which creates external access integrations, may require increased privileges. To use a different role from what is specified in your Snowflake CLI connection, append `--role ` to the end of the command, replacing ``. +**Hint**: Creating external access integrations may require increased privileges. To use a different role from what is specified in your Snowflake CLI connection, append `--role ` to the end of the command, replacing ``. ```bash snow sql -f sis_setup/looker_integration.sql -D "looker_url=" -D "client_secret=" -D "streamlit_role=" ``` +If using any other method (e.g., Git Integration), you will need to update and run [looker_integration.sql](https://github.com/Snowflake-Labs/semantic-model-generator/blob/main/sis_setup/looker_integration.sql) in Snowsight with these variables replaced. ## Local Deployment diff --git a/images/error39.png b/images/error39.png new file mode 100644 index 00000000..c0965847 Binary files /dev/null and b/images/error39.png differ diff --git a/sis_setup/sissetup_snowsightgit.sql b/sis_setup/sissetup_snowsightgit.sql new file mode 100644 index 00000000..abfdade7 --- /dev/null +++ b/sis_setup/sissetup_snowsightgit.sql @@ -0,0 +1,86 @@ +-- Please first set your Streamlit Warehouse here by replacing with a valid warehouse: + +SET my_streamlit_warehouse=; + +CREATE DATABASE IF NOT EXISTS CORTEX_ANALYST_SEMANTICS; +USE DATABASE CORTEX_ANALYST_SEMANTICS; + +-- Create API Integration for Git +CREATE OR REPLACE API INTEGRATION git_api_integration_snowflake_labs + API_PROVIDER = git_https_api + API_ALLOWED_PREFIXES = ('https://github.com/Snowflake-Labs') + ENABLED = TRUE; + +-- Create Git Repository +CREATE OR REPLACE GIT REPOSITORY git_snowflake_semantic_model_generator + API_INTEGRATION = git_api_integration_snowflake_labs + ORIGIN = 'https://github.com/Snowflake-Labs/semantic-model-generator.git'; + +ALTER GIT REPOSITORY git_snowflake_semantic_model_generator FETCH; + +-- Create Schema to host streamlit app +CREATE SCHEMA IF NOT EXISTS CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR +COMMENT = '{"origin": "sf_sit", + "name": "skimantics", + "version": {"major": 2, "minor": 0}, + "attributes": {"deployment": "sis"}}'; + +-- Create stage for App logic and 3rd party packages +CREATE OR REPLACE STAGE CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE +DIRECTORY = (ENABLE = true) +COMMENT = '{"origin": "sf_sit", + "name": "skimantics", + "version": {"major": 2, "minor": 0}, + "attributes": {"deployment": "sis"}}'; + +-- Copy Files from Git Repository into App Stage +COPY FILES + INTO @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE + FROM @CORTEX_ANALYST_SEMANTICS.PUBLIC.git_snowflake_semantic_model_generator/branches/main/app_utils/ + PATTERN='.*[.]zip'; + +COPY FILES + INTO @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/ + FROM @CORTEX_ANALYST_SEMANTICS.PUBLIC.git_snowflake_semantic_model_generator/branches/main/ + FILES = ('environment.yml', 'app.py'); + +COPY FILES + INTO @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/semantic_model_generator/ + FROM @CORTEX_ANALYST_SEMANTICS.PUBLIC.git_snowflake_semantic_model_generator/branches/main/semantic_model_generator/ + PATTERN='.*[.]py'; + +RM @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/semantic_model_generator/tests; +RM @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/semantic_model_generator/output_models; + +COPY FILES + INTO @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/images/ + FROM @CORTEX_ANALYST_SEMANTICS.PUBLIC.git_snowflake_semantic_model_generator/branches/main/images/ + PATTERN='.*[.]png'; + +COPY FILES + INTO @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/journeys/ + FROM @CORTEX_ANALYST_SEMANTICS.PUBLIC.git_snowflake_semantic_model_generator/branches/main/journeys/ + PATTERN='.*[.]py'; + +COPY FILES + INTO @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/partner/ + FROM @CORTEX_ANALYST_SEMANTICS.PUBLIC.git_snowflake_semantic_model_generator/branches/main/partner/ + PATTERN='.*[.]py'; + +COPY FILES + INTO @CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator/app_utils/ + FROM @CORTEX_ANALYST_SEMANTICS.PUBLIC.git_snowflake_semantic_model_generator/branches/main/app_utils/ + PATTERN='.*[.]py'; + +-- Create Streamlit App +CREATE OR REPLACE STREAMLIT CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.SEMANTIC_MODEL_GENERATOR +ROOT_LOCATION = '@CORTEX_ANALYST_SEMANTICS.SEMANTIC_MODEL_GENERATOR.STREAMLIT_STAGE/semantic_model_generator' +MAIN_FILE = 'app.py' +TITLE = "Semantic Model Generator" +IMPORTS = ('@cortex_analyst_semantics.semantic_model_generator.streamlit_stage/looker_sdk.zip', +'@cortex_analyst_semantics.semantic_model_generator.streamlit_stage/strictyaml.zip') +QUERY_WAREHOUSE = $my_streamlit_warehouse +COMMENT = '{"origin": "sf_sit", + "name": "skimantics", + "version": {"major": 2, "minor": 0}, + "attributes": {"deployment": "sis"}}';