Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ node_modules/

# AWS Lambda deployment artifacts
deploy/lambda_deployment.zip
deploy/lambda_package/
deploy/lambda_package/
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,21 @@ Watch the Vidcast of this app](https://app.vidcast.io/share/63e954e4-f0ae-4c20-8
- Configurable token lifetime
- Command-line automation friendly

## SDK Dependency

This CLI and Lambda consume
[`webex-byods-sdk`](https://pypi.org/project/webex-byods-sdk/) at the version
pinned in `requirements.txt`. The SDK owns BYODS API operations and
token-provider behavior; this repository owns the interactive prompts, JSON
configuration, AWS Secrets Manager adapter, and operation records.

## Token Management

This project includes automated token refresh functionality to handle service app token expiration:

**Files:**

- **`token_manager.py`** - Core token management class with smart refresh logic
- **`token_manager.py`** - CLI/Lambda adapter that creates SDK token providers
- **`refresh_token.py`** - Standalone script for manual token refresh
- **`TOKEN_MANAGEMENT.md`** - Complete setup and usage documentation

Expand Down Expand Up @@ -64,7 +72,7 @@ For complete setup instructions, see [TOKEN_MANAGEMENT.md](TOKEN_MANAGEMENT.md).

## Requirements

- Python 3.6 or higher
- Python 3.8 or higher
- **A Webex Service App** with appropriate scopes:
- `spark-admin:datasource_write` (for registration and updates)
- `spark-admin:datasource_read` (for listing/viewing)
Expand Down Expand Up @@ -259,7 +267,7 @@ The `TokenManager` class automatically detects its environment:
- **Local execution**: Uses `token-config.json` for all credentials and tokens
- **Lambda execution**: Uses AWS Secrets Manager for all credentials and tokens

This means the same code works in both environments without modification.
The adapter supplies either configuration source to the same SDK provider API.

### Monitoring and Troubleshooting

Expand Down
12 changes: 4 additions & 8 deletions TOKEN_MANAGEMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,12 +144,10 @@ python refresh_token.py

### Token Manager API

You can also use the TokenManager class directly in your code:
CLI and Lambda code can use this repository's adapter directly. It reads local
JSON or AWS Secrets Manager, then supplies explicit token providers to the SDK:

```python
# Activate virtual environment first
source venv/bin/activate

from token_manager import TokenManager

# Initialize the token manager
Expand All @@ -169,10 +167,8 @@ if not token_manager.is_token_valid():
- **Portal tokens** (Option A): Expire every 12 hours, good for development only
- **Integration tokens** (Option B): Last much longer, suitable for production
- The personal access token must have `spark:applications_token` scope
- Service app tokens are automatically updated in `token-config.json` under the `env` section:
- `WEBEX_SERVICE_APP_ACCESS_TOKEN`: The active service app token
- `WEBEX_SERVICE_APP_REFRESH_TOKEN`: Used for automatic token refresh
- **Refresh tokens** are automatically stored and used for efficient token refresh
- Service app tokens are cached in memory for the running process
- The SDK does not write credentials or token values to `token-config.json`
- Consider using environment variables for the token config in production environments

## How Refresh Tokens Work
Expand Down
264 changes: 12 additions & 252 deletions data-sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,280 +16,38 @@
import os
import sys
import json
import requests
import argparse
import uuid
import jwt
from typing import Dict, Any, List
from datetime import datetime
from token_manager import TokenManager
from webex_byods import (
WebexDataSourceClient as WebexDataSourceManager,
enhance_data_source_with_jwt,
get_token_expiration_display,
)


class WebexDataSourceManager:
"""Handle Webex Data Source operations via API"""

BASE_URL = "https://webexapis.com/v1"

def __init__(self, access_token: str):
"""Initialize with access token"""
self.access_token = access_token
self.headers = {
"Authorization": f"Bearer {access_token}",
"Accept": "application/json",
"Content-Type": "application/json",
}
self.schemas_cache = None # Cache for schemas

# Initialize token manager for automatic refresh
try:
self.token_manager = TokenManager()
except Exception:
self.token_manager = None

def _refresh_token_if_needed(self, response: requests.Response) -> bool:
"""
Check if a 401 response indicates token expiration and refresh if possible.

Args:
response: The response object to check

Returns:
bool: True if token was refreshed, False otherwise
"""
if response.status_code == 401 and self.token_manager:
try:
print("🔄 Access token appears to be expired, attempting refresh...")
new_token = self.token_manager.refresh_token()

# Update the instance with new token
self.access_token = new_token
self.headers["Authorization"] = f"Bearer {new_token}"

print("✅ Token refreshed successfully!")
return True

except Exception as e:
print(f"❌ Failed to refresh token: {e}")
return False
return False

def _make_request(self, method: str, url: str, **kwargs) -> Dict[str, Any]:
"""
Make an HTTP request with automatic token refresh on 401 errors.

Args:
method: HTTP method (GET, POST, PUT, DELETE)
url: Request URL
**kwargs: Additional arguments for requests

Returns:
Dict containing success status, data/error, and status code
"""
try:
# Make the initial request
response = requests.request(
method, url, headers=self.headers, timeout=30, **kwargs
)

# If we get a 401, try to refresh the token and retry once
if response.status_code == 401 and self._refresh_token_if_needed(response):
print("🔄 Retrying request with refreshed token...")
response = requests.request(
method, url, headers=self.headers, timeout=30, **kwargs
)

if response.status_code in [200, 201]:
return {
"success": True,
"data": response.json(),
"status_code": response.status_code,
}
else:
return {
"success": False,
"error": response.text,
"status_code": response.status_code,
}

except requests.exceptions.RequestException as e:
return {
"success": False,
"error": f"Request failed: {str(e)}",
"status_code": None,
}

def list_all_data_sources(self) -> Dict[str, Any]:
"""Retrieve all data sources"""
url = f"{self.BASE_URL}/dataSources"
return self._make_request("GET", url)

def get_data_source_details(self, data_source_id: str) -> Dict[str, Any]:
"""Retrieve details for a specific data source"""
url = f"{self.BASE_URL}/dataSources/{data_source_id}"
return self._make_request("GET", url)

def register_data_source(
self, data_source_config: Dict[str, Any]
) -> Dict[str, Any]:
"""Register a new data source"""
url = f"{self.BASE_URL}/dataSources"
return self._make_request("POST", url, json=data_source_config)

def update_data_source(
self, data_source_id: str, update_config: Dict[str, Any]
) -> Dict[str, Any]:
"""Update a data source"""
url = f"{self.BASE_URL}/dataSources/{data_source_id}"
return self._make_request("PUT", url, json=update_config)

def get_data_source_schemas(self) -> Dict[str, Any]:
"""Retrieve all available data source schemas"""
url = f"{self.BASE_URL}/dataSources/schemas"
return self._make_request("GET", url)

def load_schemas_cache(self) -> bool:
"""Load schemas into cache for friendly display"""
if self.schemas_cache is not None:
return True # Already loaded

result = self.get_data_source_schemas()
if result["success"]:
self.schemas_cache = result["data"].get("items", [])
return True
else:
print(f"Warning: Could not load schemas: {result['error']}")
self.schemas_cache = []
return False

def get_schema_display_name(self, schema_id: str) -> str:
"""Get friendly display name for a schema ID"""
if self.schemas_cache is None:
self.load_schemas_cache()

for schema in self.schemas_cache:
if schema.get("id") == schema_id:
service_type = schema.get("serviceType", "Unknown Service")
return f"{service_type} ({schema_id[:8]}...)"

return f"Unknown Schema ({schema_id[:8]}...)"

def get_available_schemas(self) -> List[Dict[str, Any]]:
"""Get list of available schemas for selection"""
if self.schemas_cache is None:
self.load_schemas_cache()

return self.schemas_cache


def load_env_token() -> str:
"""Get a fresh service app access token from token-config.json"""
def load_env_token() -> tuple:
"""Build the CLI adapter and retrieve an initial service-app token."""
# Load token-config.json from the same directory as the script
script_dir = os.path.dirname(os.path.abspath(__file__))
config_path = os.path.join(script_dir, "token-config.json")

try:
from token_manager import TokenManager

token_manager = TokenManager(config_path=config_path)

# Get fresh service app token
print("Fetching fresh service app token...")
token = token_manager.get_service_app_token()
print("Service app token retrieved successfully!")
return token

except ImportError:
print("Error: TokenManager not available")
print("Please ensure token_manager.py is in the same directory")
sys.exit(1)
return token, token_manager
except Exception as e:
print(f"Error: Could not get service app token: {e}")
print("Please ensure your token-config.json is properly configured")
sys.exit(1)


def decode_jwt_token(token: str) -> Dict[str, Any]:
"""Decode JWT token to extract audience and subject"""
try:
# Decode without verification since we just need to read the payload
decoded = jwt.decode(token, options={"verify_signature": False})
return decoded
except Exception as e:
print(f"Warning: Could not decode JWT token: {str(e)}")
return {}


def enhance_data_source_with_jwt(data_source: Dict[str, Any]) -> Dict[str, Any]:
"""Enhance data source data by decoding JWT token if present"""
enhanced = data_source.copy()

# Check if there's a JWT token field
jwt_token = data_source.get("jwtToken") or data_source.get("jwsToken")

if jwt_token:
jwt_claims = decode_jwt_token(jwt_token)

# Extract audience and subject from JWT if not present in main data
if not enhanced.get("audience") and jwt_claims.get("aud"):
enhanced["audience"] = jwt_claims["aud"]

if not enhanced.get("subject") and jwt_claims.get("sub"):
enhanced["subject"] = jwt_claims["sub"]

# Store JWT claims for reference
enhanced["jwt_claims"] = jwt_claims

return enhanced


def get_token_expiration_display(data_source: Dict[str, Any]) -> str:
"""
Get a human-readable token expiration display string by parsing the JWT token.

Args:
data_source: Data source dictionary from API response

Returns:
String showing token expiration status
"""
try:
# Get the JWT token from the data source
jws_token = data_source.get("jwsToken") or data_source.get("jwtToken")

if not jws_token:
return "No token found"

# Decode the JWT token to get expiration
decoded = jwt.decode(jws_token, options={"verify_signature": False})
exp_timestamp = decoded.get("exp")

if not exp_timestamp:
return "No expiration in token"

# Convert timestamp to datetime and calculate remaining time
exp_date = datetime.fromtimestamp(exp_timestamp)
now = datetime.now()

if exp_date > now:
time_diff = exp_date - now
total_seconds = int(time_diff.total_seconds())

if total_seconds > 3600: # More than 1 hour
hours = total_seconds // 3600
minutes = (total_seconds % 3600) // 60
return f"{hours}h {minutes}m"
elif total_seconds > 60: # More than 1 minute
minutes = total_seconds // 60
return f"{minutes}m"
else: # Less than 1 minute
return f"{total_seconds}s"
else:
return "EXPIRED"

except Exception as e:
return f"Parse error: {str(e)[:15]}"


def display_data_sources_list(
data_sources: List[Dict[str, Any]], manager: WebexDataSourceManager = None
) -> None:
Expand Down Expand Up @@ -690,10 +448,12 @@ def main():

try:
# Load access token
access_token = load_env_token()
access_token, token_manager = load_env_token()

# Create manager instance
manager = WebexDataSourceManager(access_token)
manager = WebexDataSourceManager(
token_provider=token_manager.get_token_provider()
)

print("Webex Data Source Manager")
print("=" * 30)
Expand Down
2 changes: 1 addition & 1 deletion deploy/AWS_SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -716,7 +716,7 @@ Should show the complete JSON structure with all required fields.

To update the code:

1. Make changes locally to `lambda_function.py` or `token_manager.py`
1. Make changes locally to `lambda_function.py`, `token_manager.py`, or `requirements.txt`
2. Run the packaging script: `./deploy/package_lambda.sh`
3. Upload new `lambda_deployment.zip` to Lambda
4. Test the function
Expand Down
Loading
Loading