GML-2011 Add UI page for Knowledge Graph init and Data Ingestion, and Knowledge Graph Rebuild

[prinskumar-tigergraph]

User description

Added UI code for
1-Add new Knowledge Graph
Create and initiate graph
2-Data Ingest for a KG
Upload files to server
Download files from cloud
Ingest the uploaded or ingest files
S3 bedrock configuration(placeholder in ui rightnow will integrate later)
Auto matic ingest after file upload is in progress
3-Refresh Knowledge Graph
Show warning the you can cancel or proceed

---

PR Type

Enhancement

---

Description

• Setup UI for graph create, ingest, refresh

• Server-side uploads with list/delete

• Cloud imports from S3/GCS/Azure

• New create_graph API, proxy, dependency

---

Diagram Walkthrough

flowchart LR
  uiSetup["Setup UI"]
  apiCreate["POST /{graph}/graphrag/create_graph"]
  apiUpload["POST /ui/{graph}/uploads"]
  fsUploads["uploads/{graph}/"]
  apiCloud["POST /ui/{graph}/cloud/download"]
  fsCloud["downloaded_files_cloud/{graph}/"]
  apiIngest["POST /{graph}/graphrag/create_ingest + ingest"]
  apiRefresh["GET /{graph}/graphrag/forceupdate"]

  uiSetup -- "Create Graph" --> apiCreate
  uiSetup -- "Upload files" --> apiUpload
  apiUpload -- "stores files" --> fsUploads
  uiSetup -- "Download from cloud" --> apiCloud
  apiCloud -- "writes files" --> fsCloud
  uiSetup -- "Ingest data" --> apiIngest
  uiSetup -- "Refresh graph" --> apiRefresh

Loading

File Walkthrough

	Relevant files
Enhancement	6 files supportai.py Add endpoint to create TigerGraph graph                                    +46/-0    ui.py Add server uploads and cloud download APIs                              +514/-1  Bot.tsx Reload graph list on focus/navigation                                        +28/-3    ModeToggle.tsx Add Setup navigation button with icon                                        +14/-1    main.tsx Register Setup page route                                                                +5/-0      Setup.tsx New KG Setup page with ingest workflow                                      +1437/-0
Configuration changes	5 files vite.config.ts Proxy dynamic graphrag/supportai routes to backend              +2/-1      common Remove shared common path symlink                                                +0/-1      configs Remove shared configs path symlink                                              +0/-1      common Remove redundant common symlink reference                                +0/-1      configs Remove redundant configs symlink reference                              +0/-1
Dependencies	1 files requirements.txt Add python-multipart for file uploads                                        +1/-0

---

[tg-pr-agent]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 No relevant tests

🔒 Security concerns Path traversal: The upload and delete endpoints in graphrag/app/routers/ui.py construct paths with unsanitized file.filename and query filename, enabling directory traversal and arbitrary file overwrite/deletion. Sanitize with os.path.basename, validate characters, and reject paths with separators. Injection: The create_graph endpoint in graphrag/app/routers/supportai.py directly f-strings user-controlled graphname into GSQL. Validate/escape graph names to prevent GSQL injection and enforce identifier rules. Sensitive information exposure: The cloud download endpoint accepts raw credentials (AWS keys, GCS service account JSON, Azure account key) over HTTP API. Ensure transport is HTTPS-only, avoid logging secrets (including exception traces), and document least-privilege guidance. Consider using temporary credentials or server-side stored secrets rather than accepting long-lived keys. Blocking I/O in async routes: Cloud downloads use blocking SDKs (boto3, google-cloud-storage, azure-storage-blob) inside async def, which can block the event loop. Run these in a threadpool (anyio.to_thread.run_sync / starlette.concurrency.run_in_threadpool) or make the routes synchronous to avoid DoS via long-running downloads.

⚡ Recommended focus areas for review Missing Imports The new endpoints use os (filesystem ops) and json (GCS credentials parsing) without importing them, which will raise runtime errors. Ensure required modules are imported. upload_dir = os.path.join("uploads", graphname) if not os.path.exists(upload_dir): return {"graphname": graphname, "files": [], "total_files": 0, "total_size": 0} files_info = [] total_size = 0 for filename in os.listdir(upload_dir): file_path = os.path.join(upload_dir, filename) if os.path.isfile(file_path): file_stat = os.stat(file_path) files_info.append({ "filename": filename, "size": file_stat.st_size, "modified": file_stat.st_mtime, }) total_size += file_stat.st_size return { "graphname": graphname, "files": files_info, "total_files": len(files_info), "total_size": total_size, } except Exception as e: logger.error(f"Error listing files for graph {graphname}: {e}") raise HTTPException(status_code=500, detail=f"Error listing files: {str(e)}") @router.post(route_prefix + "/{graphname}/uploads") async def upload_files( graphname: str, creds: Annotated[tuple[list[str], HTTPBasicCredentials], Depends(ui_basic_auth)], files: list[UploadFile] = File(...), overwrite: bool = False, ): """ Upload one or multiple files for a specific graphname. Files are stored in uploads/{graphname}/ directory. Parameters: - graphname: The graph name to associate files with - files: List of files to upload - overwrite: If False (default), will reject if files already exist """ try: upload_dir = os.path.join("uploads", graphname) os.makedirs(upload_dir, exist_ok=True) # Check for existing files if overwrite is False if not overwrite: existing_files = [] for file in files: file_path = os.path.join(upload_dir, file.filename) if os.path.exists(file_path): existing_files.append(file.filename) if existing_files: return { "status": "conflict", "message": "Some files already exist. Set overwrite=true to replace them.", "existing_files": existing_files, } # Save uploaded files uploaded_files = [] total_size = 0 for file in files: file_path = os.path.join(upload_dir, file.filename) # Write file to disk with open(file_path, "wb") as f: content = await file.read() f.write(content) file_size = len(content) total_size += file_size uploaded_files.append({ "filename": file.filename, "size": file_size, "path": file_path, }) logger.info(f"Uploaded file {file.filename} ({file_size} bytes) for graph {graphname}") return { "status": "success", "message": f"Successfully uploaded {len(uploaded_files)} file(s)", "graphname": graphname, "uploaded_files": uploaded_files, "total_size": total_size, } except Exception as e: exc = traceback.format_exc() logger.error(f"Error uploading files for graph {graphname}: {e}") logger.debug_pii(f"Upload error trace:\n{exc}") raise HTTPException(status_code=500, detail=f"Error uploading files: {str(e)}") @router.delete(route_prefix + "/{graphname}/uploads") async def clear_uploaded_files( graphname: str, creds: Annotated[tuple[list[str], HTTPBasicCredentials], Depends(ui_basic_auth)], filename: str | None = None, ): """ Clear uploaded files for a specific graphname. Parameters: - graphname: The graph name whose files to clear - filename: If provided, only delete this specific file. Otherwise, delete all files. """ try: upload_dir = os.path.join("uploads", graphname) if not os.path.exists(upload_dir): return { "status": "success", "message": f"No files found for graph {graphname}", "deleted_files": [], } deleted_files = [] if filename: # Delete specific file file_path = os.path.join(upload_dir, filename) if os.path.exists(file_path) and os.path.isfile(file_path): os.remove(file_path) deleted_files.append(filename) logger.info(f"Deleted file {filename} for graph {graphname}") else: raise HTTPException(status_code=404, detail=f"File {filename} not found") else: # Delete all files in the directory for filename in os.listdir(upload_dir): file_path = os.path.join(upload_dir, filename) if os.path.isfile(file_path): os.remove(file_path) deleted_files.append(filename) # Remove the directory if it's empty if not os.listdir(upload_dir): os.rmdir(upload_dir) logger.info(f"Deleted {len(deleted_files)} file(s) for graph {graphname}") return { "status": "success", "message": f"Successfully deleted {len(deleted_files)} file(s)", "graphname": graphname, "deleted_files": deleted_files, } except HTTPException: raise except Exception as e: exc = traceback.format_exc() logger.error(f"Error deleting files for graph {graphname}: {e}") logger.debug_pii(f"Delete error trace:\n{exc}") raise HTTPException(status_code=500, detail=f"Error deleting files: {str(e)}") # Cloud Storage Download Endpoints @router.post(route_prefix + "/{graphname}/cloud/download") async def download_from_cloud( graphname: str, credentials: Annotated[HTTPBase, Depends(security)], provider: str = None, # S3 parameters access_key: str = None, secret_key: str = None, bucket: str = None, region: str = None, prefix: str = "", # GCS parameters project_id: str = None, gcs_credentials_json: str = None, # Azure parameters account_name: str = None, account_key: str = None, container: str = None, ): """ Download files from cloud storage (S3, GCS, or Azure) to local directory. Parameters: - graphname: The graph name to associate downloaded files with - provider: Cloud provider (s3, gcs, azure) - For S3: access_key, secret_key, bucket, region, prefix - For GCS: project_id, gcs_credentials_json, bucket, prefix - For Azure: account_name, account_key, container, prefix """ try: download_dir = os.path.join("downloaded_files_cloud", graphname) os.makedirs(download_dir, exist_ok=True) downloaded_files = [] if provider == "s3": # Import boto3 for S3 try: import boto3 from botocore.exceptions import ClientError except ImportError: raise HTTPException( status_code=500, detail="boto3 is not installed. Please install it to use S3 downloads." ) if not all([access_key, secret_key, bucket, region]): raise HTTPException( status_code=400, detail="Missing S3 credentials: access_key, secret_key, bucket, and region are required" ) # Create S3 client s3_client = boto3.client( 's3', aws_access_key_id=access_key, aws_secret_access_key=secret_key, region_name=region ) # List and download objects try: paginator = s3_client.get_paginator('list_objects_v2') pages = paginator.paginate(Bucket=bucket, Prefix=prefix or "") for page in pages: if 'Contents' not in page: continue for obj in page['Contents']: key = obj['Key'] # Skip directories if key.endswith('/'): continue # Get filename filename = os.path.basename(key) local_path = os.path.join(download_dir, filename) # Download file s3_client.download_file(bucket, key, local_path) downloaded_files.append(filename) logger.info(f"Downloaded {key} to {local_path}") except ClientError as e: logger.error(f"S3 download error: {e}") raise HTTPException(status_code=500, detail=f"S3 error: {str(e)}") elif provider == "gcs": # Import GCS client try: from google.cloud import storage from google.oauth2 import service_account except ImportError: raise HTTPException( status_code=500, detail="google-cloud-storage is not installed. Please install it to use GCS downloads." ) if not all([project_id, gcs_credentials_json, bucket]): raise HTTPException( status_code=400, detail="Missing GCS credentials: project_id, gcs_credentials_json, and bucket are required" ) try: # Parse credentials JSON creds_dict = json.loads(gcs_credentials_json) credentials = service_account.Credentials.from_service_account_info(creds_dict) # Create GCS client gcs_client = storage.Client(project=project_id, credentials=credentials) bucket_obj = gcs_client.bucket(bucket) # List and download blobs blobs = bucket_obj.list_blobs(prefix=prefix or "") for blob in blobs: # Skip directories if blob.name.endswith('/'): continue # Get filename filename = os.path.basename(blob.name) local_path = os.path.join(download_dir, filename) # Download blob blob.download_to_filename(local_path) downloaded_files.append(filename) logger.info(f"Downloaded {blob.name} to {local_path}") except json.JSONDecodeError: raise HTTPException(status_code=400, detail="Invalid GCS credentials JSON") except Exception as e: logger.error(f"GCS download error: {e}") raise HTTPException(status_code=500, detail=f"GCS error: {str(e)}") Path Traversal File upload/delete paths use file.filename and user-supplied filename directly with os.path.join without sanitization. This allows path traversal and overwriting arbitrary files. Use os.path.basename, validate allowed characters, and reject paths containing separators. Also consider streaming writes instead of await file.read() to avoid large memory usage. for file in files: file_path = os.path.join(upload_dir, file.filename) # Write file to disk with open(file_path, "wb") as f: content = await file.read() f.write(content) file_size = len(content) total_size += file_size uploaded_files.append({ "filename": file.filename, "size": file_size, "path": file_path, }) logger.info(f"Uploaded file {file.filename} ({file_size} bytes) for graph {graphname}") return { "status": "success", "message": f"Successfully uploaded {len(uploaded_files)} file(s)", "graphname": graphname, "uploaded_files": uploaded_files, "total_size": total_size, } except Exception as e: exc = traceback.format_exc() logger.error(f"Error uploading files for graph {graphname}: {e}") logger.debug_pii(f"Upload error trace:\n{exc}") raise HTTPException(status_code=500, detail=f"Error uploading files: {str(e)}") @router.delete(route_prefix + "/{graphname}/uploads") async def clear_uploaded_files( graphname: str, creds: Annotated[tuple[list[str], HTTPBasicCredentials], Depends(ui_basic_auth)], filename: str | None = None, ): """ Clear uploaded files for a specific graphname. Parameters: - graphname: The graph name whose files to clear - filename: If provided, only delete this specific file. Otherwise, delete all files. """ try: upload_dir = os.path.join("uploads", graphname) if not os.path.exists(upload_dir): return { "status": "success", "message": f"No files found for graph {graphname}", "deleted_files": [], } deleted_files = [] if filename: # Delete specific file file_path = os.path.join(upload_dir, filename) if os.path.exists(file_path) and os.path.isfile(file_path): os.remove(file_path) deleted_files.append(filename) logger.info(f"Deleted file {filename} for graph {graphname}") else: raise HTTPException(status_code=404, detail=f"File {filename} not found") Injection Risk graphname is directly interpolated into a GSQL command (CREATE GRAPH {graphname}()), enabling potential GSQL injection or invalid identifiers. Validate/whitelist graph names (e.g., ^[A-Za-z_][A-Za-z0-9_]*$) and/or use proper quoting/escaping. LogWriter.info(f"Creating graph: {graphname}") create_query = f"CREATE GRAPH {graphname}()" result = tg_conn.gsql(create_query) LogWriter.info(f"Graph creation result: {result}") # Check if creation was successful if "error" in result.lower() or "failed" in result.lower(): if "already exists" in result.lower(): return { "status": "error", "message": f"Graph '{graphname}' already exists", "details": result } raise Exception(f"Failed to create graph: {result}")

[chengbiao-jin]

Please restore ecc/app/common and ecc/app/configs as symlinks.