diff --git a/docs/FedOps-Aggregation-method/FedOps-Aggregation-method.md b/docs/FedOps-Aggregation-method/FedOps-Aggregation-method.md index eadec023f9..17c0d199e8 100644 --- a/docs/FedOps-Aggregation-method/FedOps-Aggregation-method.md +++ b/docs/FedOps-Aggregation-method/FedOps-Aggregation-method.md @@ -1,7 +1,7 @@ --- layout: default title: FedOps Aggregation method -nav_order: 7 +nav_order: 8 has_children: true permalink: docs/FedOps-Aggregation-method --- diff --git a/docs/FedOps-Fitbit-Health-Pipeline/FedOps-Fitbit-Health-Pipeline.md b/docs/FedOps-Fitbit-Health-Pipeline/FedOps-Fitbit-Health-Pipeline.md index 0a50c9737a..496d708e27 100644 --- a/docs/FedOps-Fitbit-Health-Pipeline/FedOps-Fitbit-Health-Pipeline.md +++ b/docs/FedOps-Fitbit-Health-Pipeline/FedOps-Fitbit-Health-Pipeline.md @@ -1,7 +1,7 @@ --- layout: default title: FedOps Fitbit Health Pipeline -nav_order: 10 +nav_order: 11 has_children: true permalink: docs/FedOps-Fitbit-Health-Pipeline --- diff --git a/docs/FedOps-Hyperparameter-Optimize/FedOps-Hyperparameter-Optimize.md b/docs/FedOps-Hyperparameter-Optimize/FedOps-Hyperparameter-Optimize.md index e12259559e..d3748bc162 100644 --- a/docs/FedOps-Hyperparameter-Optimize/FedOps-Hyperparameter-Optimize.md +++ b/docs/FedOps-Hyperparameter-Optimize/FedOps-Hyperparameter-Optimize.md @@ -1,7 +1,7 @@ --- layout: default title: FedOps Clustering Tuning -nav_order: 8 +nav_order: 9 has_children: true permalink: docs/FedOps-Clustering-Tuning --- diff --git a/docs/FedOps-VLM/FedOps-VLM.md b/docs/FedOps-VLM/FedOps-VLM.md new file mode 100644 index 0000000000..ddead11cfe --- /dev/null +++ b/docs/FedOps-VLM/FedOps-VLM.md @@ -0,0 +1,8 @@ +--- +layout: default +title: FedOps VLM +nav_order: 7 +has_children: true +permalink: docs/FedOpsVLM +--- +# FedOps VLM \ No newline at end of file diff --git a/docs/FedOps-VLM/How-to-use-FedOps-VLM-Framework.md b/docs/FedOps-VLM/How-to-use-FedOps-VLM-Framework.md new file mode 100644 index 0000000000..ec7027013b --- /dev/null +++ b/docs/FedOps-VLM/How-to-use-FedOps-VLM-Framework.md @@ -0,0 +1,147 @@ +--- +layout: default +title: How to use FedOps VLM Framework? +parent: FedOps VLM +nav_order: 2 +--- +# 📝 FedOps VLM Guide + +
+ +## 1. Download the Codebase Setup + +{: .highlight } +💡 Go to https://gachon-cclab.github.io/fedopsMultimodal/ +Select Your model, Dataset, FL Configs, aggregation strategy and Click Download Codebase. + +select as below: + +- Model: ***OneVision 0.5B*** + +- Dataset: ***VQA-RAD*** + +- Aggregation Strategy: ***FedMAP*** + +
+ +![image.png](../../img/FedOps-VLM/image1.png) + + and then click: + +![image.png](../../img/FedOps-VLM/image2.png) + +now you will automatically download a zip folder with whole codebase files. +Unzip it. + +## **2. Now, the codebase include following files.** + +`conf/config.yaml` - This is where configurations reside (including the task ID) + +`client_main.py` - This is the main file the client side logics reside + +`client_manager_main.py` - This is the connection check file of client side + +`data_preparation.py` - Data loading, preprocessing logics reside here + +`models.py`, `generate_paramshape.py` - file, which will generate model’s parameter shape + +`parameter_names.json` - this is needed for the client side to distinguish parameter types + +`parameter_shapes.json` - Generated parameter shape of the model + +`setup.sh` - The file, which sets up the environment including dependencies of client side + + +## **3. Open the folder in your IDE (VSCode)** + +Open your bash terminal, + +then run + +``` +conda create -n *fedops_multimodal_env python=3.12.7 + +conda activate fedops_multimodal_env + +chmod +x setup.sh + +bash setup.sh +``` + +![image.png](../../img/FedOps-VLM/image3.png) + +and then run `python client_main.py` + +![image.png](../../img/FedOps-VLM/image4.png) + +Now open a new terminal in the same folder path, and run + +`python client_manager_main.py` + + +## **4. Server Side Code Management** + +Visit: https://ccl.gachon.ac.kr/fedops/task + +signin and create a task named task id: ***”fedopstune2”***. and do the followings + +![image.png](../../img/FedOps-VLM/image5.png) + +Leave the rest as it is. + +Go to Server Management +Press Below shown green button to create a Server pod in Fedops K8 Environment + +![image.png](../../img/FedOps-VLM/image6.png) + +Scale the Resources(memory): please do enter Scale Resources button after entering 10Gi as memory. + +![image.png](../../img/FedOps-VLM/image7.png) + +Now check the server status through below picture. keep refreshing the status and wait until you see similar to this. + +![image.png](../../img/FedOps-VLM/image8.png) + +Wait for around 6-7 mins until you see yellow colored log in Server log section below in same server management tab (shown in picture). Keep refreshing the logs to see real time log. + +![image.png](../../img/FedOps-VLM/image9.png) + +**Then, start editing server side files** + +{: .highlight } +💡 You can check server side files structure by typing file browser path as below and clicking browse button + +![image.png](../../img/FedOps-VLM/image10.png) + +{: .highlight } +💡 Paste the **file names with path** in **yellow** colored text **space** → **Copy & paste** the file content **from downloaded folder** content **into code space** → **Press save file** button. Likewise this, **do for each and every file path** mentioned below. + +filenames with path as follows: + +`/app/code/conf/config.yaml` + +`/app/code/parameter_shapes.json` -Just copy paste this content ,so it will automatically gets created + +`/app/code/parameter_names.json` -Just copy paste this content ,so it will automatically gets created + +`/app/code/server_main.py` + + +![image.png](../../img/FedOps-VLM/image11.png) + +Install below dependency as below by pressing execute: + +![image.png](../../img/FedOps-VLM/image12.png) + +## **5. Start the FL Server** + +**Start the actual server:** + +{: .highlight } +💡 Click Start FL Server button to prepare the command to run the FL server. you can then see log below says FL server created. + +{: .highlight } +💡 you must type **/app/data/logs/serverlog.txt** in File content field and press load button to see real time server side logs, and monitor server side FL global model training process. + +{: .highlight } +💡 You can stop the server by “**stop FL server**” button ,if you want to stop the server in middle. \ No newline at end of file diff --git a/docs/FedOps-VLM/What-is-FedOps-VLM-Framework.md b/docs/FedOps-VLM/What-is-FedOps-VLM-Framework.md new file mode 100644 index 0000000000..0621932a37 --- /dev/null +++ b/docs/FedOps-VLM/What-is-FedOps-VLM-Framework.md @@ -0,0 +1,59 @@ +--- +layout: default +title: What is FedOps VLM Framework? +nav_order: 1 +parent: FedOps VLM +--- +# ✅ What is FedOps VLM Framework?? +
+ +**FedOps VLM Framework** is an open, cloud-native federated learning (FL) platform purpose-built for **Vision-Language Models (VLMs)**. It bridges the gap between large-scale multimodal AI and privacy-preserving distributed training — enabling researchers and practitioners to fine-tune powerful VLMs across decentralized edge devices without ever centralizing raw data. + +Built on top of the **FedOps** ecosystem (PyPI `fedops`), the framework combines: + +- **Flower** — for robust FL communication and client–server orchestration +- **PEFT / LoRA / QLoRA** — for parameter-efficient fine-tuning of large VLMs on resource-constrained devices +- **FedMAP (Parameter-Aware Federated Aggregation)** — a novel component-aware weighted aggregation theorem that achieves tighter convergence bounds than flat averaging when LoRA and projection parameters have different learning dynamics. + +The result is a full-stack FL solution that works from a **web portal download** all the way to a **Kubernetes-managed cloud FL server** — reducing the barrier to federated VLM research to just a few configuration steps. + +--- + +## Core Concepts + +### Federated Learning for VLMs + +Traditional VLM fine-tuning requires pooling sensitive data into a central server. FedOps VLM Framework flips this: **the data never leaves the device**. Only lightweight parameter updates are sent to the server for aggregation. + +### Plugin-Based Architecture + +**Plugin-based** — models, datasets, and aggregation strategies are swappable. Pick your combination from the web portal, download, and train. No boilerplate wiring required. + +### Supported Models + +| Model | Size | Quantization | +| --- | --- | --- | +| OneVision | 0.5B | QLoRA (4-bit) | +| PhiVA | 3.8B | QLoRA (4-bit) | + +> Custom models via `models.py` + +### Supported Datasets + +| Dataset | Domain | Size | +| --- | --- | --- | +| VQA-RAD | Medical radiology | 1793 train, 451 test | +| VQAv2 | General vision | Large-scale | +| PathVQA | Pathology | ~19.7K train | +| SLAKE | Multi-organ medical | ~4.9K train, 1K test | +| ChartQA | Chart reasoning | ~18K train | +| DocVQA | Document understanding | ~10K train | + +> Custom datasets via `data_preparation.py` + +### Aggregation Strategies + +| Strategy | Description | +| --- | --- | +| FedAvg | Standard weighted average | +| FedMAP | Component-aware QP — separate λ for LoRA vs. projection layers | diff --git a/docs/FedOps-XAI/FedOps-XAI.md b/docs/FedOps-XAI/FedOps-XAI.md index 2d4b244e05..7651121d75 100644 --- a/docs/FedOps-XAI/FedOps-XAI.md +++ b/docs/FedOps-XAI/FedOps-XAI.md @@ -1,7 +1,7 @@ --- layout: default title: FedOps XAI -nav_order: 9 +nav_order: 10 has_children: true permalink: docs/FedOps-XAI --- diff --git a/docs/FlowerHub with GFedOps/FedOps-Launcher/How-to-use-FedOps-Launcher.md b/docs/FlowerHub with GFedOps/FedOps-Launcher/How-to-use-FedOps-Launcher.md index f7087b5c70..e0a44603b7 100644 --- a/docs/FlowerHub with GFedOps/FedOps-Launcher/How-to-use-FedOps-Launcher.md +++ b/docs/FlowerHub with GFedOps/FedOps-Launcher/How-to-use-FedOps-Launcher.md @@ -18,6 +18,8 @@ This guide provides step-by-step instructions for using FedOps Launcher, a FedOp ### 1. Install Docker +FedOps Launcher runs as a Docker container. Docker is required to provide an isolated execution environment for the launcher, manage the workspace volume, and run the FL client consistently across different operating systems. + [Docker Download](https://www.docker.com/get-started/) ![image.png](../../../img/FedOps-Launcher/image(1).png) @@ -36,7 +38,7 @@ This guide provides step-by-step instructions for using FedOps Launcher, a FedOp docker rm -f fedops-launcher 2>/dev/null; docker run -d --name fedops-launcher -p 5600:5600 -e WORKSPACE_DIR=/workspace -v "$HOME/fedops-workspace:/workspace" joseongjin311/fedops-launcher:v1.0.0 && echo "Now Running On: http://127.0.0.1:5600" ``` -In my case, I use VSCode in a Windows WSL environment. +The examples in this guide are performed in VSCode within a Linux environment provided by Windows WSL. Just copy the command and run it in your CLI environment. @@ -118,7 +120,7 @@ If you are logged in, you can choose a Task what you created on FedOps website. ![image.png](../../../img/FedOps-Launcher/image(12).png) -**"ssj" is my Task name, so you should select your own Task** +**In this example, "ssj" is the Task name. Select the Task created for your own environment.** Next, select the FL project to which you want to apply the Task. @@ -134,6 +136,14 @@ All settings are complete. Just click the **Run** button. ![image.png](../../../img/FedOps-Launcher/image(14).png) +### 7. Stop FedOps Launcher + +When you finish using FedOps Launcher, stop and remove the running container with the following command: + +```bash +docker rm -f fedops-launcher +``` + **Thank you.** -**You have successfully started the FedOps Launcher.** +**You have successfully learned how to use the FedOps Launcher.** \ No newline at end of file diff --git a/docs/FlowerHub with GFedOps/FedOps-Launcher/What-is-FedOps-Launcher.md b/docs/FlowerHub with GFedOps/FedOps-Launcher/What-is-FedOps-Launcher.md index 8ea8414785..77f0871a39 100644 --- a/docs/FlowerHub with GFedOps/FedOps-Launcher/What-is-FedOps-Launcher.md +++ b/docs/FlowerHub with GFedOps/FedOps-Launcher/What-is-FedOps-Launcher.md @@ -21,4 +21,4 @@ Key points: - FedOps Launcher provides a Web UI for preparing and running GFedOps Apps. - Users can import a GFedOps App from FlowerHub and create a local client project. - It supports code editing, dependency installation, FedOps task selection, and client execution in one workspace. -- Users can join a FedOps federated learning task as a real client more easily. +- Users can join a FedOps federated learning task as a real client more easily. \ No newline at end of file diff --git a/docs/FlowerHub with GFedOps/FlowerHub-with-GFedOps.md b/docs/FlowerHub with GFedOps/FlowerHub-with-GFedOps.md index 95a2d82588..f66b1c6538 100644 --- a/docs/FlowerHub with GFedOps/FlowerHub-with-GFedOps.md +++ b/docs/FlowerHub with GFedOps/FlowerHub-with-GFedOps.md @@ -1,8 +1,8 @@ --- layout: default title: FlowerHub with GFedOps -nav_order: 11 +nav_order: 12 has_children: true -permalink: docs/FedOps-Launcher +permalink: docs/FlowerHub-with-GFedOps --- # FlowerHub with GFedOps \ No newline at end of file diff --git a/docs/img/FedOps-Launcher/image(10).png b/docs/img/FedOps-Launcher/image(10).png index 35a8163639..6ea7f23d7d 100644 Binary files a/docs/img/FedOps-Launcher/image(10).png and b/docs/img/FedOps-Launcher/image(10).png differ diff --git a/docs/img/FedOps-Launcher/image(11).png b/docs/img/FedOps-Launcher/image(11).png index 7447085398..6c65dee9e2 100644 Binary files a/docs/img/FedOps-Launcher/image(11).png and b/docs/img/FedOps-Launcher/image(11).png differ diff --git a/docs/img/FedOps-Launcher/image(13).png b/docs/img/FedOps-Launcher/image(13).png index d22d244b8e..41a6ce2381 100644 Binary files a/docs/img/FedOps-Launcher/image(13).png and b/docs/img/FedOps-Launcher/image(13).png differ diff --git a/docs/img/FedOps-Launcher/image(14).png b/docs/img/FedOps-Launcher/image(14).png index c21b572fa1..7c2d49123c 100644 Binary files a/docs/img/FedOps-Launcher/image(14).png and b/docs/img/FedOps-Launcher/image(14).png differ diff --git a/docs/img/FedOps-Launcher/image(9).png b/docs/img/FedOps-Launcher/image(9).png index 8e753e4819..74a761c35f 100644 Binary files a/docs/img/FedOps-Launcher/image(9).png and b/docs/img/FedOps-Launcher/image(9).png differ diff --git a/docs/img/FedOps-VLM/image1.png b/docs/img/FedOps-VLM/image1.png new file mode 100644 index 0000000000..6e6f92283d Binary files /dev/null and b/docs/img/FedOps-VLM/image1.png differ diff --git a/docs/img/FedOps-VLM/image10.png b/docs/img/FedOps-VLM/image10.png new file mode 100644 index 0000000000..ded2f416fa Binary files /dev/null and b/docs/img/FedOps-VLM/image10.png differ diff --git a/docs/img/FedOps-VLM/image11.png b/docs/img/FedOps-VLM/image11.png new file mode 100644 index 0000000000..6b73c4b382 Binary files /dev/null and b/docs/img/FedOps-VLM/image11.png differ diff --git a/docs/img/FedOps-VLM/image12.png b/docs/img/FedOps-VLM/image12.png new file mode 100644 index 0000000000..d06f158ed4 Binary files /dev/null and b/docs/img/FedOps-VLM/image12.png differ diff --git a/docs/img/FedOps-VLM/image2.png b/docs/img/FedOps-VLM/image2.png new file mode 100644 index 0000000000..8fca5ef205 Binary files /dev/null and b/docs/img/FedOps-VLM/image2.png differ diff --git a/docs/img/FedOps-VLM/image3.png b/docs/img/FedOps-VLM/image3.png new file mode 100644 index 0000000000..7241250c84 Binary files /dev/null and b/docs/img/FedOps-VLM/image3.png differ diff --git a/docs/img/FedOps-VLM/image4.png b/docs/img/FedOps-VLM/image4.png new file mode 100644 index 0000000000..4e3d8eb59b Binary files /dev/null and b/docs/img/FedOps-VLM/image4.png differ diff --git a/docs/img/FedOps-VLM/image5.png b/docs/img/FedOps-VLM/image5.png new file mode 100644 index 0000000000..84059c642e Binary files /dev/null and b/docs/img/FedOps-VLM/image5.png differ diff --git a/docs/img/FedOps-VLM/image6.png b/docs/img/FedOps-VLM/image6.png new file mode 100644 index 0000000000..7bbbd67752 Binary files /dev/null and b/docs/img/FedOps-VLM/image6.png differ diff --git a/docs/img/FedOps-VLM/image7.png b/docs/img/FedOps-VLM/image7.png new file mode 100644 index 0000000000..3db2aee18a Binary files /dev/null and b/docs/img/FedOps-VLM/image7.png differ diff --git a/docs/img/FedOps-VLM/image8.png b/docs/img/FedOps-VLM/image8.png new file mode 100644 index 0000000000..7734d5a161 Binary files /dev/null and b/docs/img/FedOps-VLM/image8.png differ diff --git a/docs/img/FedOps-VLM/image9.png b/docs/img/FedOps-VLM/image9.png new file mode 100644 index 0000000000..42e64de70f Binary files /dev/null and b/docs/img/FedOps-VLM/image9.png differ diff --git a/docs/industry-pack.md b/docs/industry-pack.md index c6872f0e4d..0596554268 100644 --- a/docs/industry-pack.md +++ b/docs/industry-pack.md @@ -1,7 +1,7 @@ --- layout: default title: FedOps Industry Pack -nav_order: 12 +nav_order: 13 --- # FedOps Industry Pack diff --git a/docs/package.md b/docs/package.md index 19a0223d15..a2949ba902 100644 --- a/docs/package.md +++ b/docs/package.md @@ -1,7 +1,7 @@ --- layout: default title: FedOps Package -nav_order: 13 +nav_order: 14 # h2부터 h5까지 제목을 표시합니다 toc_min_heading_level: 2 toc_max_heading_level: 5