Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: FedOps Aggregation method
nav_order: 7
nav_order: 8
has_children: true
permalink: docs/FedOps-Aggregation-method
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: FedOps Fitbit Health Pipeline
nav_order: 10
nav_order: 11
has_children: true
permalink: docs/FedOps-Fitbit-Health-Pipeline
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: FedOps Clustering Tuning
nav_order: 8
nav_order: 9
has_children: true
permalink: docs/FedOps-Clustering-Tuning
---
Expand Down
8 changes: 8 additions & 0 deletions docs/FedOps-VLM/FedOps-VLM.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
layout: default
title: FedOps VLM
nav_order: 7
has_children: true
permalink: docs/FedOpsVLM
---
# FedOps VLM
147 changes: 147 additions & 0 deletions docs/FedOps-VLM/How-to-use-FedOps-VLM-Framework.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
---
layout: default
title: How to use FedOps VLM Framework?
parent: FedOps VLM
nav_order: 2
---
# 📝 FedOps VLM Guide

<br />

## 1. Download the Codebase Setup

{: .highlight }
💡 Go to https://gachon-cclab.github.io/fedopsMultimodal/
Select Your model, Dataset, FL Configs, aggregation strategy and Click Download Codebase.

select as below:

- Model: ***OneVision 0.5B***

- Dataset: ***VQA-RAD***

- Aggregation Strategy: ***FedMAP***

<br />

![image.png](../../img/FedOps-VLM/image1.png)

and then click:

![image.png](../../img/FedOps-VLM/image2.png)

now you will automatically download a zip folder with whole codebase files.
Unzip it.

## **2. Now, the codebase include following files.**

`conf/config.yaml` - This is where configurations reside (including the task ID)

`client_main.py` - This is the main file the client side logics reside

`client_manager_main.py` - This is the connection check file of client side

`data_preparation.py` - Data loading, preprocessing logics reside here

`models.py`, `generate_paramshape.py` - file, which will generate model’s parameter shape

`parameter_names.json` - this is needed for the client side to distinguish parameter types

`parameter_shapes.json` - Generated parameter shape of the model

`setup.sh` - The file, which sets up the environment including dependencies of client side


## **3. Open the folder in your IDE (VSCode)**

Open your bash terminal,

then run

```
conda create -n *fedops_multimodal_env python=3.12.7

conda activate fedops_multimodal_env

chmod +x setup.sh

bash setup.sh
```

![image.png](../../img/FedOps-VLM/image3.png)

and then run `python client_main.py`

![image.png](../../img/FedOps-VLM/image4.png)

Now open a new terminal in the same folder path, and run

`python client_manager_main.py`


## **4. Server Side Code Management**

Visit: https://ccl.gachon.ac.kr/fedops/task

signin and create a task named task id: ***”fedopstune2”***. and do the followings

![image.png](../../img/FedOps-VLM/image5.png)

Leave the rest as it is.

Go to Server Management
Press Below shown green button to create a Server pod in Fedops K8 Environment

![image.png](../../img/FedOps-VLM/image6.png)

Scale the Resources(memory): please do enter Scale Resources button after entering 10Gi as memory.

![image.png](../../img/FedOps-VLM/image7.png)

Now check the server status through below picture. keep refreshing the status and wait until you see similar to this.

![image.png](../../img/FedOps-VLM/image8.png)

Wait for around 6-7 mins until you see yellow colored log in Server log section below in same server management tab (shown in picture). Keep refreshing the logs to see real time log.

![image.png](../../img/FedOps-VLM/image9.png)

**Then, start editing server side files**

{: .highlight }
💡 You can check server side files structure by typing file browser path as below and clicking browse button

![image.png](../../img/FedOps-VLM/image10.png)

{: .highlight }
💡 Paste the **file names with path** in **yellow** colored text **space** → **Copy & paste** the file content **from downloaded folder** content **into code space** → **Press save file** button. Likewise this, **do for each and every file path** mentioned below.

filenames with path as follows:

`/app/code/conf/config.yaml`

`/app/code/parameter_shapes.json` -Just copy paste this content ,so it will automatically gets created

`/app/code/parameter_names.json` -Just copy paste this content ,so it will automatically gets created

`/app/code/server_main.py`


![image.png](../../img/FedOps-VLM/image11.png)

Install below dependency as below by pressing execute:

![image.png](../../img/FedOps-VLM/image12.png)

## **5. Start the FL Server**

**Start the actual server:**

{: .highlight }
💡 Click Start FL Server button to prepare the command to run the FL server. you can then see log below says FL server created.

{: .highlight }
💡 you must type **/app/data/logs/serverlog.txt** in File content field and press load button to see real time server side logs, and monitor server side FL global model training process.

{: .highlight }
💡 You can stop the server by “**stop FL server**” button ,if you want to stop the server in middle.
59 changes: 59 additions & 0 deletions docs/FedOps-VLM/What-is-FedOps-VLM-Framework.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
layout: default
title: What is FedOps VLM Framework?
nav_order: 1
parent: FedOps VLM
---
# ✅ What is FedOps VLM Framework??
<br />

**FedOps VLM Framework** is an open, cloud-native federated learning (FL) platform purpose-built for **Vision-Language Models (VLMs)**. It bridges the gap between large-scale multimodal AI and privacy-preserving distributed training — enabling researchers and practitioners to fine-tune powerful VLMs across decentralized edge devices without ever centralizing raw data.

Built on top of the **FedOps** ecosystem (PyPI `fedops`), the framework combines:

- **Flower** — for robust FL communication and client–server orchestration
- **PEFT / LoRA / QLoRA** — for parameter-efficient fine-tuning of large VLMs on resource-constrained devices
- **FedMAP (Parameter-Aware Federated Aggregation)** — a novel component-aware weighted aggregation theorem that achieves tighter convergence bounds than flat averaging when LoRA and projection parameters have different learning dynamics.

The result is a full-stack FL solution that works from a **web portal download** all the way to a **Kubernetes-managed cloud FL server** — reducing the barrier to federated VLM research to just a few configuration steps.

---

## Core Concepts

### Federated Learning for VLMs

Traditional VLM fine-tuning requires pooling sensitive data into a central server. FedOps VLM Framework flips this: **the data never leaves the device**. Only lightweight parameter updates are sent to the server for aggregation.

### Plugin-Based Architecture

**Plugin-based** — models, datasets, and aggregation strategies are swappable. Pick your combination from the web portal, download, and train. No boilerplate wiring required.

### Supported Models

| Model | Size | Quantization |
| --- | --- | --- |
| OneVision | 0.5B | QLoRA (4-bit) |
| PhiVA | 3.8B | QLoRA (4-bit) |

> Custom models via `models.py`

### Supported Datasets

| Dataset | Domain | Size |
| --- | --- | --- |
| VQA-RAD | Medical radiology | 1793 train, 451 test |
| VQAv2 | General vision | Large-scale |
| PathVQA | Pathology | ~19.7K train |
| SLAKE | Multi-organ medical | ~4.9K train, 1K test |
| ChartQA | Chart reasoning | ~18K train |
| DocVQA | Document understanding | ~10K train |

> Custom datasets via `data_preparation.py`

### Aggregation Strategies

| Strategy | Description |
| --- | --- |
| FedAvg | Standard weighted average |
| FedMAP | Component-aware QP — separate λ for LoRA vs. projection layers |
2 changes: 1 addition & 1 deletion docs/FedOps-XAI/FedOps-XAI.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: FedOps XAI
nav_order: 9
nav_order: 10
has_children: true
permalink: docs/FedOps-XAI
---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ This guide provides step-by-step instructions for using FedOps Launcher, a FedOp

### 1. Install Docker

FedOps Launcher runs as a Docker container. Docker is required to provide an isolated execution environment for the launcher, manage the workspace volume, and run the FL client consistently across different operating systems.

[Docker Download](https://www.docker.com/get-started/)

![image.png](../../../img/FedOps-Launcher/image(1).png)
Expand All @@ -36,7 +38,7 @@ This guide provides step-by-step instructions for using FedOps Launcher, a FedOp
docker rm -f fedops-launcher 2>/dev/null; docker run -d --name fedops-launcher -p 5600:5600 -e WORKSPACE_DIR=/workspace -v "$HOME/fedops-workspace:/workspace" joseongjin311/fedops-launcher:v1.0.0 && echo "Now Running On: http://127.0.0.1:5600"
```

In my case, I use VSCode in a Windows WSL environment.
The examples in this guide are performed in VSCode within a Linux environment provided by Windows WSL.

Just copy the command and run it in your CLI environment.

Expand Down Expand Up @@ -118,7 +120,7 @@ If you are logged in, you can choose a Task what you created on FedOps website.

![image.png](../../../img/FedOps-Launcher/image(12).png)

**"ssj" is my Task name, so you should select your own Task**
**In this example, "ssj" is the Task name. Select the Task created for your own environment.**

Next, select the FL project to which you want to apply the Task.

Expand All @@ -134,6 +136,14 @@ All settings are complete. Just click the **Run** button.

![image.png](../../../img/FedOps-Launcher/image(14).png)

### 7. Stop FedOps Launcher

When you finish using FedOps Launcher, stop and remove the running container with the following command:

```bash
docker rm -f fedops-launcher
```

**Thank you.**

**You have successfully started the FedOps Launcher.**
**You have successfully learned how to use the FedOps Launcher.**
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ Key points:
- FedOps Launcher provides a Web UI for preparing and running GFedOps Apps.
- Users can import a GFedOps App from FlowerHub and create a local client project.
- It supports code editing, dependency installation, FedOps task selection, and client execution in one workspace.
- Users can join a FedOps federated learning task as a real client more easily.
- Users can join a FedOps federated learning task as a real client more easily.
4 changes: 2 additions & 2 deletions docs/FlowerHub with GFedOps/FlowerHub-with-GFedOps.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
layout: default
title: FlowerHub with GFedOps
nav_order: 11
nav_order: 12
has_children: true
permalink: docs/FedOps-Launcher
permalink: docs/FlowerHub-with-GFedOps
---
# FlowerHub with GFedOps
Binary file modified docs/img/FedOps-Launcher/image(10).png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/FedOps-Launcher/image(11).png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/FedOps-Launcher/image(13).png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/FedOps-Launcher/image(14).png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/FedOps-Launcher/image(9).png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image10.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image11.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image12.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image7.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image8.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/FedOps-VLM/image9.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/industry-pack.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: FedOps Industry Pack
nav_order: 12
nav_order: 13
---

# FedOps Industry Pack
Expand Down
2 changes: 1 addition & 1 deletion docs/package.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: FedOps Package
nav_order: 13
nav_order: 14
# h2부터 h5까지 제목을 표시합니다
toc_min_heading_level: 2
toc_max_heading_level: 5
Expand Down
Loading