Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 32 additions & 22 deletions config/_default/menus.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,20 @@
url = "#about"
weight = 2


[[main]]
name = "Why Volcano"
url = "#required"
weight = 3

[[main]]
name = "Blog"
url = "#posts"
weight = 4


[[main]]
name = "Why Volcano"
url = "#required"
weight = 3

#[[main]]
#name = "Documentation"
#url = "docs/"
#weight = 6

name = "Documentation"
url = "/docs/"
weight = 6

[[main_right]]
name = "<i class=\"fab fa-github\" style=\"color: #000000; font-size: 1rem; line-height: 1.25\"></i>"
Expand All @@ -45,7 +41,6 @@
weight = 20

[[main_right]]

name = "<img src=\"/img/icon_slack.svg\" alt=\"slack\" style=\" font-size: 1rem; line-height: 1.25; width:20px; height:20px; margin-top: 9px;\">"
post = ""
url = "https://cloud-native.slack.com/archives/C011GJDQS0N"
Expand Down Expand Up @@ -74,10 +69,10 @@
identifier = "features"

[[docs]]
name = "User Guide"
weight = 5
identifier = "user-guide"
url = "/en/docs/user-guide/"
name = "User Guide"
weight = 5
identifier = "user-guide"
url = "/en/docs/user-guide/"

[[docs]]
name = "Ecosystem"
Expand Down Expand Up @@ -123,25 +118,40 @@ url = "/en/docs/user-guide/"
identifier = "features"

[[v1-12-0]]
name = "Ecosystem"
name = "User Guide"
weight = 5
identifier = "ecosystem"
identifier = "user-guide"
url = "/en/docs/v1-12-0/user-guide/"

# Plugins section (top-level, collapsible container)
[[v1-12-0]]
name = "Scheduler"
name = "Plugins"
weight = 6
identifier = "plugins"

# Scheduler section
[[v1-12-0]]
name = "Scheduler"
weight = 7
identifier = "scheduler"

# Continue with remaining sections
[[v1-12-0]]
name = "Ecosystem"
weight = 8
identifier = "ecosystem"
Comment on lines +139 to +142

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The weight for Ecosystem has been changed to 8, which places it after Scheduler (weight 7). However, in the latest [[docs]] menu (lines 78-80), Ecosystem has weight 6 and appears before Scheduler. This inconsistency in the sidebar structure between versions should be addressed to ensure a uniform user experience.


[[v1-12-0]]
name = "CLI"
weight = 7
weight = 9
identifier = "cli"

[[v1-12-0]]
name = "Contribution"
weight = 8
weight = 10
identifier = "contribution"


# Documentation version v1.11.0

[[v1-11-0]]
Expand Down Expand Up @@ -344,4 +354,4 @@ url = "/en/docs/user-guide/"
[[posts]]
name = "Customers"
weight = 10
identifier = "customers"
identifier = "customers"
83 changes: 83 additions & 0 deletions content/en/docs/v1-12-0/binpack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
+++
title = "BinPack"
date = 2024-01-01
lastmod = 2024-01-01
draft = false
toc = true
type = "docs"

# Add menu entry to sidebar.
linktitle = "BinPack"
[menu.v1-12-0]
parent = "plugins"
weight = 1
+++


The **BinPack Plugin** in Volcano aims to fill nodes as efficiently as possible, trying not to leave nodes idle. It scores nodes based on resource utilization and schedules pods to maximize usage, which is especially useful for small jobs or high-concurrency workloads.

## Plugin Details

- **Plugin Name:** `binpack`
- **Location in source:** `pkg/scheduler/plugins/binpack/binpack.go`
- **Purpose:** Optimizes node usage by packing pods onto fewer nodes, improving cluster resource utilization.

## Features

1. **Node Scoring**
- Scores nodes according to available resources.
- Higher scores indicate better utilization of node resources.

2. **Weighted Resource Consideration**
- Considers CPU, memory, and other resources with configurable weights.
- Different resources can be prioritized in node selection.

3. **Pod Placement**
- Fills existing nodes as much as possible before using new nodes.
- Reduces fragmentation and improves autoscaling efficiency.

4. **Integration with Scheduler**
- Injected as a plugin into Volcano's scheduling process.
- Works at the pod node-selection stage.

## Source Overview

```go
// Example snippet from binpack.go
func New(arguments framework.Arguments) framework.Plugin {
return &binPackPlugin{pluginArguments: arguments}
}
```
The full implementation includes functions for scoring nodes, handling resource weights, and integrating with the scheduling session.

## Example Configuration

To enable the BinPack plugin, add it to the scheduler configuration:

```yaml
# scheduler configuration
actions: "enqueue, allocate, preempt"
tiers:
- plugins:
- name: binpack
arguments:
binpack.cpu: 5
binpack.memory: 1
```
### Configuration Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `binpack.cpu` | Weight for CPU resources in scoring | 5 |
| `binpack.memory` | Weight for memory resources in scoring | 1 |
| `binpack.gpu` | Weight for GPU resources in scoring | 0 |

## Usage Scenarios

BinPack is particularly useful for:

- **Batch processing jobs** where maximizing node utilization reduces cost
- **Single query jobs** that run quickly and benefit from efficient packing
- **High-concurrency services** with many small pods that can be packed onto fewer nodes
- **Cloud cost optimization** where reducing node count minimizes infrastructure costs

84 changes: 84 additions & 0 deletions content/en/docs/v1-12-0/gang.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
+++
title = "Gang"
date = 2024-01-01
lastmod = 2024-01-01
draft = false
toc = true
type = "docs"

# Add menu entry to sidebar.
linktitle = "Gang"
[menu.v1-12-0]
parent = "plugins"
weight = 2
+++



{{<figure library="1" src="gang.png" title="Gang Plugin">}}


The **Gang Plugin** in Volcano enforces gang scheduling constraints, ensuring that a group of related tasks (a "gang") is scheduled together. This plugin is responsible for validating jobs, managing preemption, pipelining, and detecting job starvation.

## Plugin Details

- **Plugin Name:** `gang`
- **Location in source:** `pkg/scheduler/plugins/gang/gang.go`
- **Purpose:** Ensures gang scheduling constraints and job readiness for tasks and sub-jobs.

## Features

1. **Job Validation**
- Validates tasks and sub-jobs for gang scheduling.
- Checks minimum required tasks (`MinAvailable`) before scheduling.

2. **Preemption**
- Determines which tasks can be preempted without violating gang scheduling constraints.
- Tracks ready task counts per job to maintain scheduling guarantees.

3. **Job Ordering**
- Orders jobs and sub-jobs based on readiness.
- Ready jobs have higher priority over non-ready jobs.

4. **Pipelining**
- Allows pipelined scheduling when tasks and sub-jobs meet pipelined conditions.

5. **Starvation Detection**
- Detects jobs that cannot be scheduled due to insufficient resources.

## Source Overview

```go
// Example snippet from gang.go
func New(arguments framework.Arguments) framework.Plugin {
return &gangPlugin{pluginArguments: arguments}
}
```
The full implementation includes functions for `OnSessionOpen`, `OnSessionClose`, job validation, preemption, pipelining, and readiness checks.

## Example Configuration

To enable the Gang plugin, add it to the scheduler configuration:

```yaml
# scheduler configuration
arguments:
timeout: 600
```

### Configuration Parameters

| Parameter | Description | Default |
|-----------|-------------|---------|
| `gang.schedule.timeout` | Timeout in seconds for gang scheduling | 600 |
| `gang.schedule.retry` | Number of retry attempts for gang scheduling | 5 |

## Usage Scenarios

Gang scheduling is typically used for:

- **Distributed training jobs** where all workers must start together
- **MPI jobs** that require all processes to be running simultaneously
- **Batch processing workloads** with task dependencies
- **Big data analytics** where partial execution is not meaningful

Loading