fix: replace 'ianvs -f' with 'python benchmarking.py -f' in all example READMEs #579
+32
−32
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -246,7 +246,7 @@ Each line should be a valid JSON object with at least two fields representing th | |
| Execute the federated LLM fine-tuning benchmark: | ||
|
|
||
| ```bash | ||
| python benchmarking.py -f ./examples/federated-llm/fedllm-peft/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
| **Important**: Please change the path to `benchmarkingjob.yaml` as per your setup. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -97,8 +97,8 @@ Or you can replace the file in `yourpath/anaconda3/envs/ianvs/lib/python3.x/site | |
|
|
||
| ### Objective | ||
|
|
||
| `python benchmarking.py -f examples/government/singletask_learning_bench/objective/benchmarkingjob.yaml` | ||
|
|
||
| ### Subjective | ||
|
|
||
| `python benchmarking.py -f examples/government/singletask_learning_bench/subjective/benchmarkingjob.yaml` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -72,7 +72,7 @@ For each region's evaluation, we have four experimental designs (Type 1, 2, 3, 4 | |
|
|
||
| 2. Run the test: | ||
| ```bash | ||
| python benchmarking.py -f examples/government_rag/singletask_learning_bench/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
| ## Test Results | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -44,7 +44,7 @@ Next, download pretrained model via [[huggingface]](https://huggingface.co/optim | |
| We are now ready to run the ianvs for benchmarking image classification for high mobility scenarios on the ImageNet dataset. | ||
|
|
||
| ```python | ||
| python benchmarking.py -f ./examples/imagenet/multiedge_inference_bench/classification_job_manual.yaml | ||
|
There was a problem hiding this comment. The code block on line 46 is marked with |
||
| ``` | ||
|
|
||
| The benchmarking process takes a few minutes and varies depending on devices. | ||
|
|
@@ -75,7 +75,7 @@ Then you will find a profiler_results.yml file in the <Ianvs_HOME>/examples/imag | |
|
|
||
| Then you can run the following command to perform benchmarking: | ||
| ```shell | ||
| python benchmarking.py -f ./examples/imagenet/multiedge_inference_bench/classification_job_auto.yaml | ||
| ``` | ||
|
|
||
| After running, you will see the profit from the automatic method compared with the manual method. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -140,7 +140,7 @@ pose-estimation-llio/ | |
|
|
||
| 5. **Run Benchmarking**: | ||
| ```bash | ||
| python benchmarking.py -f examples/industrialEI/pose-estimation-llio/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -224,7 +224,7 @@ Execute the benchmark: | |
|
|
||
| ```shell | ||
| cd /ianvs | ||
| python benchmarking.py -f examples/industrialEI/single_task_learning_bench/deformable_assembly/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -36,7 +36,7 @@ pip install -r examples/llm-agent/singletask_learning_bench/requirements.txt | |
| # pipeline works. | ||
|
|
||
| # 5. Run the benchmark | ||
| python benchmarking.py -f examples/llm-agent/singletask_learning_bench/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
| First run is slow because HuggingFace downloads `Langboat/bloom-1b4-zh` (~3 GB) into `~/.cache/huggingface`. Subsequent runs use the cache and start immediately. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,11 +31,11 @@ python setup.py install | |
|
|
||
|
|
||
| ```shell | ||
| python benchmarking.py -f examples/llm-edge-benchmark-suite/single_task_bench/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
|
|
||
| ```shell | ||
| python benchmarking.py -f examples/llm-edge-benchmark-suite/single_task_bench_with_compression/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -173,7 +173,7 @@ during pipeline execution. | |
| From `$REPO_ROOT`: | ||
|
|
||
| ```bash | ||
| python benchmarking.py -f examples/llm-edge-benchmark-suite/single_task_bench/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
| Ianvs loads the configs, instantiates `LlamaCppModel`, streams inference over | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -50,7 +50,7 @@ dataset: | |||||
| Once all paths are absolute and the script is updated, execute the benchmark: | ||||||
|
|
||||||
| ```bash | ||||||
| python benchmarking.py -f ianvs/examples/llm-edge-benchmark-suite/single_task_bench_with_compression/benchmarkingjob.yaml | ||||||
|
There was a problem hiding this comment. The path contains an extra
Suggested change
|
||||||
| ``` | ||||||
|
|
||||||
| ### Expected Output | ||||||
|
|
||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -70,7 +70,7 @@ Replace the file in `yourpath/anaconda3/envs/ianvs/lib/python3.x/site-packages/s | |
|
|
||
| Run the following command: | ||
|
|
||
| `python benchmarking.py -f examples/llm/singletask_learning_bench/simple_qa/benchmarkingjob.yaml` | ||
|
|
||
| ## OpenCompass Evaluation | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -77,7 +77,7 @@ We are now ready to run the ianvs for benchmarking. | |
|
|
||
| ``` shell | ||
| cd /ianvs/project | ||
| python benchmarking.py -f examples/robot-cityscapes-synthia/lifelong_learning_bench/semantic-segmentation/benchmarkingjob.yaml | ||
| ``` | ||
|
|
||
| Finally, the user can check the result of benchmarking on the console and also in the output path( | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -130,7 +130,7 @@ To run the basic lifelong learning process: | |
|
|
||
| ```shell | ||
| cd /ianvs/project/ianvs | ||
| python benchmarking.py -f examples/robot/lifelong_learning_bench/semantic-segmentation/benchmarkingjob-simple.yaml | ||
| ``` | ||
|
|
||
| Finally, the user can check the result of benchmarking on the console and also in the output path( | ||
|
|
@@ -148,7 +148,7 @@ To run the large vision model based cloud-edge collaboration process: | |
|
|
||
| ```shell | ||
| cd /ianvs/project/ianvs | ||
| python benchmarking.py -f examples/robot/lifelong_learning_bench/semantic-segmentation/benchmarkingjob-sam.yaml | ||
| ``` | ||
|
|
||
| Finally, the user can check the result of benchmarking on the console and also in the output path( | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -107,8 +107,8 @@ Or you can replace the file in `yourpath/anaconda3/envs/ianvs/lib/python3.x/site | |
|
|
||
| ### Comment | ||
|
|
||
| `python benchmarking.py -f examples/smart_coding/smart_coding_learning_bench/comment/benchmarkingjob.yaml` | ||
|
|
||
| ### Issue | ||
|
|
||
| `python benchmarking.py -f examples/smart_coding/smart_coding_learning_bench/issue/benchmarkingjob.yaml` | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code block on line 49 is marked with
pythonsyntax highlighting, but the command inside is a shell/bash command. Please consider changing the language specifier on line 49 tobashorshellfor correct syntax highlighting.