Implement vLLM FSDP LoRA hot-swapping integration#10

Open

jacobthebanana wants to merge 92 commits intomasterfrom

jjt/lora-vllm-hotswap

Commits on Feb 26, 2024

Implemented baseline LoRA peft for one Nvidia GPU.
jacobthebanana
committed

Commits on Feb 27, 2024

Added support for saving lora adapters.
jacobthebanana
committed

Commits on Mar 11, 2024

Implemented LoRA benchmarking logic for worker.
jacobthebanana
committed
model_utils: Refactored get_lora_model to reduce interface width. (this method no longer wraps load_model_and_tokenizer)
jacobthebanana
committed
test_modelling: moved text output to data/.
jacobthebanana
committed
added example yaml config for lora benchmarking.
jacobthebanana
committed
launch_benchmark: marked qos flag as optional.
jacobthebanana
committed
launch_benchmark: added option to limit number of jobs launched.
jacobthebanana
committed
launch_benchmark: implemented torch profiler integration.
jacobthebanana
committed
Merged changes from low CPU memory usage feature (#6) into jjt/lora-benchmarking

adil-a
authored and
jacobthebanana
committed
Revised launch_benchmark.py to use new profiling path.
jacobthebanana
committed
Enabled automatic creation of data/trace folder.
jacobthebanana
committed
Added instructions for profiling tools.
jacobthebanana
committed
Merge remote-tracking branch 'origin/master' into jjt/lora-baseline
jacobthebanana
committed
Cleaned up duplicate imports from merge.
jacobthebanana
committed
Cleaned up duplicate imports from merge.
jacobthebanana
committed
Cleaned up parse_benchmark.py
jacobthebanana
committed
Integrated LoRA logic into llama_example.py.
jacobthebanana
committed
Moved lora_configs into train_parameters in config yaml. Adjusted docs/config.md accordingly.
jacobthebanana
committed

Commits on Mar 13, 2024

Commits on Mar 15, 2024

Commits on Apr 9, 2024

Implemented non-LoRA profiling and benchmarking.
jacobthebanana
committed

Commits on Apr 11, 2024

Various static typechecking and formatting fixes.
jacobthebanana
committed

Commits on Apr 15, 2024

Commits on Apr 16, 2024

Commits on Apr 17, 2024

Commits on Apr 19, 2024

Merge commit 'afbc061' from jjt/lora-baseline into jjt/lora-vllm-hotswap
jacobthebanana
committed

Commits on Apr 22, 2024

Commits on Apr 25, 2024

Commits on May 7, 2024

vllm integration [WIP]: Implemented lora hotswap.
jacobthebanana
committed

Commits on May 9, 2024

Commits on May 10, 2024

Commits on May 23, 2024

Commits on Jun 18, 2024