Fine-tuning LLMs on Palmetto 2
Workshop Description
This workshop series introduces essential concepts related to the fine-tuning of large language models (LLMs), and teaches how to fine-tune an LLM using PyTorch on Palmetto 2. Topics include: when fine-tuning is appropriate (and when it is not the right solution), parameter-efficient fine-tuning methods vs. full fine-tuning and what kind and quantity of data is required for fine-tuning. Participants will learn how to efficiently use Palmetto 2 resources to fine-tune (pre-trained) LLMs.
Prerequisites
- All workshop participants should have a Palmetto 2 cluster account. If you do not already have an account, you can visit our getting started page.
- Participants should be familiar with the Python programming language. This requirement could be fulfilled by personal projects, coursework or completion of the Introduction to Python Programming workshop series.
- Participants are expected to have experience running LLMs on Palmetto 2 or on their own workstations. These prerequisites could be satisfied by previous participation in either of our workshops: Attention, Transformers, and LLMs or Running LLMs on Palmetto 2.
Session Information
Session #1 for Spring 2025
Resources
For a self-guided version, you can read the Fine-Tuning LLMs on Palmetto 2 Workshop notebooks on our workshop site.