Index
Training GenAI will generally be domain/modality specific.
Important concepts¶
Training Objectives¶
- Next token
- Masked token
- Embedding similarity
What is done¶
- Self-supervised pre-training to predict the next token with reasonable likelihoods.
- Supervised or self-supervised Finetuning on higher quality data sets.
- These datasets may be done so recursively using simulated data.
- Automatically correcting models to more accurately train a model to produce output that is is more globally accurate to the input prompts.
Retrieval Aware Training¶
GRIT: Generative Representational Instruction Tuning
Developments The authors reveal in their paper the ability to simultaneously train generation and embedding models, revealing improved performance in both domains, and enhancement of RAG performance by not requiring separate retrieval and generation models.
Retriever-Aware Training (RAT): Are LLMs memorizing or understanding?
Retrieval aware training uses the fact that it is useful to use up-to-date information at generation time and hence considers retrievers as part of the training.
How training is done¶
- Distributed training describes the manner in which models and data can be effeciently computed with.
Automatically Correcting¶
Foundationally, the use of reinforcement learning with human feedback (RLHF) has enabled highly successful models that are aligned with tasks and requirements. The automated improvement of GenAI can be bbroken down into improving the models during training time and then during generation time.
Distributed Training¶
References¶
To filter¶
Training variations¶
Fairness Enablement¶
Using Knowledge Links¶
- LinkBERT places in the context window hyperlinked references to achieve better performance and is a drop-in replacement for BERT models.
Fine Tuning¶
Using examples to fine-tune a model can reduce the number of tokens needed to achieve a sufficiently reasonable response. Can be expensive to retrain though.