Skip to content

Index

Training GenAI will generally be domain/modality specific.

Important concepts

Training Objectives

  • Next token
  • Masked token
  • Embedding similarity

What is done

  • Self-supervised pre-training to predict the next token with reasonable likelihoods.
  • Supervised or self-supervised Finetuning on higher quality data sets.
  • Automatically correcting models to more accurately train a model to produce output that is is more globally accurate to the input prompts.

Retrieval Aware Training

??? abstract "GitHub Repo stars GRIT: Generative Representational Instruction Tuning" grit-lm

**Developments** The authors reveal in their [paper](https://arxiv.org/pdf/2402.09906.pdf) the ability to simultaneously train generation and embedding models, revealing improved performance in both domains, and enhancement of RAG performance by not requiring separate retrieval and generation models.

<img width="564" alt="image" src="https://github.com/ianderrington/genai/assets/76016868/f2411adc-e760-4e50-9c2f-637ea159e40c">
<img width="571" alt="image" src="https://github.com/ianderrington/genai/assets/76016868/9f3001fd-968b-4f8e-9658-dce3bdbfb333">

<img width="565" alt="image" src="https://github.com/ianderrington/genai/assets/76016868/7a14ce3b-193c-4605-aced-75c2f1a5afcd">
<img width="553" alt="image" src="https://github.com/ianderrington/genai/assets/76016868/98380c59-7308-449c-8592-6643e3fb7198">
Retriever-Aware Training (RAT): Are LLMs memorizing or understanding?

Retrieval aware training uses the fact that it is useful to use up-to-date information at generation time and hence considers retrievers as part of the training. image

How training is done

  • Distributed training describes the manner in which models and data can be effeciently computed with.

Automatically Correcting

Foundationally, the use of reinforcement learning with human feedback (RLHF) has enabled highly successful models that are aligned with tasks and requirements. The automated improvement of GenAI can be bbroken down into improving the models during training time and then during generation time.

Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies

Developments The authors reveal a comprehensive set of solutions to iteratively improve models. image

Distributed Training

Distributed Training

References

To filter

Training variations

Fairness Enablement

  • LinkBERT places in the context window hyperlinked references to achieve better performance and is a drop-in replacement for BERT models.

Fine Tuning

Using examples to fine-tune a model can reduce the number of tokens needed to achieve a sufficiently reasonable response. Can be expensive to retrain though.

Symbol Tuning Improves in-context learning in Language Models

image