Skip to content

Frameworks

The big-bang like expansion of AI has led to a surge in services, methods, frameworks, and tools that enhance the creation and deployment of models from start to finish. Although there are end-to-end providers for generating valuable GenAI solutions, there is immense value in implementing and experimenting with your own stacks.

Additionally, there are useful libraries and tools worth exploring.

tldr; Here are the prominent frameworks

  • Langchain is an early system with a principled design that allows for extensive applications to be built with it.
  • Llama Ecosystem is a community of Llama-focused modelers, based on the Meta model called Llama, Llama-2, and beyond.
  • A number of others.

The rapid development in Generative AI tooling makes it challenging to keep up with the development and deprecation of powerful frameworks and tools. Some of the mentioned references may not be fully completed, or even nascent repos to build their intended purposes (described here). Please let us know if we are missing anything here.

Layer 1: Foundation

Starting with base programming languages, increasingly higher-level frameworks enable training and calling of AI models. Higher-level orchestration libraries and platforms allow creating and evaluating chains, agents, and systems that sometimes use visual interfaces. These can often be augmented with various tools/packages/repositories. On top of these involve mostly or all-complete frameworks and platforms that enable nearly complete.

Base languages

Prominent languages include python, C++/CUDA, and Javascript.

AI software libraries

PyTorch is a popular python-focused system for creating and using AI.
Tensorflow is a popular multi-language eco-system for creating and using AI.
JAX is a library enabling composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
spAcy is a library for advanced Natural Language Processing in Python and Cython.

APIs based model usage

Interaction and Orchestration Frameworks and Languages

Handling the inputs/outputs to GenAI in a consistent and reliable manner has spurred the creation of software libraries that can work with GenAI that is called as a service, or hosted locally.

LangChain

  • Open source Large user community Extensive integrations Enterprise expansions with LangSmith, LangChainHub and more

Langchain Is a thorough python and javascript orchestration language for adaptable, memory and tooling-equipped calls that can enable agentic AI.

GitHub Repo stars LangServe will provide a hosted version of LangServe for one-click deployments of LangChain applications.

GitHub Repo stars OpenGPTs Provides an open-source effort to integrate multiple LLMs, and builds upon Langchain, LangServe, and LangSmith

Their Stack

image

They are building Lang Smith for more Low-code solutions for agentic needs.

Tutorials:

Higher level

Pytorch Lightning Enables model training with Pytorch and minimizes the boilerplate

Model parallelism

??? abstract "Pytorch Lightning THunder

GitHub Repo stars Deep Speed (by MSFT) empowers ChatGPT-like model training with a single click, offering 15x speedup over SOTA RLHF systems with unprecedented cost reduction at all scales

Blog on Deepspeed Ulysses image

DeepSpeed-Ulysses uses a simple, portable, and effective methodology for enabling highly efficient and scalable LLM training with extremely long sequence lengths "DeepSpeed-Ulysses partitions individual samples along the sequence dimension among participating GPU. Then right before the attention computation, it employs all-to-all communication collective on the partitioned queries, keys and values such that each GPU receives the full sequence but only for a non-overlapping subset of the attention heads. This allows the participating GPUs to compute attention for different attention heads in parallel. Finally, DeepSpeed-Ulysses employs another all-to-all to gather the results along the attention heads while re-partitioning along the sequence dimension." Ulysses Tutorial here