Orchestration

Interaction and Orchestration Frameworks and SDKs

Handling the inputs/outputs to GenAI in a consistent and reliable manner has spurred the creation of software libraries that can work with GenAI that is called as a service, or hosted locally.

LangChain

LangChain is an open source SDK that allows for creation and management of chat and RAG based interactions. It has a large user community emphasizing extensions to multiple types of models and documents. It has enterprise offerings with LangSmith for observability, LangServe for serving. It also can enable multi-agent interactions with LangGraph.

LangChain

A thorough python and javascript orchestration language for adaptable, memory and tooling-equipped calls that can enable agentic AI.

LangServe

Provides a hosted version of LangServe for one-click deployments of LangChain applications.

OpenGPTs

Open-source effort to integrate multiple LLMs, building upon LangChain, LangServe, and LangSmith.

**LangChain Stack** ![image](https://github.com/ianderrington/genai/assets/76016868/c66bf027-8556-43e6-8e73-de59c5e58d95)

LangSmith

Low-code solutions for agentic needs.

LangGraph

The first agent IDE for visual development.

Langflow

Visual programming interface for LangChain.

Awesome LangChain

Curated list of LangChain tools and resources.

**Tutorials:**

GPT and PDFs

Tutorial on working with PDFs using GPT-4 and LangChain.

LangChain Prompt Templates

Guide to using prompt templates effectively.

Deep Learn LangChain

Comprehensive course on LangChain development.

### Other SDKs

Semantic Kernel

Microsoft's framework for integrating AI with software applications.

EmbedChain

Framework to easily create LLM powered bots over any dataset.

txtai

All-in-one embeddings database for semantic search, LLM orchestration and language model workflows.

### Language-like Interfaces

LMQL

Query language that enables simplified representations of chats and agents with minimal code.

### Control Libraries * Guidance * RELM * Outlines

Retrieval Augmentation

RAGAS

Framework for evaluating Retrieval Augmented Generation (RAG) pipelines.

### Llama Index

Create Llama

CLI tool for quickly starting new LlamaIndex applications.

LlamaIndex

Orchestration framework with multiple connectors.

Llama Lab

Flexible tools for using and indexing various data sources.

LLaMA2-Accessory

Open-source toolkit for pretraining, finetuning and deployment of LLMs.

### Enterprise Solutions

Haystack

E2E LLM orchestration framework by DeepSet:

Scalable search and retrieval
Evaluation pipelines
REST API deployment

Griptape

Enterprise alternative to LangChain:

Commercial support
Cloud optimization
Security features

### Monitoring and Observability

Langfuse

Open Source LLM Engineering platform with traces, evals, and prompt management.

AgentOps

Monitoring and analytics for AI agents.

LangSmith

Debugging and monitoring for LangChain applications.

Helicone

Usage tracking and analytics for LLM applications.

### Additional Tools

Flowise

Visual workflow builder for LLM applications.

Chain Forge

Data flow prompt engineering environment.

LocalAI

Drop-in replacement REST API compatible with OpenAI specifications.

Open Agent

Microservices approach to AGI development.

DSPY

Framework for solving advanced tasks with language models.

Orchestration and Integration Frameworks