Orchestrating
Orchestration¶
Interaction and Orchestration Frameworks and SDKs¶
Handling the inputs/outputs to GenAI in a consistent and reliable manner has spurred the creation of software libraries that can work with GenAI that is called as a service, or hosted locally.
LangChain¶
Langchain is an open source SDK that allows for creation and management of chat and RAG based interactions. It has a Large user community emphasizing extensions to multiple types of models and documents. It has entrprise offerings with Langsmith for observability, Langserve for serving. It alos can enable multi-agent interactions with lang-graph.
Langchain Is a thorough python and javascript orchestration language for adaptable, memory and tooling-equipped calls that can enable agentic AI.
LangServe will provide a hosted version of LangServe for one-click deployments of LangChain applications.
OpenGPTs Provides an open-source effort to integrate multiple LLMs, and builds upon Langchain, LangServe, and LangSmith
Their Stack
They are building Lang Smith for more Low-code solutions for agentic needs.
- Langchain service deployment
- Awesome Langchain
- Langflow
- Toolkit Generates LangChain plugins for javascript. May be deprecated.
Tutorials:
Other SDKs¶
EmbedChain is a framework to easily create LLM powered bots over any dataset.
Example:
import os
from embedchain import Llama2App
os.environ['REPLICATE_API_TOKEN'] = "REPLICATE API TOKEN"
zuck_bot = Llama2App()
# Embed your data
zuck_bot.add("youtube_video", "https://www.youtube.com/watch?v=Ff4fRgnuFgQ")
zuck_bot.add("web_page", "https://en.wikipedia.org/wiki/Mark_Zuckerberg")
# Nice, your bot is ready now. Start asking questions to your bot.
zuck_bot.query("Who is Mark Zuckerberg?")
# Answer: Mark Zuckerberg is an American internet entrepreneur and business magnate. He is the co-founder and CEO of Facebook.
txtai 'is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows.
- Flowise
- Chain Forge A data flow prompt engineering environment for evaluating ana analyzing LLM responses
- llm-chain ChatGPT and Alpaca support. Agentic with bash commands.n
- Agent Flow
- Auto Chain
- Chatall To interact with multiple chatbots at the same time.
- LocalAI drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing.
Open Agent IN DEVELOPMENT Microservices approach to AGI. Modular components for AI apps or AGI agents
DSPY is a framework for solving advanced tasks with language models and retrieval models
Useful for exploring automatic prompt opteimization.
Language-like interfaces¶
LMQL is a query language that enables simplified representations of chats and agents with minimal code.
"Greet LMQL:[GREETINGS]\n" where stops_at(GREETINGS, ".") and not "\n" in GREETINGS
if "Hi there" in GREETINGS:
"Can you reformulate your greeting in the speech of \
victorian-era English: [VIC_GREETINGS]\n" where stops_at(VIC_GREETINGS, ".")
"Analyse what part of this response makes it typically victorian:\n"
for i in range(4):
"-[THOUGHT]\n" where stops_at(THOUGHT, ".")
"To summarize:[SUMMARY]"
Control libraries¶
- Guidance
- RELM
- Outlines
Retrieval Augmentation focus¶
is a framework that helps you evaluate your Retrieval Augmented Generation (RAG) RAGAS pipelines
Llama Index¶
Create Llama The easiest way to get started with LlamaIndex is by using create-llama.
This CLI tool enables you to quickly start building a new LlamaIndex application, with everything set up for you.
Llamaindex Provides an orchestration framework for with multiple connectors
Llama Lab enables flexible tools to use and indesx various tools
??? abstract " LLaMA2-Accessory: An Open-source Toolkit for LLM Development 🚀
🚀LLaMA2-Accessory is an open-source toolkit for pretraining, finetuning and deployment of Large Language Models (LLMs) and multimodal LLMs. This repo is mainly inherited from LLaMA-Adapter with more advanced features.🧠
-
LlamaGPT A self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.
-
GPT LLM Training Generates and trains fine-tuned LLAMA-2 LLMs for specific tasks.
-
llama index at Github for integrating data ingestion and models.
- LlamaHub (community library of data loaders)
- LlamaLab (cutting-edge AGI projects using LlamaIndex)
Haystack¶
Haystack is an e2e llm orchestration framework that allows a number of versatile interactions.
Open source by DeepSet Designed for scaleable search and retrieval Evaluation pipelines for system eval Deployable as REST API
Griptape¶
Griptape an enterprise alternative to Langchain
Open source / managemed Commercial Support Optimized for scalability and cloud Encryption, access control, security
Others¶
AGent Smith AI makes it easy to instantiate AI agents that can safely and easily call APIs and locally defined functions to interact with the world.
Monarch Assistant Uses AGent Smith for RAG purposes