Members-Only

Recent Talks & Demos are for members only

Exclusive feed

You must be an AI Tinkerers active member to view these talks and demos.

Sign in View FAQ

December 12, 2023 · Seattle

Orca

Overview

Orca is an (in development) LLM orchestration written in Rust. It allows for LLM orchestration with a focus on allowing local LLM inference in both your machine and (in the future), running fully on your browser using WebAssembly! No internet connection or API calls are required.

Links

https://github.com/scrippt-tech/orca
Rust LLM Orchestrator framework implements Qdrant vector store and sequential pipelines.
https://github.com/huggingface/candle
Candle: Minimalist Rust ML framework enabling fast CUDA, WASM, and CPU inference.

Tech stack