This is a demonstration project showcasing how to run Large Language Models locally in the browser using Transformers.js and WebGPU-acceleration. It uses SmolLM2-1.7B-Instruct model with 4-bit quantization for efficient execution.
Would you like to try chatting with a locally-running LLM? Click below to load the model!