Run any Huggingface model locally

Adithya S K
10 min readJan 1, 2024

A guide/colab notebook to quantize LLMs in GGUF formate to run them locally

Code Updated : 29th December 2023

Introduction

In the ever-evolving landscape of natural language processing, language models are at the forefront, playing a central role in comprehending and generating human-like text. The increasing complexity and size of these models…

--

--

Adithya S K

Post blogs about Gen AI | Cloud | Web Dev | Founder @CognitiveLab spending time fine-tuning LLMs ,Diffusion models and developing production ready application