Will this model run on your hardware?

Can I run Phi-3-mini-4k-instruct locally?

Open this page in a browser to check it against your own GPU or Mac: the tool detects your hardware locally and shows the best quantization that fits, the memory it needs and an estimated speed.

Quantizations and file sizes

QuantizationGGUF file
Q8_04.1 GB
Q6_K3.1 GB
Q5_K_M2.8 GB
Q4_K_M2.4 GB
Q3_K_M2.0 GB
Q2_K1.4 GB

Run it locally

ollama run hf.co/second-state/Phi-3-mini-4k-instruct-GGUF:Q4_K_M
llama serve -hf second-state/Phi-3-mini-4k-instruct-GGUF:Q4_K_M

LM Studio: open second-state/Phi-3-mini-4k-instruct-GGUF