Optimize Edge LLMs: Finetuning Llama.cpp Quantization

The Edge is Not Your Cloud Cluster [ 4.291032] oom-killer: gfp_mask=0x270cc(GFP_HIGHUSER|__GFP_NOWARN|__GFP_NOREMAP), order=0, oom_score_adj=0 [ 4.293112] CPU: 6 PID: 1421 Comm: llama-server Tainted: P O 5.15.0-1040-tegra #1 [ 4.293121] Hardware name: NVIDIA Jetson Orin Nano Developer Kit [ 4.301984] Node 0: the_net_mem: 3042456KB total, 12KB free, 3042444KB anon, 0KB file, 0KB shmem [ 4.303212] Out of … Read more