No, I could only get 2.7B to run on 8GB VRam unfortunatly. | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		RHab on May 3, 2023 \| parent \| context \| favorite \| on: Replit's new Code LLM: Open Source, 77% smaller th... No, I could only get 2.7B to run on 8GB VRam unfortunatly.

amasad on May 3, 2023 [–]

it is 2.7B

tarruda on May 3, 2023 | [–]

Loading seems to have worked on my laptop's RTX 3070, `nvidia-smi` shows `5188MiB / 8192MiB` in memory usage.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact