Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

sharding is a bad solution for any databases, especially vector database. See https://milvus.io/blog/why-manual-sharding-is-a-bad-idea-for...


That's an ad, plain and simple, and as such can't be taken seriously.

> When we reached 100 million vector embeddings, query latency spiked to over a second, something far beyond what our customers would tolerate.

I don't know what else they were doing but 1000 msec latency for 100 million embeddings... that's definitely not on pgvector.


What’s the largest deployment you’ve maintained in production?

If you’ve had the chance to manage one—especially while handling concurrent insert and search operations—you’ve probably felt the pain firsthand and understand just how critical scalability really is.

Insert into PGvector is super slow when data accumulate and the index build will take forever


Depends on the hardware, sharding layout, bulk/individual insert and table bloat. Looks like you are missing pg_repack..


The article talks about manual sharding, not sharding in general.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: