This looks very cool! I maintain a FreeCAD workbench for 3d model version control[0], and it currently uses git as the VCS, simply because that's what I was already using. Thinking long-term, I see it eventually morphing into a broader PDM (Product Data Management) system, perhaps even PLM (Product Lifecycle Management). Lore has a lot of the requirements already built-in, like centralized locking (better than SVN), and it's better suited for for binary files. I implemented the git backend as a protocol/port so it'd be pretty easy to swap it out. I'll be watching Lore closely.
I use Google Maps on Graphene. It works perfectly. You still get the benefits of the rest of the phone being degoogled. Just allow it to access your location only when you're actively using the app. When it's closed, it's closed.
I don't understand why no labs create dedicated models per industry/expert. E.g. physics, electronics, chemistry, etc. Each model would be much smaller and better suitable for running locally. Everyone is trying to cram everything into a single model.
It's cool seeing all the engineering that goes into optimizing performance of diffs. I'm working on a FreeCAD workbench that generates diffs on CAD model trees[0], and although my bottlenecks are a bit different, I can still implement some of your optimizations down the line if needed (such as deferred syntax highlighting).
My main bottleneck is that I do a complete diff on all open + changed documents in the repository up front, because due to how document properties are stored, I won't know if the file has meaningful changes until I compute the full diff (FreeCAD may save the document, but not have anything meaningful change.)
FreeCAD is really the only serious contender for CAD on Linux. I love how everything is hackable via Python APIs. Every release seems to fix more and more UX issues. I have very high hopes for it to eventually start attracting more commercial usage, much like KiCad and Blender did in their spaces. We need more open software like that.
>I love how everything is hackable via Python APIs.
It is hackable with languages other than Python too. The Python interpreter has no restrictions to access and execute everything it likes. Contrast this with Gimp, which it's Scheme interpreter cannot access anything except objects inside Gimp.
This opens some security hole risks in Freecad, given that plugins can execute arbitrary code in the user's machine, but that means one can write C, C++, Rust and automate the gui.
Freecad btw, very unfortunately, it loads each .so external library only once, and it will load the library only with different name or version number. I lost 5 hours messing up with Python versions and .venv installations till I figured that out.
Almost purely for amusement, I played around with letting Claude generate Python code for FreeCAD. I actually got a coherent part out of it, that I sent out for fabrication. The practical motivation is that my needs are extremely basic (plates, brackets, etc) but I have some physical issues that prevent me from using GUI based CAD directly.
I know what it's like growing up in a religious atmosphere. Since you've been trained to think inside the box, you realize the box is nonsense. The problem is thinking that the only valid alternative is that there is no creator and everything is just random (the opposite extreme.) Look into other philosophies like advaita vedanta, buddhism, or progressive science like another commenter mentioned about nassim haramein.
I didn't say there's no creator. I'm pretty confident that the God defined by these religions doesn't exist. Since then, I have lived comfortably with knowing enough that I don't know and might never know.
> Jan 21 update: llama.cpp fixed a bug that caused looping and poor outputs. We updated the GGUFs - please re-download the model for much better outputs.
Yes! This update works great. Seems to be pretty good at first glance. I'll have to setup an interesting task and see how different models approach the problem.
That's around 350,000 tokens in a day. I don't track my Claude/Codex usage, but Kilocode with the free Grok model does and I'm using between 3.3M and 50M tokens in a day (plus additional usage in Claude + Codex + Mistral Vibe + Amp Coder)
I'm trying to imagine a use case where I'd want this. Maybe running some small coding task overnight? But it just doesn't seem very useful.
Essentially migrating codebases, implementing features, as well as all of the referencing of existing code and writing tests and various automation scripts that are needed to ensure that the code changes are okay. Over 95% of those tokens are reads, since often there’s a need for a lot of consistency and iteration.
It works pretty well if you’re not limited by a tight budget.
I only run small models (70b at my hardware gets me around 10-20 TOPS) for just random things (personal assistant kind of thing) but not for coding tasks.
For coding related tasks I consume 30-80M tokens per day and I want something as fast as it gets
Hard disagree. The difference in performance is not something you'll notice if you actually use these cards. In AI benchmarks, the RTX 3090 beats the RTX 4080 SUPER, despite the latter having native BF16 support. 736GiB/s (4080) memory bandwidth vs 936 GiB/s (3090) plays a major role. Additionally, the 3090 is not only the last NVIDIA consumer card to support SLI.
It's also unbeatable in price to performance as the next best 24GiB card would be the 4090 which, even used, is almost tripple the price these days while only offering about 25%-30% more performance in real-world AI workloads.
You can basically get an SLI-linked dual 3090 setup for less money than a single used 4090 and get about the same or even more performance and double the available VRAM.
If you run fp32 maybe but no sane person does that. The tensor performance of the 3090 is also abysmal. If you run bf16 or fp8 stay away from obsolete cards. Its barely usable for llms and borderline garbage tier on video and image gen.
> The tensor performance of the 3090 is also abysmal.
I for one compared my 50-series card's performance to my 3090 and didn't see "abysmal performance" on the older card at all. In fact, in actual real-world use (quantised models only, no one runs big fp32 models locally), the difference in performance isn't very noticeable at all. But I'm sure you'll be able to provide actual numbers (TTFT, TPS) to prove me wrong. I don't use diffusion models, so there might be a substantial difference there (I doubt it, though), but for LLMs I can tell you for a fact that you're just wrong.
To be clear, we are not discussing small toy models but to be fair I also don't use consumer cards. Benchmarks are out there (phoronix, runpod, hugginface or from Nvidias own presentation) and they say it's at least 2x on high and nearly 4x on low precision, which is comparable to the uplift I see on my 6000 cards, if you don't see the performance uplift everyone else sees there is something wrong with your setup and I don't have the time to debug it.
> To be clear, we are not discussing small toy models but to be fair I also don't use consumer cards.
> if you don't see the performance uplift everyone else sees there is something wrong with your setup and I don't have the time to debug it.
Read these two statements and think about what might be the issue. I only run what you call "toy models" (good enough for my purposes), so of course your experience is fundamentally different from mine. Spending 5 figures on hardware just to run models locally is usually a bad investment. Repurposing old hardware OTOH is just fine to play with local models and optimise them for specific applications and workflows.
[0] https://github.com/eblanshey/HistoryWorkbench