Quantizing Gemma 4's MTP drafters

Christina Sørensen — Fri, 29 May 2026 00:00:00 +0000

A while after Gemma 4 came out, Google released a set of Multi-Token-Prediction (MTP) drafters for it. Pairing a large model (the verifier) with a tiny MTP drafter (the assistant) enables speculative decoding: the small model guesses the next few tokens, and the big model verifies them in one parallel pass. When it works, the throughput gain is most of the way to free.

To put it shortly: MTPs are one of the rare free lunches we get with LLMs. Or well, on paper, like with free lunches, that’s the assumption.

These are notes on getting that working locally. The previous post</a> was about why this should have been easier than it was.

If you try to convert the Gemma 4 MTP drafter weights using mainline ggml-org/llama.cpp</code>, the converter rejects them:
Model Gemma4AssistantForCausalLM is not supported </code></pre>Mainline supports the base Gemma 4 models, but it does not know what a gemma4_assistant</code> is. The converter lacks the class, and the runtime lacks the MTP API. The runtime PR #18886</a> has been in draft for months. A companion converter PR #22738</a> was opened and closed unmerged with one of four tasks done. The working path is a fork: ikawrakow/ik_llama.cpp</code></a>, specifically the feat/gemma-4-mtp</code> branch from PR #1744</a>. The resulting GGUFs only load in that fork. Downstream wrappers — Ollama, LM Studio, jan.ai, llama-cpp-python</code> — will all reject them until they backport the changes. This is the cost of running new architectures the week they ship: you swap inference backends to match the model, not the other way around. Drafters are tiny. The E2B drafter’s MTP head is 78M parameters; the 31B’s drafter is roughly 470M; the 26B-A4B’s is about 408M. Their job is to predict tokens accurately enough that the verifier accepts them, so acceptance rate dominates everything else. Aggressive quantization causes the verifier to reject more drafts, and the pipeline becomes slower than running the verifier alone. The quantization policy for drafters is the inverse of what you’d do for a large base model: Do use: bf16</code>, f16</code>, Q8_0</code>, Q6_K</code>, Q5_K_M</code>, Q4_K_M</code>.</li> Do not use: IQ1_*</code>, IQ2_*</code>, Q2_K</code>, Q3_*</code>. These cost more rejected drafts than they save in load time.</li> </ul> imatrix</code> calibration is not available for these drafters at the moment. PR #1744 builds the gemma4_mtp</code> drafter graph with a hardcoded GGML_ASSERT(has_target_ctx)</code>, so standalone llama-imatrix</code> runs abort in llama_decode</code> before producing anything. The IK fork’s signature quants (IQ4_KS</code>, IQ4_KSS</code>, IQ5_KS</code>) derive their precision-per-bit advantage from imatrix-guided scale selection, so without an imatrix they collapse to roughly K-quant quality at the same bit budget. Shipping them under those labels would be misleading. Quantizing directly from bf16</code>/f16</code> to Q4_K_M</code>/Q8_0</code> is the realistic option until the tooling catches up. At ≥4 bits the precision gap from missing the imatrix is small — single-digit percent on most benchmarks. At Q3 and below it would matter, which is the other reason Q3 isn’t a sensible target here. One empirical surprise: on a 4060 Laptop GPU, pairing unsloth’s Gemma-4-E4B-it-Q4_K_M.gguf</code> verifier against both a Q8_0</code> and a Q4_K_M</code> drafter at --draft-max 3</code>, the Q4_K_M</code> drafter beat the Q8_0</code> by ~13% in total throughput. The smaller drafter’s faster draft step outweighed the acceptance-rate cost. The “always pick the highest-bit drafter” rule assumes you are memory-bandwidth-rich and compute-poor; on a laptop GPU you are neither. Benchmark your own silicon. llama-quantize</code> silently embeds the absolute on-disk paths of your imatrix file and calibration corpus into the GGUF metadata. There is no flag to disable this at quantize time. The *-f16.gguf</code> intermediate is clean; only quantized files have the leak. Inspect any quantized GGUF and the two offending keys are sitting there: nix-shell -p python3Packages.gguf --run \ 'gguf-dump --no-tensors model.gguf' </code></pre> Key</th> What it leaks</th></tr></thead> quantize.imatrix.file</code></td> Absolute path to the imatrix file on the build host</td></tr> quantize.imatrix.dataset</code></td> Absolute path (and filename) of the calibration corpus</td></tr> </tbody></table> Uploading the raw output of a quantization pipeline publishes your infrastructure layout — usernames, ZFS pool names, directory structure — alongside the weights. The fix is gguf_new_metadata</code> from the Python gguf</code> package, which rewrites the file with selected keys removed without touching the tensors: nix-shell -p python3Packages.gguf --run ' for f in /srv/models/<MODEL>-GGUF/*.gguf; do [ "$f" = "*-f16.gguf" ] && continue python3 -m gguf.scripts.gguf_new_metadata \ --remove-metadata quantize.imatrix.file \ --remove-metadata quantize.imatrix.dataset \ --force \ "$f" "$f.clean" done ' </code></pre> Each rewrite runs at ~5–15s per GiB and shrinks the file by exactly the size of the two strings removed (≈190 bytes). Verify before uploading: gguf-dump --no-tensors model-clean.gguf | grep -v INFO </code></pre> kv_count</code> should be exactly 2 less than the original. The imatrix.*_count</code> integer fields stay (those carry no path information). A smoke test confirms the rewritten file still loads: llama-completion -m model-clean.gguf --jinja \ -p 'The capital of France is' -n 20 --temp 0 </code></pre> Coherent output means the rewrite didn’t corrupt anything. Sanitized quants in hand, the next step is pairing. A drafter is not a standalone language model: it has to be paired with a verifier that shares its exact vocabulary. For Gemma 4 that vocabulary is 262,144 tokens. The size of that vocabulary is also the reason the MTP speedup applies to structured output. The drafter sees the same <|tool_call|></code>, <|tool_response|></code>, and <|channel|></code> tokens as the verifier, so tool-calling agents and structured-generation workflows benefit on the full response, not just the natural-language parts. The vocabulary itself isn’t quantized — it lives in a separate GGUF section. A working ik_llama.cpp</code> invocation: llama-server \ --model gemma-4-31B-it-Q8_0.gguf \ --model-draft gemma-4-31B-it-assistant-Q8_0.gguf \ --spec-type mtp \ --draft-max 3 \ --draft-p-min 0.0 \ -ngld 99 \ --n-gpu-layers 99 \ --ctx-size 32768 \ -ctk q8_0 -ctv q8_0 \ -b 1024 -ub 1024 \ --jinja </code></pre> The relevant levers: --spec-type mtp</code> enables the speculative-decoding path from PR #1744.</li> --model-draft</code> is the drafter GGUF; its vocabulary must match the verifier.</li> --draft-max 3</code> sets the speculative chain length. Values from 1 to 4 are all reasonable; --spec-autotune</code> will probe and pick one for your workload.</li> --draft-p-min 0.0</code> accepts all drafts; raising it shortens chains on difficult prompts.</li> --jinja</code> applies the model’s chat template, including Gemma 4’s tool-call format.</li> </ul> PR #1744’s reference numbers, 31B verifier + 31B drafter at Q8_0/Q8_0 on a data-center GPU: Run</th> Throughput</th> Acceptance</th></tr></thead> Baseline (no MTP)</td> ~21 t/s</td> —</td></tr> MTP --draft-max 1</code></td> ~35 t/s</td> ~89%</td></tr> MTP --draft-max 2</code></td> ~44 t/s</td> ~83%</td></tr> MTP --draft-max 3</code></td> ~49 t/s</td> ~74%</td></tr> MTP --draft-max 4</code></td> ~49 t/s</td> ~64%</td></tr> </tbody></table> A bit more than 2× throughput, after all of the above. The throughput gains are real, and on the right hardware they are substantial. The work it takes to surface them is also real: undocumented CLI divergences, an obscure fork, manual metadata sanitization, quantization rules that invert the usual advice. None of this is in the model card or the upstream README. The path to a working setup is reconstructed from PR threads and stack traces. The next post in this series takes the result and runs it on a 2016 Xeon with no GPU, which is where the command line stops being a paragraph. Open weights are not open source Christina Sørensen — Tue, 26 May 2026 00:00:00 +0000 I’ve been quantizing Gemma 4 and it’s MTP drafters recently. The next two posts are about how that went. This one is about why it was so much work. When a lab releases a model under an open license, what they almost always release is the weights. Sometimes a tokenizer. Occasionally a short technical report. Rarely the training data, data-processing code, training code, evaluation harness, or the recipes used to produce the published quantizations. People still call this open source, and I think that’s misleading. If you can’t rebuild it, you can’t fork it. If you can’t fork it, the word doesn’t mean what it used to. Calling it open is like calling an executable binary open because you can execute it. Unless you consider assembly code open source, I think it’s really obvious why this misses the mark, and is exellent marketing spin. The people releasing models don’t seem to feel responsible for whether anyone can run them, or have good results. Its more like box ticking. Here it is? There is no shared vocabulary for how a model should be packaged, or which inference engine it expects, or what quantization recipe was used to make the file on whatever hub they pick. The settings that actually decide whether a model is usable on a given machine — mlock</code>, KV-cache dtype, flash attention, MoE routing, draft model pairing — live in PR descriptions and Discord screenshots. The model card lists the benchmark scores and links a HuggingFace space. So you download the weights, get a result that’s slow or incoherent or both, and you don’t know whether the model is bad, your quantization is bad, your runtime is bad, or you’ve just configured something wrong. Most people, reasonably, conclude it’s just their hardware, or that open models don’t actually work for normal users. The tools that have grown up to paper over this make the problem worse, not better. Ollama is the obvious example. It picks a quantization, runtime, context and batch sizes for you, and doesn’t tell you what it picked or why. When the output is bad, there’s no thread to pull. You can’t tell if you’re holding it wrong because the thing was designed so you can’t hold it at all. It’s just a blackbox! This wouldn’t matter much if the defaults were good. They aren’t. A default Q4_0 of a small model with no imatrix calibration and a mismatched chat template will be much worse than the same model run carefully, and the user has no way of knowing the difference exists. AND NO ONE REALLY TALKS ABOUT HOW MUCH THIS MATTERS! Concretely: a while after Gemma 4 came out, Google released a set of Multi-Token-Prediction drafters for it, designed to make the base models faster via speculative decoding. The obvious thing to do is download the GGUF and run it under llama.cpp</code>. That doesn’t work, and the reasons it doesn’t work are a fair sample of the genre: Mainline llama.cpp</code> doesn’t know what a Gemma4AssistantForCausalLM</code> is. The converter rejects the drafter outright. To convert it, you have to find ikawrakow/ik_llama.cpp</code>, on a feature branch, from a PR that hasn’t merged.</li> llama-imatrix</code> aborts on that branch’s MTP graph because the build hardcodes an assertion that a target context is bound. So you skip imatrix calibration, and your Q4 quants are noticeably worse than the ones every other model on the hub ships with.</li> llama-quantize</code> writes the absolute on-disk paths of your imatrix file and calibration corpus into the GGUF metadata. There’s no flag to turn this off. If you upload without scrubbing, you publish your home directory layout next to the weights.</li> </ul> None of this is in the model card. None of it is in the upstream README. You find it by trying things and reading stack traces. No one does this. Seriously. I mean yes. Part of it is that they just use what the people that did bother to do it made, fair. But there’s like 4 quants for this on huggingface at the time of writing, one is mine. FOUR! For the state of the art MTP/MOE models! The hardware to run a 30B-class MoE at reading speed isn’t exotic. A five-year-old workstation with enough RAM will do it. The bottleneck isn’t the silicon, it’s everything between the weights file and a working configuration: which fork, which quant, which flags, which template, which version of which template. Ohh, and getting it all running, good luck unless you’re… literally an infrastructure engineer? Most people give up somewhere in the middle and go back to the API, and I don’t blame them. I do think the labs releasing these models could make this much less bad than they have. Publishing the imatrix corpus would help. Publishing the quantization recipe would help. Picking one reference runtime and keeping the model working on it would help. Treating the deployment story as part of the release, rather than something the community is expected to reconstruct, would help. Until that changes, “open source” for language models mostly means “someone else has to figure out how to run it, and none of the freedoms that used to mean open source are actually present”, and the people best positioned to figure it out are charging by the token. I’ll write up the actual quantization steps — which flags, which traps, what the metadata leak looks like, how to scrub it — and what it took to run the result on a 2016 Xeon with no GPU, in two follow-up posts this week. Thanks for reading :) Another NixOS Election Cycle Christina Sørensen — Thu, 02 Oct 2025 00:00:00 +0000 And here I thought we’d have a boring election cycle</h2> If you’ve not been following NixOS governance, recently the moderation team decided to all collectively resign. I’ve been informed that I’ve been one of the people that certain steering committee members have wanted to moderate harder, specifically because I’ve been calling out their undisclosed conflicts of interest, and providing actual level headed, direct criticisms that they’ve yet managed to answer. This is despite their alleged pledge to respond to community concerns. One of the people I’ve been led to believe specifically has been pushing for aggressively moderating me has yet to respond to any messages I’ve sent them in private, which to me is even more odd, they’re not rude, and could likely have cleared up any legitimate moderation issues. I think that, looking back at the previous steering committee, and even watching their talks at NixCon, it was quite clear that most of them were not really interested in governance. Paraphrasing, most of them said a version of “I’m not the most competent person to be here”, and those were the ones that at least have the insight to make that realization. I think that many were there simply to add a new line to their resume. What does it mean for me running?</h2> Nothing. I’ll run. If enough people vote for me, I’ll do what I set out to do in my candidate statement. If I’m not elected, I’ll put my focus elsewhere, and make myself available to any elected Steering Committee members on policy and governance questions, just like the last cycle. I don’t think that the Steering Committee is the only (or even the most effective) way for me to improve the project and ecosystem. But it is an option I’d be foolish to throw away. And I have much I want to do. Here is the start. Process</h4> I’ve read the Steering Committee retrospective from Gabriella</a>. To me, the challenges raised here are of primary concern. Currently, the steering committee isn’t leading the community. And if the final instance required for decisive action in the project is indescisive, then what’s the point? So that’s my primary concern. The steering committee should actually get things done. I think we can solve this. Transparency</h4> Meeting notes and employers of board and steering committee memeber should be made public - to the extend possible. There should be public logs of both. Everything should default to being documented in public, including decisions and how members vote. If the steering committee finds certain subjects too sensitive, they may be redacted, but even then, the fact that a vote or discussion took place should be documented. Who voted for and against - even if we can’t say for what - should be written down for everyone to see. The steering committee must become auditable and accountable to everyone in the project. Unblocking Adoption</h4> Flakes aren’t unstable. That’s mainly a naming convention at this point. It is a major disadvantage for the project, and the official installer adds friction. Flakes should be installed by default. Defaults should be sane. Many decisions in the project aren’t made based on what’s best for the project, but where measures to alleviate immediate harm. Now those same measures have themselves become harmful. By engaging with the community, and listening for places where their colleagues are blocked or choosing alternatives to official tools, I’ll correct any blockers with the steering committee and community, and ensure we become the best choice. I think Nix is a paradigm shift in how we do computers, and I want to convince the rest of the world of that. I want to work with Nix for the rest of my career, and for that to become a reality, I’m running for the steering committee again. A final question: Will you go the distance</h2> I know a lot of people are scared that they’ll end up voting for someone that will turn passive and be checked out. I am scared of voting for such people. It was a major pain in the last election, that even if I was aligned with a candidate and they got elected, they just didn’t actually push for anything, and seemed to essentially give up. If that’s something you’re scared will happen, I think specially if you’re a long time community member, the best proof I can give that it wont happen if you vote for me is that… I’m still here. I’d not want to waste your vote. But I’d also like to temper your expectations. I can only do so much. If you vote for a bunch of people that are not gonna go the distance, then my primary role becomes keeping you updated on why the process remains broken. I can be a voice for accountability, but I cannot be the entire committee. Tracking Sattelites with Consumer Components Christina Sørensen — Sun, 05 Jan 2025 06:00:00 +0000 A Global Network of Ground Stations</h2> At 38c3</a> I attended a workshop on building a sattelite ground stations for communicating with LoRa based sattelites, typically used for research or weather probes. Generally speaking, these are so called “cubesats”</a>, which are small, relatively inexpensive sattelites usually built up of off the shelf components in a modular fashion. In fact, it turns out that with one of these inexpensive sattelites, a small launch would actually only cost in the range of $50,000-$100,000, not a small amount, but still, a lot more accessible than what you would initially imagine. Combine this with LoRa (literally Long Range) — a proprietary radio communication technique — and you have the components for a globe spanning network of sattelites and amatur ground stations. This has lead to a suprising amount of these sattelites currently in low-earth orbit around the globe</a>. Typically these aren’t geostationary (meaning located over the same point on the planet at all times), and so, it is useful for e.g. a smaller university to have access to a globe spanning network of sattelite stations. TinyGS</h2> Enter TinyGS, tiny ground stations</a>. TinyGS is an open network of Ground stations distributed across the globe, run by amateurs. It allows operating LoRa based weather probes, sattelites, and other “flying objects”, using consumer electronics. The basic idea is that you get an ESP32 as well as a some sort of LoRa module, an antenna, and flash the tinygs firmware onto it. Then you just connect it to the tinygs servers using MQTT, and you will be able to opearte your board on the tinygs website, as well as being able to contribute to the global sattelite tracking effort. Build your own Ground Station</h2> Hardware</h3> If you wanna build your own, the easiest possible route is just to acquire a LoRa32 V2.1_1.6 - 433MHZ CH9102F</code> board. Usually you can find these online, and make sure that you get one with a small antenna. You’ll need to flash the firmware, the easiest way is to use the TinyGS web flashing tool</a>, and sorry, but you’ll need to use a chrome based browser for this. Setup</h3> What’s worse, you’ll also need to install telegram, and create an account, as the TinyGS projects communications mainly happen there, and that means the registration bot for your new board also lives on telegram. Join the TinyGS telegram group here</a>, and then start a conversation with the tinygs personal bot here</a>. Also, make sure you join the TinyGS community chat in the group first, as the bot will require you to be a member before signing you up. When you are in that chat, send the /start</code> command to the bot for some info about it’s usage. You’re mainly just interested in the /mqtt</code> command, which gives you your credentials for the mqtt server. Now with your credentials, and your participation in the TinyGS community (hi!), you’ll want to plug in your flashed board. After a few moments, it should start it’s own WiFi AP, that you can connect to. On the screen of your board, it should show the IP address that you’ll then want to connect to on the boards WiFi. This will give you a configuration inteface. Here, you enter the configuration, and then configure a ground station name, create an admin password (to be used with the admin</code> account), enter the WiFI SSID and Password that you want the board to use for internet connectivity, your lattitude and longitude (this doesn’t have to be exact, but should be vaguely correct withing 25km~ because it is used to determine what sattelites to try and connect to), and a timezone. Further, you’ll want to add your MQTT credentials from the telegram bot. If you’ve done everything correctly, you’ll receive a message on telegram from the bot, like I did in the above screenshot. You’re now on the network. Start operating your sattelite</h3> After you’ve setup your station, you’ll want to /login</code> with the bot to be able to operate it fully. This will give you access to even more information about your device, and the ability to edit it’s description and image, and to control how it is tracking sattelites, or setup manual tracking if you wanna just track something specific, or you don’t like the automatic selection. Below that, you’ll find a log of the packets you have recieved so far. Notice here that the red circle indicated a CRC checksum error, meaning you recieved a faulty package. Also notice that there are two packets with a CRC error that were caught by just one station (meaning just your station). In this case, if you click on the packet, you’ll get the text that: Unrecognized packet. This packet does not match the known structure of this satellite. It might be an unknown packet or just terrestrial noise. </blockquote> However, for those that don’t have a CRC error, that’s a real sattelite packet you’ve caught. Well done!. If you click on it you can even read the details about what you caught. For example, the HOD-HOD-1A</code> capture from above provides the following information when you click on it. As you can see, it shows what the content of the packet was, both as text, hexadecimal, and parsed json that you can expand and browse through. Below that, you’ll see a list of other stations that caught the packet. The view to the side shows the geographical location of the stations that caught the sattelite communication. I wanna capture more packets!</h3> As you will likely realize after running your station for a few days, you’re probably not capturing a lot of packets. This is because the default antenna doesn’t really perform well, and likely, you’ll also ideally want to make sure that your station is outside, and isn’t obstructed, and that weather conditions are favorable (anecdotally, I also tend to capture much more packets at night). I’ll not elaborate more on this in this post, as it is something I’m working on myself, but I may follow up later with a separate post on what it takes to increase the amount of packets you capture. In short, you’ll want a 1/4-wave ground plane antenna. Obligatory Conclusion</h2> With inexpensive consumer hardware, it is possible to build your own sattelite ground station, and join a globe spanning network of space enthusiasts capturing sattelite data. That is pretty cool :3 Conventional Commits Change my Brain Christina Sørensen — Wed, 24 Jul 2024 00:00:00 +0000 What is a conventional commit?</h2> In case you’re not aware, conventional commits</a> is a standard format for writing commit summaries. It looks like this. build(deps): bump git2 from 0.18.3 to 0.19.0 Bumps [git2](https://github.com/rust-lang/git2-rs) from 0.18.3 to 0.19.0. - [Changelog](https://github.com/rust-lang/git2-rs/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/git2-rs/compare/git2-0.18.3...git2-0.19.0) --- updated-dependencies: - dependency-name: git2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> </code></pre> This example is from dependabot</a>, which does conform to conventional commits. Notice that only the first line, the commit summary, is part of the conventional commits spec here. The last line is a “footer”[1]</a>, or technically, a trailer</a> added by the git commit -s</code> flag, -s</code> short for --signoff</code>. The exact spec looks like this: <type>[optional scope]: <description> [optional body] [optional footer(s)] </code></pre> Unlike many project specific standards, such as that of angular</a> or nixpkgs</a>, what sets conventional commits apart is that it aims to be a universal commit standard, a bit like how semantic versioning</a> seeks to be a universal software versioning standard. In the same sense as semantic versioning provides meaning through version numbers, such as a major</code> number increasing meaning a breaking change, conventional commits also seeks to encode information about the state of a project in the commit summaries, and provides enough information to reasonably tell when it’s time to do a patch, minor, or major version bump. It further also provides ways to specify if a commit contains a breaking change, via a git footer. It may be easiest to explain this by just showing you the spec. The commit contains the following structural elements, to communicate intent to the consumers of your library: fix: a commit of the type fix</code> patches a bug in your codebase (this correlates with PATCH</code></a> in Semantic Versioning).</li> feat: a commit of the type feat</code> introduces a new feature to the codebase (this correlates with MINOR</code></a> in Semantic Versioning).</li> BREAKING CHANGE: a commit that has a footer BREAKING CHANGE:</code>, or appends a !</code> after the type/scope, introduces a breaking API change (correlating with MAJOR</code></a> in Semantic Versioning). A BREAKING CHANGE can be part of commits of any type.</li> types other than fix:</code> and feat:</code> are allowed, for example @commitlint/config-conventional</a> (based on the Angular convention</a>) recommends build:</code>, chore:</code>, ci:</code>, docs:</code>, style:</code>, refactor:</code>, perf:</code>, test:</code>, and others.</li> footers other than BREAKING CHANGE: <description></code> may be provided and follow a convention similar to git trailer format</a>.</li> </ol> Additional types are not mandated by the Conventional Commits specification, and have no implicit effect in Semantic Versioning (unless they include a BREAKING CHANGE). A scope may be provided to a commit’s type, to provide additional contextual information and is contained within parenthesis, e.g., feat(parser): add ability to parse arrays</code>. </blockquote> So basically, if there is a type fix</code>, that’s a PATCH</code> bump to your semver, a feat</code> is a MINOR</code> bump, and a BREAKING CHANGE:</code> footer or a !</code> after the type/scope is a MAJOR</code> bump. If we for instance have commit with the summary fix: change windows-only imports to be windows-only</code>, that’s a PATCH</code> change, and if we are on version 3.2.48</code>, then next version will have to be at least 3.2.49</code>. If we instead had a summary feat: add --no-|show-symlinks flags for filtering output</code>, then that would a MINOR</code> change, and again, if we were on 3.2.48</code>, next release would be 3.3.0</code>. Lastly, if we had a summary feat(flags)!: add --classify=always,auto,never</code>, even thou it has a feat</code> type, the !</code> indicates this is a breaking change, and so we must cut a new MAJOR</code> version on next release if we include this commit, meaning that if we were on 3.2.48</code>, we’d go to 4.0.0</code>. Yes. Breaking changes means a new major version. Yes, that might seem sever to you if you’re not used to being at the helm of something being used by a lot of people, but that’s how we make these sausages (at least if we’re actually trying to do it correctly). The guy who authored the semver spec, Tom Preston-Werner</a> (also known for co-founding GitHub</a> and creating gravatar</a>) has a blog post about this called Major Version Numbers are Not Sacred</a>. What people end up doing in practice is building up a lot of breaking changes and then releasing them all at once, to not constantly destroy the API/ABI and UX of their software. How this ties in with a more continuous development is a very interesting question, e.g. how does eza</a> manage to release a new version every week, yet also work on breaking changes like config files or moving to a new command line parser (specifically clap</a>). I think such a discussion is far out of scope for this post, but I may write about it in the future. Conventional Commits and Linguistic Relativity</h2> If you were to ask wikipedia what “Behavioral Design” means, it would reply: Behavioural design is a sub-category of design, which is concerned with how design can shape, or be used to influence human behaviour. </blockquote> Now, how is that relevant to us as developers, maintainers, or just hobbyist hackers? Well you see, the design of conventional commits, whether intentional or not, has a lot of decisions that end up shaping how contributors interact with your project. When I initially forked eza</code>, I mostly moved us to conventional commits fairly early because I was concerned with a maintenance burden. I knew that exa</code> had been a project of maintainer burnout, and what I was trying to do was create longevity for the project. That meant that if there was something I could offload to someone or something else, I would, including changelogs. And yes, conventional commits means I don’t have to write changelogs, I can automate it, and it means I don’t have to think too hard about what version is next, it’s automatically generated. In fact, I can release eza every week because a release is just running two scripts, doing a cargo publish</code>, and a nix-update eza</code> in the nixpkgs</a> repo and opening a PR. While that release velocity is great, I found another, much more interesting perk of using conventional commits. Namely that conventional commits lead to better commits, better commit summaries, better PRs, and better structured commits. For instance, if a project just allows a commit summary to be an arbitrary string, what you’ll often end up with is either something like made extensions work</code>, a single commit that encompasses vast refactorings of the codebase, introduces a feature, fixes 3 different bugs, and is one big monolithic and unrevievable mess. On the other hand, it also saves you from the series of wip</code>, wip 2</code>, formatted code and added quantum cryptography</code>, made compile</code>. These are equally unreviewable, and completely destroy your git log. Now, you may have the discipline of projects like nixpkgs</a> of asking contributors to rebase their commits, so that they each represent unique atomic changes. But even then, what conventional commits gives you is a language to express what these changes actually are. Before I had conventional commits as something I could just write without looking at the spec, I would genuinly struggle putting words to what my changes did. It made it hard to create good PRs to projects. Now, it’s much easier. I just go: refactor(parser): used enums instead of bools in match logic</code></li> fix(parser): capital letters not differentiated</code></li> fix(parser): long lines ignored</code></li> feat(parser): add long flags</code></li> docs(parser): document long flag usage</code></li> style: change flag rendering</code></li> ci: check parser flags workflow</code></li> </ul> Not only does this make my PR much more structured, but by getting into that habit, I find that my work also becomes much more organized. Instead of just working on some hodgepodge mix of things at the same time and committing either to save my work or to put all of it together in a disorganized sack of a single commit, I find that I am much more able to dissect a problem into discrete solvable steps. I’d even go as far as saying this makes you a better programmer, as breaking down a large problem into small solvable chunks is a great problem solving strategy. What conventional commits does is it gives you this language to describe the steps it takes to solve a problem, and getting familiar with that vocabulary not only makes you a better programmer, but it makes it much easier to work with you, and it heavily reduced the burden on maintainers. Linguistic relativity is the idea that the structure of a language influences the speakers world view and cognition, changing how you interact with your surroundings. Linguistic relativity is a hypothesis, but I find it pretty easy to prove the weak case that if you can’t signify something, you will struggle to operate with it. In other words, conventional commits get you merged faster, and makes your code better. Hopefully you’re sold by this point. There are some tools for dealing with these, such as git-footers</a>. Personally, I don’t really use those. But it’s good to know they exist. ↩</a> </li> </ol> rustls and aws-sdk seems incompatible with FOSS Christina Sørensen — Mon, 17 Jun 2024 14:00:00 +0000 I should preface this with the fact that I am by no means a lawyer or anything like it. Nothing here constitutes a legal argument or should be taken as true. I may be completely wrong or have missed crucial details. As part of a rust project I’ve been working on at dayjob, I’ve had to audit all dependencies for license compatibility with the AGPLv3, our preferred license. Sadly, I encountered what I think is a major ecosystem blocker, and we will likely not be able to license the code under AGPLv3, or any GPL license for that matter, having to go with a more permissive, but compatible license until the problem this post is about is fixed. Which is a shame, and something I wanna bring more attention to. A fairly fundamental crate in rust is ring</a>, providing “safe, fast, small crypto using Rust”. To understand how fundamental it is, consider that other crates such as rustls</a>, the aws-sdk-rust</a> crate, and sct</a> make use of ring. And this is by no means an exhaustive list. The problem with ring, as you may well notice if you dare read its rather convoluted license</a>[1]</a>, is that amongst it’s licenses is the OpenSSL license. The problem with the OpenSSL license is it’s equivalent to a BSD4 clause license. That is, it contains the problematic advertisement clause. * 3. All advertising materials mentioning features or use of this * software must display the following acknowledgment: * "This product includes software developed by the OpenSSL Project * for use in the OpenSSL Toolkit. (http://www.openssl.org/)" </code></pre> This clause is well known</a> for being GPL-incompatible, and indeed, if you look up the OpenSSL license on the gnu.org</a> list of gpl compatible licenses</a>, it is indeed the case that the license is considered GPL-incompatible. But so what? Big deal? Big deal… yes. This means that if you want to interface with s3 storage[2]</a>, or use rustls, you’ll be GPL incompatible! For example, consider the following GNU Affero General Public License Version 3 projects that use crates in the aws-sdk</code> suite, or rustls</code>. zed</a></li> Signal-Calling-Service</a></li> garage s3</a></li> quickwit</a></li> rustdesk</a></li> </ul> And these are just some quick examples I found with a quick github search, there is probably much more software currently out there in the rust ecosystem that’s using a license that its crates are fundamentally incompatible with[3]</a>. While this is a problem the ring</code> crate seems to be aware of</a>, it’s sad to see that there is such a large wound on the FOSS compatibility of the rust crate ecosystem. Not to put any blame anywhere. I am well aware that if this was easy to fix, it wouldn’t be a problem anymore, and anyone wanting to criticize the ring devs should spend the effort helping them fix the situation, rather than complaining, they seem eager to solve this. But it is something I think needs more awareness. Update 2024-06-19: We’ll try to go with EUPL-1.2, as it is compatible with all BSD licenses. Also, it seems to have many other useful properties. This does not make the problem disappear however, but it does create a path forward for a stronger license for rust projects affected that want SaaS loophole protections. This is a problem the ring devs are working on. Let it be a lesson to start your project off using REUSE</a> or paying the price down the line >:3 ↩</a> </li> Yes, there are other crates than the aws-sdk, but when investigating this for the project in question, none seem to be tennable solutions. Vet your dependencies! ↩</a> </li> Yes, there are potential ways to mitigate this, but by a quick skim, I haven’t seen any of those implemented in these projects, and implementing them would likely be both architecturally ugly, and horribly inefficient. For… reasons, I’ll not actually elaborate on how you could potentially circumvent licenses like this, because I’m really not a lawyer. ↩</a> </li> </ol> Merge or Rebase Christina Sørensen — Wed, 29 May 2024 13:40:00 +0000 This isn’t the article that’s gonna explain merge vs rebase for people that haven’t used git before (sorry). Rather, this is gonna be about some more meta things for maintainers, release engineers, and other people who care about development process. Git gives you a lot of options. How you use it isn’t prescribed in advance. There are a lot of ways that people might use git differently, but here, I’m more seeking to investigate some of the ways of using git that are relevant to people in charge of git repositories with a non-trivial amount of contributors, like myself. While I enjoy having these things done differently between projects I’m working in and part of maintaining — as it gives some diversity to the methods I get to use in my work, and let’s me continuously evaluate multiple methods[1]</a> in practice — I do think it makes sense that for a single project, there is consistency in the way it’s developed, so that expectations can be set, and procedures can be learned and become second nature. There are two main questions for a project to decide when it comes to merging via git. How to merge feature branch changes from PRs.</li> How to merge newest changes into outdated branches.</li> </ol> I’ll discuss both, and then summariese my observations. Merging branches</h2> When some developer starts working on a software feature, especially in open source, those changes, typicall, will be made in a separate branch that has branched off from the main branch. This is often called a feature branch. This allows the developer to work on their feature in isolation, and, at some point, put those changes back into the main branch, for all people to use. This is what some may call trunk based development, although the lines get fuzzy about how long a PR should live. That discussion is out of scope for this post, but I’d love to follow up with a post about that at some point. When moving changes from your feature branch into main, you can have potentially three possible options for merging. Fast Forward</li> Rebase and Fast Forward</li> Merge</li> </ul> For fast forward, main may not have changed while you were working, and so you can “fast forward” your changes on top of main, without having to worry about any conflicts. This is, for both developers and maintainers, extremely easy, with some reservations that will be discussed later. Another option is to take the changes from the updated main branch, and git pull --rebase origin main</code> them into your feature branch, placing your changes on top of the latest changes from main. As you may guess, this allows you to git merge --ff-only</code> your changes into the main branch. The final strategy is the classical merge. You may decide to merge your changes, and the changes in main together, using a special commit that represents the operation of merging the two together, a merge commit. Such a commit has special properties, such as having two parents, and being the result of the unification of two separate branches. Of course, if you’re not doing just a simple fast forward, either a rebase-ff or merge may require that you solve potential code conflicts. For instance, someone may have changed a line that you have also changed, leading to the two versions colliding. For the merge, you resolve this by making the merge commit define the merged version, for a rebase, you’ll go through your changes and adjust them so they fit cleanly onto the branch you’re pulling. One important difference to note for us, is between the two, merging into main typically gets done by the maintainer, where as rebasing gets done by the developer working on the feature branch. Rebasing also has some downsides. For example, if the person rebasing doesn’t have the PGP private key of the people who’s commit are included in the rebase, they cannot sign the commits. This means that if Alice made 3 commits and signed them with her PGP signature, Bob can’t later rebase those changes onto some other commit without breaking that signature. This is because, while the code doesn’t change, the commit revision does, and so the signature is wrong. Merging does not have this issue, as a merge is simply the actual merge commit, containing the new state of the branch, that potentially specifies how to rearrange everything so as to not cause any conflicts. Thus it doesn’t suffer from breaking the signatures. Now, if you’re using rebase for merges into main, as we do in eza, you may still want a signature verifying that a trusted person has indeed verified these changes are up to whatever standard they guarantee (eza at best guarantees that it runs on my linux box that you should be able to install it on NixOS[2]</a>). The way we solve this is by tagging all our releases with a signed tag, making sure you know that I’ve personally said that this is the codebase I want you to run for a given release. But even then, it’s coping. It’s not as nice as having a tree full of the signatures of everyone that’s committed, right on their commits. Another point raised against rebases is they rewrite history. This is true, unlike a merge that just merges two branches and keeps both in history, the rebase will make it appear as thou the feature branch was developed on top of the main branch. I don’t personally think this is a huge concern, except the loss of potential pretty git log --graph</code> outputs. One of the annoyances of merges, in comparison, is that they’re very noisy. Consider this output from nixpkgs: git log --oneline bfb7a882678e Merge pull request #314109 from trofi/githooks.tests-fix-eval 47bdef656cc5 Merge pull request #311260 from purepani/update-svelte-language-tools fc165a03b23a Merge pull request #309524 from r-ryantm/auto-update/enet 3de810d52cbe Merge pull request #309724 from r-ryantm/auto-update/secp256k1 1d6a2f5a4d4d Merge pull request #309787 from r-ryantm/auto-update/python311Packages.minio a36fa5451c44 Merge pull request #309426 from Rconybea/add-sphinxcontrib-ditaa 890f8e436a56 Merge pull request #309476 from r-ryantm/auto-update/goperf dfa36c1d67f5 Merge pull request #312557 from r-ryantm/auto-update/intune-portal 7d8ed5ce921d Merge pull request #291853 from greaka/grafana d0a20d7c5955 Merge pull request #314112 from khaneliman/bicep 4d2462511f06 Merge pull request #314099 from mrkline/snapper-and-borgbackup-doc-fix 3e3ac0e7baa8 Merge pull request #305516 from OPNA2608/init/lomiri/ayatana-indicator-display a46ce7c77d9d svelte-language-server: convert to buildNpmPackage 7962cbb2326b Merge pull request #314082 from superherointj/bicep-0.27.1 5771dbfa7d2f bicep: fix updater script 2db4e7d035f8 Merge pull request #313729 from pluiedev/zhf-24.05/gobang 44744fc83f3c githooks.tests: fix eval 0f3add331c6a segger-jlink: 794l -> 796b </code></pre> This… isn’t nice, and makes history way harder to read. Compare to eza: git log --oneline 22dbeec2 (HEAD -> main, origin/main, origin/HEAD) build(deps): bump libc from 0.2.154 to 0.2.155 82e4f0f3 build(deps): bump trycmd from 0.15.1 to 0.15.2 bbede6d4 (tag: v0.18.16) chore: release eza v0.18.16 70891fa1 docs(README): use 3 columns for packaging status badge e8e0b6da fix: change windows-only imports to be windows-only 15528d8b docs(install): fix typo in `INSTALL.md` 26e5943a docs: update INSTALL.md 20241984 build(deps): bump DeterminateSystems/nix-installer-action from 10 to 11 feff970d docs(man): replace decay with color-scale a1c36389 build(deps): bump DeterminateSystems/flake-checker-action from 5 to 7 7437c2a1 (tag: v0.18.15) chore: release eza v0.18.15 f1ef455b feat(devtools): add optional tag argument to deb-package.sh 23502d3c fix(devtools): correct command for latest tag in deb-package.sh 292cf7ec feat(devtools): return to original commit at the end of deb-package.sh 4f542eaf docs(reaedme): add some keywords for benefit of ctrl-f 6d6612d2 docs(readme): move heading out of collapsed section abe9f587 docs(readme): correct heading levels in markdown 01a9c2aa docs(readme): add how to find man pages in terminal and online. Partly fixes #967 f78e4bb6 (tag: v0.18.14) chore: release eza v0.18.14 01919cdd build(deps): bump palette from 0.7.5 to 0.7.6 414f70ba build(deps): bump unicode-width from 0.1.11 to 0.1.12 1554472a build(deps): bump libc from 0.2.153 to 0.2.154 bebd39c0 build(deps): bump uzers from 0.11.3 to 0.12.0 df7d51fc feat: add icon for "cron.minutely" directory 8b7dc5f5 (tag: v0.18.13) chore: release eza v0.18.13 c0df8ecd feat: generate completion/manpage tarballs on release 8afb5cc2 (tag: v0.18.12) fix: checking for deref flag in file_name a4782d1d feat: add scheme filetype and icons 87b36785 fix: allow unused imports for freebsd 99562e3a (tag: v0.18.11) chore: release eza v0.18.11 07f67708 fix: build aarch64, arm without libgit2 17733e9a fix(netbsd): enable the rule only for NetBSD. 7664a1fb ci: bump NetBSD version to 10.0 462fc344 fix(netbsd): fix clippy lints 75f1f8cf (tag: v0.18.10) chore: release eza v0.18.10 </code></pre> Isn’t that a lot nicer? I think so. Of course, what may be lost is that it’s no longer obvious from what PR a change originated. It also isn’t obvious to me when some feature branch was created, how long it was worked on and so on. This isn’t… really that important either, but it’s nice to have in logs. But you can just look at branches if you need to know this for tooling or something. For nixpkgs, you get a to have a visual overview of how branches end up in main. It looks like this: git log --oneline --graph * 4d4571b20a29 (HEAD -> devpi-loadcredential) nixos/devpi-server: fix loading credentials as DynamicUser * bfb7a882678e Merge pull request #314109 from trofi/githooks.tests-fix-eval |\ | * 44744fc83f3c githooks.tests: fix eval * | 47bdef656cc5 Merge pull request #311260 from purepani/update-svelte-language-tools |\ \ | * | a46ce7c77d9d svelte-language-server: convert to buildNpmPackage * | | fc165a03b23a Merge pull request #309524 from r-ryantm/auto-update/enet |\ \ \ | * | | 8e729c70a119 enet: 1.3.17 -> 1.3.18 * | | | 3de810d52cbe Merge pull request #309724 from r-ryantm/auto-update/secp256k1 |\ \ \ \ | * | | | f96a90834c5e secp256k1: 0.4.1 -> 0.5.0 * | | | | 1d6a2f5a4d4d Merge pull request #309787 from r-ryantm/auto-update/python311Packages.minio |\ \ \ \ \ | * | | | | 24d453186792 python311Packages.minio: 7.2.6 -> 7.2.7 * | | | | | a36fa5451c44 Merge pull request #309426 from Rconybea/add-sphinxcontrib-ditaa |\ \ \ \ \ \ | * | | | | | 197d6261ebf8 python311Packages.sphinxcontrib-ditaa: init at 1.0.2 | * | | | | | 51de558f5634 maintainers: add rconybea * | | | | | | 890f8e436a56 Merge pull request #309476 from r-ryantm/auto-update/goperf |\ \ \ \ \ \ \ | * | | | | | | 8eea66b3f69a goperf: 0-unstable-2023-11-08 -> 0-unstable-2024-05-10 * | | | | | | | dfa36c1d67f5 Merge pull request #312557 from r-ryantm/auto-update/intune-portal |\ \ \ \ \ \ \ \ | * | | | | | | | 010c4a334eaf intune-portal: 1.2404.23-jammy -> 1.2404.25-jammy * | | | | | | | | 7d8ed5ce921d Merge pull request #291853 from greaka/grafana |\ \ \ \ \ \ \ \ \ | * | | | | | | | | 254dbdcc6296 grafanaPlugins.grafana-oncall-app: init at 1.5.1 | * | | | | | | | | 0e5f44658ee6 maintainers/team-list: add fslabs | * | | | | | | | | 8d6f8c9ed75a maintainers: add lpostula | * | | | | | | | | 7bda925dacb2 maintainers: add greaka * | | | | | | | | | d0a20d7c5955 Merge pull request #314112 from khaneliman/bicep </code></pre> Another problem worth considering is that a rebase may — as Astrid</a> pointed out — results in main being in a state where a git bisect</code> of a build command may fail, if you haven’t ensured that all commits in the feature branch still worked after being rebased on top of the latest main. Now this is rare, but rare isn’t really a good argument for not caring here, as breaking once would be extremely annoying for people working on a bugfix, leading to false git dissect</code> positives. There are solutions to this. You could (perhaps) have each commit in a branch run against CI. This may be outside of your compute budget thou. Another option is to run your build command, e.g. cargo build</code> on each commit after a rebase, but then, this kind of checking step usually falls on the maintainer, and we, as maintainers, should seek to limit our own burdens as much as possible (to prevent burnout[3]</a>, the killer of most FOSS projects). To eza, this isn’t fatal, but a very interesting point worth keeping in mind. Updating a branch</h2> Of course, since there may be multiple people all working on separate branches, there is another problem. One person might have worked on something, at the same time as you worked on something else, and then gotten their changes merged before you had a chance. But if that happens, your changes might now conflict with what was in the main branch when you started, and worse, their changes may make ruin your work, or even make it unable to compile! The first choice a project should make, is whether or not to allow outdated feature branches to be merged. The problem with tolerating outdated branches is you will not be able to fix any problems related to integrating the code with the main codebase before merging it. This puts the burden of ensuring the code works on the maintainer, rather than the developer, and this is bad! If you do choose to not allow this, another question faces you: how should the main branch be merged into the feature branch when the feature branch is out of date with the main branch. Your options are either to merge the main branch into your feature branch, or to rebase your changes on top of main, making it like if you had started working on the latest changes. To me, it’s a more simple dilemma, merges shouldn’t exist if they don’t meaningfully represent merging two things together. Simply updating your branch is what rebase does best, and having merges both from feature branches into main and from main into your feature branch is absolutely tasteless. This clutters your history for no reason. It also makes it harder for you to fix your commits when the reviewer tells you that you had a faulty commit 3 commits before you merged main into your feature branch and kept working for 4 more commits. In general, rebasing with merge commits is tedious, and we only consider merges viable for the main branch under the premise that we won’t later need to rebase main, because everything that gets into main is in a state of being up to standard. Sadly, code forges lack a feature to only allow updating branches with the projects preferred standard, unlike for merges where rebase, squash, and merge are all options, that all can be disabled by the repositories owner(s). Summary</h2> So what where the tradeoffs? Merging feature branches into main Merges. Preserves history.</li> Gives you git log --graph</code>.</li> Doesn’t break PGP signatures.</li> Creates ugly git log</code>.</li> Gives weak protection that all commits build (if you have CI)</li> Puts burden of verification on maintainer.</li> </ul> </li> Rebase. Rewrites history.</li> No git log --graph</code>.</li> Breaks PGP signatures.</li> Creates useful git log</code> output.</li> Needs more care if you want all commits to be validated.</li> Puts burden of verification on committer (mostly).</li> </ul> </li> Fast Forward. Like a rebase, but with less drawback.</li> </ul> </li> </ul> </li> Updating your feature branch. Merges. merges make git rebase -i</code> difficult, when asked to change your commits.</li> It makes your git log</code> unreadable.</li> Doesn’t provide any benefits.</li> Mostly just a quirk that git even allows this, it doesn’t make sense.</li> </ul> </li> Rebase. The preffered choice.</li> Fits like a glove for updating your feature branch.</li> </ul> </li> </ul> </li> </ul> As a maintainer, remember that what where really trying to do if offload any non-maintainer burdens onto relavant roles, like developers and reviewers. We may wear multiple hats, often we are the final reviewers, but even then, it shouldn’t be our job to ensure a clean merge, we should meerly verify it. The main goal is always: maintainers shouldn’t need to fix the feature branch! This also applies for formatters. I personally mostly use alejandra for nix, but dayjob uses nixfmt (the RFC 166 version). I just hit a format button before commiting anyways, so it’s just cool to see the different options and feel out which ones I truly prefer. ↩</a> </li> Because I don’t have any MacOS, Windows, or BSD boxes to test on, and we don’t have any maintainers that do with the spoons to be involved in giving such guarantees. ↩</a> </li> Consider reading this excellent resource https://opensource.guide/maintaining-balance-for-open-source-maintainers/</a>, and this mozilla worksheet https://docs.google.com/document/d/1esQQBJXQi1x_-1AcRVPiCRAEQYO4Qlvali0ylCvKa_s/edit?pli=1#heading=h.xd8n2v3f4866</a>. ↩</a> </li> </ol> Grok Git Repos Christina Sørensen — Wed, 15 May 2024 00:00:00 +0000 Introduction</h1> Git is often perceived as a necessary evil, a tool to be tolerated rather than mastered. But treating your primary work tool as a mystery is a vulnerability. When the abstraction leaks — and it always does — the developer who understands the internals can surgically repair the damage. In contrast, those who rely on surface-level knowledge often leave behind a sloppy timeline. A history riddled with vague messages and botched merges signals a lack of care that code quality alone cannot hide. Unfortunately, we often learn git through folklore — memorizing obscure commands to save us when things go wrong, without understanding why they work. We rely on luck rather than logic. To escape this ritualistic dependency, we must strip away the interface. To truly grok git, we will ignore the high-level commands and build a repository from the ground up, proving that the magic is actually just simple, elegant engineering. Creating a Git Repository from Scratch</h1> Let's create a folder for us to work in. $ mkdir grok $ cd grok </code></pre> Now, we have a folder. A folder isn't a git repository, as we can verify. $ git status fatal: not a git repository (or any of the parent directories): .git </code></pre> As the status</code> subcommand tells us, neither is the directory we currently created, of any of it parent directories a git repository. If that wasn't the case for you, that would be because you created the folder grok</code> as a subdirectory of a git repository. </blockquote> The error seems to mention a .git</code> folder. Turns out, when speaking of a git repository, we are actually referring to this folder in most cases[1]</a>. So, in theory, it should be simple enough to create a git repository right, just create a .git</code> folder and we are good to go? $ mkdir .git $ git status fatal: not a git repository (or any of the parent directories): .git </code></pre> Ohh, apparently not so. If we were to look up the definition of a repository in the git glossary</a>, we would see the following. A collection of refs together with an object database containing all objects which are reachable from the refs, possibly accompanied by meta data from one or more porcelains. A repository can share an object database with other repositories via alternates mechanism. </blockquote> Ahh, simple. Seems like we need to have some refs and an object database with all the objects refs can reach. Also, there might be meta data from porcelains? Although the definition may seem complex, the underlying concept is straightforward. Rather than summarizing it briefly, we will demonstrate its simplicity through a practical, step-by-step example. First, we need an object database. Well, database sounds a bit hard, we could just make a folder for it instead in our git repository. $ mkdir .git/objects </code></pre> Now, we also need a collection of refs. No issue, just make a folder for them. $ mkdir -p .git/refs/heads </code></pre> What are heads? Don't worry, we'll get to it. Now we have the bare minimum for a repository thou, right? Well, let's try to find out! $ git status fatal: not a git repository (or any of the parent directories): .git </code></pre> It appears we are missing a critical component. $ echo "ref: refs/" > .git/HEAD </code></pre> Now, is this finally a repository? Let's see what git thinks the status is. $ git status On branch main No commits yet nothing to commit (create/copy files and use "git add" to track) </code></pre> Seems like it is! …So now we have a git repository, that was easy. Committing to a Repository</h1> Creating Objects</h2> While creating a git repository manually is an interesting exercise, it is of little value unless we can populate it with files. We likely will want to fill this repository with text files. For the occasion, you can use fortune to generate something, but if you use what I got, you'll be able to compare your hashes to the ones in the article. "If a listener nods his head when you're explaining your program, wake him up. </blockquote> That is very wise indeed. We want to add this to the repository. But how? Well, one way to add it is the following, using the git hash-object</code> subcommand. $ echo "If a listener nods his head when you're explaining your program, wake him up." | git hash-object --stdin -w 665e95f1674e9466cb429bdfebaf1b8792ef0eec </code></pre> Okay, two questions now. What is 665e95f1674e9466cb429bdfebaf1b8792ef0eec</code>, and what did hash-object</code> just do? We can inspect the git repository to investigate. Examining the directory tree structure provides some insight. $ tree .git .git/ ├── HEAD ├── objects │ └── 66 │ └── 5e95f1674e9466cb429bdfebaf1b8792ef0eec └── refs └── heads 4 directories, 2 files </code></pre> Here, we see that our object “database” in the objetcs folder has seen some change. There is a folder, 66</code>, with a file that has an equally vexing name, 5e95f1674e9466cb429bdfebaf1b8792ef0eec</code>. I wonder what is in this file. Let's try to take a look at it. $ cat .git/objects/66/5e95f1674e9466cb429bdfebaf1b8792ef0eec WKX Y$b `݆!rB-8}ɢ,xSOŋE598}y? % </code></pre> The file does not contain plain text; it appears to be in a binary format. We can use the file</code> command to determine its type. $ file .git/objects/66/5e95f1674e9466cb429bdfebaf1b8792ef0eec .git/objects/66/5e95f1674e9466cb429bdfebaf1b8792ef0eec: zlib compressed data </code></pre> Apparently, it's some data compressed with zlib. That means it's likely DEFLATE compressed (RFC 1951) in a zlib wrapper (RFC 1950)^{[2]</a>.} There is actually a neat hack we can use to view this data[3]</a>. $ printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - .git/objects/66/5e95f1674e9466cb429bdfebaf1b8792ef0eec | gzip -dc blob 78If a listener nods his head when you're explaining your program, wake him up. gzip: stdin: unexpected end of file </code></pre> We have located our string. The output begins with blob</code>, which specifies the git object’s file type, followed by 78</code>, indicating the blob’s size in bytes[4]</a>. Objects in git have 3 primary types: blob</li> tree</li> commit</li> </ul> When we ran the hash-object</code> subcommand, we created a blob. Regarding the name, that is just the sha1 hash (RFC 3174) of the object, and some other metadata and such[5]</a>. The actual output of the command was this hash. It is 40 characters long. The name of the directory in the object database is the first two characters of the hash, that is 66</code>, and the actual object files name is the 38 other characters. Manually decompressing objects for inspection is inefficient. Instead, we can use the hash 665e95f1674e9466cb429bdfebaf1b8792ef0eec</code> to inspect the newly created object. Inspecting Objects</h2> We do this with the git cat-file</code> subcommand. To see the file type of an object, we use the type flag -t</code>. $ git cat-file -t 665e95f1674e9466cb429bdfebaf1b8792ef0eec blob </code></pre> That means that the type (-t</code>) of the object we made is a blob, as we saw by inspecting it. What if we wanna see the contents of the blob? We use -p</code> for print. $ git cat-file -p 665e95f1674e9466cb429bdfebaf1b8792ef0eec If a listener nods his head when you're explaining your program, wake him up. </code></pre> This procedure is significantly more convenient than the manual decompression hack used earlier. Also, another useful flag to know is -s</code>, for size. $ git cat-file -s 665e95f1674e9466cb429bdfebaf1b8792ef0eec 78 </code></pre> This matches the 78</code> observed earlier, confirming the object size in bytes. Index</h2> Okay, enough about the blob. It seems like we have added an object – with some text – into the git repository now. We might wonder if this is reflected in the git status</code> of the repo. $ git status On branch main No commits yet nothing to commit (create/copy files and use "git add" to track) </code></pre> Adding an object to the database stores the data, but it does not tell git where that file belongs in your project. This is the purpose of the index (or staging area): to build the tree structure for the next commit. Without being registered in the index, our blob is essentially a “loose” object. If it remains unreferenced, it will eventually be removed by git gc</code> (garbage collection). To actually commit our object, we must first register it in the index. git update-index --add --cacheinfo 100633 665e95f1674e9466cb429bdfebaf1b8792ef0eec truth.txt </code></pre> The long string is the hash of the blob object we created, and truth.txt</code> is a fitting filename. What about 100633</code>? Looking at man git update-index</code> we see that… --cacheinfo <mode>,<object>,<path>, --cacheinfo <mode> <object> <path> Directly insert the specified info into the index. For backward compatibility, you can also give these three arguments as three separate parameters, but new users are encouraged to use a single-parameter form. </code></pre> Although the documentation suggests the single-parameter form for new users, we have used the separate parameter form here for clarity. The value 100633</code> is the mode of the file. In git, this mode is a 6-digit octal number that defines the file type and permissions, similar to standard UNIX file modes but restricted to a few specific values. But crucially, for blobs in git, we only have 3 modes available^{[6]</a>:} 100644 a normal file</li> 100755 a executable file</li> 120000 a symbolic link</li> </ul> Let's take a look at the repo structure again. $ tree .git .git/ ├── HEAD ├── index ├── objects │ └── 66 │ └── 5e95f1674e9466cb429bdfebaf1b8792ef0eec └── refs └── heads 4 directories, 3 files </code></pre> Here we notice that there now is an index</code> file. So all we did was add some index file</code>? …well, slow down. Let's do a sanity check with git status</code>. $ git status On branch main No commits yet Changes to be committed: (use "git rm --cached <file>..." to unstage) new file: truth.txt Changes not staged for commit: (use "git add/rm <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) deleted: truth.txt </code></pre> Aha! So we did something, we staged (added) the truth.txt file… but we also have unstaged changes saying we deleted it? Well, yes. We have told git that the blob object we created is in the working-tree, and it has the actual blob for this in its object database… but apparently can't find the file we told it about. This makes sense, since we just created the blob from stdin and didn't actually add any files. Because the file exists in the index but is missing from the actual working directory, git correctly assumes it has been removed. Hmm, while we are at it, let's take a look at .git/index</code>, just to see what that is about. $ cat .git/index DIRCf^gNfB truth.txtqblr33il% </code></pre> This confirms it is not a text file. We can verify its type using the file</code> command. $ file .git/index .git/index: Git index, version 2, 1 entries </code></pre> The output identifies it as a Git index, version 2</code> with 1 entries</code>. This single entry corresponds to the file we just added. To see what this actually looks like on disk, we can inspect the raw binary data. $ hexdump -C .git/index 44 49 52 43 00 00 00 02 00 00 00 01 00 00 00 00 |DIRC............| 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00 00 00 00 00 00 81 a4 00 00 00 00 00 00 00 00 |................| 00 00 00 00 66 5e 95 f1 67 4e 94 66 cb 42 9b df |....f^..gN.f.B..| eb af 1b 87 92 ef 0e ec 00 09 74 72 75 74 68 2e |..........truth.| 74 78 74 00 c1 b9 71 62 b4 ab c1 c2 6c 72 33 33 |txt...qb....lr33| 92 69 92 99 ac 6c 8e ff |.i...l..| </code></pre> (I removed the offset from the output) The first 4 bytes, DIRC</code>, are the “magic number” — a constant value used to identify the file format (short for “Directory Cache”). Following some padding bytes[7]</a>, the 02</code> indicates the version number. The rest of the file contains the entry data, including the filename truth.txt</code>. If we were to add another file, likely patterns would emerge in the binary structure. Investigating how the index looks in a larger project is an excellent way to learn more, so I have included it in the Exercises section below. Tree Objects</h2> Back on track, we've now made the index thingy, and we'll have to write it with the write-tree</code> subcommand. $ git write-tree a6325f064bac723691f20c0b1ed2bea82a1728fd </code></pre> git status</code> does not seem to have changed, but if we check what's in repo's file structure, we'll see something did change. $ tree .git .git ├── HEAD ├── index ├── objects │ ├── 66 │ │ └── 5e95f1674e9466cb429bdfebaf1b8792ef0eec │ └── a6 │ └── 325f064bac723691f20c0b1ed2bea82a1728fd └── refs └── heads 5 directories, 4 files </code></pre> We notice the a6325f064bac723691f20c0b1ed2bea82a1728fd</code> sha-1 hash refers to a new object. Interesting, wonder what this is. To thoroughly understand this object, we will inspect it manually by examining the raw file. This is the “hard way,” but it offers the most insight into the underlying data structure. $ file .git/objects/a6/325f064bac723691f20c0b1ed2bea82a1728fd .git/objects/a6/325f064bac723691f20c0b1ed2bea82a1728fd: zlib compressed data </code></pre> Okay, that's zlib compressed data. We know that, so let's use that same hack to figure out what is inside of this blob. $ printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - .git/objects/a6/325f064bac723691f20c0b1ed2bea82a1728fd | gzip -dc tree 37100644 truth.txtf^gNfB gzip: stdin: unexpected end of file </code></pre> This object is not a blob, but a tree — one of the three primary git object types. The number 37100644</code> initially appears confusing, but it is actually two separate values concatenated: the size (37</code> bytes) and the file mode (100644</code>). Following this are the filename and a binary sequence f^gNfB</code>. The sequence f^gNfB</code> matches data we previously observed in the hexdump</code> of the .git/index</code> file. Identifying its specific purpose is left as an exercise for the reader. Let's look at the contents the canonical way. $ git cat-file -p a6325f064bac723691f20c0b1ed2bea82a1728fd 100644 blob 665e95f1674e9466cb429bdfebaf1b8792ef0eec truth.txt </code></pre> An Experiment with Tree Objects</h2> An important concept to explore is how a tree object handles multiple files. To demonstrate this, we will create another blob. $ echo "AMOGUS" | git hash-object --stdin -w f58617716d903fb842b5606a335ff1406b9a21d3 </code></pre> And add it to our index file. $ git update-index --add --cacheinfo 100633 f58617716d903fb842b5606a335ff1406b9a21d3 amogus.txt </code></pre> Now, let's look at the repository. We have 3 objects in our objects database. $ tree .git .git ├── HEAD ├── index ├── objects │ ├── 66 │ │ └── 5e95f1674e9466cb429bdfebaf1b8792ef0eec │ ├── a6 │ │ └── 325f064bac723691f20c0b1ed2bea82a1728fd │ └── f5 │ └── 8617716d903fb842b5606a335ff1406b9a21d3 └── refs └── heads 6 directories, 5 files </code></pre> Let's dump the index binary. $ hexdump -C .git/index 44 49 52 43 00 00 00 02 00 00 00 02 00 00 00 00 |DIRC............| 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00 00 00 00 00 00 81 a4 00 00 00 00 00 00 00 00 |................| 00 00 00 00 f5 86 17 71 6d 90 3f b8 42 b5 60 6a |.......qm.?.B.`j| 33 5f f1 40 6b 9a 21 d3 00 0a 61 6d 6f 67 75 73 |3_.@k.!...amogus| 2e 74 78 74 00 00 00 00 00 00 00 00 00 00 00 00 |.txt............| 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00 00 00 00 00 00 81 a4 00 00 00 00 00 00 00 00 |................| 00 00 00 00 66 5e 95 f1 67 4e 94 66 cb 42 9b df |....f^..gN.f.B..| eb af 1b 87 92 ef 0e ec 00 09 74 72 75 74 68 2e |..........truth.| 74 78 74 00 54 52 45 45 00 00 00 06 00 2d 31 20 |txt.TREE.....-1 | 30 0a 94 c6 2b 91 4b 84 9d 9a 2e 3c 20 e4 3e 93 |0...+.K....< .>.| 1b 69 3f 19 3d bd |.i?.=.| </code></pre> Nothing too spectacular here, we notice that we see both amogus.txt and truth.txt. We also see TREE at the end of the file, that must be our tree object. However, what happens when we run write-tree</code>? $ git write-tree aee76412ed220742aeaf02ca1c50519bcea013e1 </code></pre> Let's dump the index again. $ hexdump -C .git/index 44 49 52 43 00 00 00 02 00 00 00 02 00 00 00 00 |DIRC............| 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00 00 00 00 00 00 81 a4 00 00 00 00 00 00 00 00 |................| 00 00 00 00 f5 86 17 71 6d 90 3f b8 42 b5 60 6a |.......qm.?.B.`j| 33 5f f1 40 6b 9a 21 d3 00 0a 61 6d 6f 67 75 73 |3_.@k.!...amogus| 2e 74 78 74 00 00 00 00 00 00 00 00 00 00 00 00 |.txt............| 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00 00 00 00 00 00 81 a4 00 00 00 00 00 00 00 00 |................| 00 00 00 00 66 5e 95 f1 67 4e 94 66 cb 42 9b df |....f^..gN.f.B..| eb af 1b 87 92 ef 0e ec 00 09 74 72 75 74 68 2e |..........truth.| 74 78 74 00 54 52 45 45 00 00 00 19 00 32 20 30 |txt.TREE.....2 0| 0a ae e7 64 12 ed 22 07 42 ae af 02 ca 1c 50 51 |...d..".B.....PQ| 9b ce a0 13 e1 07 2b 99 c5 b6 0e 3d 53 33 c9 21 |......+....=S3.!| dd b1 75 41 41 84 b1 d8 ec |..uAA....| </code></pre> The data following the TREE</code> identifier has increased in size. We will now inspect the contents of this newly written tree. $ git cat-file -p aee76412ed220742aeaf02ca1c50519bcea013e1 100644 blob f58617716d903fb842b5606a335ff1406b9a21d3 amogus.txt 100644 blob 665e95f1674e9466cb429bdfebaf1b8792ef0eec truth.txt </code></pre> Now we seem to have two files inside of the tree. Time to Commit</h2> So now, it's time to create a commit. Above, when we did write tree, we created the tree</code> object aee76412ed220742aeaf02ca1c50519bcea013e1</code>. This includes both of the blobs we made. So how do we commit that? It's actually really simple, we just use git commit-tree</code>. git commit-tree aee76412ed220742aeaf02ca1c50519bcea013e1 -m "initial commit" 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6 </code></pre> If we now look at the .git</code> repository, we will see a new object, 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6</code> created. We can verify the type of this new object. git cat-file -t 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6 commit </code></pre> Unsurprisingly, it's a commit object. If we look what is inside, we see something that may look familiar to people that regularly use git. git cat-file -p 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6 tree aee76412ed220742aeaf02ca1c50519bcea013e1 author Christina Sørensen <christina@cafkafk.com> 1715792150 +0200 committer Christina Sørensen <christina@cafkafk.com> 1715792150 +0200 initial commit </code></pre> A fun thing to do now is to try and run git log</code>. $ git log fatal: your current branch appears to be broken </code></pre> Seems like our branch is broken hu? Let's fix that. Let's quickly remind ourself of the contents of .git/HEAD</code>. $ cat .git/HEAD ref: refs/ </code></pre> This doesn't refer to anything. We can easily fix this, let's make a new branch. But what git subcommand will we use this time? None actually, as it turns out, branches are just files in .git/refs/heads/</code> that contain the sha-1 hash of some commit. The name of the file becomes the name of the branch. $ echo 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6 > .git/refs/heads/main </code></pre> Now we just need to switch to that branch. Instead of doing git switch main</code> we can just change the reference in the repository directory .git</code>. $ echo "ref: refs/heads/main" > .git/HEAD </code></pre> If you're following along, and you have some PS1 git branch feature, you may have noticed something incredible just after running that command. First, if we run git status</code> now, we see that there no longer are any changes to be committed. $ git status On branch main Changes not staged for commit: (use "git add/rm <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) deleted: amogus.txt deleted: truth.txt no changes added to commit (use "git add" and/or "git commit -a") </code></pre> Also, we can now run git log</code>! git log commit 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6 (HEAD -> main) Author: Christina Sørensen <christina@cafkafk.com> Date: Wed May 15 18:55:50 2024 +0200 initial commit </code></pre> Something else that's cool is we can run git log --format=raw</code>, and see output similar to what we got from git cat-file -p</code> on the commit objects sha-1. $ git log --format=raw commit 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6 tree aee76412ed220742aeaf02ca1c50519bcea013e1 author Christina Sørensen <christina@cafkafk.com> 1715792150 +0200 committer Christina Sørensen <christina@cafkafk.com> 1715792150 +0200 initial commit $ git cat-file -p 87a1aa833dccca5ea503e9a7ff81c51fe82c85c6 tree aee76412ed220742aeaf02ca1c50519bcea013e1 author Christina Sørensen <christina@cafkafk.com> 1715792150 +0200 committer Christina Sørensen <christina@cafkafk.com> 1715792150 +0200 initial commit </code></pre> But what about those files that are deleted? We can solve that with git checkout</code> like this. $ git checkout HEAD -- amogus.txt truth.txt </code></pre> Now if we run git status</code> we get. $ git status On branch main nothing to commit, working tree clean </code></pre> With this, we have successfully created a commit from scratch. Conclusions</h1> Starting from an empty directory, we have successfully initialized a repository, staged files, created a branch, and committed our work — all without using the standard git commands. Is this efficient? Certainly not. However, by bypassing the porcelain commands, we have exposed the elegant simplicity of git’s plumbing. Understanding how objects, trees, and references interact transforms the system from a black box into a logical machine. This deeper understanding will help you internalize the tool, making complex operations intuitive rather than terrifying. There are still open questions: how does a pull</code> work? How do we manually rebase</code>? That is a topic for another day — or perhaps a therapy session. For now, we will leave them as an exercise for the curious reader. Exercises</h1> What does the hexdump -C</code> look like of the .git/index</code> if we add another blob with hash-object</code> and update-index</code>?</li> Can you discern any patterns from this. What more can you learn about the index file format.</li> What does the zlib compressed data look like inside of the tree object after if we add another blob?</li> What does the .git/index</code> look like after running update-index</code> with a tree blob in the object database?</li> What does the .git/index</code> look like after running update-index</code> and write-tree</code> with a tree blob in the object database?</li> Can you figure out what the myserious f^gNfB</code> means?</li> What does the hexdump -C</code> look like of the .git/index</code> inside a larger project.</li> Can you discern any patterns from this. What more can you learn about the index file format.</li> Does changing the size of the blob from 78 to some other number inside of the zlib compressed data influence git cat-file -s</code>? You will need to decompress and recompress the data from and into the proper zlib format^{[8]</a>.</li> </ul> Footnotes</h1> We don't have to put our git repo in .git</code>. We could use the GIT_DIR</code> environment variable, or the --git-dir=<path></code> flag. ↩</a> </li> the zlib wrapper (RFC 1950) — unlike gzip wrapper (RFC 1952) — doesn't store file name and other file system information, which is fine, considering how git manages this elsewhere. ↩</a> </li> From this unix.stachexchange answer</a>. Here, we concatenate the gzip magic number and compression method, and concatenate (the actual reason for cat existing) this with the file. We then pipe it into gzip, who can now understand and decompress it. Still, we didn't finish the file with the 8 byte footer, so gzip gets confused, but that doesn't matter, we get to see the data regardless. ↩</a> </li> https://git-scm.com/book/en/v2/Git-Internals-Git-Objects</a> ↩</a> </li> If you're interested in finding out how the hash is generated, start here</a>. ↩</a> </li> Read more herehttps://git-scm.com/book/sv/v2/Git-Internals-Git-Objects ↩</a> </li> The mortal enemy of C has many names: empty bytes, null bytes, nop bytes. ↩</a> </li> One approach could be to use the hack, with the additional padding at the end, to extract the file. Then, after changing the number, compressing it again, and removing the prepend and appended gzip magic numbers. ↩</a> </li> </ol>} Institutional Math Christina Sørensen — Fri, 10 Nov 2023 14:03:00 +0000 Foreword from the future</h3> As 2025 is coming to an end… I’ve been reading over some of my old posts. The tone here is obviously on the more polemic side, but it would be a disservice to my former self to erase the record of my views on academia when I was on the inside. I think it’s worthwhile to conserve not just the more reasoned and polished version of “why did you leave academia” — that I now often find myself regurgitating — but also conserving the “emotion” behind that decision. Because any after the fact rationalization aside, the fundamental decision wasn’t rational, it was emotional. Experienced. Not up for argument. It was an incommensurable experience of a normal science that wasn’t suited to my taste. It was the observation of the fundamental sin in the heart of academia as an institution. If nothing else, then as a warning to my future self to never make the fault of Epimetheus again. Before Newtons birthday seems an apt time to spit on the grave of the enlightenment. So /spit</code>, cheers, and onward! I’ve been wanting to write something terse, dense, with academic heft on this subject a long time. Sadly, I fear I’ll never write such a thing. But I need to speak my heart. I left the mathematical asylum, but I need some closure, and eating Curry isn’t cutting it anymore. Grigori Perelman rejected the fields medal, and the Clay Institute awarded monetary prize for solving one of the “Millenium Problems”. Evariste Galois laid dormant for 10 years after his death in the drawers of Liouville. Gauss and Newton let their personal quibbles turn into abuse of power. Grothendieck left the whole thing behind, burned all his papers. Kaczynski became a domestic terrorist. Lovely field indeed! I’m not interested in money or fame; I don’t want to be on display like an animal in a zoo. – Grigori Perelman</cite> </blockquote> The QED project is but ruins — visions of better times. If the proof isn’t surveyable, then it isn’t solvable! Mathematics isn’t just syntactics, no, they’re new age incarnations on the platonic plane of ideas, they’re universal truths of the universe!… Nonsense. Mathematics is circlejerking on tautological systems. Lockhart can lament all he wants, at least in primary school the problems are trivial, and if you’re done fast you can do something actually useful with the research stipend called public school. At the prestigious academy, the queen of sciences is growing ever more complicated year by year… but ironically, the material taught is the perfect problems for proof systems, we’re teaching students only to replicate. Since the primary concern of these problems is less work for the validator, the true test of your “learning outcome” is your ability to solve solved problems systematically. We must preserve the human validator, by turning said human into a mistaken mockery of the computer. Science, knowledge, the university itself. Open is its doors to any and all. Come research for Boeing and Lockheed, at the faculty of partitioning of 7 hole toruses by means of ballistics, or if it feels too theoretical to indirectly maim and kill people in foreign countries, why not study applied mathematics, and develop social models for companies like Palantir? Profiling, what’s that? Do you not believe in the infallible probabilities? “We need a new Bourbaki” we young people may say to each other, but we all know well that we’d rather get ours and leave this corpse rotting and dying. If anything, what’s needed isn’t more fuel on the fire. Institutional science is systematically pissing in the super soaker wielded by various tyrants, profiteers and oligarchies around the planar graph we all live on. If we are to change mathematics, we should get to the roots of the problem, like we have always done. The foundation on which we built has a crisis, and that foundation isn’t mathematical, it’s social. And for myself, for my mathematical circlejerking? I keep it between friends, private, like good sex. The smut we produce we encrypt and keep safe so as to not show something profane to the plutocrats of the academy. Encrypting — hoping to be kept in those drawers until saner heads emerge to open it. RIME Christina Sørensen — Thu, 09 Nov 2023 16:02:00 +0000 This isn’t a tutorial, it’s more a “here’s what that thing I’ve been working on is about.”[1]</a> </blockquote> I like Nix a lot. It’s a so called “obvious good idea”. Even with much of its baggage, some technical, some cultural, it’s probably one of the most interesting things currently happening in software, and you’re making a mistake if you’re not paying attention to it. My desktop, laptop, build server, and even my IRC-bounce server in the “cloud” runs Nix. This makes it easy for me to provision my systems from a single source of version controlled truth. What has made that even better is that all the software projects I maintain include a so called Nix flake, and I make sure to upstream the projects I could imagine people would find some value in to nixpkgs every week on release. One particular project I maintain is called eza, it’s that fork of exa, that you may or may not even be aware that you’re running if you’re a former exa user[2]</a>. Eza relies heavily on Nix. The first way Nix showed up was to help us remove vagrant, a product from HashiCorp, the company famous for making terraform. This was partially because I was uncomfortable with the vagrant business license, but further because Nix could replace it while providing unique benefits. But we do so much more with Nix. The Nix flake sets up the development environment for contributors, in a way that matches the same environment as I myself develop on effortlessly for devs. It helps ensure that commits follow conventional commits, and checks if the code conforms to a battery of linters. It further makes it easy to format all source, and helps build eza in a declarative way. In general, I like to think that we at eza run a tight ship. This is necessary, because we’re working in a surprisingly hard domain, where there is a lot of side-effects — what else would you expect from something so closely tied to the file system. This means that we enforce semantic versioning as well, and generate changelogs that make breaking changes clear. Now with all that in mind, let me quickly explain the lifecycle of a eza version. We start on Thursday, after a release was just made. For most of the week, we slowly and steadily merge pull requests. Then, when Thursday comes, based on whether or not a breaking change was introduced (as is easily parsed from our commit summaries thanks to conventional commits), we determine the next version of eza automatically, and justfile recipes (think make files but less pain) cross compiles binaries for the platforms we support, and cuts and tags the new release, as well as making a “proper” github release. Then, after all that is done, there’s only two things left. First, I run cargo publish</code>, to send the newest eza version to crates.io, from which it is immediately available for download via cargo install eza</code>[3]</a>. Secondly, I update the nixpkgs package (that I, reasonably enough, am the maintainer of), and await someone with commit bit set to merge it. Now, it is often merged quite quickly these days, I think that there is a small group of commiters who’s gotten used to looking for my batch of software updates and merge them without much fuss. But even with a same day merge, due to the nature of nixpkgs, this newest version will actually first be available in some days, usually four to five. This is because the nixpkgs unstable branch lags behind the main branch[4]</a> by a bit. The reasons for why this is are overly technical, but sufficient to say that even while nixpkgs-unstable is the most up to date package distribution in aggregate, compared to arch-extra, which usually has the newest version of eza within a day, for packages like eza, it lags behind. There of course is a potential solution to this, one that I did use for bit. See, since eza has a flake.nix</code> file, one can just refer to that directly in their NixOS config, by making it a flake input (if one does use flakes that is). This is great, that way, you don’t have to wait for nixpkgs-unstable to catch up with our latest release, but there is a catch. Flake inputs have no notion of versioning. Now, if you’ve been keeping your ear to the ground, there has, withing the last few month been two major improvements on this front. Firstly came flakehub, a proprietary platform by many NixOS veterans. Later came the, seemingly spiteful flakestry.dev, by other NixOS veteran. Between the lines, it’s farily clear that this is a direct reaction to flakehub being closed source. The selling point of both of these platforms is the following: you add a github action, and whenever you make a proper github release, that action will push your flake to the respective platform. The problem with both however, at least to me, as someone that distributes software to a fair few amount of users, but even further, just as someone that likes to tinker with my system and loves to make it all neat is the following: People need to add a github action to their repos.</li> Only repos that have done this, and has released after, can actually be used.</li> This only works for github.</li> If this brittle setup breaks in the future, of if releases stop coming, you’ll have to move to a different source.</li> </ul> And I tried to see if these limitations weren’t too annoying for me to use for some months. But aside from the technical limitations, the closed nature of flakehub got to me (at the time, flakestry.dev didn’t exist). Having had this in the back of my head lead to many shower thoughts, which lead to additional shower thoughts, which lead to generally very long showers. Some weeks ago, those shower thoughts collided into a sudden, immediate hacking run, and rime was born. Now, three weeks later, we’re sorta at a place where all the essential features we want are in place. Those things include supporting not just github, but gitlab, codeberg, forgejo, gitea, sourcehut, and even flakehub[5]</a>, supporting semver requirements, whether that be getting any version that is a version $\le 0.3.48$, or just only getting minor and patch updates, but not major version bumps without manual intervention. And like, a lot more features. But really, what makes it great is that I can just specify: { description = "My Configuration"; inputs = { eza.url = "http://rime.cx/v1/github/cafkafk/eza.tar.gz"; ... your other inputs } outputs = { ... all your cool outputs } } </code></pre> And like that, the eza flake input will always be the latest released (excluding pre-releases) version. This really puts nixpkgs into perspective for me. For projects that already have their own flakes, the value of nixpkgs mainly becomes it being a hand crafted manually updated package whenever there is a new version. And while I’m determined to make eza available to as many users as possible, and having it in nixpkgs will obviously help that… for my own use at least, it’s just so much more neat to have it as a flake input, but without having to include unreleased, potentially unstable changes. It clearly to me also opens some routes to a more decentralized NixOS landscape, since it adds a sort of layer between the latest raw flake and the users computer, reminding me of the layer of snow that build up when you place something sturdy in a hostile, cold climate with a lot of snow(flakes). A layer called rime. Although much of the work has been done by Gergely Nagy</a>, who I’ve been able to successfully nerdsnipe. ↩</a> </li> Since distributions shim in eza in the place of exa nowadays. ↩</a> </li> Although, sadly, it’s impossible to include man pages and completions via crates.io. ↩</a> </li> Technically, the “master” branch, but let’s not indulge that anachronism. ↩</a> </li> And it’s written to be easily extendible, so adding new forges is trivial. ↩</a> </li> </ol> Adding dollar sign (apply) to nix Christina Sørensen — Thu, 03 Aug 2023 14:34:33 +0000 This post is very low quality for comedic effect. </blockquote> A detailed list of instructions</h2> Take one nix</a>, apply this patch</a>[1]</a>: What does this do? It adds a new token $</code> to the nix languages parser and exposes it as __dollar</code>, essentially by copying the way this works for __lessThan</code>. Hack on your nix: nix develop ./bootstrap.sh ./configure $configureFlags --prefix=$(pwd)/outputs/out make -j $NIX_BUILD_CORES outputs/out/bin/nix repl </code></pre> Now in the nix repl</code> that just opened: nix-repl> __dollar = x: y: builtins.foldl' (a: b: a b) x y </code></pre> This makes $</code> an infix operator that’s essentially just a generic apply. Now just create a toy example to demonstrate: nix-repl> concat = x: y: z: (x + y + z) nix-repl> concat $ ["literally" " " "dollar sign"] </code></pre> And your repl will say: "literally dollar sign" </code></pre> Tadaa! That’s all it takes to add the $</code> operator to nix, just like Haskell. The end $\text{\(^ヮ^)/}$ diff --git a/src/libexpr/parser.y b/src/libexpr/parser.y index 201370b90..4b26af623 100644 --- a/src/libexpr/parser.y +++ b/src/libexpr/parser.y @@ -353,6 +353,7 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, ParseData * data, const char * err %left AND %nonassoc EQ NEQ %nonassoc '<' '>' LEQ GEQ +%nonassoc '$' DOL %right UPDATE %left NOT %left '+' '-' @@ -411,6 +412,8 @@ expr_op | expr_op LEQ expr_op { $$ = new ExprOpNot(new ExprCall(makeCurPos(@2, data), new ExprVar(data->symbols.create("__lessThan")), {$3, $1})); } | expr_op '>' expr_op { $$ = new ExprCall(makeCurPos(@2, data), new ExprVar(data->symbols.create("__lessThan")), {$3, $1}); } | expr_op GEQ expr_op { $$ = new ExprOpNot(new ExprCall(makeCurPos(@2, data), new ExprVar(data->symbols.create("__lessThan")), {$1, $3})); } + | expr_op '$' expr_op { $$ = new ExprCall(makeCurPos(@2, data), new ExprVar(data->symbols.create("__dollar")), {$1, $3}); } + | expr_op DOL expr_op { $$ = new ExprOpNot(new ExprCall(makeCurPos(@2, data), new ExprVar(data->symbols.create("__dollar")), {$3, $1})); } | expr_op AND expr_op { $$ = new ExprOpAnd(makeCurPos(@2, data), $1, $3); } | expr_op OR expr_op { $$ = new ExprOpOr(makeCurPos(@2, data), $1, $3); } | expr_op IMPL expr_op { $$ = new ExprOpImpl(makeCurPos(@2, data), $1, $3); } diff --git a/src/libexpr/primops.cc b/src/libexpr/primops.cc index ddf529b9e..00622a142 100644 --- a/src/libexpr/primops.cc +++ b/src/libexpr/primops.cc @@ -3583,6 +3583,24 @@ static RegisterPrimOp primop_lessThan({ .fun = prim_lessThan, }); +static void prim_dollar(EvalState & state, const PosIdx pos, Value * * args, Value & v) +{ + state.forceValue(*args[0], pos); + state.forceValue(*args[1], pos); + // pos is exact here, no need for a message. + CompareValues comp(state, noPos, ""); + v.mkBool(comp(args[0], args[1])); +} + +static RegisterPrimOp primop_dollar({ + .name = "__dollar", + .args = {"e1", "e2"}, + .doc = R"( + Just like haskell :) + )", + .fun = prim_dollar, +}); + /************************************************************* * String manipulation </code></pre> ↩</a></li> </ol> The Pull-Request Heist Christina Sørensen — Thu, 03 Aug 2023 14:34:33 +0000 Your effort to remain what you are is what limits you. – The Puppet Master</cite> </blockquote> The Beginning</h2> May 3, 2014. Benjamin “Ogham” Sago has just made his first commit to Exa</a>. This will go on to be one of the most used ls alternatives, with ~22k stars, 655 forks, and 151 contributors. For almost a decade, he maintains exa. Yet, around 2021, activity is drastically slowing. Eventually, the project falls dead. With 63 open pull requests, 276 issues, and some major bugs, the project is on its deathbed. The users are begging</a> for activity. At this point, other projects are being advertised in the issues for various features. Features that — mind you — are already finished in the pull-requests, but without anyone to merge. The situation is dire, are we all doomed to use LSDeluxe</a> and endure its awful color scheme?[1]</a> That’s where I somehow came in. The rabbit hole yak-shave</h2> The terminal emulator kitty is really neat. So is exa, and I’ve been using exa for a fairly long time. Never have I given it much more thought than installing it from nixpkgs, I’m for the most part blisfully unaware of the projects state (which speaks to it’s quality no doubt). Kitty has a feature. You can type ctrl+shift+e</code>, and it highlights links with letters, a bit like vim easy-motions, and allows you to effortlessly open them in your browser. As a keyboard only extremist, this pleases me. Of course, the thing is, when I exa some directory, I can’t “easy motion” for files. And I want that. So I look into it. Turns out, kitty has that feature! And so does ls! ls --hyperlink=auto </code></pre> Neat! Then, tragedy… exa doesn’t. What! A bit of searching reaveals to me a PR on exa. It’s been there since April. It’s awsome! But it isn’t merged. Why? I orient myself in the issues. I try to figure out who to complain to. But there is none, the project looks dead. I dip my toes in some forks, see if I can get this done. Seems trivial. Then, my partners has to help sister move. I have the weekend free. The nerd snipe tinglies are coming on. A hacking run</a> is imminent — whether I like it or not. Onto the programming</h2> Fork other forks, we’ll write our own! Call it eza, old habits die hard, and the x is next to the z on my keyboard. This will suffice. In one bold move we have already solved half of the hard problems of computer science[2]</a>, I’d say that’s a good start. Now we just have to get those pull request over in our repo. One way to do this is the following: git fetch \ git@github.com:ogham/exa.git/pull/<pr-id>/head:<target-branch> </code></pre> This way, we can take a pull request branch from exa, and copy it into a branch in our repo. The problem is, there’s 63 pull-requests!?! We’d have to do this for each of them, push the branch to github, then slowly click through the webui and make that pull request, one at a time. As a keyboard only extremist, this displeases me. There is no way we’re doing this. If only we had a list of all those repo-ids, we’d be able to at least automate the fetching. Then, lightbulb. Github has a REST-API</a> ripe for our queries[3]</a>. A bearer token is created. An absolute unit of a curl request is crafted. The world pauses for a moment as we lay the fate of the project into the wire. curl -L \ -H "Accept: application/vnd.github+json" \ -H "Authorization: Bearer <the-bearer-token>" \ -H "X-GitHub-Api-Version: 2022-11-28" \ 'https://api.github.com/repos/ogham/exa/pulls?per_page=100' > open-prs-full.json </code></pre> Then we wait for the response. After the Suspense</h2> Well. It worked. cat open-prs-full.json | head -n 10 </code></pre> [ { "url": "https://api.github.com/repos/ogham/exa/pulls/1221", "id": 1446083658, "node_id": "PR_kwDOATIZtM5WMXxK", "html_url": "https://github.com/ogham/exa/pull/1221", "diff_url": "https://github.com/ogham/exa/pull/1221.diff", "patch_url": "https://github.com/ogham/exa/pull/1221.patch", "issue_url": "https://api.github.com/repos/ogham/exa/issues/1221", "number": 1221, ... </code></pre> Was this what we needed? We do see a number</code> property there, and if we look in the html_url</code> property, we see that it matches the expected url for that pull request. Now here is the part where I’d shown off my jq prowess, but I have no such thing. So instead, I make do with some regex. cat open-prs-full.json \ | rg -i 'pulls/\d+"' -o \ | tr -d "pulls/" \ | uniq \ | tee pr-ids.txt \ | wc -l </code></pre> 63 </code></pre> The number of lines is right, but is the output? cat pr-ids | head -n 5 </code></pre> 1221" 1219" 1210" 1204" 1202" </code></pre> Ahh, close enough, a little adjustment and we are there: cat open-prs-full.json \ | rg -i 'pulls/\d+"' -o \ | tr -d "pulls/" \ | tr -d '"' \ > pr-ids.txt </code></pre> Now did we get it? cat pr-ids.txt | head -n 5 </code></pre> 1221 1221 1219 1219 1210 </code></pre> Seems we did! The Part Where We Jam Those Pull-Requests into GitHub</h2> At this point, the estrofem tablet under my tongue had dissolved, and I’d woken from my groggy morning state. A purely performative crack of the knuckles were in order, as I leaned over my desk like a hawk that’s caught sight of a mouse. If you’re a keyboard-only extremist like me, web-ui is a nuisance to be solved. GitHub has gh</code>, or the GitHub CLI. It is extremely useful to sync your nixpkgs fork, and send your package definition pull requests. But today, old reliable was gonna learn some new tricks. Then I paused. And stared. In my head flashed visions of old Bell Labs informational videos</a> on the Unix Operating System. The voice of Brain W. Kerninghan echoed in my head — “I think the notion of pipelining is the fundamental contribution of the system…”. The hacking cgi was front and center as I said out loud “It’s a UNIX system, I know this!”. Temporarily absolved of my mortal coil, I resided in waves of abstraction and shell, time and language lost their meaning to the beauty of the baud. It was as if every bit moved in concert with my thoughts, as if the collective will of every electron in my sillicon rock filled out the sails of my spaceship gently surfing on the stream of raw information entering and exiting me. As I slowly raised my cup of tea to my mouth, not for one moment breaking this intimate emergence and embodiment of me in my code, in my will, in my craft, before me it appeared, at the end of time and space itself. The final destination, the one truth of programming, the zen was ever closer. Then the tea — hotter than a NVidia GPU running KDE — removed my taste buds faster than a Arch maintainer removes system critical boot code. What had been my ticket to enlightenment turned into a leaky abstraction all over my desk. However, I had tried so hard and gone so far, there was no way that in the end, it wouldn’t really matter. Now — armed with a more modest disposition, xargs, and the GitHub CLI — I went to work. Soon, emerged the command that would seal my fate: cat pr-ids.txt \ | xargs -I% -n1 sh -c \ 'git fetch git@github.com:ogham/exa.git pull/%/head:pr-%; git checkout pr-%; git push; gh pr create --title \'(exa PR) %\' --body \'github.com/ogham/exa/pulls/%\'' </code></pre> As a smut novel once said[4]</a>: There comes a time in every woman’s life where she reaches a crossroads. A chance to choose between two paths. One is unknown, filled with potential and struggles that demand growth and change, but could lead to new beginnings. The other is familiar, grooved with well-worn ruts and established twists, muddy holes and isolation. The path that, while lonely, fits like a glove. When — if — I hit enter, I’d be down the road of committing myself to maintaining the damn thing. Were I not to, I’d never downlive the internal shame. Were I to do it, I’d have allocated quite a substantial amount of time in my life to pesky users, stubborn contributors, and the chore and dull drag of changelogs, version bumps, and releases. As I hit enter, and as the pr tab on the repo slowly grew in the web interface, I hoped this wouldn’t be a mistake. The Aftermath</h2> Over the next two days, me and sbatial</a> would go onto hack on eza. When monday came, 63 had been reduced to 18. Hyperlinks, SE-linux contexts, git repo statuses, human readable relative dates, more security fixes than I have fingers to count them on. And the darn grid bug, the pesky bugger had been swatted and was now but a stain on the git history. With flakes, CI, and sheer will, we had overcome. Trying out eza was as imple as running: cargo install eza </code></pre> Or if one was so inclined: nix run github:cafkafk/eza </code></pre> But had we won? The battle, no doubt, but not the war. Reflections on DevOps and Infra</h2> In the excellent talk at Chaos Communication Conference 36: From Managerial Feudalism to the Revolt of the Caring Classes</a>, the late David Graeber summarizes a core problem in modernity as eloquently as only he could: [Whenever] I say, talk to a Marxist theorist, wherever, they try to explain value, which is what they always like to do, they always take the example of a [bottle]. […] Well, look at this bottle, you know. [It] takes a certain amount of socially necessary labor time to produce this, say it takes, you know, this much time, this much resources. It’s always about production of stuff. But a teacup or a bottle. […] You produce a cup once, you wash it like 10000 times. Most work isn’t actually about producing new things, it’s about maintaining things. […] So I think that what we need to do is we need to start over. We need to realign. First of all, think about [work], not as producers, but as carers. </blockquote> I spend four years of my life studying “the queen of the sciences”, mathematics herself. One view that I particularly enjoyed were that of the toolmaker. Sure, we were studying abstract structures, but that’s being a student. A mathematician was a tool maker. That view might have a stink of applied mathmatics, but I think it’s true even for those who’d call themselves pure[5]</a>. This too is true in DevOps, in systems programming, in infrastructure. At the end of the day, we’re toolmakers. In my mispend youth, in the grips of startup culture, the idea of the elusive 10x developer, while just a mirage was still very alluring. Regardless of the obviously wrong and problematic discurse related to it, there is no doubt that some people move mountains, and others don’t. I wanted to move mountains. What I’ve later came to learn is that’s just an illusion. The idea of the lone genious, burning the midnight oil is just a myth. John Carmack, Linus Torvalds, George Hotz, they’re just a personality cult. And this isn’t just some clever insight I’ve come to, it’s stolen</a>. But I like my iteration on it. A certain Archmage of Infrastructure has the right idea about being a “force multiplier”. What really matters to move these mountains is to enable those that want to move them. The most dedicated cyclist in the world will cycle less than an average bikesmith will enable others to do. In this creating of tools, and in this care work of maintaining them, we are force multipliers, and in our multiplication of force, the impossible becomes possible. This is the point of DevOps. This is the point of infra. This is the next level of abstractions hackers should strive for if they wanna boil the ocean. Post-Credit Scene</h2> From 5:00-19:00, for two days, I had hacked. 3 meals a day, hydrating, and a walk in nature had supported this endeavor[6]</a>. At last, a knock on the door, my partner had come home. After a hug and some pleasantrise, came the question: So, what did you do with your weekend? </blockquote> I eagerly typed eza</code> in my terminal emulator, and showed how there were now letters next to the entries, and clicking them copied the text. What should have been pure awe was a look of slight concern, followed by a shoulder shrug and a kiss on the forehead. That’s all you did while I was gone? </blockquote> It’s all love, I bet it’s just a bad screenshot on their git repo <3 ↩</a> </li> See the first quote here</a>. ↩</a> </li> And we’d never scrape a website, that’s rude :^) ↩</a> </li> “Arabella Stone, Nashville’s darling, is eager to shuck her prim-and-proper rep, and a few wild months spent checking items off her ’ summer bucket list’ is the way to do it. First up: kiss the man she’s crushed on since she wore a training bra, the bad boy of country himself, Blue bassist Charlie Tucker. For Charlie, a beautiful woman flirting with him isn’t out of the norm—but a beautiful woman bolting after the hottest kiss of his life sure is. And when he finds out his kiss-and-run Cinderella is none other than Arabella Stone, daughter of his label’s CEO, he knows he’s in trouble. Because not only is she a Stone, she’s also his employee for the next few months at the recording studio he just bought. Over the course of one thrilling summer, Arabella and Charlie chip away at her bucket list and fight the simmering attraction between them…knowing that once it’s all over, so is their time together.” Steamy. ↩</a> </li> Which really should be called theoretical. ↩</a> </li> All this rise and grind never sleep never eat never enjoy your life stuff is stupid. If you wanna be even like mildly consistent, you gotta get this right. Dumb stuff like hacking is like the last thing you get to in mazlows hierachy[7]</a> for a reason. ↩</a> </li> Yes, I know it’s “debunked”, it’s a working model geez louise. ↩</a> </li> </ol> Warts and all Christina Sørensen — Mon, 24 Jul 2023 16:10:10 +0000 Learning</h2> The better you get the worse you get. The more I learn the more I learn that I don’t know much. This can be overwhelming at times. I think it’s pretty normal to feel this way. But I also see it can easily become a limiting type of negative feedback. Looking back at old projects, old essays, old anything, I see flaws. I was so naive. Simultaneously, I feel like I see how my output has gotten better. But realizing this negative feedback loop — as in keeping with the theme — just makes you even more insecure! Because, if this model is true (and it generally seems to be), then this post I’m writing right now will seem so bad to me in a year… not to imagine 10 years! Knowing this, it’s easy to fall into perfectionism. There’s always something to improve after all. But this is dangerous. Because, while it might be the case that in 10 years, I can easily see all the flaws, right now I can’t. And it seems to be my experience that no matter how much I correct, no matter how much time I spend trying to make something perfect, I only realize what was actually missing much later. Worst, things don’t even make it out the door. Perfectionism can kill projects. When that happens, I don’t realize the flaws. The projects I never finished because I didn’t know how to explain it the right way or didn’t know just how to say what I wanted to say perfectly, those I still don’t know how to improve. Granted, they might just still be out of my reach, and thus simply never get finished because of this, but I feel everything I do is out of my reach, even when doing it. And then it’s done, and I hate it a little. And then a year goes by, and I hate it a lot. I have to finish things to improve. Incomplete things are in a constant state of improvement. They’re too maleable to be at fault[1]</a>. They’re beyond critique because they’re meerly in progress, they’re not solid enough to grasp and criticise. Ideally, I shouldn’t get discouraged by feeling stupid. In fact, I’m starting to feel stupid, and I’m kinda glad about that, because I know how much I had to learn to be this dumb. But I dislike everyone’s complex with wanting to be the wisest of all the G(r)eeks by pretending that they don’t know anything. Knowing that you know nothing is a paradox, and a platitude. He who knows, does not speak. He who speaks, does not know. – Lao Tsu</cite> </blockquote> This is a dumb quote. It should really be: Those who speaks, does not know. Those who knows, does not speak, and plateaus. </blockquote> You can’t know things, you can only know them more. So knowing, isn’t a destination, but a process. But it’s a painful one. Self Redaction</h2> I feel like contemporary culture expects a self redaction from the individual of their history. You’re not supposed to fail in public. In a spectacle, there are just dunkers and those that are dunked on. Being wrong is death of your socially distributed ego. What this false internet reality leaves out is the numerous failures that lie at the end of everything even remotely sort of cool. Without this ever in the public eye, the culture perpetuates and intensifies perfectionism in us all. I think that to some extend, this isn’t just a moral issue, but also an individual one. The idea of being unable to fail in public means you can’t grow in public either. This I find limiting. Further, for those around us, the network effects of being imperfect seem to be worthwhile. In fact, I’d argue that in a time of flawless, but false public personas, being not just a finished product but a evolving history is a much more potent and intersting identity to embody. And it’s one that anyone with any sense should find preferable to the superficially flawles. I feel like I’ve already made numerous public mistakes, and I don’t see that slowing down, and maybe I shouldn’t want that, each one is a sign of me going in the right direction. I don’t know how to end this. Reference implementations and their consequences have been a disaster for the human race. ↩</a> </li> </ol>

Obligatory Conclusion</h2>
With inexpensive consumer hardware, it is possible to build your own sattelite ground station, and join a globe spanning network of space enthusiasts capturing sattelite data.</p>
That is pretty cool :3</p>

point.free

Quantizing Gemma 4's MTP drafters

Open weights are not open source

Another NixOS Election Cycle

Tracking Sattelites with Consumer Components

Conventional Commits Change my Brain

rustls and aws-sdk seems incompatible with FOSS

Merge or Rebase

Grok Git Repos

Institutional Math

RIME

Adding dollar sign (apply) to nix

The Pull-Request Heist

Warts and all

point.free

Quantizing Gemma 4's MTP drafters

Open weights are not open source

Another NixOS Election Cycle

Tracking Sattelites with Consumer Components

Hardware</h3> If you wanna build your own, the easiest possible route is just to acquire a LoRa32 V2.1_1.6 - 433MHZ CH9102F</code> board. Usually you can find these online, and make sure that you get one with a small antenna.</p> </p>

Obligatory Conclusion</h2> With inexpensive consumer hardware, it is possible to build your own sattelite ground station, and join a globe spanning network of space enthusiasts capturing sattelite data.</p> That is pretty cool :3</p>

Conventional Commits Change my Brain

rustls and aws-sdk seems incompatible with FOSS

Merge or Rebase

Summary</h2> So what where the tradeoffs?</p>

Grok Git Repos

An Experiment with Tree Objects</h2> An important concept to explore is how a tree object handles multiple files. To demonstrate this, we will create another blob.</p>

Footnotes</h1>

Institutional Math

RIME

Adding dollar sign (apply) to nix

The Pull-Request Heist

After the Suspense</h2> Well. It worked.</p>

Warts and all

Obligatory Conclusion</h2>
With inexpensive consumer hardware, it is possible to build your own sattelite ground station, and join a globe spanning network of space enthusiasts capturing sattelite data.</p>
That is pretty cool :3</p>