roguelazer's website

Yesterday around 10am, I got my second shot of the ModeRNA COVID-19 vaccine (unsurprisingly, I got my first show four weeks ago); at this point I'm in the countdown to be "fully vaccinated"¹. So far, the side effects have been pretty mild:

• I was maybe a little tired yesterday afternoon
• Yesterday evening I had an elevated temperature of 99.7°F, but no clinical fever
• This morning my arm feels like I lost a really intense game of Punch Buggy

I know the pandemic is nowhere near over and people are dying by the thousands around the world, but I'm certainly happy that my chances of serious illness are now very close to zilch.

Speaking of the pandemic being nowhere near over, let me just say how bizarre I find last week's CDC guidance on face masks and social distancing... 48% of the US is currently vaccinated, with some states as low as 33%. Even here in Alameda County, only 60% of eligible adults are vaccinated so far. I've been spending the last year watching an endless sequence of men² who certainly were not vaccinated refuse to take any precautions; now the CDC is saying that I'm all of a sudden supposed to trust that all the newly-unmasked people around me aren't in the 40% who haven't gotten vaccinated? Yeah, right! I know that I, when I am fully-vaccinated, will not be at much risk, but I'd rather not risk that my infant son join the increasing number of pediatric COVID-19 cases. I'm glad that California is being a little more conservative and waiting until June 15th. It's a damn shame that nobody was able to figure out a reliable "vaccine passport" system yet³; Mozilla has a good article on how this could be done that, unfortunately, I don't think any US state will ever bother to implement.

As usual, stay safe out there, readers.

Just Finished

For All Mankind Season Two. This season had a lot of slow moments, and more of Karen than I really wanted but holy shit did those final two episodes deliver. If you've bought an Apple product in the last couple of years, you get this for free, so you might as well watch it!

Half-Way Through

Babylon 5 Season Three. I'd never seen any B5 and $SPOUSE and I decided to watch it over lockdown. It's pretty fun seeing how much DS9 stole from this show. My main complaint is that I don't really care about any of the human characters and really just want to watch the "Londo and G'Kar screw up the galaxy" show.

Just Started

Mythic Quest Season One. We're always on the hunt for a half-hour comedy to watch during nap-time, and this one's pretty good. Unlike Silicon Valley, this doesn't remind me of my day job too much. Came into it expecting Danny Pudi to be the main draw but Charlotte Nicdao really steals the show. Season Two comes out in a few weeks!

As of yesterday morning, I've gotten my first shot of the ModeRNA¹ COVID-19 vaccine!

I'd been trying to get an appointment since Berkeley opened them up to all adults on April 9th, and had no luck until 3am on Friday morning, when I managed to get one of the spots released by the CVS near my house (they were all gone a few minutes later). I was surprised to find when I arrived at CVS that, despite the intense competition to get vaccinated, supply is so tight that they were only being issued 10 doses (one vial) per day.

It's so frustrating that vaccines are still so hard to get here, and yet doses are sitting unused around the country. I don't know what the solution is to convince rural Republicans to get vaccinated, but I hope someone comes up with something to prevent the ultra-conservative parts of America from serving as a breeding ground for weird SARS-CoV-2 variants.

Anyhow, while I'm writing a post on COVID-19, here are some fun links:

• Reverse Engineering the source code of the BioNTech/Pfizer SARS-CoV-2 Vaccine: I assume everyone's read this one already but it's a great explanation of something I know very little about
• Exploring the Supply Chain of the Pfizer/BioNTech and Moderna COVID-19 vaccines: Another very nitty-gritty cool article
• Deep Cleaning Isn’t a Victimless Crime: I feel like all of my news sources have been making it clear that this virus spreads through the air and not through surfaces for the last, like, 10 months, but I guess a lot of people have gotten hung up on crazy and useless "deep cleaning". Hopefully we can get past that and stop making the entire world smell like bleach. I guess this Lancet article has been very influential?

At this point I'm kind of vacillating between optimism that we're all going to be vaccinated and safe soon, and deep pessimism that the right wing media echo chamber has created an insurmountable barrier of misinformed people with a near-pathological antipathy towards science and public health, even if we do make it through this pandemic, the next one is going to be some real Dark Ages shit.

Fun stuff.

Stay safe out there.

Here's a fun game for you: what do you expect to be the state of the filesystem after running the following commands in an empty directory on a Linux system?

$ touch foo:bar
$ tar -cpf foo:bar.tar foo:bar
$ rm foo:bar
$ tar -xpf foo:bar.tar

Do you expect the directory to contain the files foo:bar and foo:bar.tar?

What if I told you that instead the directory would only contain foo:bar.tar and stderr would say

tar (child): Cannot connect to foo: resolve failed

Yep! It turns out that GNU tar, if passed a filename containing a colon, treats the part before the colon as a hostname and attempts to connect to that host over rsh to download the part of the file after the colon. If you're not familiar with it, rsh is the completely-insecure predecessor to SSH. It's been at least 20 years since any reasonable system has shipped with RSH. This behavior is documented in Chapter 9.1 of the GNU tar manual but nobody I polled had ever heard of it.

Anyhow, GNU tar has a --force-local option to disable this behavior. If you ever process tar files whose names you do not completely control, or which might for some reason contain a colon, you should pass --force-local before every invocation.

An enterprising security researcher could probably have a lot of fun experimenting with vendor crap like security cameras and routers that take tar files with user-controlled filenames for firmware upgrades and see how many of them can be persuaded to establish a rsh shell to some attacker-controlled device...

I went into my office yesterday for the first time in a few months to pick some stuff up. We got notified a couple of days ago to get any personal property out of the office before Thanksgiving or else it'd be thrown out, so I guess we're moving out of the office. It was a pretty eerie place to be; even now, 8 months later, most people haven't been back and it kind of looks like the entire office was abducted by aliens in early March.

Despite how weird it is, I still miss working out of the office.

It's 9:45am on a cloudless Wednesday morning here in summertime California.

My light meter reads 14 lux pointed directly at the sky — about as much as the middle of the night (with local light pollution), or about 1/1000 as much as a normal morning. They tell me that the skies are this apocalyptic shade of red due to the smoke from the 16 major wildfires currently burning in California¹. Local news has plenty to say on the cause if you want to read up. Damn if it isn't disconcerting to live through, though.

What world did I bring my son into? Remember, this is probably going to be one of the cooler years over the next century... Vote, friends. Vote like your lives depend on it.

Hello hypothetical readers; sorry for my absence, but I've spent the last weeks pretty well busy — as I foreshadowed in May, I have a son now!

It's been quite an adjustment. Isaac was born on August 7, 2020 after a very long¹ labor, but he's happy and healthy now. I got 3 weeks off of work, which I spent with my wife doing intensive child care. Let me tell you, I have a whole new respect for single parents. Caring for an infant with just the two of us (since none of our families or friends can visit or help out due to COVID-19) is hard work. He needs to be fed every 2–3 hours, changed every 0.1 – 3 hours, and while he does sleep a lot, it's not really the kind of deep sleep where you can just put him in a crib and do other work. As of a couple of days ago, I'm back at work², but my wife still has six or so more weeks of parental leave. We're still trying to do our best to spread the load, which means I'm spending most nights up all night³. Again, much props to all the single parents out there. Also: much props to grocery delivery. Things would be a lot harder if I had to gird myself against the pandemic and spend an hour getting groceries every time we ran out of something.

I guess you should prepare yourself for lots more baby pictures in the future.

Sidebar: The American Healthcare System is Truly Awful

My son's delivery was at the nearby hospital, which is owned by Sutter Health Group. Sutter Health also owns every other hospital in the area except the Kaiser in Oakland, which is only accessible to people on Kaiser insurance. I'm noting this first because, like most people in America, we didn't have any realistic choice about what hospital to go to, unless we were willing to drive to a different county (potentially an hour or more drive with traffic to get to UCSF).

Keeping in mind that Sutter Health has a monopoly on delivery centers in the area, you can imagine my surprise when we received the first bill for the delivery, in the amount of $103,736. No, that's not a typo. Nope, it's not missing a decimal point. One hundred and three thousand dollars. More than the average Californian makes in a year before tax. Yes, I have pretty good insurance and they're covering most of that⁴, but Jesus Fucking Christ, we were in the hospital for less than a week, in a labor & delivery ward (no ICU, etc) and had no unusual procedures performed. We weren't even in a very good hospital; there was a concerning level of dirt and grime, the cafeteria wasn't open on weekends, and a scary amount of the medical equipment was broken and had to be replaced during our stay. The doctors and nurses (especially the nurses) were good, but we spent maybe a total of four hours over the entire stay interacting with a doctor, and until the very end over the delivery⁵ we only saw a nurse once every two or three hours⁶. This isn't exactly concierge care!

I was always in favor of socialized medicine, but after seeing how (a) awful the care is, and (b) how incredibly, comically, expensive it is for a hospital stay that basically every human being goes through, I just want to take a moment to offer an emphatic middle finger to every Roger-Ailes-brainwashed voter and politician who's kept us in this nightmare system of private medicine while the rest of the world moved on to treating medical care as a right instead of a privilege.

PSA for folks in the Bay Area: The University of California Botanical Garden has re-opened for limited, outdoor-only visits. Seems like a nice (and reasonably) safe place to go during the pandemic. It's July, so not much is in bloom, but it's still beautiful.

So, my etcd post was cross-posted to Hacker News and Reddit and probably more places. For the good of my own sanity, I'm not going to try to read through the comment threads on those other sites; generally, I try to avoid Hacker News anyway¹. Just some brief, off-the-cuff follow-ups:

• Yes, I recognize that the post from the other day was, uh, inflammatory. I did not write it as a persuasive thinkpiece targeted at the critical 18-25 demo or whatever. It's just some notes while I was in the process of rewriting some software from the etcd v2 API to the v3 API².
• This is my damned website and I'm going to be as snarky as I want, Anonymous Coward from 13 hours ago.
• All nine million of you who wrote to me are absolutely right: as a user of open-source software I have the right to fork it if they decide to pull out the v2 API. I'm not super-interested in becoming the de facto maintainer of a database, particularly not one in a language that don't use very often, but I do have that right and ability. Thank you for writing to me.
• Non-sarcastic thanks to everyone who pointed out typos and issues in the original post. Fun fact: Layer 3 and Layer 4 in the OSI model are not the same thing.
• If you agree with me that simplicity is a virtue in software architecture and we embrace too many things that look like 90's-era Microsoft APIs, how about really driving it home by giving some money or time to Black Lives Matter, the SPLC, the ACLU, or other organizations working to simplify and improve our civil lives³?
• If you vehemently disagree with me and think I'm human scum for not embracing the glorious combination of systemd, kubernetes, and the Registry Hive, why don't you really pwn me by donating money or time to an organization whose website probably runs on some k8s cluster somewhere like Black Lives Matter, the SPLC, or the ACLU?

Many thanks to DreamHost for successfully seeing this static-HTML website through a big traffic spike. I've been a customer for like 15 years and so far so good.

Once upon a time in 2013, there was a tool called etcd which was a really lightweight database written around the Raft consensus algorithm. This tool was originally written in 2013 for a bullshit unsuccessful project called CoreOS Container Linux that was EOL'd several years ago, but that doesn't really matter --- etcd was greater than its original use-case. Etcd provided a convenient and simple set of primitives (set a key, get a key, set-only-if-unchanged, watch-for-changes) with a drop-dead simple HTTP API on top of them. I have built a number of tools using etcd as a lightweight consensus store behind them and it's absolutely a pleasure to work with.

In 2015, an unrelated tool called Kubernetes was released by Google (but, really, by Xooglers). I would go so far as to say that Kubernetes (or, as the "cool kids" say, k8s) is the worst thing to happen to system administration since systemd. It's a comprehensive suite that promises to simplify operating clusters of software and give something like the experience of Google's borg cluster manager. What it really does is:

1. Add hundreds of new failure modes to your software
2. Move you from writing portable software configuration to writing thousands of lines of k8s-specific YAML
3. Ensnare you in a mesh of questionably-good¹ patterns like containerization and software defined networking

If you are running a truly enormous system and want to have off-the-shelf orchestration for it, Kubernetes may be the tool for you. For 99.9% of people out there, it's just an extra layer of complexity that adds almost nothing of value.

I digress, though; this is a story about etcd. And, unfortunately, our stories come together because Kubernetes was quickly changed to use etcd as its state store. Thus began the rapid decline of etcd.

With the massive influx of Kubernetes users came, of course, a large number of Xooglers who decided to infect etcd with Google technologies, as is their way[^infection]². Etcd's simple HTTP API was replaced by a "gRPC"³ version; the simple internal data model was replaced by a dense and non-orthogonal data model with different types for leases, locks, transactions, and plain-old-keys. etcd 3.2 added back a tiny subset of the HTTP API through the "gRPC Gateway", but not enough to implement any of the rich applications built on top of the original API. The v2 API lives on for now, but upstream threatens to remove it in every new version and there will surely come a time when it'll be removed entirely.

That's it. That's the story. Popular modern technology is taken over by expats from a megacorp and made worse in the service of a hyper-specialized (and just plain over-hyped) orchestration platform. That's the world today. Anything that has a simple and elegant feature-set ends up coöpted by people who just want to build big ungainly architecture and ends up inheriting features from whatever megacorp the coöpters came from⁴. The software development world would prefer to use their multi-gigabyte IDEs running on ElectronJS to build thousand-dependency Java applications targeting ungainly APIs on hard-to-operate systems than support something simpler and better. Quality is, alas, a dying art.