Posts in Category "programming"

Surprising Change in Python 3.7.6

Here's a surprising change for you: Python 3.7.6 (ostensibly, a patch bugfix release) totally changed how URLs are parsed by Python programs.

As of Python 3.7.5, a URL like foo:8888 would be parsed into the following:

>>> urllib.parse.urlparse('foo:8888')
ParseResult(scheme='', netloc='', path='foo:8888', params='', query='', fragment='')

As of Python 3.7.6, foo is now detected as the scheme:

>>> urllib.parse.urlparse('foo:8888')
ParseResult(scheme='foo', netloc='', path='8888', params='', query='', fragment='')

This will cause massive chaos if you are ever parsing URLs with ports in them but without schemes …

read more

Vim Setup: 2019

Neovim Icon Vim icon

It's been a little while since I posted about my editor configuration1, and I thought I might post what I'm using now. I guess the most notable change is that (after much prodding from my coworker Drew Ditthardt) I've switched from Vim to Neovim. Neovim is a vim-compatible editor written in C and Lua (as opposed to Vim, which is written in C, Vimscript, and prayers). I upgraded to Vim 8 last year and have had a few too many segmentation faults in the editor, so I decided to switch to something where more functionality was implemented in a memory-safe language. So far, Neovim has been pretty good to me, although the new process model means that it's pretty hard to write functions which invoke an external process which takes interactive input from a user.

As is probably expected for this sort of thing, here's a couple of screenshots; the first is of VimR, and the second is from NeoVim in, both editing files from rust-mysql-binlog:

VimR editing a file Neovim editing a file

read more

Serialization Format Performance

Most of the work done in actual programming jobs is taking structured data in some particular format from one system, slightly tweaking it, and sending it off to some other system. When exchanging data between different processes, it's almost always necessary to serialize it into a series of bytes which can be sent across a dumb byte-oriented transport (such as TCP). There are hundreds upon hundreds of different serialization formats out there, but I just wanted to talk about a few of the most common that folks use with the Python programming language.

read more

Beating the Compiler

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

— Donald Knuth

Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest.

— Rob Pike

We spend a lot of our time in the modern, web services-driven technology industry ignoring performance issues. What's the point of micro-optimizing a 3ms function call when each request spends 8 or 9 seconds inside the SQLAlchemy ORM? Well, sometimes it's nice to practice those optimizion skills anyway, and today I'm going to walk you through micro-optimizing a simple problem and we'll see just how much better we can do than a naive solution… even though we probably normally shouldn't.

read more

dnsextd, TCP, and IPv6

Hello interested parties. dnsextd (in my git repository) now supports TCP. It was actually sort of an amusing bug. I guess the dnsextd code must date back to PowerPC, because it had an extra ntoh call which on little-endian systems would cause TCP requests to fail. It's fixed in the "tcp" branch of my git repository. In case you're curious as to how to get that repository, you would use the following:

% git clone //
% cd mDNSResponder-214
% git branch --track tcp origin/tcp
% git checkout tcp

If you need more setup help, Dynamic DNS: Part …

read more