Serialization Format Performance

Posted Sun 31 January 2016 11:53 under category programming

Most of the work done in actual programming jobs is taking structured data in some particular format from one system, slightly tweaking it, and sending it off to some other system. When exchanging data between different processes, it's almost always necessary to serialize it into a series of bytes which can be sent across a dumb byte-oriented transport (such as TCP). There are hundreds upon hundreds of different serialization formats out there, but I just wanted to talk about a few of the most common that folks use with the Python programming language.

read more

Beating the Compiler

Posted Sat 28 February 2015 14:33 under category programming

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

— Donald Knuth

Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest.

— Rob Pike

We spend a lot of our time in the modern, web services-driven technology industry ignoring performance issues. What's the point of micro-optimizing a 3ms function call when each request spends 8 or 9 seconds inside the SQLAlchemy ORM? Well, sometimes it's nice to practice those optimizion skills anyway, and today I'm going to walk you through micro-optimizing a simple problem and we'll see just how much better we can do than a naive solution... even though we probably normally shouldn't.

read more