John Titor
@johntitor

From the future.

Joined September 2020

Whenever we speak about indexes, especially in PostgreSQL context, there is a lot to talk about: B-tree, Hash, GiST, SP-GiST, GIN, BRIN, RUM. But what if I tell you that even the first item in this list alone hiding astonishing number of interesting details and years of research? In this blog post I’ll try to prove this statement, and we will be concerned mostly with B-tree as a data structure.

So! Rust futures! Easy peasy lemon squeezy. Until it’s not. So let’s do the easy thing, and then instead of waiting for the hard thing to sneak up on us, we’ll go for it intentionally.

OpenTTD is a free, open-source recreation of the Chris Sawyer masterpiece: Transport Tycoon. I lost many hours playing this after opening it for Christmas in 1994. (alongside Outpost, and Global Domination – thanks, dad!)

The three goals for this project:

  • Share steps that beginners can follow to take apart any large open-source project

  • Do it by example with OpenTTD

  • Add an active, intermediate-difficulty project to my library of decoded work


This project deviates from my usual practice of documenting every line of code. It’s not practical to take apart over 200,000 lines, especially for an actively developed project. Instead, I will pick and choose the key ideas that an aspiring contributor may want to dig in to. Above all, this exercise should be transferable to other projects.

Here’s the plan of attack:

  1. The setup

  2. Code overview

  3. Organization

  4. Game Object Model

  5. Engine core

  6. Startup & Initialization

  7. Game loop

  8. Honorable Mentions

  9. Appendix

Read More

If you ask me to name the most misunderstood aspect of Python, I will answer without a second thought: the Python import system. Just remember how many times you used relative imports and got something like ImportError: attempted relative import with no known parent package; or tried to figure out how to structure a project so that all the imports work correctly; or hacked sys.path when you couldn’t find a better solution. Every Python programmer experienced something like this, and popular StackOverflow questions, such us Importing files from different folder (1822 votes), Relative imports in Python 3 (1064 votes) and Relative imports for the billionth time (993 votes), are a good indicator of that.

The Python import system doesn’t just seem complicated – it is complicated. So even though the documentation is really good, it doesn’t give you the full picture of what’s going on. The only way to get such a picture is to study what happens behind the scenes when Python executes an import statement. And that’s what we’re going to do today.

Read More

Welcome to part 1 of Modern C++ for C Programmers, please see the introduction for the goals and context of this series.

In this part we start with C++ features that you can use to spice up your code ‘line by line’, without immediately having to use all 1400 pages of ‘The C++ Programming Language’.

Various code samples discussed here can be found on GitHub.

Reflection is often presented as a feature that makes software harder to understand. In this article, I will present ways to approximate some level of static reflection in pure C++, thanks to C++17 and C++20 features, show how that tool can considerably simplify a class of programs and libraries, and more generally enable ontologies to be specified and implemented in code.

I’ve been working on my new Rust side-project for several months now, and I’ve got some learnings to share. The project is called pq - it’s a command-line tool to parse and query log files as time series. It comes with its own domain-specific language that is highly influenced by PromQL.

Have you ever envisioned the daily C preprocessor as a tool for some decent metaprogramming?

Have you ever envisioned the C preprocessor as a tool that can improve the correctness, clarity, and overall maintainability of your code, when used sanely?

I did. And I have done everything dependent on me to make it real.

Meet Metalang99, a simple functional language that allows you to create complex metaprograms. It represents a header-only macro library, so everything you need to set it up is -Imetalang99/include and a C99 compiler ^
. However, today I shall focus only on two accompanying libraries – Datatype99 and Interface99. Being implemented atop of Metalang99, they unleash the potential of preprocessor metaprogramming at the full scale, and therefore are more useful for an average C programmer.

I shall also address a few captious questions regarding compilation times, compilation errors, and applicability of my method to the real world.

Nuff said, let us dive into it!

Read More

The Stockfish project strongly believes in free and open-source software and data. Collaboration is what made this engine the strongest chess engine in the world. We license our software using the GNU General Public License, Version 3 (GPL) with the intent to guarantee all chess enthusiasts the freedom to use, share and change all versions of the program.

Unfortunately, not everybody shares this vision of openness. We have come to realize that ChessBase concealed from their customers Stockfish as the true origin of key parts of their products (see also earlier blog posts by us and the joint Lichess, Leela Chess Zero, and Stockfish teams). Indeed, few customers know they obtained a modified version of Stockfish when they paid for Fat Fritz 2 or Houdini 6 - both Stockfish derivatives - and they thus have good reason to be upset. ChessBase repeatedly violated central obligations of the GPL, which ensures that the user of the software is informed of their rights. These rights are explicit in the license and include access to the corresponding sources, and the right to reproduce, modify and distribute GPLed programs royalty-free.

Read More

WebAssembly threads support is one of the most important performance additions to WebAssembly. It allows you to either run parts of your code in parallel on separate cores, or the same code over independent parts of the input data, scaling it to as many cores as the user has and significantly reducing the overall execution time.

In this article you will learn how to use WebAssembly threads to bring multithreaded applications written in languages like C, C++, and Rust to the web.

An important feature of transactional databases like SQLite is “atomic commit”. Atomic commit means that either all database changes within a single transaction occur or none of them occur. With atomic commit, it is as if many different writes to different sections of the database file occur instantaneously and simultaneously. Real hardware serializes writes to mass storage, and writing a single sector takes a finite amount of time. So it is impossible to truly write many different sectors of a database file simultaneously and/or instantaneously. But the atomic commit logic within SQLite makes it appear as if the changes for a transaction are all written instantaneously and simultaneously.

SQLite has the important property that transactions appear to be atomic even if the transaction is interrupted by an operating system crash or power failure.

This article describes the techniques used by SQLite to create the illusion of atomic commit.

Read More

Integration of Lua as a first-class language in the latest release of Neovim provides a good alternative to developers to use a more proper programming language to configure Neovim and develop plugins. Vimscript can get a bit messy to manage when the code base grows. Lua is also relatively easy to learn compared to Emacs Lisp which is used to configure Emacs.

I have been trying to configure Neovim using Lua. There are many developers who had done this, and it seems each one has very different ways of coding the configurations. This is a bit challenging for a beginner to understand.

In this article, I am going to set up a very simple solution that hopefully is easier for beginners to understand, and they can further engineer or custom the solution based on their preferences.

Read More

Shell languages such as Bash excel at certain tasks, such as gluing programs together or quickly automating a set of command line steps. In contrast to those strengths, using a Shell to parse an INI config file is a bit like writing a poem in mud, you might succeed, but the result will probably be inscrutable and your swear jar will be full! As this wonderful Stack Overflow post attests there are many different ways to parse an INI file in Bash, but few of the answers provided are elegant.

So if you have a task poorly suited to Bash, what are your options?

  1. Choose another language for the task? (Perhaps sensible, but not always fun.)

  2. Write a custom Bash builtin to extend Bash for the task? (Spoiler, this is the route we will choose!)

Read More

Many years ago, Peter Norvig wrote a beautiful article about creating a lisp interpreter in Python. It’s the most fun tutorial I’ve seen, not just because it teaches you about my favorite language family (Lisp), but because it cuts through to the essence of interpreters, is fun to follow and quick to finish.

Recently, I had some time and wanted to learn Rust. It’s a beautiful systems language, and I’ve seen some great work come out from those who adopt it. I thought, what better way to learn Rust, than to create a lisp interpreter in it?

Hence, Risp — a lisp in rust — was born. In this essay you and I will follow along with Norvig’s Lispy, but instead of Python, we’ll do it in Rust 🙂.

Read More

The current Go language memory model was written in 2009, with minor updates since. It is clear that there are at least a few details that we should add to the current memory model, among them an explicit endorsement of race detectors and a clear statement of how the APIs in sync/atomic synchronize programs.

This post restates Go’s overall philosophy and the current memory model and then outlines the relatively small adjustments I believe that we should make to the Go memory model. It assumes the background presented in the earlier posts “Hardware Memory Models” and “Programming Language Memory Models.”

I have opened a GitHub discussion to collect feedback on the ideas presented here. Based on that feedback, I intend to prepare a formal Go proposal later this month. The use of GitHub discussions is itself a bit of an experiment, continuing to try to find a reasonable way to scale discussions of important changes.

Read More