It’s common knowledge that Rust code is slow to compile. But I have a strong gut feeling that most Rust code out there compiles much slower than it could.
As an example, one fairly recent post says:
With Rust, on the other hand, it takes between 15 and 45 minutes to run a CI pipeline, depending on your project and the power of your CI servers.
This doesn’t make sense to me. rust-analyzer CI takes 8 minutes on GitHub actions. It is a fairly large and complex project with 200k lines of own code and 1 million lines of dependencies on top.
It is true that Rust is slow to compile in a rather fundamental way. It picked “slow compiler” in the generic dilemma, and its overall philosophy prioritizes runtime over compile time (an excellent series of posts about that: 1, 2, 3, 4). But
rustcis not a slow compiler — it implements the most advanced incremental compilation in industrial compilers, it takes advantage of compilation model based on proper modules (crates), and it has been meticulously optimized. Fast to compile Rust projects are a reality, even if they are not common. Admittedly, some care and domain knowledge is required to do that.
So let’s take a closer look at what did it take for us to keep the compilation time within reasonable bounds for rust-analyzer!
To complete the Traits milestone, I need to finish my work on super-traits, dynamic trait objects, and operator overloading support. The remaining items will be closed out by these higher-level features. Traits were always going to be the most challenging piece of work for me; I also expect gaps that may be missed, especially around auto-traits. I have also found interesting cases like qualified types, as I describe in a section below, that caused me to slow down a bit.
Though I have gone over my initial deadline for traits, I will need to use some extra time to complete the remaining work for this milestone. My initial expectation is to target the 20th of September to complete this work. I believe this is ok considering conference prep work and failing to plan vacation time over the summer. Though the community has done a stellar job recently by completing work such as module support and unions. They are even working on enum support now, which saves time in future milestones.
Thank you to everyone who continues to support and work on the compiler.
Using async Rust libraries is usually easy. It’s just like using normal Rust code, with a little
.awaithere and there. But writing your own async libraries can be hard. The first time I tried this, I got really confused by arcane, esoteric syntax like
Pin<&mut Self>. I had never seen these types before, and I didn’t understand what they were doing. Now that I understand them, I’ve written the explainer I wish I could have read back then. In this post, we’re gonna learn
What Futures are
What self-referential types are
Why they were unsafe
How Pin/Unpin made them safe
Using Pin/Unpin to write tricky nested futures
Iterators are part of Rust’s secret sauce. They power things from the humble for-loop to the elegant iterator chain, but have you ever stopped to think how they work?
Let’s find out more about Rust’s iterators by implementing our own versions of common iterators and reading the standard library’s source code.
In this article, I’ll share my experience with organizing large Rust projects. This is in no way authoritative — just some tips I’ve discovered through trial and error.
Cargo, Rust’s build system, follows convention over configuration principle. It provides a set of good defaults for small projects, and it is especially well-tailored for public crates.io libraries. The defaults are not perfect, but they are good enough. The resulting ecosystem-wide consistency is also welcome.
However, Cargo is less opinionated when it comes to large, multi-crate projects, organized as a Cargo workspace. Workspaces are flexible — Cargo doesn’t have a preferred layout for them. As a result, people try different things, with varying degrees of success.
It turns out that finite state machines are useful for things other than expressing computation. Finite state machines can also be used to compactly represent ordered sets or maps of strings that can be searched very quickly.
In this article, I will teach you about finite state machines as a data structure for representing ordered sets and maps. This includes introducing an implementation written in Rust called the
fstcrate. It comes with complete API documentation. I will also show you how to build them using a simple command line tool. Finally, I will discuss a few experiments culminating in indexing over 1,600,000,000 URLs (134 GB) from the July 2015 Common Crawl Archive.
The technique presented in this article is also how Lucene represents a part of its inverted index.
Along the way, we will talk about memory maps, automaton intersection with regular expressions, fuzzy searching with Levenshtein distance and streaming set operations.
I recently wrote the most complex procedural Rust macro I’ve ever attempted. This post tries to outline the problems I’ve encountered and tells how I overcame them.
MeiliSearch is a fast, feature-rich full-text search engine. Its built on top of the LMDB key-value store and lives as a 35 MB binary when installed on Ubuntu or MacOS. It comes with a built-in client, server and WebUI. Features such as stemming, stop words, synonyms, ranking, filters and faceting all work out of the box, use sensible defaults and can be easily customised.
In a previous post, I tried to provide a general overview of the work required to implement a full-featured GUI toolkit. Spoiler: there’s quite a bit of it.
One of the major strengths of Rust is how easy it is to build and share components, using cargo and crates.io. In this post, I would like to try and enumerate several of the subprojects involved in building a desktop GUI framework: these projects can be thought of as infrastructure, and ideally they can live as independent crates shared by various GUI projects.
Let’s begin by saying: this is a very exciting post. Some people reading this will be overwhelmingly thrilled; some will have no idea what GATs (generic associated types) are; others might be in disbelief. The RFC for this feature did get opened in April of 2016 (and merged about a year and a half later). In fact, this RFC even predates const generics (which an MVP of was recently stabilized). Don’t let this fool you though: it is a powerful feature; and the reactions to the tracking issue on Github should maybe give you an idea of its popularity (it is the most upvoted issue on the Rust repository):
So! Rust futures! Easy peasy lemon squeezy. Until it’s not. So let’s do the easy thing, and then instead of waiting for the hard thing to sneak up on us, we’ll go for it intentionally.
I’ve been working on my new Rust side-project for several months now, and I’ve got some learnings to share. The project is called
pq- it’s a command-line tool to parse and query log files as time series. It comes with its own domain-specific language that is highly influenced by PromQL.
When learning Rust, understanding the difference between statically and dynamically sized types seems critical. There are some good discussions out there already (e.g. here and here). Whilst these explain the mechanics, they didn’t tell me why its done like this in Rust. The articles made sense, but I was still confused! Eventually I had my “eureka” moment, so I figured I should share that.
There’s a lot of tribal knowledge surrounding #[inline] attribute in Rust. I often find myself teaching how it works, so I finally decided to write this down.
Caveat Emptor: this is what I know, not necessary what is true. Additionally, exact semantics of #[inline] is not set in stone and may change in future Rust versions.
Today I’m happy to announce a new era in Tor implementation.
Over the past year or so, we’ve been working on “Arti”, a project to rewrite Tor in Rust. Thanks to funding from Zcash Open Major Grants (ZOMG), we can finally put the Arti project up in our priorities list, and devote more time to it.
Below I’ll talk about why we’re doing this project, what it means for Tor users and operators, where it’s going in the future, and how people can help.
Rust supports a number of type coercions, which implicitly convert one type to another. As in any language with coercion, there is a trade-off made between clarity when reading and ease of writing. While disagreement may be had about whether Rust’s list of supported coercions is best, there is value in learning the coercions available, as some are central to functioning or idiomatic Rust code. In this post, I describe what coercions are possible, and where they can happen.
This is the patch series to add support for Rust as a second languageto the Linux kernel.