This post was originally posted on my Facebook wall on: August 26th, 2016. It also was posted on my original blog in 2017.
tl;dr Rust is my new favorite programming language.
I'll be honest about this post upfront. This post is very much a rant about things I have personally loved about a newish programming language. Although I've been using rust for a couple months now, I understand this is still a fairly new language. You could even as a very vague definition call this an attempt to get engineers to write in Rust, but to be honest I don’t know too many engineers anymore working at a web company who like writing in systems languages.
With that out of the way. Let’s begin shall we? So what exactly is Rust? What is it’s goals/ideals/life goals/favorite snack? I mean what made me get interested in Rust? Why is it so cool that I’m literally writing my first blog post about it, and named by blog after it when I haven’t written a post/named a blog about the language I’ve written most in my life (C++)?
Well it all started with the inkling of an idea a couple months ago. I remember reading a specific blog post about Servo, which Mozilla is developing as a brand new layout engine for the browser (Think of Blink to Chrome, or Gecko to Firefox). (Also yes, that’s the same Mozilla who makes FireFox). I remember reading the article where it was my sort of first introduction to Rust. In the article it’s described as:
Rust focuses on safety and speed: its security measures do not impact it at run-time as the safety
mechanisms are in the language by design. For example, variables in Rust have an owner and a lifetime; they can be borrowed by another owner. When a variable is being used by one owner, it cannot be used by another. This is supposed to help enforce memory safety and stop data races between threads.
It also forces the programmer to stop and think about their software's design – Rust is not something for novices to pick up and quickly bash out code on.
This, hopefully, means fewer security bugs in Rust code, which for Servo means fewer exploitable holes in
an application that has to deal with potentially hostile data every moment of the day. Servo hopes to avoid the usual use-after-free() and buffer overflows bugs present in other software, particularly browsers and
their plugins, and thus give users a more secure window to the internet.
Next I ran into a solution at work that was an interesting problem to solve (that admittedly is one of the two large projects I’m working on). I can’t go too much into it, but I can give a high level overview of the problem. Which admittedly seems simple at first, but has quite a few interesting catches.
The problem is essentially this. We have a source of events. These events need to be processed in real time (or as close to it as possible. The maximum delay is 5 seconds behind real time, and even this in my eyes is unacceptable. The greatest time difference I would be even remotely acceptable with is 3 seconds and even that is too slow for my tastes). The producer of these events can be wildly random. I’m talking we can get anywhere from about 10 events per second, to over 1000 events per second. (Over 1,000 events per second is absolutely insane if you think about it. For reference 1,000 events per second * 60 is 60,000 events per minute * 60 is 360,000 events per hour times 24 hours is >86 million events per day, this happens with quite some frequency). The way these events are delivered are over Webhooks. Processing these normally isn’t considered too much of a challenge. After all you just split your traffic across the world, and you can handle considerably more traffic than this. However the problem was this is an internal only project. Meaning we can’t just spin up a more than a couple app servers to just handle this load (like no more than 3 really would be good).
So the pipeline for this was built out using Apache Kafka, but we needed a way to turn those HTTP Events -> Apache Kafka messages. Essentially we needed a proxy. So I had to build a webserver! Oh boy! Everyone’s like first real boy personal project in Computer Science right (I mean who honestly doesn’t create a blog as soon as they think they know things)? However we had a few extra challenges:
- The Web Server needed to be extremely fault tolerant. If Apache Kafka ever was going through an upgrade (or even worse down! :gasp:) the webserver should be able to log these, and replay them.
- I wanted to split the sending of messages to kafka, separate from the receiving of messages. I don't want to drop events because kafka is having problems, or taking awhile to respond. (Yay multithreading! Everyone's favorite problem).
- This solution should be able to be monitored easily. If something goes wrong. I want to know about it, and I want it to be easy to track down the problem.
- The system should be easy to manage. This internal project isn’t vital (not even close), and as such doesn’t get a lot of developer/operations attention (and shouldn’t). We don’t want this sucking up everyone’s time.
So I started looking at building a solution for this. This didn’t seem like just an “us” problem, but I couldn’t find anything that was lightweight, could handle this load, and obviously was an http -> kafka proxy. So I started looking at building out an implementation for us. After all we had a lot of wiggle room to try something new here. Originally I was planning on building our implementation in GoLang (written by google, and similar to Rust in a way). However I found it lacking in a couple areas that didn’t make me as a developer very excited. I won’t go into the details of why, but I just didn't really like it, and didn't think it would be the right choice. I ended up seeing quite a few threads mentioning Rust as a viable competitor, and even SOME articles that would say they aren’t competitors.
Eventually I decided it would be worth an actual look-see. So I hopped on over to the Official Docs of Rust, and went through their book. Eventually coming out on the other side a changed person. Some of the ideas they had were absolutely amazing. Things like:
- Memory Safety built into the compiler. Wait I don’t have to worry about multi-threading bugs that can be the absolute bane of all existence? Things are guaranteed to work in an environment when any number of things can be modifying the same value at the same time?
- Dependencies are smartly designed? All dependencies have features, and I can only pull out the features I need? I don’t have to manually update them like I do in C/C++ (and make sure there’s a compatible file for each OS)? I don’t have the problem of NodeJS where dependencies can be updated with breaking changes even if I don’t want them too? They get locked into place? I can check out my dependencies from open source code places, and don’t actually have to publish to a third party service? It’s like someone finally figured out how to do this right! They're not just stuffed in "import" statements all over, and I have to dig for where they all are?
- Documentation gets checked at compile time too? Meaning every single piece of documentation will work? So I’ll never run into documentation that is outdated? Like the documentation works 100% of the time? I don’t have to worry about version conflicts or anything it just works?
- My code gets compiled down to x86 assembly directly with performance in mind? Meaning like I get the performance of C code, and not some minor performance hits in languages like Go, or even worse interpreted languages?
- Multithreading is easy as long as you understand the language? Wow never thought that would be able to be said, but due to the ownership/type system + a great built in threading system. It almost seems like a dream. As long as you get the ownership system (which turns out can be quite painful if you don’t understand what it’s doing, and why, but if you stick with it and ask the community. It's totally possible to master, and not as hard as people make it seem) this becomes a breeze.
- A good automated testing framework built into the language? Like just default? I don’t have to go find a library to do it for me? I just have good assertions, and a test runner? What’s more I can include my tests in the same file as the source file (or in a separate directory) so tests are easy to find what they’re actually testing?
Basically it sounded like this language had everything I had been looking for in a language for years, and sure enough after enough messing around with it I was right! Seriously after spending a couple weeks in rust I’ve absolutely fallen in love. In papyrus words:
In theory rust language is pretty awesome, and if you like low level languages I really think you’ll like rust if you give it a chance.