• How to make an ARM virtual machine (armhf/aarch64)

    I noticed that very few people seem to know how to create a full ARM virtual machine, so I decided to create a quick guide.

    This tutorial will use aarch64 and Debian as examples, but the same methodology should work for 32-bit ARM and other distributions. The instructions can also be adapted to create a simple chroot.

    (Read more...)
  • Run a Linux Program on a Different Network Interface

    Sometimes, you have multiple Internet connections, whether physical or virtual, and you want a few programs to access the Internet through one connection without making it the default gateway. For example, if you want a program to connect to the Internet through a VPN, but without forcing the entire system’s traffic through the VPN as well.

    The traditional way to do this is with packet marking with iptables and an ip rule to force marked packets through a different routing table to send the traffic to the correct destination. However, as the source IP was selected before routing, an SNAT rule in iptables is required to change the source IP. This is ugly and clearly a hack.

    However, since around 2013, Linux has introduced networking namespaces, which can be managed via ip netns as part of the iproute2 package. We can easily exploit this feature to achieve the desired goal with minimal fuss.

    (Read more...)
  • Desktop Audio Visualizer with i3 and Cava on WSL

    After seeing pictures of people running desktop audio visualizers on Reddit, I started to think if it is possible to replicate the effect on my i3-gaps setup running on Windows Subsystem for Linux (WSL).

    (Read more...)
  • Install Debian on a VPS Provider without Debian Images

    Recently, I came across a VPS provider that does not provide Debian images. This is rather annoying since I much prefer a fresh minimal install of Debian over a “minimal” Ubuntu image that still has a lot of stuff that I don’t want.

    Naturally, I decided to install Debian anyways, and came up with an approach to do so.

    If you are feeling particularly bold, you can try running my pre-made scripts that would convert a fresh Ubuntu install to a fresh Debian install.

    To use the scripts, you should download either the UEFI version or the BIOS version, depending on whether your current OS is using BIOS or UEFI.

    At the top of the script, change the variables to match your system configuration. The most important one being BOOT_DRIVE so that grub would be installed on the correct device.

    The scripts will prompt you for a root password and SSH keys. Once the script finishes, the system will be rebooted and you should be able to SSH into the now-Debian machine as root via the SSH keys.

    If you don’t feel like using the script, I am also providing manual instructions. This also explains how the scripts work.

    (Read more...)
  • Using Unordered Data Structures on C++ std::pair

    In many situations, it seems fairly natural to use std::unordered_set and std::unordered_map on std::pair. Here’s an example of what you might be tempted to do:

    #include <unordered_set>
    
    int main(void) {
        std::unordered_set<std::pair<int, int>> test;
    }
    

    However, std::pair is not hashable by default, so a simple snippet like the above would not work.

    There are many proposals online to define a pairhash class and explicitly specify it as the hash function as a template parameter to std::unordered_set and std::unordered_map.

    This is not a bad idea. In fact, if you are writing a library, you should probably do this. But we can do better…

    (Read more...)
  • Simple NDP Proxy to Route Your IPv6 VPN Addresses

    If you tried setting up an IPv6-capable VPN on a VPS provider that gave you an IP range to play with, perhaps a /64 or larger, you would want to assign some of the IPv6 addresses you have to your clients. In this post, we suppose that you have the range 2001:db8::/64.

    This should be a simple process: enable the sysctl option net.ipv6.conf.all.forwarding to 1 (or whatever the equivalent is on your system), use DHCPv6 or SLAAC to assign the addresses to the clients, and then your client should have working IPv6.

    The Problem

    Unfortunately, this is not so simple. Most VPS providers are not actually routing the entire subnet 2001:db8::/64 to you. Rather, they just connect a number of VPSes onto the same virtual Ethernet network and rely on the Neighbour Discovery Protocol (NDP) to find the router.

    (Read more...)
  • On Invalidation of Aggressively Cached Static Sites

    I have always wanted to make this website load fast everywhere in the world, despite the server being in Montréal, Canada, without investing heavily. It shouldn’t be hard: after all, it is just a bunch of static files, generated with Jekyll.

    Cloudflare brings a free CDN. You can set a page rule to aggressively cache your website on their CDN edge nodes, allowing your site to load as if it is hosted locally, even if you are half a world away.

    There is just a little problem: how do you efficiently purge the cache when you update your site? It is quite easy to purge the entire cache on Cloudflare, but that is rather inefficient: most of your assets probably did not change, and now they will all have to be fetched again.

    Today I decided to tackle this problem by creating purge-static, a tool designed to purge your CDN cache. It can purge your Cloudflare cache for you. You can get started by running pip install purge-static.

    (Read more...)
  • Optimize MySQL/MariaDB Queries with STRAIGHT_JOIN

    Recently, I had to deal with an issue on DMOJ, where a page displaying 100 out of around a million elements took over 10 seconds to load. Naturally, I started investigating the issue.

    (Note: DMOJ uses MariaDB, but the same problem, as well as the eventual solution, should work the same on MySQL as well.)

    The first course of action, of course, was to see what the database was trying to do, by running an EXPLAIN query. For those of you who don’t know, if you have a query of the form SELECT x FROM y WHERE z, running EXPLAIN SELECT x FROM y WHERE z would show what the query is doing, without actually executing the query.

    A quick look at the EXPLAIN output showed that MariaDB first did a filter on a 2000 row table, and then joined in the table with a million elements. Then, the 100 wanted rows were filtered out. This query plan was quite horrifying.

    (Read more...)
  • Python 2 on Windows: Unicode Command Lines with subprocess

    On Windows, using Python 2’s subprocess module to launch a process with a unicode command line that is not strictly from the currently active ANSI code page (i.e. encoding mbcs) will be mangled. All characters that cannot be encoded by mbcs will, in fact, be replaced with ?.

    Obviously, this is can be resolved by switching to Python 3, but sometimes, converting to Python 3 is not yet an option. A terrifying prospect in 2018, but a problem nonetheless.

    I present the module uniprocess, which defines its custom version of Popen and friends to work around the problem. I hope it proves useful to you.

    (Read more...)
  • The fast way to install nginx.org debs on Debian

    I personally prefer the nginx.org packages for nginx over the ones that comes with Debian. They are usually newer and have a more sane amount of dependencies. I also prefer the conf.d system over the sites-available and sites-enabled system.

    The main challenge in installing these packages on Debian is the trouble you have to go through to get the PGP keys and sources.list set up. nginx.org does not present a good setup script. This has become a repetitive and annoying experience, so I present a series of commands to set it up quickly.

    For stable:

    curl https://nginx.org/keys/nginx_signing.key | sudo apt-key add -
    (codename="$(dpkg --status tzdata | grep Provides | cut -f2 -d'-')"; echo; for deb in deb deb-src; do echo $deb http://nginx.org/packages/debian/ "$codename" nginx; done) | sudo tee -a /etc/apt/sources.list
    sudo apt update && sudo apt install nginx
    

    For mainline:

    curl https://nginx.org/keys/nginx_signing.key | sudo apt-key add -
    (codename="$(dpkg --status tzdata | grep Provides | cut -f2 -d'-')"; echo; for deb in deb deb-src; do echo $deb http://nginx.org/packages/mainline/debian/ "$codename" nginx; done) | sudo tee -a /etc/apt/sources.list
    sudo apt update && sudo apt install nginx
    
    (Read more...)