yaymukund’s weblog

Lix: path does not exist and cannot be created

I use Lix on aarch64-darwin without home-manager or nix-darwin. When I tried to upgrade my installation, I saw the error:

$ nix upgrade-nix
=> error: path '/nix/store/ggppip3wfx4vbdc0kxsdniw118viymdg-lix-2.93.2' does not exist and cannot be created

After going down the rabbithole, I figured out the issue. My nix.conf was missing two crucial lines:

# append to /etc/nix/nix.conf
substituters = https://cache.nixos.org https://cache.lix.systems
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.lix.systems:aBnZUw8zA7H35Cz2RyKFVs3H4PlGTLawyY5KRbvJR8o=

For context, these lines are normally added by the Lix installer, but they weren’t in my conf. I think this is because my install predates lix-installer, but I am not sure.

After adding those lines, I was able to reboot the daemon and upgrade successfully:

$ sudo launchctl kickstart -k system/org.nixos.nix-daemon
$ sudo nix upgrade-nix
$ nix --version
nix (Lix, like Nix) 2.93.2
System type: aarch64-darwin

Hopefully, this saves someone else some time.

Wrapping Rust's log crate

Rust’s log crate crate is a thin macro interface for logging that can plug in to many different backends. Its functions can be called just like println!:

let you = "cow".to_string();
let name = "moo".to_string();

// There's also log::log, log::warn, and so on.
log::info!("Hello {animal}, {} to you", greeting);

I wanted to wrap it to add a prefix [farm] so the output looks like:

[farm] Hello cow, moo to you

Easy, right? 😅

My first attempt was a macro that tried to concat! the prefix:

macro_rules! warn {
    ($str:literal) => (::log::warn!(concat!("[farm] ", $str)));
    ($str:literal, $($args:tt)*) => (::log::warn!(concat!("[farm] ", $str), $($args)*))
}

But it doesn’t work:

error: there is no argument named `animal`
  --> src/log.rs:3:51
   |
3  |     ($str:literal, $($args:tt)*) => (::log::warn!(concat!("[farm] ", $str), $(...
   |                                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
  ::: src/main.rs:46:5
   |
46 |     warn!("Hello {animal}");
   |     -------------------- in this macro invocation

Keep in mind that animal was previously defined with a let-statement as in the example code, so it should be in the calling context. Fortunately, there’s a helpful note:

= note: did you intend to capture a variable `animal` from the surrounding scope?
= note: to avoid ambiguity, `format_args!` cannot capture variables when the format string is expanded from a macro

The solution is straightforward. We just call format_args! ourselves!

// It works!
macro_rules! warn {
    ($str:literal) => (::log::warn!("[farm] {}", format_args!($str)));
    ($str:literal, $($args:tt)*) => (::log::warn!("[farm] {}", format_args!($str, $($args)*)))
}

Hacking a GPO 746 rotary phone

Rotary dial telephones are a fantastic way to start learning about electronics. They’re built to be disassembled and serviced (because they were originally rented!). There are online communities of engineers and enthusiasts who are happy to to share what they know. Most of all, they’re beautiful objects that inspire curiosity and nostalgia.

Here’s what I’ll be making:

It’s a GPO (General Post Office) 746 rotary dial telephone that plays a random song each time you pick up the handset.

Overview

The logical core of the GPO 746 is the D92732 circuit board. It connects all of the different components— the microphone, rotary dial, ringer bells, switch hook, and handset speaker.

For this project, I only cared about the last two:

  1. The switch hook signals when the handset has been picked up.
  2. The handset receiver plays music.

Instead of the circuit board, I used the Teensy to drive all the logic. Instead of powering it by a cord to the wall, I used a 3xAA battery pack to provide 4.5V, as the Teensy docs suggest.

Getting inside the phone

I loosened the screw at the back of the telephone with a flathead screwdriver. It was held in place with a spring and nut. This is explained in more detail here.

I took a photo of the circuit…

…and then disconnected all the wires!

Brute-forcing the switch hook terminals

I found the D92732 circuit diagrams, but eventually opted for a brute-force approach.

From reading online, I learned that there are a pair of terminals that connect to the switch hook. I used a multimeter to check connectivity between every pair of terminals.

That’s 19 choose 2 = 171 combinations.

In my phone, it happened to be terminals T3 and T6. They connected when the switch hook was released, and disconnected when the switch hook was pressed. If you’re following along, they may be different terminals if you’re on another version of the GPO 746. The meaning may also be reversed (i.e. disconnected = released, connected = pressed).

The handset receiver

The handset cord has four wires leaving it: two for the receiver and two for the microphone. To identify the wires for the receiver, I just unscrewed the receiver-end of the handset and noted the colors.

Wiring the terminals

Then I made the following four connections to the terminals:

  1. The red and green handset receiver wires connect at T11 and T13 respectively.
  2. The brown and black switch hook wires connect at T3 and T6 respectively.

I chose T11 and T13 for the handset receiver wires because they happened to be “free.” That is, they didn’t seem to be connected to anything else.

Wiring the audio breakout board

Red goes to the sleeve and green goes to the tip. You may be able to switch these around— I don’t think they are polarised— but I did not test it.

Then, I soldered the Teensy Audio Shield to the Teensy using pins headers and connected the audio breakout board to the Teensy Audio Shield using a short 3.5mm audio cable.

Wiring the Teensy

  1. Red and white connect the battery pack to Teensy’s ground and Vin (3.6 to 5.5 volts) pins respectively.
  2. Brown and black connect the switch hook terminals to Teensy’s Input 0 and ground respectively.

The Teensy Pins reference is excellent. The ground pins are interchangeable, but be sure to use the Vin (3.6 to 5.5 volts) and not the 3.3V input because the battery pack will exceed 3.3V.

Battery pack

Finally, I connected the red and black wires from the Teensy to a 3xAA (4.5V) battery pack and toggle switch.

The complete circuit

Software considerations

I’ve appended the code for reference.

The PJRC website has instructions on flashing the Teensy 4. I disconnected the Teensy from the battery pack. Then, I used a micro-B USB cable to connect it to my computer and used teensy-loader-cli to transfer a compiled image.

I wrapped everything in a Nix flake so you can see exactly what is needed.

Costs

I ended up spending about £300 for everything, but it could definitely be a lot cheaper:

  £54 TS101 soldering iron
+ £52 Omnifixo "third hand" tool
+ £42 Ratcheted crimper
+ £11 Wirestrippers
+ £6 Wirecutters

= £165 for tools

  £65 GPO 746
+ £22 Teensy
+ £14 Audio adapter
+ £11 microSD card
+ £10 22 AWG Wire
+ £7 Heat shrink Tubing
+ £6 3xAA Battery packs
+ £5 Audio breakout board
+ £5 Toggle switches
+ £1 Header pins

= £311 total

If you’re trying to save money, another option is to visit a local hackerspace. They may provide tools and components for free or very cheap.

Final thoughts

After starting this project, I also felt emboldened to fix some things around the house: a damaged power cord on my hand blender, an electric toothbrush, the battery on my headphone amp.

As for the telephone, I’d like to get the rotary dial, ringer, and microphone working. The ringer may need a 9V battery. It seems like a hassle to require two kinds of batteries, so that’s an open question.

Overall, I’m very pleased with how it turned out.

Inspiration & resources

Appendix

Tools

Materials

Optional

  • 3xAA Battery pack: To power it without a USB cable.
  • Toggle switch for the battery: Why drain the battery when it’s not in use?
  • Pin headers and jumper cable: These let me easily switch the inputs/outputs. of the Teensy without soldering - useful for reuse and experimentation.
  • 3.5mm jack breakout board: Lets you avoid cutting an audio cable.
  • Fork terminals: Connects wire to terminals. Must match wire and terminal sizes.

Code

For the latest version of this code, please check the repository.

#include <Arduino.h>
#include <Audio.h>
#include <AudioStream.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <Bounce2.h>

#define DEBOUNCE_INTERVAL_MS 40
#define SERIAL_WAIT_TIMEOUT_MS 5000
#define SERIAL_BAUD_RATE 9600

// Pins
#define SDCARD_CS_PIN 10
#define SDCARD_MOSI_PIN 11
#define SDCARD_SCK_PIN 14
#define HOOK_PIN 0

#define DEBUG false

#define debug_printf(...) \
  do { if (DEBUG) Serial.printf(__VA_ARGS__); } while (0)

// Handset audio output.
AudioOutputI2S audio_out;
// Play files from the SD card.
AudioPlaySdWav play_wav;
// 
// There are two connections for L and R speakers.
AudioConnection wav_out_0(play_wav, 0, audio_out, 0);
AudioConnection wav_out_1(play_wav, 0, audio_out, 1);

// The chip on the Teensy Audio Shield that lets you control the microphone
// input and headphone output.
AudioControlSGTL5000 sgtl5000;

// A debounced "button" for the hook switch.
Bounce2::Button hook_switch_button = Bounce2::Button();

// Upload these to the SD card.
const char* FILENAMES[] = { "foo.wav", "bar.wav", "baz.wav" };

void setup_debugging() {
  if (!DEBUG) {
    return;
  }

  Serial.begin(SERIAL_BAUD_RATE);

  while (!Serial && millis() < SERIAL_WAIT_TIMEOUT_MS) {
    // Wait for the serial port to connect.
  }

  debug_printf("AUDIO_BLOCK_SAMPLES is %d.\n", AUDIO_BLOCK_SAMPLES);

  if (CrashReport) {
    /* print info (hope Serial Monitor windows is open) */
    Serial.print(CrashReport);
  }
}

extern "C" void setup(void) {
  setup_debugging();

  // Initialise random.
  uint32_t seed = micros();
  debug_printf("random seed is %d.\n", seed);
  randomSeed(seed);

  // 120 blocks of audio memory from which you construct audio_block_t.
  AudioMemory(120);

  // Enable headphone/mic.
  sgtl5000.enable();
  sgtl5000.volume(0.5);

  // Setup SD card.
  SPI.setMOSI(SDCARD_MOSI_PIN);
  SPI.setSCK(SDCARD_SCK_PIN);

  hook_switch_button.attach(HOOK_PIN, INPUT_PULLUP);
  hook_switch_button.interval(DEBOUNCE_INTERVAL_MS);
  hook_switch_button.setPressedState(HIGH);


  if (!(SD.begin(SDCARD_CS_PIN))) {
    while (1) {
      debug_printf("Unable to access the SD card.\n");
      delay(1000);
    }
  }

  debug_printf("HIGH is %d\n", HIGH);
  debug_printf("LOW is %d\n", LOW);
}

extern "C" int main(void) {
  setup();

  while (1) {
    hook_switch_button.update();

    if (hook_switch_button.changed()) {
      debug_printf("Hook switch is now %d.\n", hook_switch_button.read());
    }

    if (hook_switch_button.isPressed() && play_wav.isPlaying()) {
      play_wav.stop();
    }

    if (hook_switch_button.fell() && play_wav.isStopped()) {
      int index = random(std::size(FILENAMES));
      debug_printf("Playing track #%d: %s.\n", index, FILENAMES[index]);
      play_wav.play(FILENAMES[index]);
    }
  }
}

Cross-compiling NixOS for a Raspberry Pi

I had to install NixOS on my Raspberry Pi 4, Model B recently. I didn’t have the HDMI→ micro HDMI cable so I decided to install it headlessly. This is a fairly intricate setup because I wanted to:

  1. Cross-compile from aarch64-darwinaarch64-linux.
  2. Remote build using nixbuild.net to speed up build times.
  3. Build from my memory-constrained 1GB Raspberry Pi.*

It involved a few gotchas which I want to document here.

* Note: Although it is possible to offload the compilation to nixbuild, you still need memory on the Pi to evaluate the nix code. There is an open issue for eval memory usage which may alleviate this.

Making a NixOS SD Image

First, make a flake.nix that produces the SD image:

{
  inputs = {
    nixos-generators.url = "github:nix-community/nixos-generators";
    nixos-hardware.url = "github:NixOS/nixos-hardware/master";
    nixpkgs.url = "nixpkgs/nixos-unstable";
  };

  outputs =
    { nixos-generators
    , nixos-hardware
    , nixpkgs
    }: {
      # This produces the install ISO.
      packages.aarch64-linux.installer-sd-image =
        nixos-generators.nixosGenerate {
          specialArgs = { inherit dotfiles-private; };
          system = "aarch64-linux";
          format = "sd-aarch64-installer";
          modules = [
            ./modules/hardware-configuration.nix
            nixos-hardware.nixosModules.raspberry-pi-4
            ./modules/base.nix
            ./modules/builder.nix
            ./modules/networking.nix
            ./modules/users.nix

            # Anything else you like...
          ];
        };
    };
}

Onto the modules…

modules/base.nix

{ pkgs, ... }: {
  programs.ssh.extraConfig = ''
    Host nixbuild
        HostName eu.nixbuild.net
        User root
        PubKeyAcceptedKeyTypes ssh-ed25519
        ServerAliveInterval 60
        IPQoS throughput
        IdentitiesOnly yes
        IdentityFile ~/.ssh/nixbuild

    # SSH config for your favorite code forge, needed so you can clone your
    # repository containing flake.nix for rebuilds.
  '';

  # Not strictly necessary, but nice to have.
  boot.tmp.useTmpfs = true;
  boot.tmp.tmpfsSize = "50%"; # Depends on the size of your storage.

  # Reduces writes to hardware memory, which increases the lifespan
  # of an SSD.
  zramSwap.enable = true;
  zramSwap.memoryPercent = 150;

  # Needed for rebuilding on the Pi. You might not need this with more
  #memory, but my Pi only has 1GB.
  swapDevices = [{
    device = "/swapfile";
    size = 2048;
  }];
}

modules/builder.nix

The remote builder lets us do two things:

  1. Cross-compile the SD image from a different architecture (aarch64-darwin in my case).
  2. Remote-build from the Raspberry Pi 4B. Compiling things locally on a Pi takes longer.

I (happily) use nixbuild.net, but you don’t have to. Any builder will do, as long as it can build aarch64-linux.

{
  nix.settings = {
    trusted-users = [ "my_username" ];
    builders-use-substitutes = true;
  };
  nix.distributedBuilds = true;
  nix.buildMachines = [{
    hostName = "eu.nixbuild.net";
    sshUser = "root";
    sshKey = "/home/my_username/.ssh/nixbuild";
    systems = [ "aarch64-linux" ];
    maxJobs = 100;
    speedFactor = 2;
    supportedFeatures = [
      "benchmark"
      "big-parallel"
    ];
  }];
}

modules/networking.nix

It’s important to get this right with a headless setup or else you won’t be able to SSH to diagnose any other issues. You probably want to use a secrets management system to configure the WiFi passkey.

{ ... }: {
  # Setup wifi
  networking = {
    hostName = "my_hostname";
    wireless.enable = true;
    useDHCP = false;
    interfaces.wlan0.useDHCP = true;
    wireless.networks = {
      my_ssid.pskRaw = "...";
    };
  };

  # And expose via SSH
  programs.ssh.startAgent = true;
  services.openssh = {
    enable = true;
    settings = {
      PasswordAuthentication = false;
      KbdInteractiveAuthentication = false;
    };
  };

  users.users."my_username".openssh.authorizedKeys.keys = [
    "ssd-ed25519 ..." # public key
  ];
}

modules/users.nix

{
  users.users.my_username = {
    isNormalUser = true;
    home = "/home/my_username";
    extraGroups = [
      "wheel"
      "networkmanager"
      "audio"
      "video"
    ];
  };

  security.sudo.execWheelOnly = true;

  # don't require password for sudo
  security.sudo.extraRules = [{
    users = [ "my_username" ];
    commands = [{
      command = "ALL";
      options = [ "NOPASSWD" ];
    }];
  }];
}

modules/hardware-configuration.nix

I don’t think there’s a good way to generate this before installing. Luckily, lots of people with Raspberry Pi 4Bs have put their hardware-configuration.nix online. Any of them should work. Here’s mine:

# Do not modify this file!  It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations.  Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/installer/scan/not-detected.nix")
    ];

  boot.initrd.availableKernelModules = [ "xhci_pci" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "/dev/disk/by-uuid/44444444-4444-4444-8888-888888888888";
      fsType = "ext4";
    };

  swapDevices = [ ];

  # Enables DHCP on each ethernet and wireless interface. In case of scripted networking
  # (the default) this is the recommended approach. When using systemd-networkd it's
  # still possible to use this option, but it's recommended to use it in conjunction
  # with explicit per-interface declarations with `networking.interfaces.<interface>.useDHCP`.
  networking.useDHCP = lib.mkDefault true;
  # networking.interfaces.end0.useDHCP = lib.mkDefault true;
  # networking.interfaces.wlan0.useDHCP = lib.mkDefault true;

  nixpkgs.hostPlatform = lib.mkDefault "aarch64-linux";
  powerManagement.cpuFreqGovernor = lib.mkDefault "ondemand";
}

Once you have SSH access, you can generate it with nixos-generate-config to verify it matches.

Putting it all together

  1. (From aarch64-darwin) Build the SD image.
    # -max-jobs 0: needed to force remote building for cross-compilation.
    # -system aarch64-linux: we need to override this bc we're on darwin.
    nix build \
        --max-jobs 0 \
        --system aarch64-linux \
        .#installer-sd-image
    
    zstd \
        -d result/sd-image/*.img.zst \
        -o installer-sd-image.img
    
  2. (From aarch64-darwin) Write it to your SD card.
    diskutil unmountDisk /dev/diskN
    sudo dd \
        if=..path/to/installer-sd-image.img \
        of=/dev/diskN \
        status=progress bs=1M
    diskutil eject /dev/diskN
    
  3. Put the SD card in your Raspberry Pi and start it up. It should appear on your local network.
  4. (From aarch64-darwin) ssh my_hostname and you should see it.

Rebuilding locally on the Pi

To rebuild on the Pi, there are a few more steps.

First, you’ll need to add the non-SD build target to your flake.nix:

nixosConfigurations.dave = nixpkgs.lib.nixosSystem {
  specialArgs = { inherit dotfiles-private; };
  system = "aarch64-linux";
  modules = [
    ./modules/hardware-configuration.nix
    nixos-hardware.nixosModules.raspberry-pi-4
    ./modules/base.nix
    ./modules/builder.nix
    ./modules/networking.nix
    ./modules/users.nix

    # Anything else you like...
  ];
};

Then, a few manual steps:

  1. ssh into your Pi and ssh-keygen -t ed25519 /.ssh/nixbuild
  2. ssh into your Pi and ssh-keygen -t ed25519 /root/.ssh/nixbuild (as root)
  3. Add the public key to your nixbuild.net account.
  4. git clone your config on the Pi.

(I’m not sure why both root and non-root keys are needed for nixos-rebuild to do its thing here. If you know, please tell me.)

Then you should be able to run:

nixos-rebuild switch \
    --use-remote-sudo \
    --max-jobs 0 \
    --flake /path/to/dir/containing/my/flake/

Takeaways

Nice things

  • Everything works— remote builds are fast, headless setup was successful.
  • Most of this is in Nix rather than in state, so redoing everything from scratch is simple.

Potential improvements

  • With a little bit more work, you could move the SSH keys and git clone into the config.
  • Use something like deploy-rs to remote deploy and we can skip setting up SSH keys on the Pi altogether. This seems ideal.

References

Using Nix on Flakes on OSX

I use Nix Flakes on OSX to setup my development environment. I’ve not seen anyone else document this approach. Maybe you will find it useful.

What’s in a development environment?

By “development environment,” I mean three things:

  1. Adding and mutating shell environment variables (e.g. $EDITOR)
  2. Installing command line applications (e.g. /usr/bin/nvim)
  3. Adding config files (e.g. $HOME/.config/nvim/init.lua)

Unfortunately, 2 and 3 are “impure” according to Nix because they require access to mutable paths. But there are simple workarounds:

  • Instead of installing binaries to /usr/bin/, I can install it to the store and add it to the $PATH. For example, instead of installing /usr/bin/nvim, I would install /nix/store/abc123-nvim/bin/nvim.
  • Instead of adding config files, I can wrap a binary to point to the store. For example, instead of generating $HOME/.config/nvim/init.lua, I’d:
    • Generate /nix/store/abc123-init.lua/init.lua.
    • Generate /nix/store/abc123-nvim-wrapped/bin/nvim, which just does nvim -u /nix/store/abc123-init.lua/init.lua $@. The -u flag lets you pass a path to the config, and $@ forwards arguments.
    • Add /nix/store/abc123-nvim-wrapped/bin to $PATH.

So if I can mutate environment variables— including $PATH— then I can do everything!

But first, I need to explain Flakes a little bit.

A Nix Flakes primer

Sorry, I feel like every Nix article that touches on Flakes has to explain Flakes from scratch. I’ll try and stick with what’s relevant to what I’m doing. If you’re interested in a deep dive, I recommend Xe Iaso’s Nix Flakes: an Introduction.

Flakes, at their core, are a configuration format for the Nix toolchain. This format accepts inputs, which are dependencies that live in the Nix store, and produces outputs, which are read by various tools. For example, the nix CLI tool’s nix build subcommand builds the packages.default output for the flake.

See? That wasn’t so bad, was it? If this still seems a bit abstract, read on for an example.

Note: In versions of nix prior to 2.7, packages.default was known as defaultPackage. If you care about compatibility with old versions, you may want to alias it to point to packages.default.

Designing a development environment

Using Flakes, I need to mutate environment variables. To do this, I’ll use a little-known command called nix print-dev-env:

nix print-dev-env - print shell code that can be sourced by bash to reproduce the build environment of a derivation

If you run nix print-dev-env, it will build the packages.default output of your current flake.nix.

This approach has two steps:

  1. Make a packages.default output that mutates shell environment variables as desired. For example, it should add /nix/store/abc123-nvim-wrapped/bin to the $PATH.
  2. Source the output of nix print-dev-env in my development shell.

Putting the pieces together

To construct the packages.default output, you can use pkgs.mkShell:

# In flake.nix
let
  neovim-with-config = neovim.override {
    customRC = ''
      lua << EOF
        -- init.lua goes here
      EOF
    '';
  };
in 
  {
    outputs = flake-utils.lib.eachDefaultSystem (_system: {
        packages.default = pkgs.mkShell {
          packages = with pkgs; [
            neovim-with-config
            # anything else
          ];

          shellHook = ''
            # Optionally, inject other stuff into your shell
            # environment.
          '';
        };
      });
  }

Since the shell requires neovim-with-config, its ‘build environment’ will append /nix/store/abc123-neovim-with-config/bin/ to $PATH. That’s exactly what we want.

And finally, source the output of nix print-dev-env:

# `print-dev-env` assumes bash. It mutates env variables such as
# `LINENO` that # are immutable in zsh, so I need to exclude them.
# This is annoying, but in practice works fine.
$ nix print-dev-env \
  | grep -v LINENO \
  | grep -v EPOCHSECONDS \
  | grep -v EPOCHREALTIME \
  > $HOME/development-configuration.zsh 

$ echo 'source $HOME/development-configuration.zsh' >> $HOME/.zshrc

If you inspect development-configuration.zsh, you’ll see a giant RC file that includes:

PATH='...:/nix/store/abc123-neovim-with-config/bin:...'

Indeed, running nvim works as expected. We have set up a development environment using Nix Flakes!

Full dotfiles

If you want to see my full dotfiles, it lives on sourcehut. Here’s a link to where I define packages.default and here’s where I run print-dev-env

Scoring an animation with Orca

In this post, I’ll demonstrate how to score an animation using Orca.

What is Orca?

Orca is an esoteric programming language for composing music. An Orca program is somewhere between a circuit diagram and an ASCII roguelike. But you don’t need to know either of those things to get started— in an interview with its creator Devine Lu Linvega of the programming duo Hundred Rabbits:

I was always kind of aiming, I guess, at children. I was like, if you can just open the page and put that in front of a kid, could they figure it out? It wouldn’t take that many keystrokes until they figure out which… like [the operator] E will start moving, and through the act of playing they’ll find their way without having to read the documentation.

— from Devine’s interview on the Future of Coding podcast

Some resources to get started:

The final score

The completed score, consisting of all the techniques described in this post.

Although it might seem complicated— especially if you’re not familiar with Orca— this program is actually the result of building on a few core ideas. As you read this, I hope it will feel like a natural progression to go from one step to the next.

Ok, now let’s start at the beginning.

What is a score?

Scoring an animation requires timing sounds to events on screen. For example, when a piece of glass shatters in the animation, there should be a crash sound. Let’s say this happens on frame 21. Using Orca, how do you play a sound on the 21st frame and then never again?

The answer to this was not obvious to me. Most Orca compositions, at the time of writing this, consisted of loops. I could not find any examples that did what I wanted. But since we have access to the clock frame using the C operator, this feels like it should be possible.

Here’s one way to do it:

Learning how to wait

Note: Throughout this post, I will refer to hexadecimal numbers using the prefix 0x. For example, 0x10 is decimal 16.

Note 2: I will refer to the last digit in a hexadecimal number as the “ones’ digit” and the second-to-last digit as the “sixteens’ digit.” For example, 0x10‘s ones’ digit is 0 and its sixteens’ digit is 1.

Ok, let’s begin.

First, use a pair of hexadecimal clocks Cf and fCf:

  • The frame count is at the bottom right, a monotonically increasing number.
  • Cf mods the frame count by 0xf, or 16, to output the ones’ digit, a number from 0x0 to 0xe.
  • fCf also divides the frame count by 0xf, or 16, to output the sixteens’ digit.

Next, check the outputs using F. The F operator will output a bang (*) if their inputs on either side are equal.

Finally, AND the outputs of both the Fs into a single *:

  • Y, the ‘yumper’ operator, just copies the input horizontally.
  • f here is a lowercase F that only operates on a bang.

On frame 0x15, the f outputs a bang.

Note: If the f were uppercase, it would incorrectly output a bang when both inputs are empty (i.e. when neither ones’ nor sixteens’ digits matched.)

At this point, I posted my timer to the Orca thread on lines, asking the community if there was a simpler way to do this. Devine responded, suggesting this condensed version that uses one fewer operator.

Limitations

Here, it is important to note that this timer does not actually “bang once and then never again” as originally promised. Since the timer only checks the ones’ and sixteens’ digits of the frame number, it will bang at 0x015, 0x115, 0x215, …— every 0x100 or 256 frames. This ends up being about 25 seconds at 120 bpm.

My animation happens to be 15s long, so this was not a problem for me. If anyone finds an elegant solution to the original problem as stated, I’d love to see it.

Wiring up sounds

Now, the fun part. Use this timer to schedule different sounds.

Playing a note

This can be used to play a single midi note.

Toggling a loop

First, consider a simple drum loop.

Then, use an X to set the note’s velocity, effectively turning the loop on or off. Here, it sets the velocity to 0x7.

It’s a bit silly to use X to set a constant value like this, but it should make sense in the next step.

Finally, automate this using timers:

  • The drum loop begins muted, its velocity 0x0.
  • The first timer at frame 0x15 turns on the drum loop by setting its velocity to 0xa.
  • The second timer at frame 0x28 turns it off again by setting the velocity to 0x0.

Timing a sequence of notes

Play a sequence of notes by combining operators Z and T:

  • The Z counts up from 1 to 5, exactly once.
  • This determines the note output by T.
  • The output of T is fed to :, the midi operator.

Use a timer to control when the sequence plays:

  • Z starts at 5, unlike the previous example, so it doesn’t immediately play.
  • The timer fires at frame 0x15 to activate x.
  • x sets the Z back to 0 and causes it to play.

Timers in practice

At this point, if you go back to the beginning and see the final composition, you may notice some differences from the examples. I omitted details to keep the examples small and easy to understand:

  • Instead of repeating fCf and Cf, the sixteens’ and ones’ digits are stored in variables a and b and read using Va and Vb, respectively.
  • I often use a timer triggered at frame 0x00 to reset the initial state of timed sequences or loops. This allows me to easily replay the composition from the beginning using Ctrl+R.

Conclusion

Here’s the completed looping animation:

A hand reaches to water a plant but drops the cup, which shatters. The plant droops, the mug pieces rise and hover in the air above the plant, before falling into the pot which we now see contains a worm. The worm is sliced in half by a piece of mug, its worm blood spreading around the dirt, before gathering and traveling up the stalk of the plant. Zooming in, we see a new stem emerge. Its bud sprouts, revealing it has grown pieces of the same mug fragments. Zooming out all the way, we see the plant is standing up once again. It has an extra stem, which the hand plucks. A worm squirms out of the pot and offscreen, and the entire animation loops.

(If you want to see more of my art, please follow my instagram.)

This is only a small taste of what Orca’s capable of doing, but I hope it’s a fun read. If you notice any mistakes in this article or want to share feedback, please reach out. I’d like to do more of these in the future.

Restic + Backblaze B2 on NixOS

While NixOS fully supports making restic backups using Backblaze, I couldn’t find documentation for it. From browsing configs on GitHub, many people seem to also use rclone but I’d rather not introduce another dependency.

Here’s how I did it:

{ config, pkgs, ... }:
{
  environment.systemPackages = [ pkgs.restic ];

  services.restic.backups.myaccount = {
    initialize = true;
    # since this uses an `agenix` secret that's only readable to the
    # root user, we need to run this script as root. If your
    # environment is configured differently, you may be able to do:
    #
    # user = "myuser
    #
    passwordFile = config.age.secrets.my_backups_pw.path;
    # what to backup.
    paths = ["/home/myusername"];
    # the name of your repository.
    repository = "b2:my_repo_name";
    timerConfig = {
      # backup every 1d
      OnUnitActiveSec = "1d";
    };


    # keep 7 daily, 5 weekly, and 10 annual backups
    pruneOpts = [
      "--keep-daily 7"
      "--keep-weekly 5"
      "--keep-yearly 10"
    ];
  };

  # Instead of doing this, you may alternatively hijack the
  # `awsS3Credentials` argument to pass along these environment
  # vars.
  #
  # If you specified a user above, you need to change it to:
  # systemd.services.user.restic-backups-myaccount = { ... }
  #
  systemd.services.restic-backups-myaccount = {
    environment = {
      B2_ACCOUNT_ID = "my_account_id_abc123";
      B2_ACCOUNT_KEY = "my_account_key_def456";
    };
  };

}

Overriding packages in NixOS

In NixOS, it’s sometimes desirable to override a package in order to extend or modify its behavior. For example, I override my Neovim to add plugins so they get all the benefits of being in the nix store. Here’s how I do it.

# in configuration.nix
nixpkgs.overlays = [
  (import ./overlays.nix)
];

# in overlays.nix
self: super: {
  neovim-mukund = self.callPackage ./packages/neovim-mukund.nix {};
}

# finally, in packages/neovim-mukund.nix
{ pkgs }:
  neovim.override {
    vimAlias = true;
    viAlias = true;
    configure = {
      packages.mukund-plugins = with pkgs.vimPlugins; {
        start = [
          ale
          fzf-vim
          # ...
        ];
      };
    };
  }

# putting it all together
environment.systemPackages = [
  neovim-mukund
];

Bonus: Installing a single package from main

If you need to install a single package from the main branch but keep the rest of your code on your nix channel (usually the main channel or nixos-unstable), then try this:

# in packages/neovim-mukund.nix
let neovim-master = (import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/master.tar.gz") {}).neovim
in
  environment.systemPackage = [
    neovim-master
  ]

This time, I’m fetching and installing from the master.tar.gz file. This is handy if there’s an update upstream that you want to use immediately. For example, I often use this when Discord releases an update. Nixpkgs usually merges the version bump fairly quickly, but it doesn’t reach the release channels for many days during which Discord is unusable.

References

Rust Magic

This is a list of places in Rust where implementing a trait or using a struct affects the syntax of your code. I think of these features as “magical” because using them can change the behavior of very basic Rust code (let, for-in, &, etc.).

What follows is a small list of (hopefully) illustrative examples, and a short epilogue pointing you to more articles if this topic interests you.


Contents


struct Foo {
    text: String,
}

impl Drop for Foo {
    fn drop(&mut self) {
        println!("{} was dropped", self.text);
    }
}

fn main() {
    let mut foo = Some(Foo {
        text: String::from("the old value"),
    });

    // this calls the drop() we wrote above
    foo = None;
}
struct MyCustomStrings(Vec<String>);

impl IntoIterator for MyCustomStrings {
    type Item = String;
    type IntoIter = std::vec::IntoIter<Self::Item>;

    fn into_iter(self) -> Self::IntoIter {
        self.0.into_iter()
    }
}

fn main() {
    let my_custom_strings = MyCustomStrings(vec![
        String::from("one"),
        String::from("two"),
        String::from("three"),
    ]);

    // We can use for-in with our struct
    //
    // prints "one", "two", "three"
    for a_string in my_custom_strings {
        println!("{}", a_string);
    }
}
use std::ops::Deref;

struct Smart<T> {
    inner: T,
}

// You can implement `DerefMut` to coerce exclusive references (&mut).
impl<T> Deref for Smart<T> {
    type Target = T;

    fn deref(&self) -> &Self::Target {
        &self.inner
    }
}

fn main() {
    let text = Smart {
        inner: String::from("what did you say?"),
    };

    // The `impl Deref` lets us invoke the `&str` method
    // `to_uppercase()` on a `&Smart<String>`
    println!("{}", &text.to_uppercase());
}
use std::fmt;

struct Goat {
    name: String,
}

impl fmt::Display for Goat {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "a goat named {}", self.name)
    }
}

fn main() {
    let goat = Goat {
        name: String::from("Yagi"),
    };

    // This invokes our `Display`'s `fmt()`
    println!("{}", goat);
}
#[derive(Clone, Copy, Debug)]
struct Point {
    x: usize,
    y: usize,
}

fn main() {
    let point_a = Point { x: 1, y: 2 };
    let point_b = point_a;

    // point_a is still valid because it was copied rather than moved.
    println!("{:?}", point_a);
}
// Notes:
// * This works very similarly with Option<T>
// * We need to derive(Debug) to use the error in a Result.
//
#[derive(Debug)]
struct SomeError;

fn uh_oh() -> Result<(), SomeError> {
    Err(SomeError)
}

fn main() -> Result<(), SomeError> {
    // The following line desugars to:
    //
    // match uh_oh() {
    //     Ok(v) => v,
    //     Err(SomeError) => return Err(SomeError),
    // }
    //
    uh_oh()?;

    Ok(())
}

Epilogue

When I first started compiling this list, I asked around in the Rust community discord. scottmcm from the Rust Language team introduced me to the concept of lang items. If you search for articles on this topic, you get some fantastic resources:

So what is a lang item? Lang items are a way for the stdlib (and libcore) to define types, traits, functions, and other items which the compiler needs to know about.

Rust Tidbits: What is a Lang Item? by Manish Goregaokar

Not all lang items are magical, but most magical things are lang items. If you want a deeper or more comprehensive understanding, I recommend reading Manish’s article in its entirety.

How to configure API Gateway v2 using Terraform

Here’s how you wire up an AWS lambda into an HTTP API using Terraform and AWS’s API Gateway v2 resources.

When you terraform apply this, it’ll spit out an API URL. You can GET / against that API URL to run your lambda:

resource "aws_iam_role" "plants" {
  name = "iam_plant_api"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "lambda.amazonaws.com"
        ]
      }
    }
  ]
}
EOF
}

# This presumes you have a zip file get_water_level.zip
# which contains a get_water_level.js file which exports
# a `handler` function
resource "aws_lambda_function" "get_water_level" {
  filename = "get_water_level.zip"
  function_name = "get_water_level"
  publish = true
  role = aws_iam_role.plants.arn
  handler = "get_water_level.handler"
  source_code_hash = filebase64sha256("get_water_level.zip")
  runtime = "nodejs12.x"
}

resource "aws_apigatewayv2_api" "plants" {
  name          = "http-plants"
  protocol_type = "HTTP"
}

resource "aws_apigatewayv2_stage" "plants_prod" {
  api_id = aws_apigatewayv2_api.plants.id
  name = "$default"
  auto_deploy = true
}

resource "aws_apigatewayv2_integration" "get_water_level" {
  api_id = aws_apigatewayv2_api.plants.id
  integration_type = "AWS_PROXY"
  integration_method = "POST"
  integration_uri = aws_lambda_function.get_water_level.invoke_arn
}

resource "aws_apigatewayv2_route" "get_water_level" {
  api_id = aws_apigatewayv2_api.plants.id
  route_key = "GET /"
  target = "integrations/${aws_apigatewayv2_integration.get_water_level.id}"
}

resource "aws_lambda_permission" "get_water_level" {
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.get_water_level.arn
  principal     = "apigateway.amazonaws.com"
  source_arn = "${aws_apigatewayv2_stage.plants_prod.execution_arn}/*"
}

output "api_url" {
  value = aws_apigatewayv2_stage.plants_prod.invoke_url
}

Notes

  1. Anyone with the URL will be able to invoke your lambda. If you want access control or rate limiting, you’ll need to add that.

  2. Without the aws_lambda_permission, your API Gateway won’t have permission to invoke the lambda and it’ll 500.

  3. The aws_apigatewayv2_stage is a staging environment (e.g. development, production, test). You must have at least one stage, or else calls to your API will fail with “Not Found”.

  4. The aws_lambda_permission lets any route on your API’s $default stage invoke the lambda. If you want to restrict it to a particular route, you can make the source_arn more specific.

Resources