yaymukund’s weblog ^_^

Overriding packages in NixOS

In NixOS, it’s sometimes desirable to override a package in order to extend or modify its behavior. For example, I override my Neovim to add plugins so they get all the benefits of being in the nix store. Here’s how I do it.

# in configuration.nix
nixpkgs.overlays = [
  (import ./overlays.nix)
];

# in overlays.nix
self: super: {
  neovim-mukund = self.callPackage ./packages/neovim-mukund.nix {};
}

# finally, in packages/neovim-mukund.nix
{ pkgs }:
  neovim.override {
    vimAlias = true;
    viAlias = true;
    configure = {
      packages.mukund-plugins = with pkgs.vimPlugins; {
        start = [
          ale
          fzf-vim
          # ...
        ];
      };
    };
  }

# putting it all together
environment.systemPackages = [
  neovim-mukund
];

Bonus: Installing a single package from main

If you need to install a single package from the main branch but keep the rest of your code on your nix channel (usually the main channel or nixos-unstable), then try this:

# in packages/neovim-mukund.nix
let neovim-master = (import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/master.tar.gz") {}).neovim
in
  environment.systemPackage = [
    neovim-master
  ]

This time, I’m fetching and installing from the master.tar.gz file. This is handy if there’s an update upstream that you want to use immediately. For example, I often use this when Discord releases an update. Nixpkgs usually merges the version bump fairly quickly, but it doesn’t reach the release channels for many days during which Discord is unusable.

References

Rust magic

This is a list of places in Rust where implementing a trait or using a struct affects the syntax of your code. I think of these features as “magical” because using them can change the behavior of very basic Rust code (let, for-in, &, etc.).

What follows is a small list of (hopefully) illustrative examples, and a short epilogue pointing you to more articles if this topic interests you.


Contents


struct Foo {
    text: String,
}

impl Drop for Foo {
    fn drop(&mut self) {
        println!("{} was dropped", self.text);
    }
}

fn main() {
    let mut foo = Some(Foo {
        text: String::from("the old value"),
    });

    // this calls the drop() we wrote above
    foo = None;
}
struct MyCustomStrings(Vec<String>);

impl IntoIterator for MyCustomStrings {
    type Item = String;
    type IntoIter = std::vec::IntoIter<Self::Item>;

    fn into_iter(self) -> Self::IntoIter {
        self.0.into_iter()
    }
}

fn main() {
    let my_custom_strings = MyCustomStrings(vec![
        String::from("one"),
        String::from("two"),
        String::from("three"),
    ]);

    // We can use for-in with our struct
    //
    // prints "one", "two", "three"
    for a_string in my_custom_strings {
        println!("{}", a_string);
    }
}
use std::ops::Deref;

struct Smart<T> {
    inner: T,
}

// You can implement `DerefMut` to coerce exclusive references (&mut).
impl<T> Deref for Smart<T> {
    type Target = T;

    fn deref(&self) -> &Self::Target {
        &self.inner
    }
}

fn main() {
    let text = Smart {
        inner: String::from("what did you say?"),
    };

    // The `impl Deref` lets us invoke the `&str` method
    // `to_uppercase()` on a `&Smart<String>`
    println!("{}", &text.to_uppercase());
}
use std::fmt;

struct Goat {
    name: String,
}

impl fmt::Display for Goat {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "a goat named {}", self.name)
    }
}

fn main() {
    let goat = Goat {
        name: String::from("Yagi"),
    };

    // This invokes our `Display`'s `fmt()`
    println!("{}", goat);
}
#[derive(Clone, Copy, Debug)]
struct Point {
    x: usize,
    y: usize,
}

fn main() {
    let point_a = Point { x: 1, y: 2 };
    let point_b = point_a;

    // point_a is still valid because it was copied rather than moved.
    println!("{:?}", point_a);
}
// Notes:
// * This works very similarly with Option<T>
// * We need to derive(Debug) to use the error in a Result.
//
#[derive(Debug)]
struct SomeError;

fn uh_oh() -> Result<(), SomeError> {
    Err(SomeError)
}

fn main() -> Result<(), SomeError> {
    // The following line desugars to:
    //
    // match uh_oh() {
    //     Ok(v) => v,
    //     Err(SomeError) => return Err(SomeError),
    // }
    //
    uh_oh()?;

    Ok(())
}

Epilogue

When I first started compiling this list, I asked around in the Rust community discord. scottmcm from the Rust Language team introduced me to the concept of lang items. If you search for articles on this topic, you get some fantastic resources:

So what is a lang item? Lang items are a way for the stdlib (and libcore) to define types, traits, functions, and other items which the compiler needs to know about.

Rust Tidbits: What is a Lang Item? by Manish Goregaokar

Not all lang items are magical, but most magical things are lang items. If you want a deeper or more comprehensive understanding, I recommend reading Manish’s article in its entirety.

How to configure API Gateway v2 using Terraform

Here’s how you wire up an AWS lambda into an HTTP API using Terraform and AWS’s API Gateway v2 resources.

When you terraform apply this, it’ll spit out an API URL. You can GET / against that API URL to run your lambda:

resource "aws_iam_role" "plants" {
  name = "iam_plant_api"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "lambda.amazonaws.com"
        ]
      }
    }
  ]
}
EOF
}

# This presumes you have a zip file get_water_level.zip
# which contains a get_water_level.js file which exports
# a `handler` function
resource "aws_lambda_function" "get_water_level" {
  filename = "get_water_level.zip"
  function_name = "get_water_level"
  publish = true
  role = aws_iam_role.plants.arn
  handler = "get_water_level.handler"
  source_code_hash = filebase64sha256("get_water_level.zip")
  runtime = "nodejs12.x"
}

resource "aws_apigatewayv2_api" "plants" {
  name          = "http-plants"
  protocol_type = "HTTP"
}

resource "aws_apigatewayv2_stage" "plants_prod" {
  api_id = aws_apigatewayv2_api.plants.id
  name = "$default"
  auto_deploy = true
}

resource "aws_apigatewayv2_integration" "get_water_level" {
  api_id = aws_apigatewayv2_api.plants.id
  integration_type = "AWS_PROXY"
  integration_method = "POST"
  integration_uri = aws_lambda_function.get_water_level.invoke_arn
}

resource "aws_apigatewayv2_route" "get_water_level" {
  api_id = aws_apigatewayv2_api.plants.id
  route_key = "GET /"
  target = "integrations/${aws_apigatewayv2_integration.get_water_level.id}"
}

resource "aws_lambda_permission" "get_water_level" {
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.get_water_level.arn
  principal     = "apigateway.amazonaws.com"
  source_arn = "${aws_apigatewayv2_stage.plants_prod.execution_arn}/*"
}

output "api_url" {
  value = aws_apigatewayv2_stage.plants_prod.invoke_url
}

Notes

  1. Anyone with the URL will be able to invoke your lambda. If you want access control or rate limiting, you’ll need to add that.

  2. Without the aws_lambda_permission, your API Gateway won’t have permission to invoke the lambda and it’ll 500.

  3. The aws_apigatewayv2_stage is a staging environment (e.g. development, production, test). You must have at least one stage, or else calls to your API will fail with “Not Found”.

  4. The aws_lambda_permission lets any route on your API’s $default stage invoke the lambda. If you want to restrict it to a particular route, you can make the source_arn more specific.

Resources

I decided to catalogue examples of engineers– primarily software developers– being asked to change a name to avoid being racist, sexist, transphobic, ableist, or otherwise bigoted. About half of these examples come from responses to my AskMetaFilter question.

Note: This list is not any sort of exhaustive or representative sample. I offer it as a starting point for anyone interested in reading more about how tech communities respond. If you find any links that should be added, please don’t hesitate to send me an email.



There’s a TON of 2020 Pull Requests about this, so I am going to pick some of them.

2020

Panicking, unsafe, and you

In Jon Gjengset’s Demystifying Unsafe Code talk at Rust NYC, he gives a very interesting example of unsafe code. Here’s the link– please go and watch it– but I’ve transcribed it here along with my paraphrased explanation.

impl<T> Vec<T> {
  /// apply `f` to every element of `us`, and extend `self`
  /// with the result. for example:
  ///
  /// names.extend_map([user1, user2], |u| u.name())
  ///
  fn extend_map<U, F>(&mut self, us: &[U], mut f: F)
  where F: FnMut(&U) -> T {
    // reserve capacity in the Vec all at once, for perf (?)
    self.reserve(us.len());

    // set the length() manually.
    let cur_len = self.len();
    unsafe { self.set_len(cur_len + us.len()) };

    // insert the items by writing to the memory location
    let into = unsafe { self.as_mut_ptr().add(cur_len) };
    for u in us {
      unsafe { std::ptr::write(into, f(u)) }; into += 1;
    }
  }
}

If f panics, then Rust will unwind the stack and drop every item in Vec before dropping the Vec itself. However, since you’ve unsafely set its length with set_len(), it will try to call drop on array indices that you haven’t written to yet! In other words, it will just call garbage memory. Apparently, this is why it can be challenging to write Vec::drain() implementations.

A Few Things Like These (Ippatiyum Cila Vicayankal)

Among birds, I like crows very much.
It's true; it is a thieving creature
tactfully snatching away the eats from the hands of children.
In deed, it is a foolish creature
visiting and perching on the compound wall of the house
and caws at the oddest hours.
Even then
isn't it my friend
who looks at me and calls out to me
in my village where I crawled as a baby and grew up
and also in this city planted from elsewhere?

– Cinnakkapali (Translated by Nirmal Selvamony)

There were three or four crows standing on a branch in the trees outside our balcony, so this poem feels very timely.

Source: Oikopoetrics and Tamil Poetry by Nirmal Selvanomy

Sleeping Alone

Listening
to the neighbor's dog
bark
and bark.
He must
be very big,
his mouth as big
as my head,
his head
as big
as a Macy's Day float,
his corridor throat
commanding me
to hear:
I am one
dog
in the lonely bowl
of night
and there
is so
much
dark.

– Amy Miller, 2018

I want to hug a dog.

How to spy on your Rust code

In the following code, how can you test that Player is making the correct API calls?

struct Player<'a> {
  api: &'a Api,
}

impl<'a> Player<'a> {
  pub fn new(api: &'a Api) -> Player<'a> {
    Player { api }
  }

  pub fn play(&self, song: &str) {
    self.api.sing(song);
  }

  pub fn stop(&self) {
    self.api.shush();
  }
}

The answer? Make it generic!

struct Player<'a, T> {
  api: &'a T,
}

impl<'a, T> Player<'a, T>
where
  T: PlayerApi,
{
  pub fn new(api: &'a T) -> Player<'a, T> {
    Player { api }
  }

  // ...
}

trait PlayerApi {
  // Default trait implementation uses `Api`
  fn sing(&self, url: &str);
  fn shush(&self);
}

impl PlayerApi for Api {
  fn sing(&self, url: &str) {
    Api::sing(self, url)
  }

  fn shush(&self) {
    Api::shush(self)
  }
}

Then you can easily spy on it:

#[cfg(test)]
module Test {
  struct ApiSpy {
    pub invocations: Vec<String>,
    api: Api,
  }

  impl ApiSpy {
    pub fn new() -> ApiSpy {
      ApiSpy { api: Api::New() }
    }
  }

  impl PlayerApi for ApiSpy {
    fn sing(&self, url: &str) {
      self.invocations.push('play');
      self.api.sing(url)
    }
  }

  #[test]
  fn test_play() {
    let api = ApiMock::new();
    let player = Player::new(&api);
    player.play("my_url");
    assert_eq!(api.invocations[0], "play");
  }
}

That’s it!

How can I assert that I passed the correct arguments?

You can store the arguments in the ApiSpy. For example, here’s how I mocked the mpv api and used it in a test.

I don’t want to define a trait for my API just for tests!

Defining a trait for your external APIs is good for reasons beyond testing. But if you still still don’t want to make your API generic, then you could use conditional compilation instead..

What if I don’t want to execute API code in tests?

If you want to mock instead of spy, you have a couple options:

Wireless printers on swaywm

Here’s how you add a wireless printer to Sway. You’ll need the following tools:

  1. CUPS, the “standards-based, open source printing system”
  2. Avahi for wireless support
  3. nss-mdns, so we can refer to the printer as <hostname>.local (e.g. myprinter.local)
# Install the packages
$ yay -S cups avahi nss-mdns

# Start the cups service
$ systemctl start org.cups.cupsd.service

# Ensure you can see the CUPS web interface at localhost:631

# Start the Avahi daemon
$ systemctl start avahi-daemon.service

Now, enable hostname resolution in Avahi by following the instructions on the Arch Wiki.

# Find the printer on the local network
$ sudo lpinfo --include-schemes dnssd -v

# You should see something like:
# network dnssd://Canon%20MG5700%20series._ipp._tcp.local/?uuid=<some uuid>

# Add the printer to CUPS
$ sudo lpadmin \
  -p short-name-eg-canon-md5750 \
  -D "Full Name (e.g. 'Canon MG5750 Laserjet')" \
  -L "Location (e.g. 'Living Room') (Optional)" \
  -v dnssd://Canon%20MG5700...

If you did everything right so far, you should be able to see the printer in the CUPS web interface (localhost:631). If you click on the printer’s name, it should say “Idle, Accepting Jobs.”

Now, you can try to print your document. You can track your print job from the CUPS web interface. If something goes wrong, you can see the error there.

Bonus! Add custom printer drivers

To access advanced features of your printer, you may need to install a custom driver. First, find the relevant drivers by searching the AUR for your printer model. For me, it was the cnijfilter2-mg7700 package.

Once you’ve done that, tell CUPS to assign the custom driver to the printer:

# Find the driver's ppd file
$ lpinfo -m
# It will be something like lsb/usr/canonmg5700.ppd

# And assign it in CUPS
$ sudo lpadmin -p canon-md5750 -m lsb/usr/canonmg5700.ppd

That’s it!

Troubleshooting

My printer in CUPS is listed as disabled or not accepting new jobs

Make sure you run:

$ sudo cupsenable $short_name
$ sudo cupsaccept $short_name

You may be able to do this from the web interface, but I was getting permission denied errors.

My print job fails because it can’t find ghostscript

You need to install ghostscript. In Arch, it’s just yay -S ghostscript.

References

By translating these poems, we aim to memorialize Xu, share some of his excellent literary work, and spread awareness that the harsh conditions, struggles and aspirations of Chinese migrant workers (including but not limited to Foxconn) have not diminished since the more widely-publicized spate of 18 attempted Foxconn suicides in 2010, resulting in 14 deaths. Insiders report that thereafter, although the frequency of suicides decreased (mainly due to Foxconn’s installation of nets making it more difficult for workers to jump from their dormitories, along with the development of workers’ collective resistance), such suicides have continued to the present. Including Xu Lizhi, at least 8 cases have been reported in the media since 2010, but insiders say that many other cases go unreported. We hope that in the future, workers in Foxconn and elsewhere manage to find ways around such companies’ military-style discipline and surveillance, come together, and forge collective paths out of this capitalist world of death, into a world worth living in. Don’t give up!

Source: libcom.org/blog/xulizhi-foxconn-suicide-poetry

On My Deathbed

I want to take another look at the ocean, behold the vastness of tears from half a lifetime
I want to climb another mountain, try to call back the soul that I’ve lost
I want to touch the sky, feel that blueness so light
But I can’t do any of this, so I’m leaving this world
Everyone who’s heard of me
Shouldn’t be surprised at my leaving
Even less should you sigh or grieve
I was fine when I came, and fine when I left.

– Xu Lizhi, 30 September 2014