Hi! I’m David. Here are some things I’ve learned…

January 29, 2024 - Appending content to a database using Notion's API

Sometimes, I want to visualize some of my experimental results, or share them with others. One of the easiest ways to do this is using notion: which is capable of handling a variety of data types, and can be published to a relatively nice looking website, with minimal effort. Step 1: Creating a notion integration To do this, you first have to create a notion integration, which you can do here....

January 29, 2024 · 3 min · 554 words

January 24, 2024 - Using UUID for Unique Identifiers in Python

In lots of situations, we may need a unique identifier for an object, for example, when running a database transformation, we may want to create a unique key for each record, or when creating a database, we might want a unique key for each object in the database. In these situations, I’ve seen a lot of people use some variant of the following code: import random import string def generate_random_unique_identifier(length=10): return ''....

January 24, 2024 · 2 min · 378 words

January 23, 2024 - Detecting COCO Objects with Detectron2 API

Sometimes you need to run an object detector as part of a larger system. One of the best tools for bringing in a pre-trained detector may be the detectron2 framework from Meta. Installation To install detectron2, it’s as easy as a pip install: python -m pip install git+https://github.com/facebookresearch/detectron2.git Detectron2 API It’s amazingly easy to being to extract objects from images using their pre-trained models. First, you get a model from the model zoo:...

January 23, 2024 · 2 min · 265 words

January 22, 2024 - Handling basePath in next.js client components

It turns out that hosting a next.js application in a subdirectory isn’t as easy as it should be. There are two major issues that I’ve run into when doing this: (1) Handling links to local pages and (2) Handling links to static assets. The right way to handle this is to set the basePath in the next.config.js file. This will cause the next.js router to prepend the basePath to all links:...

January 22, 2024 · 2 min · 273 words

January 19, 2024 - Cloudfront Egress is pretty cheap!

This isn’t really a TIL, but it turns out that Amazon provides 1TB of free egress from S3 buckets with a CloudFront CDN front-end (instead of the standard 100GB available for free with S3 alone). To get started, I followed this tutorial – but it assumes a pretty high level of prior understanding of AWS, so if you’ve never used CloudFront, AWS, or S3 before, it might be best to google around for another tutorial....

January 19, 2024 · 1 min · 139 words

January 11, 2024 - Configuring AWS CLI Endpoint URL

So, for a long time, it wasn’t possible to configure the AWS CLI to use a custom endpoint url by default. Since I usually use Wasabi for my S3 storage, I had to use the --endpoint-url flag every time I wanted to use the CLI, pretty annoying, right? Well, turns out at some point Amazon added a set of useful configs to their CLI. To configure the CLI to use a custom endpoint url, you can add the following to your ~/....

January 11, 2024 · 1 min · 102 words

May 12, 2023 - Testing for Multivariate Normality with a Henze-Zirkler Test (In Python)

If you’re just interested in the code, it’s super easy to use pingouin to do this: import numpy as np import pingouin as pg data = np.random.normal(size=(100, 3)) output = pg.multivariate_normality(data, alpha=.05) print(output.hz, output.pval, output.normal) Sometimes, it’s useful to know when a sample looks a lot like a normal distribution. In a single dimension, we can use the function from scipy scipy.stats.normaltest to test whether a sample is normal. This test combines tests from D’Agostino and Pearson which measure the skew and kurtosis of the sample, and report when those differ from a similar normal population....

May 12, 2023 · 2 min · 286 words

December 07, 2022 - How to make a tree with a parent pointer in Rust

I was working on the Advent of Code (day 7) today, which I’m using to learn a bit more about Rust, and while I was able to quickly solve the problem with python (thanks to the fact that python allows references to everything everywhere), I had a hard time solving the problem with Rust, thanks to the inability to keep around references to a “parent” in the tree. This inconvenience is by design: Rust tries to ensure memory safety by forbidding you from doing things that might potentially be unsafe....

December 7, 2022 · 7 min · 1299 words

December 02, 2022 (2) - Encoding secrets for Kubernetes

Kubernetes uses base64 encoded secrets, but it’s important to make sure that when encoding the secrets, you don’t include any extra newlines - tldr; use echo -n instead of echo # Works > echo -n "my secret" | base64 bXkgc2VjcmV0 # Doesn't work > echo "my secret" | base64 bXkgc2VjcmV0Cg== Otherwise, you could be in for a world of pain :)

December 2, 2022 · 1 min · 61 words

December 02, 2022 - How to use python-dotenv to easily manage env vars

So, I had a project recently where I have a bunch of API keys, but I didn’t want to accidentally commit them to a repo (since that would be bad…). But I also wanted the convencience of just in-lining the code, and not have to worry about sourcing environment variables every time I ran the code (since this would inevitably cause annoyances). To solve this, I found a handy tool - python-dotenv, a package which handles all of the complicated bits....

December 2, 2022 · 1 min · 152 words