From 5562e320736812d1ad309cfaf73383512a87858d Mon Sep 17 00:00:00 2001 From: Owen Jacobson Date: Mon, 18 Dec 2023 19:41:51 -0500 Subject: Migrate to Hugo. This is a big and somewhat complicated decision, but the crux of it is this: The _mkdocs_ tool embeds a ton of "I am writing a manual" assumptions about document structure. These assumptions include that there is a single, sitewide TOC, that a top nav bar is broadly an appropriate way to skip around in the document, and numerous others. They serve that use case well, but that's not really what this site _is_, or how I intend it to be approached. I'm trying for something more blog-esque (and deliberately a bit haphazard). Hugo is an experiment. This commit migrates most pages to it, but it does drop a few; this is a convenient excuse to forget items I'd prefer not to continue publishing. --- docs/code/commit-messages.md | 13 -- docs/code/configuring-browser-apps.md | 98 --------- docs/code/index.md | 15 -- docs/code/tools-convention.md | 173 --------------- docs/code/users-rolegraph-privs.md | 88 -------- docs/git/config.md | 22 -- docs/git/detached-sigs.md | 172 --------------- docs/git/index.md | 14 -- docs/git/pull-request-workflow.md | 91 -------- docs/git/scratch.md | 55 ----- docs/git/stop-using-git-pull-to-deploy.md | 98 --------- docs/git/survival.md | 81 ------- docs/hire-me.md | 49 ----- docs/index.md | 25 --- docs/nomic/index.md | 10 - docs/nomic/notes.md | 90 -------- docs/nomic/rules.md | 153 -------------- docs/papers.md | 35 ---- docs/resume.md | 43 ---- .../Satisfactory - Phase 2 Final Report.pdf | Bin 4392718 -> 0 bytes docs/satisfactory/way-of-building.md | 232 --------------------- 21 files changed, 1557 deletions(-) delete mode 100644 docs/code/commit-messages.md delete mode 100644 docs/code/configuring-browser-apps.md delete mode 100644 docs/code/index.md delete mode 100644 docs/code/tools-convention.md delete mode 100644 docs/code/users-rolegraph-privs.md delete mode 100644 docs/git/config.md delete mode 100644 docs/git/detached-sigs.md delete mode 100644 docs/git/index.md delete mode 100644 docs/git/pull-request-workflow.md delete mode 100644 docs/git/scratch.md delete mode 100644 docs/git/stop-using-git-pull-to-deploy.md delete mode 100644 docs/git/survival.md delete mode 100644 docs/hire-me.md delete mode 100644 docs/index.md delete mode 100644 docs/nomic/index.md delete mode 100644 docs/nomic/notes.md delete mode 100644 docs/nomic/rules.md delete mode 100644 docs/papers.md delete mode 100644 docs/resume.md delete mode 100644 docs/satisfactory/Satisfactory - Phase 2 Final Report.pdf delete mode 100644 docs/satisfactory/way-of-building.md (limited to 'docs') diff --git a/docs/code/commit-messages.md b/docs/code/commit-messages.md deleted file mode 100644 index 532b2c6..0000000 --- a/docs/code/commit-messages.md +++ /dev/null @@ -1,13 +0,0 @@ -# Writing Good Commit Messages - -Rule zero: “good” is defined by the standards of the project you're on. Have a look at what the existing messages look like, and try to emulate that first before doing anything else. - -Having said that, here are some principles I've found helpful and broadly applicable. - -* Treat the first line of the message as a one-sentence summary. Most SCM systems have an “overview” command that shows shortened commit messages in bulk, so making the very beginning of the message meaningful helps make those modes more useful for finding specific commits. _It's okay for this to be a “what” description_ if the rest of the message is a “why” description. - -* Fill out the rest of the message with prose outlining why you made the change. Don't reiterate the contents of the change in great detail if you can avoid it: anyone who needs that can read the diff themselves, or reach out to ask for help understanding the change. A good rationale sets context for the problem being solved and addresses the ways the proposed change alters that context. - -* If you use an issue tracker (and you should), include whatever issue-linking notes it supports right at the start of the message, where it'll be visible even in summarized commit logs. If your tracker has absurdly long issue-linking syntax, or doesn't support issue links in commits at all, include a short issue identifier at the front of the message and put the long part somewhere out of the way, such as on a line of its own at the end of the message. - -* If you need rich commit messages (links, lists, and so on), pick one markup language and stick with it. It'll be easier to write useful commit formatters if you only have to deal with one syntax, rather than four. Personally, I use Markdown when I can, or a reduced subset of Markdown, as it's something most developers I interact with will be at least passing familiar with. diff --git a/docs/code/configuring-browser-apps.md b/docs/code/configuring-browser-apps.md deleted file mode 100644 index 8869a61..0000000 --- a/docs/code/configuring-browser-apps.md +++ /dev/null @@ -1,98 +0,0 @@ -# Configuring Browser Apps - -I've found myself in he unexpected situation of having to write a lot of -browser apps/single page apps this year. I have some thoughts on configuration. - -## Why Bother - -* Centralize environment-dependent facts to simplify management & testing -* Make it easy to manage app secrets. - - [@wlonk](https://twitter.com/wlonk) adds: - - > “Secrets”? What this means in a browser app is a bit different. - - Which is unpleasantly true. In a freestanding browser app, a “secret” is only as secret as your users and their network connections choose to make it, i.e., not very secret at all. Maybe that should read “make it easy to manage app _tokens_ and _identities_,” instead. - -* Keep config data & API tokens out of app's source control -* Integration point for external config sources (Aerobatic, Heroku, etc) -* The forces described in [12 Factor App: Dependencies](http://12factor.net/dependencies) and, to a lesser extent, [12 Factor App: Configuration](http://12factor.net/config) apply just as well to web client apps as they do to freestanding services. - -## What Gets Configured - -Yes: - -* Base URLs of backend services -* Tokens and client IDs for various APIs - -No: - -* “Environments” (sorry, Ember folks - I know Ember thought this through carefully, but whole-env configs make it easy to miss settings in prod or test, and encourage patterns like “all devs use the same backends”) - -## Delivering Configuration - -There are a few ways to get configuration into the app. - -### Globals - -```html - - - - -``` - -* Easy to consume: it's just globals, so `window.appConfig.foo` will read them. - * This requires some discipline to use well. -* Have to generate a script to set them. - * This can be a `` tag or a standalone config script loaded with `` in them. That may be an unlikely edge case, but that only makes it a nastier trap for administrators. - - [There are more edge cases](https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify). I strongly suspect that a hazard-free implementation requires a full-blown JS source generator. I had a look at building something out of [escodegen](https://github.com/estools/escodegen) and [estemplate](https://github.com/estools/estemplate), but - - 1. `escodegen`'s node version [doesn't generate browser-safe code](https://github.com/estools/escodegen/issues/298), so string literals with `` or `` in them still break the page, and - - 2. converting javascript values into parse trees to feed to `estemplate` is some seriously tedious code. - -### Data Attributes and Link Elements - -```html - - - - -``` - -* Flat values only. This is probably a good thing in the grand, since flat configurations are easier to reason about and much easier to document, but it makes namespacing trickier than it needs to be for groups of related config values (URL + token for a single service, for example). -* Have to generate the DOM to set them. - * This is only practical given server-side templates or DOM rendering. You can't do this with bare nginx, unless you pre-generate pages at deployment time. - -### Config API Endpoint - -```js -fetch('/config') /* {"FOO_URL": …, "FOO_TOKEN": …} */ - .then(response => response.json()) - .then(json => someConfigurableService); -``` - -* Works even with “dumb” servers (nginx, CloudFront) as the endpoint can be a generated JSON file on disk. If you can generate files, you can generate a JSON endpoint. -* Requires an additional request to fetch the configuration, and logic for injecting config data into all the relevant configurable places in the code. - * This request can't happen until all the app code has loaded. - * It's very tempting to write the config to a global. This produces some hilarious race conditions. - -### Cookies - -See for example [clientconfig](https://github.com/henrikjoreteg/clientconfig): - -```js -var config = require('clientconfig'); -``` - -* Easy to consume given the right tools; tricky to do right from scratch. -* Requires server-side support to send the correct cookie. Some servers will allow you to generate the right cookie once and store it in a config file; others will need custom logic, which means (effectively) you need an app server. -* Cookies persist and get re-sent on subsequent requests, even if the server stops delivering config cookies. Client code has to manage the cookie lifecycle carefully (clientconfig does this automatically) -* Size limits constrain how much configuration you can do. diff --git a/docs/code/index.md b/docs/code/index.md deleted file mode 100644 index e0efe37..0000000 --- a/docs/code/index.md +++ /dev/null @@ -1,15 +0,0 @@ -# Code - -Pieces of code and code-adjacent work, with or without exposition, that don't quite fit into the library ecosystem, but which I enjoyed writing. - -* [Using the shell environment as a project tool](tools-convention.md) — A general and less tooling-intensive approach to automating routine project tasks. - -* [A Users, Roles & Privileges Scheme Using Graphs](users-rolegraph-privs.md) — An SQL schema and associated queries for handling permissions when roles can nest arbitrarily. - -* [Configuring Browser Apps](configuring-browser-apps.md) — Notes on the available techniques for delivering runtime configuration to code running in a user's browser, and the tradeoffs involved. - -* [Writing Good Commit Messages](commit-messages.md) — A style guide. - -* [Some collected advice about Git](../git/index.md) — Not the source control tool we want, but definitely the source control tool we've got, and I think we should make the best of it. - -I also maintain a [Github account](https://github.com/ojacobson/) for more substantial projects. diff --git a/docs/code/tools-convention.md b/docs/code/tools-convention.md deleted file mode 100644 index 4d1175e..0000000 --- a/docs/code/tools-convention.md +++ /dev/null @@ -1,173 +0,0 @@ -# The `tools` directory - -## Background - -It's fairly common for the authors and maintainers of a software project to accumulate routine tasks as the project proceeds. Common examples of this: - -* Building or packaging the code to prepare it for delivery. -* Running the test suite. -* Compiling project documentation. -* Deploying services to the cloud. - -Many projects also sprout more esoteric, project-specific tasks - as I write this, I'm working on one where "delete an AMI from AWS" is one of the things I've had to automate. - -Many software projects also end up relying on a build tool - `make`, classically, but there are myriad others. These are tools that are designed to solve specific classes of routine problems (in fact, generally the ones listed above). - -I've often seen these two factors converge in projects in the form of a build tool script that contains extensive task-specific support, or in some cases where a build tool such as `make` is pressed into service as a generic task service and does none of the work it was designed to do in the first place. - -This has a couple of consequences. - -First, the configuration language for these tools is invariably unique to the tool. `make` is the least prone to this, but `Makefile`s require careful understanding of both shell and make variables, Make's unique implicit rule system, and the huge number of variables and rules implied by the use of `make` in the first place. - -Adding utility tasks to projects using scons, Gradle, Cargo, Maven, or any other, more modern build tool involves similar assumptions and often a very specific programming language and execution model. Even tools like Gradle, which use a general-purpose language, impose a unqiue dialect of that language which is intended to facilitate the kinds of build tasks the tool is designed around. - -So, writing and understanding the utility tasks that exist requires specific skills, which are orthogonal to the skills needed to understand or implement the project itself. - -Second, it creates a dependency on that tool for those tasks. They can only be executed automatically when the tool is available, and can only be executed by hand (with all that entails), or reimplemented, when the tool is absent. Often, build tools are not expected in the end-user environment. Projects either end up having to different approaches for tasks that might run in both development and end-user environements versus tasks that run in only one of the two. - -## The `tools` approach - -To address those consequences, I've started putting routine tasks into my project as shell scripts, in a `tools` directory. - -The shell is a widely-deployed, general-purpose automation tool that handles many routine administration tasks well. For software which will be delivered to a unix-like end environment, developers can be confident that a shell will be present, and the skills to write basic shell scripts are generally already part of a unix programmers' repertoire. - -These scripts follow a few principles to ensure that they remain manageable: - -* Always use `set -e`, use `set -o pipefail` if appropriate. A tool that needs to run multiple steps should abort, naturally and automatically, if any of those steps fail. - -* Usually chdir to the project's root directory (generally with `cd "$(dirname "$0")/.."`). Normally, manipulating the cwd makes shell scripts brittle and hard to deploy, but in this use case, it ensures that the script can reliably manipulate the project's code. This step can be skipped if the tool doesn't do anything with the project, but it's usually worth keeping it anyways just for consistency. - -* Include a block comment near the top with a usage summary: - - ## tools/example - ## tools/example FILENAME - ## - ## Runs the examples. If FILENAME is provided, then only the - ## example in FILENAME will actually be executed. - -* Minimize the use of arguments. Ideally, use only either no arguments, one argument, or an arbitrary number of arguments. Under no circumstances use options or flags. - -* Minimize the use of shell constructs other than running commands. This means minimizing the use of `if`, `for`, `while`, and `set` constructs, shell functions, `$()` and backticks, complex pipelines, `eval`, reading from the environment, and otherwise. This is not a hard and fast rule, and the underlying intent is to keep the tool script as straightforward and as easily-understood as possible. - -* Where a tool needs to run multiple steps, generally break those steps out into independent tool scripts. Processes are cheap; clarity is valuable. - -More generally, the intention is to encapsulate the commands needed to perform routine tasks, not to be complex software in their own right. - -If you're using a project-specific shell environment (with [`direnv`](https://direnv.net) or similar), add `tools` to your `PATH` so that you can run tool scripts from anywhere in the project. Being able to type `build` and have the project build, regardless of where you're looking at that moment, is very helpful. For `direnv`'s `.envrc` this can be done using `PATH_add tools`. - -## Tradeoffs - -No approach is perfect, and this approach has its own consequences: - -* Tasks can't participate in dependency-based ordering on their own. They're just shell scripts. This can be troubling if a tool does something that needs to take place during ordinary builds. - -* There's always the temptation to Add Features to tools scripts, and it takes steady effort and careful judgment as to how to do so while hewing to the goals of the approach. For example, with Docker, I've had situations where I end up with two tools with nearly-identical Docker command lines (and thus code duplication to maintain) because I preferred to avoid adding an optional debug mode to an existing tools script. - -* If you're not using `direnv`, the tools directory is only easily available to your shell if your `pwd` is the project root. This is awkward, and `sh` derivatives don't provide any convenient way to "search upwards" for commands. - -* Tool scripts need to take care to set their own `pwd` appropriately, which can take some thinking. What directory is appropriate depends on what the tool script needs to accomplish, so the right answer isn't always obvious. A convention of "always cd to the project root" covers most cases, but there are exceptions where the user's `pwd` or some other directory may be more appropriate. - -## Examples - -This site is built using two tools. `tools/build`: - -```bash -#!/bin/bash -e - -cd "$(dirname "$0")/.." - -## tools/build -## -## Converts the content in docs/ into a deployable website in site/ - -exec mkdocs build -``` - -And `tools/publish`: - -```bash -#!/bin/bash -e - -cd "$(dirname "$0")/.." - -## tools/publish -## -## Publishes site/ to the S3 bucket hosting grimoire.ca - -exec aws s3 sync --delete site/ s3://grimoire.ca/ -``` - -Another project I'm working on has a tool to run all CI checks - `tools/check`: - -```bash -#!/bin/bash -ex - -# tools/checks -# -# Runs all code checks. If you're automating testing, call this rather than -# invoking a test command directly; if you're adding a test command, add it here -# or to one of the tools called from this script. - -tools/check-tests -tools/check-lints -tools/check-dependencies -``` - -This, in turn, runs the following: - -* `tools/check-tests`: - - #!/bin/bash -ex - - # tools/check-tests - # - # Checks that the code in this project passes incorrectness checks. - - cargo build --locked --all-targets - cargo test - -* `tools/check-lints`: - - #!/bin/bash -ex - - # tools/check-lints - # - # Checks that the code in this project passes style checks. - - cargo fmt -- --check - cargo clippy -- --deny warnings - -* `tools/check-dependencies`: - - #!/bin/bash -ex - - # check-dependencies - # - # Checks that the dependencies in this project are all in use. - - cargo udeps --locked --all-targets - -Yet another project uses a tool to run tests against a container: - -```bash -#!/bin/bash -e - -cd "$(dirname "$0")/.." - -## tools/check -## -## Runs tests on the docker image. - -VERSION="$(detect-version)" - -py.test \ - --image-version "${VERSION}" -``` - -## Alternatives - -* NAME's [`just`](https://github.com/casey/just) stores shell snippets in a project-wide `Justfile`. - - This allows for reliable generation of project-specific help and usage information from the `Justfile` and allows `just foo` to work from any directory in the project without project-specific shell configuration. - - On the other hand, it's another tool developers would need to install to get started on a project. `Justfile` entries can't be run without `just` (a problem `make` also has). diff --git a/docs/code/users-rolegraph-privs.md b/docs/code/users-rolegraph-privs.md deleted file mode 100644 index d36f430..0000000 --- a/docs/code/users-rolegraph-privs.md +++ /dev/null @@ -1,88 +0,0 @@ -# A Users, Roles & Privileges Scheme Using Graphs - -The basic elements: - -* Every agent that can interact with a system is represented by a **user**. -* Every capability the system has is authorized by a distinct **privilege**. -* Each user has a list of zero or more **roles**. - * Roles can **imply** further roles. This relationship is transitive: if role A implies role B, then a member of role A is a member of role B; if role B also implies role C, then a member of role A is also a member of role C. It helps if the resulting role graph is acyclic, but it's not necessary. - * Roles can **grant** privileges. - -A user's privileges are the union of the privileges granted by the transitive closure of their roles. - -```sql -create table "user" ( - username varchar - primary key - -- credentials &c -); - -create table role ( - name varchar - primary key -); - -create table role_member ( - role varchar - not null - references role, - member varchar - not null - references "user", - primary key (role, member) -); - -create table role_implies ( - role varchar - not null - references role, - implied_role varchar - not null -); - -create table privilege ( - privilege varchar - primary key -); - -create table role_grants ( - role varchar - not null - references role, - privilege varchar - not null - references privilege, - primary key (role, privilege) -); -``` - -If your database supports recursive CTEs, this schema can be queried in one shot, since we can have the database do all the graph-walking along roles: - -```sql -with recursive user_roles (role) AS ( - select - role - from - role_member - where - member = 'SOME USERNAME' - union - select - implied_role as role - from - user_roles - join role_implies on - user_roles.role = role_implies.role -) -select distinct - role_grants.privilege as privilege -from - user_roles - join role_grants on - user_roles.role = role_grants.role -order by privilege; -``` - -If not, you'll need to pull the entire graph into memory and manipulate it there: this schema doesn't give you any easy handles to identify only the roles transitively included in the role of interest, and repeatedly querying for each step of the graph requires an IO roundtrip at each step, burning whole milliseconds along the way. - -Realistic use cases should have fairly simple graphs: elemental privileges are grouped into concrete roles, which are in turn grouped into abstracted roles (by department, for example), which are in turn granted to users. If the average user is in tens of roles and has hundreds of privileges, the entire dataset fits in memory, and PostgreSQL performs well. In PostgreSQL, the above schema handles ~10k privileges and ~10k roles with randomly-generated graph relationships in around 100ms on my laptop, which is pretty slow but not intolerable. Perverse cases (interconnected total subgraphs, deeply-nested linear graphs) can take absurd time but do not reflect any likely permissions scheme. diff --git a/docs/git/config.md b/docs/git/config.md deleted file mode 100644 index 456d580..0000000 --- a/docs/git/config.md +++ /dev/null @@ -1,22 +0,0 @@ -# git-config Settings You Want - -Git comes with some fairly [lkml](http://www.tux.org/lkml/)-specific configuration defaults. You should fix this. All of the items below can be set either for your entire login account (`git config --global`) or for a specific repository (`git config`). - -Full documentation is under `git help config`, unless otherwise stated. - -* `git config user.name 'Your Full Name'` and `git config user.email 'your-email@example.com'`, obviously. Git will remind you about this if you forget. - -* `git config merge.defaultToUpstream true` - causes an unqualified `git merge` to merge the current branch's configured upstream branch, rather than being an error. This makes `git merge` much more consistent with `git rebase`, and as the two tools fill very similar workflow niches, it's nice to have them behave similarly. - -* `git config rebase.autosquash true` - causes `git rebase -i` to parse magic comments created by `git commit --squash=some-hash` and `git commit --fixup=some-hash` and reorder the commit list before presenting it for further editing. See the descriptions of “squash” and “fixup” in `git help rebase` for details; autosquash makes amending commits other than the most recent easier and less error-prone. - -* `git config branch.autosetupmerge always` - newly-created branches whose start point is a branch (`git checkout master -b some-feature`, `git branch some-feature origin/develop`, and so on) will be configured to have the start point branch as their upstream. By default (with `true` rather than `always`) this only happens when the start point is a remote-tracking branch. - -* `git config rerere.enabled true` - enable “reuse recorded resolution.” The `git help rerere` docs explain it pretty well, but the short version is that git can record how you resolve conflicts during a “test” merge and reuse the same approach when resolving the same conflict later, in a “real” merge. - -## For advanced users - -A few things are nice when you're getting started, but become annoying when -you no longer need them. - -* `git config advice.detachedHead` - if you already understand the difference between having a branch checked out and having a commit checked out, and already understand what “detached head” means, the warning on every `git checkout ...some detached thing...` isn't helping anyone. This is also useful repositories used for deployment, where specific commits (from tags, for example) are regularly checked out. diff --git a/docs/git/detached-sigs.md b/docs/git/detached-sigs.md deleted file mode 100644 index 0b8a386..0000000 --- a/docs/git/detached-sigs.md +++ /dev/null @@ -1,172 +0,0 @@ -# Notes Towards Detached Signatures in Git - -Git supports a limited form of object authentication: specific object categories in Git's internal model can have GPG signatures embedded in them, allowing the authorship of the objects to be verified using GPG's underlying trust model. Tag signatures can be used to verify the authenticity and integrity of the _snapshot associated with a tag_, and the authenticity of the tag itself, filling a niche broadly similar to code signing in binary distribution systems. Commit signatures can be used to verify the authenticity of the _snapshot associated with the commit_, and the authorship of the commit itself. (Conventionally, commit signatures are assumed to also authenticate either the entire line of history leading to a commit, or the diff between the commit and its first parent, or both.) - -Git's existing system has some tradeoffs. - -* Signatures are embedded within the objects they sign. The signature is part of the object's identity; since Git is content-addressed, this means that an object can neither be retroactively signed nor retroactively stripped of its signature without modifying the object's identity. Git's distributed model means that these sorts of identity changes are both complicated and easily detected. - -* Commit signatures are second-class citizens. They're a relatively recent addition to the Git suite, and both the implementation and the social conventions around them continue to evolve. - -* Only some objects can be signed. While Git has relatively weak rules about workflow, the signature system assumes you're using one of Git's more widespread workflows by limiting your options to at most one signature, and by restricting signatures to tags and commits (leaving out blobs, trees, and refs). - -I believe it would be useful from an authentication standpoint to add "detached" signatures to Git, to allow users to make these tradeoffs differently if desired. These signatures would be stored as separate (blob) objects in a dedicated `refs` namespace, supporting retroactive signatures, multiple signatures for a given object, "policy" signatures, and authentication of arbitrary objects. - -The following notes are partially guided by Git's one existing "detached metadata" facility, `git notes`. Similarities are intentional; divergences will be noted where appropriate. Detached signatures are meant to interoperate with existing Git workflow as much as possible: in particular, they can be fetched and pushed like any other bit of Git metadata. - -A detached signature cryptographically binds three facts together into an assertion whose authenticity can be checked by anyone with access to the signatory's keys: - -1. An object (in the Git sense; a commit, tag, tree, or blob), -2. A policy label, and -3. A signatory (a person or agent making the assertion). - -These assertions can be published separately from or in tandem with the objects they apply to. - -## Policies - -Taking a hint from Monotone, every signature includes a "policy" identifying how the signature is meant to be interpreted. Policies are arbitrary strings; their meaning is entirely defined by tooling and convention, not by this draft. - -This draft uses a single policy, `author`, for its examples. A signature under the `author` policy implies that the signatory had a hand in the authorship of the designated object. (This is compatible with existing interpretations of signed tags and commits.) (Authorship under this model is strictly self-attested: you can claim authorship of anything, and you cannot assert anyone else's authorship.) - -The Monotone documentation suggests a number of other useful policies related to testing and release status, automated build results, and numerous other factors. Use your imagination. - -## What's In A Signature - -Detached signatures cover the disk representation of an object, as given by - -```bash -git cat-file -``` - -For most of Git's object types, this means that the signed content is plain text. For `tree` objects, the signed content is the awful binary representation of the tree, _not_ the pretty representation given by `git ls-tree` or `git show`. - -Detached signatures include the "policy" identifier in the signed content, to prevent others from tampering with policy choices via `refs` hackery. (This will make more sense momentarily.) The policy identifier is prepended to the signed content, terminated by a zero byte (as with Git's own type identifiers, but without a length field as length checks are performed by signing and again when the signature is stored in Git). - -To generate the _complete_ signable version of an object, use something equivalent to the following shell snippet: - -```bash -# generate-signable POLICY TYPE SHA1 -function generate-signable() { - printf '%s\0' "$1" - git cat-file "$2" "$3" -} -``` - -(In the process of writing this, I discovered how hard it is to get Unix's C-derived shell tools to emit a zero byte.) - -## Signature Storage and Naming - -We assume that a userid will sign an object at most once. - -Each signature is stored in an independent blob object in the repository it applies to. The signature object (described above) is stored in Git, and its hash recorded in `refs/signatures///`. - -```bash -# sign POLICY TYPE SHA1 FINGERPRINT -function sign() { - local SIG_HASH=$( - generate-signable "$@" | - gpg --batch --no-tty --sign -u "$4" | - git hash-object --stdin -w -t blob - ) - git update-ref "refs/signatures/$1/$3/$4" -} -``` - -Stored signatures always use the complete fingerprint to identify keys, to minimize the risk of colliding key IDs while avoiding the need to store full keys in the `refs` naming hierarchy. - -The policy name can be reliably extracted from the ref, as the trailing part has a fixed length (in both path segments and bytes) and each ref begins with a fixed, constant prefix `refs/signatures/`. - -## Signature Verification - -Given a signature ref as described above, we can verify and authenticate the signature and bind it to the associated object and policy by performing the following check: - -1. Pick apart the ref into policy, SHA1, and key fingerprint parts. -2. Reconstruct the signed body as above, using the policy name extracted from the ref. -3. Retrieve the signature from the ref and combine it with the object itself. -4. Verify that the policy in the stored signature matches the policy in the ref. -5. Verify the signature with GPG: - - ```bash - # verify-gpg POLICY TYPE SHA1 FINGERPRINT - verify-gpg() { - { - git cat-file "$2" "$3" - git cat-file "refs/signatures/$1/$3/$4" - } | gpg --batch --no-tty --verify - } - ``` - -6. Verify the key fingerprint of the signing key matches the key fingerprint in the ref itself. - -The specific rules for verifying the signature in GPG are left up to the user to define; for example, some sites may want to auto-retrieve keys and use a web of trust from some known roots to determine which keys are trusted, while others may wish to maintain a specific, known keyring containing all signing keys for each policy, and skip the web of trust entirely. This can be accomplished via `git-config`, given some work, and via `gpg.conf`. - -## Distributing Signatures - -Since each signature is stored in a separate ref, and since signatures are _not_ expected to be amended once published, the following refspec can be used with `git fetch` and `git push` to distribute signatures: - -``` -refs/signatures/*:refs/signatures/* -``` - -Note the lack of a `+` decoration; we explicitly do not want to auto-replace modified signatures, normally; explicit user action should be required. - -## Workflow Notes - -There are two verification workflows for signatures: "static" verification, where the repository itself already contains all the refs and objects needed for signature verification, and "pre-receive" verification, where an object and its associated signature may be being uploaded at the same time. - -_It is impractical to verify signatures on the fly from an `update` hook_. Only `pre-receive` hooks can usefully accept or reject ref changes depending on whether the push contains a signature for the pushed objects. (Git does not provide a good mechanism for ensuring that signature objects are pushed before their subjects.) Correctly verifying object signatures during `pre-receive` regardless of ref order is far too complicated to summarize here. - -## Attacks - -### Lies of Omission - -It's trivial to hide signatures by deleting the signature refs. Similarly, anyone with access to a repository can delete any or all detached signatures from it without otherwise invalidating the signed objects. - -Since signatures are mostly static, sites following the recommended no-force policy for signature publication should only be affected if relatively recent signatures are deleted. Older signatures should be available in one or more of the repository users' loca repositories; once created, a signature can be legitimately obtained from anywhere, not only from the original signatory. - -The signature naming protocol is designed to resist most other forms of assertion tampering, but straight-up omission is hard to prevent. - -### Unwarranted Certification - -The `policy` system allows any signatory to assert any policy. While centralized signature distribution points such as "release" repositories can make meaningful decisions about which signatures they choose to accept, publish, and propagate, there's no way to determine after the fact whether a policy assertion was obtained from a legitimate source or a malicious one with no grounds for asserting the policy. - -For example, I could, right now, sign an `all-tests-pass` policy assertion for the Linux kernel. While there's no chance on Earth that the LKML team would propagate that assertion, if I can convince you to fetch signatures from my repository, you will fetch my bogus assertion. If `all-tests-pass` is a meaningful policy assertion for the Linux kernel, then you will have very few options besides believing that I assert that all tests have passed. - -### Ambigiuous Policy - -This is an ongoing problem with crypto policy systems and user interfaces generally, but this design does _nothing_ to ensure that policies are interpreted uniformly by all participants in a repository. In particular, there's no mechanism described for distributing either prose or programmatic policy definitions and checks. All policy information is out of band. - -Git already has ambiguity problems around commit signing: there are multiple ways to interpret a signature on a commit: - -1. I assert that this snapshot and commit message were authored as described in this commit's metadata. (In this interpretation, the signature's authenticity guarantees do _not_ transitively apply to parents.) - -2. I assert that this snapshot and commit message were authored as described in this commit's metadata, based on exactly the parent commits described. (In this interpretation, the signature's authenticity guarantees _do_ transitively apply to parents. This is the interpretation favoured by XXX LINK HERE XXX.) - -3. I assert that this _diff_ and commit message was authored as described in this commit's metadata. (No assertions about the _snapshot_ are made whatsoever, and assertions about parentage are barely sensical at all. This meshes with widespread, diff-oriented policies.) - -### Grafts and Replacements - -Git permits post-hoc replacement of arbitrary objects via both the grafts system (via an untracked, non-distributed file in `.git`, though some repositories distribute graft lists for end-users to manually apply) and the replacements system (via `refs/replace/`, which can optionally be fetched or pushed). The interaction between these two systems and signature verification needs to be _very_ closely considered; I've not yet done so. - -Cases of note: - -* Neither signature nor subject replaced - the "normal" case -* Signature not replaced, subject replaced (by graft, by replacement, by both) -* Signature replaced, subject not replaced -* Both signature and subject replaced - -It's tempting to outright disable `git replace` during signing and verification, but this will have surprising effects when signing a ref-ish instead of a bare hash. Since this is the _normal_ case, I think this merits more thought. (I'm also not aware of a way to disable grafts without modifying `.git`, and having the two replacement mechanisms treated differently may be dangerous.) - -### No Signed Refs - -I mentioned early in this draft that Git's existing signing system doesn't support signing refs themselves; since refs are an important piece of Git's workflow ecosystem, this may be a major omission. Unfortunately, this proposal doesn't address that. - -## Possible Refinements - -* Monotone's certificate system is key+value based, rather than label-based. This might be useful; while small pools of related values can be asserted using mutually exclusive policy labels (whose mutual exclusion is a matter of local interpretation), larger pools of related values rapidly become impractical under the proposed system. - - For example, this proposal would be inappropriate for directly asserting third-party authorship; the asserted author would have to appear in the policy name itself, exposing the user to a potentially very large number of similar policy labels. - -* Ref signing via a manifest (a tree constellation whose paths are ref names and whose blobs sign the refs' values). Consider cribbing DNSSEC here for things like lightweight absence assertions, too. - -* Describe how this should interact with commit-duplicating and commit-rewriting workflows. diff --git a/docs/git/index.md b/docs/git/index.md deleted file mode 100644 index 5cfc4d8..0000000 --- a/docs/git/index.md +++ /dev/null @@ -1,14 +0,0 @@ -# Collected Advice about Git - -* [git-config Settings You Want](config.md) — Git is highly configurable, and the defaults have gotten drastically better over the years, but there are still some non-default behaviours that I've found make life better. - -* [Notes Towards Detached Signatures in Git](detached-sigs.md) — An idea I had, but never fully developed, for implementing after-the-fact object signing on top of Git. This was based on a similar feature in Monotone, which I'd found very effective for annotating commits on the fly. - -* [Life With Pull Requests](pull-request-workflow.md) — Some notes I made while getting up to speed with pull requests to help my team come to grips with the workflows. - -* [Git Is Not Magic](scratch.md) — An exploration of Git's on-disk data structures and the design choices taken very early in Git's existence. - -* [Stop using `git pull` for deployment!](stop-using-git-pull-to-deploy.md) — Describing the least-painful way to use Git as a deployment tool I had worked out, circa 2014. Written in an aversarial style as a response to repeated ”why don't we just”s that, while well-intentioned, came from an incomplete understanding of what `git pull` does. - -* [Git Survival Guide](survival.md) — Some words of caution about Git, `git`'s preferred workflows, and various recoverable mistakes. - diff --git a/docs/git/pull-request-workflow.md b/docs/git/pull-request-workflow.md deleted file mode 100644 index 2d3e2c0..0000000 --- a/docs/git/pull-request-workflow.md +++ /dev/null @@ -1,91 +0,0 @@ -# Life With Pull Requests - -I've been party to a number of discussions with folks contributing to pull-request-based projects on Github (and other hosts, but mostly Github). Because of Git's innate flexibility, there are lots of ways to work with pull requests. Here's mine. - -I use a couple of naming conventions here that are not stock `git`: - -* `origin` is the repository to which you _publish_ proposed changes, and - -* `upstream` is the repository from which you receive ongoing development, and - which will receive your changes if they are accepted. - -## One-time setup - -Do these things once, when starting out on a project. Keep the results around for later. - -I'll be referring to the original project repository as `upstream` and pretending its push URL is `UPSTREAM-URL` below. In real life, the URL will often be something like `git@github.com:someguy/project.git`. - -### Fork the project - -Use the repo manager's forking tool to create a copy of the project in your own namespace. This generally creates your copy with a bunch of useless tat; feel free to ignore all of this, as the only purpose of this copy is to provide somewhere for _you_ to publish _your_ changes. - -We'll be calling this repository `origin` later. Assume it has a URL, which I'll abbreviate `ORIGIN-URL`, for `git push` to use. - -(You can leave this step for later, but if you know you're going to do it, why not get it out of the way?) - -### Clone the project and configure it - -You'll need a clone locally to do work in. Create one from `origin`: - -```bash -git clone ORIGIN-URL some-local-name -``` - -While you're here, `cd` into it and add the original project as a remote: - -```bash -cd some-local-name -git remote add upstream UPSTREAM-URL -``` - -## Feature process - -Do these things for each feature you work on. To switch features, just use `git checkout my-feature`. - -### Create a new feature branch locally - -We use `upstream`'s `master` branch here, so that your feature includes all of `upstream`'s state initially. We also need to make sure our local cache of `upstream`'s state is correct: - -```bash -git fetch upstream -git checkout upstream/master -b my-feature -``` - -### Do work - -If you need my help here, stop now. - -### Integrate upstream changes - -If you find yourself needing something that's been added upstream, use _rebase_ to integrate it to avoid littering your feature branch with “meaningless” merge commits. - -```bash -git checkout my-feature -git fetch upstream -git rebase upstream/master -``` - -### Publish your branch - -When you're “done,” publish your branch to your personal repository: - -```bash -git push origin my-feature -``` - -Then visit your copy in your repo manager's web UI and create a pull request for `my-feature`. - -### Integrating feedback - -Very likely, your proposed changes will need work. If you use history-editing to integrate feedback, you will need to use `--force` when updating the branch: - -```bash -git push --force origin my-feature -``` - -This is safe provided two things are true: - -1. **The branch has not yet been merged to the upstream repo.** -2. You are only force-pushing to your fork, not to the upstream repo. - -Generally, no other users will have work based on your pull request, so force-pushing history won't cause problems. diff --git a/docs/git/scratch.md b/docs/git/scratch.md deleted file mode 100644 index e912a1e..0000000 --- a/docs/git/scratch.md +++ /dev/null @@ -1,55 +0,0 @@ -# Git Is Not Magic - -I'm bored. Let's make a git repository out of whole cloth. - -Git repos are stored in .git: - - fakegit$ mkdir .git - -They have a “symbolic ref” (which are text files, see [`man -git-symbolic-ref`](http://jk.gs/git-symbolic-ref.html)) named `HEAD`, pointing -to the currently checked-out branch. Let's use `master`. Branches are refs -under `refs/heads` (see [`man git-branch`](http://jk.gs/git-branch.html)): - - fakegit ((unknown))$ echo 'ref: refs/heads/master' > .git/HEAD - -The have an object database and a refs database, both of which are simple -directories (see [`man -gitrepository-layout`](http://jk.gs/gitrepository-layout.html) and [`man -gitrevisions`](http://jk.gs/gitrevisions.html)). Let's also enable the reflog, -because it's a great safety net if you use history-editing tools in git: - - fakegit ((ref: re...))$ mkdir .git/refs .git/objects .git/logs - fakegit (master #)$ - -Now `__git_ps1`, at least, is convinced that we have a working git repository. -Does it work? - - fakegit (master #)$ echo 'Hello, world!' > hello.txt - fakegit (master #)$ git add hello.txt - fakegit (master #)$ git commit -m 'Initial commit' - [master (root-commit) 975307b] Initial commit - 1 file changed, 1 insertion(+) - create mode 100644 hello.txt - - fakegit (master)$ git log - commit 975307ba0485bff92e295e3379a952aff013c688 - Author: Owen Jacobson - Date: Wed Feb 6 10:07:07 2013 -0500 - - Initial commit - -[Eeyup](https://www.youtube.com/watch?v=3VwVpaWUu30). - ------ - -Should you do this? **Of course not.** Anywhere you could run these commands, -you could instead run `git init` or `git clone`, which set up a number of -other structures, including `.git/config` and any unusual permissions options. -The key part here is that a directory's identity as “a git repository” is -entirely a function of its contents, not of having been blessed into being by -`git` itself. - -You can infer a lot from this: for example, you can infer that it's “safe” to -move git repositories around using FS tools, or to back them up with the same -tools, for example. This is not as obvious to everyone as you might hope; people diff --git a/docs/git/stop-using-git-pull-to-deploy.md b/docs/git/stop-using-git-pull-to-deploy.md deleted file mode 100644 index 078c95b..0000000 --- a/docs/git/stop-using-git-pull-to-deploy.md +++ /dev/null @@ -1,98 +0,0 @@ -# Stop using `git pull` for deployment! - -## The problem - -* You have a Git repository containing your project. -* You want to “deploy” that code when it changes. -* You'd rather not download the entire project from scratch for each - deployment. - -## The antipattern - -“I know, I'll use `git pull` in my deployment script!” - -Stop doing this. Stop teaching other people to do this. It's wrong, and it -will eventually lead to deploying something you didn't want. - -Deployment should be based on predictable, known versions of your code. -Ideally, every deployable version has a tag (and you deploy exactly that tag), -but even less formal processes, where you deploy a branch tip, should still be -deploying exactly the code designated for release. `git pull`, however, can -introduce new commits. - -`git pull` is a two-step process: - -1. Fetch the current branch's designated upstream remote, to obtain all of the - remote's new commits. -2. Merge the current branch's designated upstream branch into the current - branch. - -The merge commit means the actual deployed tree might _not_ be identical to -the intended deployment tree. Local changes (intentional or otherwise) will be -preserved (and merged) into the deployment, for example; once this happens, -the actual deployed commit will _never_ match the intended commit. - -`git pull` will approximate the right thing “by accident”: if the current -local branch (generally `master`) for people using `git pull` is always clean, -and always tracks the desired deployment branch, then `git pull` will update -to the intended commit exactly. This is pretty fragile, though; many git -commands can cause the local branch to diverge from its upstream branch, and -once that happens, `git pull` will always create new commits. You can patch -around the fragility a bit using the `--ff-only` option, but that only tells -you when your deployment environment has diverged and doesn't fix it. - -## The right pattern - -Quoting [Sitaram Chamarty](http://gitolite.com/the-list-and-irc/deploy.html): - -> Here's what we expect from a deployment tool. Note the rule numbers -- -> we'll be referring to some of them simply by number later. -> -> 1. All files in the branch being deployed should be copied to the -> deployment directory. -> -> 2. Files that were deleted in the git repo since the last deployment -> should get deleted from the deployment directory. -> -> 3. Any changes to tracked files in the deployment directory after the -> last deployment should be ignored when following rules 1 and 2. -> -> However, sometimes you might want to detect such changes and abort if -> you found any. -> -> 4. Untracked files in the deploy directory should be left alone. -> -> Again, some people might want to detect this and abort the deployment. - -Sitaram's own documentation talks about how to accomplish these when -“deploying” straight out of a bare repository. That's unwise (not to mention -impractical) in most cases; deployment should use a dedicated clone of the -canonical repository. - -I also disagree with point 3, preferring to keep deployment-related changes -outside of tracked files. This makes it much easier to argue that the changes -introduced to configure the project for deployment do not introduce new bugs -or other surprise features. - -My deployment process, given a dedicated clone at `$DEPLOY_TREE`, is as -follows: - - cd "${DEPLOY_TREE}" - git fetch --all - git checkout --force "${TARGET}" - # Following two lines only required if you use submodules - git submodule sync - git submodule update --init --recursive - # Follow with actual deployment steps (run fabric/capistrano/make/etc) - -`$TARGET` is either a tag name (`v1.2.1`) or a remote branch name -(`origin/master`), but could also be a commit hash or anything else Git -recognizes as a revision. This will detach the head of the `$DEPLOY_TREE` -repository, which is fine as no new changes should be authored in this -repository (so the local branches are irrelevant). The warning Git emits when -`HEAD` becomes detached is unimportant in this case. - -The tracked contents of `$DEPLOY_TREE` will end up identical to the desired -commit, discarding local changes. The pattern above is very similar to what -most continuous integration servers use when building from Git repositories, -for much the same reason. diff --git a/docs/git/survival.md b/docs/git/survival.md deleted file mode 100644 index 60d1b62..0000000 --- a/docs/git/survival.md +++ /dev/null @@ -1,81 +0,0 @@ -# Git Survival Guide - -I think the `git` UI is pretty awful, and encourages using Git in ways that -will screw you. Here are a few things I've picked up that have saved my bacon. - -* You will inevitably need to understand Git's “internals” to make use of it - as an SCM tool. Accept this early. If you think your SCM tool should not - expose you to so much plumbing, [don't](http://mercurial.selenic.com) - [use](http://bazaar.canonical.com) [Git](http://subversion.apache.org). - * Git weenies will claim that this plumbing is what gives Git all of its - extra power. This is true; it gives Git the power to get you out of - situations you wouldn't be in without Git. -* `git log --graph --decorate --oneline --color --all` -* Run `git fetch` habitually. Stale remote-tracking branches lead to sadness. -* `git push` and `git pull` are **not symmetric**. `git push`'s - opposite operation is `git fetch`. (`git pull` is equivalent to `git fetch` - followed by `git merge`, more or less). -* [Git configuration values don't always have the best defaults](config). -* The upstream branch of `foo` is `foo@{u}`. The upstream branch of your - checked-out branch is `HEAD@{u}` or `@{u}`. This is documented in `git help - revisions`. -* You probably don't want to use a merge operation (such as `git pull`) to - integrate upstream changes into topic branches. The resulting history can be - very confusing to follow, especially if you integrate upstream changes - frequently. - * You can leave topic branches “real” relatively safely. You can do - a test merge to see if they still work cleanly post-integration without - actually integrating upstream into the branch permanently. - * You can use `git rebase` or `git pull --rebase` to transplant your - branch to a new, more recent starting point that includes the changes - you want to integrate. This makes the upstream changes a permanent part - of your branch, just like `git merge` or `git pull` would, but generates - an easier-to-follow history. Conflict resolution will happen as normal. -* Example test merge, using `origin/master` as the upstream branch and `foo` - as the candidate for integration: - - git fetch origin - git checkout origin/master -b test-merge-foo - git merge foo - # run tests, examine files - git diff origin/master..HEAD - - To discard the test merge, delete the branch after checking out some other - branch: - - git checkout foo - git branch -D test-merge-foo - - You can combine this with `git rerere` to save time resolving conflicts in - a later “real,” permanent merge. - -* You can use `git checkout -p` to build new, tidy commits out of a branch - laden with “wip” commits: - - git fetch - git checkout $(git merge-base origin/master foo) -b foo-cleaner-history - git checkout -p foo -- paths/to/files - # pick out changes from the presented patch that form a coherent commit - # repeat 'git checkout -p foo --' steps for related files to build up - # the new commit - git commit - # repeat 'git checkout -p foo --' and 'git commit' steps until no diffs remain - - * Gotcha: `git checkout -p` will do nothing for files that are being - created. Use `git checkout`, instead, and edit the file if necessary. - Thanks, Git. - * Gotcha: The new, clean branch must diverge from its upstream branch - (`origin/master`, in the example above) at exactly the same point, or - the diffs presented by `git checkout -p foo` will include chunks that - revert changes on the upstream branch since the “dirty” branch was - created. The easiest way to find this point is with `git merge-base`. - -## Useful Resources - -That is, resoures that can help you solve problems or understand things, not -resources that reiterate the man pages for you. - -* Sitaram Chamarty's [git concepts - simplified](http://sitaramc.github.com/gcs/) -* Tv's [Git for Computer - Scientists](http://eagain.net/articles/git-for-computer-scientists) diff --git a/docs/hire-me.md b/docs/hire-me.md deleted file mode 100644 index 5b46bba..0000000 --- a/docs/hire-me.md +++ /dev/null @@ -1,49 +0,0 @@ -# Hire Me - -I'm always interested in hearing from people and organizations that I can help, whether that means coming in for a few days to talk about end-to-end testing or joining your organization full-time to help turn an idea into reality. - -I live in and around Toronto. I am more than happy to work remotely, and I can probably help your organization learn to integrate remote work if it doesn't already know how. - -## For Fun - -I regularly mentor people new to programming, teaching them how to craft working systems. This is less about teaching people to write code and more about teaching them why we care about source control, how to think about configuration, how to and why to automate testing, and how to think about software systems and data flow at a higher level. I strongly believe that software development needs a formal apprenticeship program, and mentoring has done a lot to validate that belief. - -## Ada Support (2022-2023) - -As an engineering manager at Ada, I lead a team of engineers to build an internal platform for chat applications. Our goal was to enable growth into new markets, by making it possible to extend Ada's product in novel ways based on the needs of new customers. - -During my tenure the team set out on building an event processing system based on Kafka, intended to decouple the company's in-house chat frontend from the response generation services and to become the common interface for other customer service platforms, so that Ada could intervene to assist customers via email, phone, and other services our customers might already be using. - -## Heroku/Salesforce (2015-2022) - -In my time with Heroku (and with Salesforce, Heroku's parent organization), I've contributed to the delivery and operation of services that let developers bring their ideas to life on the internet, both as a developer and as a manager. I've been involved in maintaining and expanding existing features, exploring and developing new products, and in cultivating my peers and my team as people and as developers. - -* As an engineering manager (2018 to 2022), I've been responsible for building and supporting an effective, unified team across multiple continents. Moving into management was motivated by a desire to act as a force multiplier, which I've brought to life through coaching, process management, facilitating ongoing discussions about the direction and health of the team, and through actively being involved in my reports' progress as developers. - - Each of the teams I've worked on has been responsible for both developing and operating a mature product, delivered at scale via the internet, to a diverse range of customers. My team has served everyone from single developers working on hobby projects all the way up to billion-dollar enterprises who selected Heroku as their platform of choice for the enterprise. - - Those teams have been comprised of everything from unique, hard-to-replace domain experts to interns on their first outing. In order to organize and lead, I take a disciplined approach to communication, emphasizing clarity and empathy. I provide as much flexibility around scheduling as the organization can spare, to enable my teams to work when they're at their best. And, as my team's ambassador to the organization, I gather up the disparate and sometimes highly-speculative streams of work in flight to present as a coherent roadmap against organizational goals. - - I've also been responsible for the huge range of work that Salesforce expects from line management, including performance management and coaching, compensation planning, hiring and interviewing, balancing on-call schedules against burnout and retention risks, and skilling up the team to handle the parts of all of these processes that can be delegated, while keeping their time free to do the things they're good at as much as is possible. - -* As a lead developer (2015-2018), I worked on the [Heroku build system](https://devcenter.heroku.com/articles/git), which ingests code from end users and deploys that code to applications running on the Heroku platform. As part of that work, we implemented a number of features to control abuse, support language-specific features and needs, and to develop [new ways to deploy code](https://devcenter.heroku.com/articles/build-docker-images-heroku-yml) to Heroku. - -## FreshBooks (2009-2014) - -During the five years I was with the company, it grew from a 20-person one-room organization to a healthy, growing two-hundred-person technology company. As an early employee, I had my hand in many, many projects and helped the development team absorb the massive cultural changes that come with growth, while also building a SaaS product that let others realize their dreams. Some highlights: - -* As the team's database administrator, I was responsible for balancing concerns about reliability and availability against the need to deliver new services and functional improvements for customers. Alongside the operations team, I handled capacity planning, reliability, outage planning, and performance monitoring. Alongside the development team, I was responsible for designing processes tooling and providing advice on the most effective ways to use MySQL to accomplish their goals. - -* As an ops toolsmith, I worked extensively on deployment automation and standardizing process for internal services. I created a standard development VM to ensure developers had an environment consistent with reality, I automated packaging and rollout to testing servers, I explored options around platform-as-a-service products to look for fit, and more. As part of this work, I built training materials and ran sessions to teach other developers how to think like a sysadmin, covering Linux, Puppet, virtualization, and other topics. - -## Riptown Media (2006-2009) - -Riptown Media was an software development company tasked with building and operating a suite of gambling systems for a single client. I was brought on board as a Java developer, and rapidly expanded my role to encompass other fields. - -* As the primary developer for poker-room back office and anti-fraud tools, I worked with the customer support and business intelligence teams to better understand their daily needs and frustrations, so that I could turn those into meaningful improvements to their tools and processes. These improvements, in turn, lead to measurable changes in the frequency and length of customer support calls, in fraud rates, and in the percieved value of internal customer intelligence. - -* As a lead developer, my team put together the server half of an in-house casino gaming platform. We worked in tight collaboration with the client team, in-house and third-party testers, and interaction designers, and delivered our first game in under six months. Our platform was meant to reduce our reliance on third-party “white label” games vendors; internally, it was a success. Our game received zero customer-reported defects during its initial run. - -## Contact Me - -You can get in touch by email at owen@grimoire.ca. I'd love to hear from you. diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 81104c3..0000000 --- a/docs/index.md +++ /dev/null @@ -1,25 +0,0 @@ -# Owen Jacobson - -* [Hire Me](hire-me.md). - - I've been a professional software developer since the early 2000s and an enthusiastic amateur even longer, and a manager of developers since 2019. I'm also deeply interested in organizational dynamics and group consensus: software, like ourselves, lives in a society, and both serves the needs of and serves to help shape that society. - -* [Code](code/index.md). - - I program computers. I have done so all of my adult life, and expect to do so as long as I can string concepts together. Like many lifelong programmers, I periodically write up interesting things I've developed, collaborated on, or run across. - - My larger projects are on [Github](https://github.com/ojacobson/). - -* [Papers of Note](papers.md). - - Computer science and development-adjacent papers and academic works I encourage people to read. - -* [A More Satisfactory Way of Building](satisfactory/way-of-building.md). - - What's a nerd without a factory game fascination? - - Satisfactory is a game about destroying the ecosystem of a pristine world, for profit. You are put in the shoes of a faceless corporate drone, given tools with rapidly increasing agency for extract and exploiting natural resources, and situated within an incentive structure that rewards you for doing so as efficiently as possible. - - It's great fun. - -The best way to contact me is [by email](mailto:owen@grimoire.ca), but [I'm present in many places](https://wheretofind.me/@owen). diff --git a/docs/nomic/index.md b/docs/nomic/index.md deleted file mode 100644 index 8e28e6e..0000000 --- a/docs/nomic/index.md +++ /dev/null @@ -1,10 +0,0 @@ -# Nomic - -[Nomic](https://legacy.earlham.edu/~peters/nomic.htm) is a game invented in 1982 by Peter Suber, as an appendix to his PhD thesis _The Paradox of Self-Amendment_. In Nomic, the primary move available to the players is to change the rules of the game in a structured way. Nomic itself was intended as a minimalist study of procedural law, but it has been played very successfully by many groups over the years. - -I first played Nomic through [Agora](http://www.dfw.net/~ccarroll/agora/), a long-running Nomic of a heavily procedural bent (as opposed to variants like BlogNomic, that have developed in much more whimsical directions). I've found the game, and the communities that have sprung up around the game, deeply fascinating as a way to examine how groups reach consensus and exercise decisions. - -I briefly experimented with the notion of running a procedural Nomic - a mini-Agora - via Github, and produced two documents: - -* [Notes Towards Initial Rules for a Github Nomic](notes.md) -* [Github Nomic Rules](rules.md) diff --git a/docs/nomic/notes.md b/docs/nomic/notes.md deleted file mode 100644 index 7ff43e5..0000000 --- a/docs/nomic/notes.md +++ /dev/null @@ -1,90 +0,0 @@ -# Notes Towards Initial Rules for a Github Nomic - -This document is not part of the rules of a Nomic, and is present solely as a guide to the design of [this initial ruleset](rules.md), for play on Github. It should be removed before the game starts, and at no time should it be consulted to guide gameplay directly. - -Peter Suber's [Nomic](http://legacy.earlham.edu/~peters/writing/nomic.htm) is a game of rule-making for one or more players. For details on the rationale behind the game and the reasons the game might be interesting, see Suber's own description. - -# Changes from Suber's Rules - -## Format - -I've marked up Suber's rules into Markdown, one of Github's “native” text markup formats. This highly-structured format produces quite readable results when viewed through the Github website, and allows useful things like HTML links that point to specific rules. - -I've also made some diff-friendliness choices around the structure of those Markdown documents. For want of a better idea, the source documents are line-broken with one sentence per line, so that diffs naturally span whole sentences rather than arbitrarily-wrapped text (or unwrapped text). Since Github automatically recombines sequences of non-blank lines into a single HTML paragraph, the rendering on the web site is still quite readable. - -I have not codified this format in the rules themselves. - -## Asynchrony - -In its original form, Nomic is appropriate for face-to-face play. The rules assume that it is practical for the players to identify one another using out-of-game context, and that it is practical for the players to take turns. Each player is expected to wait indefinitely (or, more likely, to apply non-game social pressure) if the preceding player takes inordinately long to complete their turn. Similarly, Judgement interrupts the flow of game play and brings turns to a stop. - -This Nomic is to be played on Github, and the players are _not_ likely to be present simultaneously, or to be willing to wait indefinitely. - -It's possible for Suber's original Nomic rules to be amended, following themselves, into a form suitable for asynchronous play. This has happened several times: for examples, see [Agora](http://agoranomic.org/) and [BlogNomic](http://blognomic.com/), though there are a multitude of others. However, this process of amendment takes _time_, and, starting from Suber's initial rules, would require a period of one-turn-at-a-time rule-changes before the game could be played more naturally in the Github format. This period is not very interesting, and is incredibly demanding of the initial players' attention spans. - -In the interests of preserving the players' time, I have modified Suber's initial ruleset to replace sequential play with a simple asynchronous model of play. In summary: - -* Every player can begin a turn at any time, even during another player's (or players') turn, so long as they aren't already taking a turn. -* Actions can be resolved in any order, depending on which proposals players choose to vote on, and in what order. -* The initial rules allow for players to end their turns without gathering every vote, once gameplay has proceeded far enough for non-unanimous votes to be possible. - -I have attempted to leave the rules as close to Suber's original rules as possible otherwise while implementing this change to the initial ruleset. I have faith that the process of playing Nomic will correct any deficiencies, or, failing that, will clearly identify where these changes break the game entirely. - -I have, as far as I am able, emulated Suber's preference for succinctness over thoroughness, and resisted the urge to fix or clarify rules even where defects seem obvious to me. In spite of my temptation to remove it, I have even left the notion of “winning” intact. - -## Rule-numbering - -The intent of this Nomic is to explore the suitability of Github's suite of tools for proposing, reviewing, and accepting changes to a corpus of text are suitable for self-governed rulemaking processes, as modelled by Nomic. Note that this is a test of Github, not of Git: it is appropriate and intended that the players rely on non-Git elements of Github's workflow (issues, wiki pages, Github Pages, and so on), and similarly it is appropriate and intended that the authentic copy of the game in play is the Github project hosting it, not the Git repo the project contains, and certainly not forks of the project or other clones of the repository. - -To support this intention, I have re-labelled the initial rules with negative numbers, rather than digits, so that proposals can be numbered starting from 1 without colliding with existing rules, and so that they can be numbered by their Pull Requests and Github issue numbers. (A previous version of these rules used Roman numerals for the initial rules. However, correctly accounting for the priority of new rules over initial rules, following Suber, required more changes than I was comfortable making to Suber's ruleset.) I have made it explicit in these initial rules that Github, not the players, assigns numbers to proposals. This is the only rule which mentions Github by name. I have not explicitly specified that the proposals should be implemented through pull requests; this is an intentional opportunity for player creativity. - -## Projects & Ideas - -A small personal collection of other ideas to explore: - -### Repeal or replace the victory criteria entirely - -“Winning” is not an objective I'm personally interested in, and Suber's race to 200 points by popularity of proposal is structurally quite dull. If the game is to have a victory condition, it should be built from the ground up to meet the players' motivations, rather than being retrofitted onto the points-based system. - -### Codify the use of Git commits, rather than prose, for rules-changes - -This is unstated in this ruleset, despite being part of my intention for playing. So is the relationship between proposals and the Git repository underpinning the Github project hosting the game. - -### Clarify the immigration and exit procedures - -The question of who the players _are_, or how one becomes a player, is left intentionally vague. In Suber's original rules, it appears that the players are those who are engaged in playing the game: tautological on paper, but inherently obvious by simple observation of the playing-space. - -On Github, the answer to this question may not be so simple. A public repository is _visible_ to anyone with an internet connection, and will accept _proposed_ pull requests (and issue reports) equally freely. This suggests that either everyone is, inherently, a player, or that player-ness is somehow a function of engaging with the game. I leave it to the players to resolve this situation to their own satisfaction, but my suggestion is to track player-ness using repository collaborators or organization member accounts. - -### Figure out how to regulate the use of Github features - -Nomic, as written, largely revolves around sequential proposals. That's fine as far as it goes, but Github has a very wide array of project management features - and that set of features changes over time, outside the control of the players, as Github roll out improvements (and, sometimes, break things). - -Features of probable interest: - -* The `gh-pages` branch and associated web site. -* Issue and pull request tagging and approval settings. -* Third-party integrations. -* Whether to store non-rule state, as such arises, in the repository, or in the wiki, or elsewhere. -* Pull request reactions and approvals. -* The mutability of most Github features. - -### Expand the rules-change process to permit a single proposal to amend many rules - -This is a standard rules patch, as Suber's initial rule-set is (I believe intentionally) very restrictive. - -This may turn out to be less relevant on Github, if players are allowed to submit turns in rapid succession with themselves. - -### Transition from immediate amendment to a system of sessions - -Why not? Parliamentary procedure is fun, right? - -In an asynchronous environment, the discrete phases of a session system (where proposals are gathered, then debated, then voted upon, then enacted as a unit) might be a better fit for the Github mode of play. - -### Evaluate other models of proposal vetting besides majority vote - -Github open source projects regularly have a small core team of maintainers supporting a larger group of users. Is it possible to mirror this structure in Nomic? Is it wise to do so? - -I suspect this is only possible with an inordinately large number of players, but Github could, at least in principle, support that number of players. - -Note that this is a fairly standard Nomic passtime. diff --git a/docs/nomic/rules.md b/docs/nomic/rules.md deleted file mode 100644 index 912b4c8..0000000 --- a/docs/nomic/rules.md +++ /dev/null @@ -1,153 +0,0 @@ -# Github Nomic Rules - -## Immutable Rules - -### Rule -216. - -All players must always abide by all the rules then in effect, in the form in which they are then in effect. The rules in the Initial Set are in effect whenever a game begins. The Initial Set consists of rules -216 through -201 (immutable) and rules -112 through -101 (mutable). - -### Rule -215. - -Initially, rules -216 through -201 are immutable, and rules -112 through -101 are mutable. Rules subsequently enacted or transmuted (that is, changed from immutable to mutable or vice versa) may be immutable or mutable regardless of their numbers, and rules in the Initial Set may be transmuted regardless of their numbers. - -### Rule -214. - -A rule-change is any of the following: - -1. the enactment, repeal, or amendment of a mutable rule; - -2. the enactment, repeal, or amendment of an amendment of a mutable rule; or - -3. the transmutation of an immutable rule into a mutable rule or vice versa. - -(Note: This definition implies that, at least initially, all new rules are mutable; immutable rules, as long as they are immutable, may not be amended or repealed; mutable rules, as long as they are mutable, may be amended or repealed; any rule of any status may be transmuted; no rule is absolutely immune to change.) - -### Rule -213. - -All rule-changes proposed in the proper way shall be voted on. They will be adopted if and only if they receive the required number of votes. - -### Rule -212. - -Every player is an eligible voter. - -### Rule -211. - -All proposed rule-changes shall be written down before they are voted on. If they are adopted, they shall guide play in the form in which they were voted on. - -### Rule -210. - -No rule-change may take effect earlier than the moment of the completion of the vote that adopted it, even if its wording explicitly states otherwise. No rule-change may have retroactive application. - -### Rule -209. - -Each proposed rule-change shall be given a number for reference. The numbers shall be assigned by Github, so that each rule-change proposed in the proper way shall receive the a distinct integer from all prior proposals, whether or not the proposal is adopted. - -If a rule is repealed and reenacted, it receives the number of the proposal to reenact it. If a rule is amended or transmuted, it receives the number of the proposal to amend or transmute it. If an amendment is amended or repealed, the entire rule of which it is a part receives the number of the proposal to amend or repeal the amendment. - -### Rule -208. - -Rule-changes that transmute immutable rules into mutable rules may be adopted if and only if the vote is unanimous among the eligible voters. Transmutation shall not be implied, but must be stated explicitly in a proposal to take effect. - -### Rule -207. - -In a conflict between a mutable and an immutable rule, the immutable rule takes precedence and the mutable rule shall be entirely void. For the purposes of this rule a proposal to transmute an immutable rule does not "conflict" with that immutable rule. - -### Rule -206. - -If a rule-change as proposed is unclear, ambiguous, paradoxical, or destructive of play, or if it arguably consists of two or more rule-changes compounded or is an amendment that makes no difference, or if it is otherwise of questionable value, then the other players may suggest amendments or argue against the proposal before the vote. A reasonable time must be allowed for this debate. The proponent decides the final form in which the proposal is to be voted on and, unless the Judge has been asked to do so, also decides the time to end debate and vote. - -### Rule -205. - -The state of affairs that constitutes winning may not be altered from achieving _n_ points to any other state of affairs. The magnitude of _n_ and the means of earning points may be changed, and rules that establish a winner when play cannot continue may be enacted and (while they are mutable) be amended or repealed. - -### Rule -204. - -A player always has the option to forfeit the game rather than continue to play or incur a game penalty. No penalty worse than losing, in the judgment of the player to incur it, may be imposed. - -### Rule -203. - -There must always be at least one mutable rule. The adoption of rule-changes must never become completely impermissible. - -### Rule -202. - -Rule-changes that affect rules needed to allow or apply rule-changes are as permissible as other rule-changes. Even rule-changes that amend or repeal their own authority are permissible. No rule-change or type of move is impermissible solely on account of the self-reference or self-application of a rule. - -### Rule -201. - -Whatever is not prohibited or regulated by a rule is permitted and unregulated, with the sole exception of changing the rules, which is permitted only when a rule or set of rules explicitly or implicitly permits it. - -## Mutable Rules - -### Rule -112. - -A player may begin a turn at any time that suits them. Turns may overlap: one player may begin a turn while another player's is in progress. No player may begin a turn unless all of their previous turns have ended. - -All players begin with zero points. - -### Rule -111. - -One turn consists of two parts in this order: - -1. proposing one rule-change and having it voted on, and - -2. scoring the proposal and adding that score to the proposing player's score. - -A proposal is scored by taking the proposal number, adding nine to it, multiplying the result by the fraction of favourable votes the proposal received, and rounding that result to the nearest integer. - -(This scoring system yields a number between 0 and 10 for the first proposal, with the upper limit increasing by one for each new proposal; more points are awarded for more popular proposals.) - -### Rule -110. - -A rule-change is adopted if and only if the vote in favour is unanimous among the eligible voters. If this rule is not amended before each player has had two turns, it automatically changes to require only a simple majority. - -If and when rule-changes can only be adopted unanimously, the voting may be ended as soon as an opposing vote is counted. If and when rule-changes can be adopted by simple majority, the voting may be ended as soon as a simple majority in favour or a simple majority against is counted. - -### Rule -109. - -If and when rule-changes can be adopted without unanimity, the players who vote against winning proposals shall receive 10 points each. - -### Rule -108. - -An adopted rule-change takes full effect at the moment of the completion of the vote that adopted it. - -### Rule -107. - -When a proposed rule-change is defeated, the player who proposed it loses 10 points. - -### Rule -106. - -Each player always has exactly one vote. - -### Rule -105. - -The winner is the first player to achieve 200 (positive) points. - -### Rule -104. - -At no time may there be more than 25 mutable rules. - -### Rule -103. - -If two or more mutable rules conflict with one another, or if two or more immutable rules conflict with one another, then the rule with the lowest ordinal number takes precedence. - -If at least one of the rules in conflict explicitly says of itself that it defers to another rule (or type of rule) or takes precedence over another rule (or type of rule), then such provisions shall supersede the numerical method for determining precedence. - -If two or more rules claim to take precedence over one another or to defer to one another, then the numerical method again governs. - -### Rule -102. - -If players disagree about the legality of a move or the interpretation or application of a rule, then the player moving may ask any other player to be the Judge and decide the question. Disagreement for the purposes of this rule may be created by the insistence of any player. This process is called invoking Judgment. - -When Judgment has been invoked, no player may begin his or her turn without the consent of a majority of the other players. - -The Judge's Judgment may be overruled only by a unanimous vote of the other players taken before the next turn is begun. If a Judge's Judgment is overruled, then the Judge may ask any player other than the moving player, and other than any player who has already been the Judge for the question, to become the new Judge for the question, and so on, except that no player is to be Judge during his or her own turn or during the turn of a team-mate. - -Unless a Judge is overruled, one Judge settles all questions arising from the game until the next turn is begun, including questions as to his or her own legitimacy and jurisdiction as Judge. - -New Judges are not bound by the decisions of old Judges. New Judges may, however, settle only those questions on which the players currently disagree and that affect the completion of the turn in which Judgment was invoked. All decisions by Judges shall be in accordance with all the rules then in effect; but when the rules are silent, inconsistent, or unclear on the point at issue, then the Judge shall consider game-custom and the spirit of the game before applying other standards. - -### Rule -101. - -If the rules are changed so that further play is impossible, or if the legality of a move cannot be determined with finality, or if by the Judge's best reasoning, not overruled, a move appears equally legal and illegal, then the first player unable to complete a turn is the winner. - -This rule takes precedence over every other rule determining the winner. diff --git a/docs/papers.md b/docs/papers.md deleted file mode 100644 index 37d2d84..0000000 --- a/docs/papers.md +++ /dev/null @@ -1,35 +0,0 @@ -# Papers of Note - -* Perlman, Radia (1985). “[An Algorithm for Distributed Computation of a Spanning Tree in an Extended LAN][1]”. ACM SIGCOMM Computer Communication Review. 15 (4): 44–53. doi:10.1145/318951.319004. - -* [The related Algorhyme][2], also by Perlman. - -* Richard Barbrook, Andy Cameron. "[The Californian Ideology][11]" (1995). An important read to understand how the current internet came to be what it is, and about the beliefs of the people who made it that way. - -* Guy Lewis Steele, Jr.. “[Debunking the 'Expensive Procedure Call' Myth, or, Procedure Call Implementations Considered Harmful, or, Lambda: The Ultimate GOTO][3]”. MIT AI Lab. AI Lab Memo AIM-443. October 1977. - -* [What Every Computer Scientist Should Know About Floating-Point Arithmetic][4], by David Goldberg, published in the March, 1991 issue of Computing Surveys. Copyright 1991, Association for Computing Machinery, Inc. - -* [RFC 1925][5]. - -* [Regular Expression Matching Can Be Simple And Fast][10], Russ Cox's empirical research into degenerate cases in common regular expression implementations and a proposed implementation based on Thomson's NFA construction. - -* [The above-cited Thomson NFA paper][6] on regular expressions. - -* [The Eight Fallacies of Distributed Computing][7]. - -* [HAKMEM][8] is another good one. It's _dense_ but rewarding. - -* Kahan, William (January 1965), “[Further remarks on reducing truncation errors][9]”, Communications of the ACM, 8 (1): 40, doi:10.1145/363707.363723 - -[1]: https://www.researchgate.net/publication/238778689_An_Algorithm_for_Distributed_computation_of_a_Spanning_Tree_in_an_Extended_LAN -[2]: http://etherealmind.com/algorhyme-radia-perlman/ -[3]: https://dspace.mit.edu/bitstream/handle/1721.1/5753/AIM-443.pdf -[4]: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html -[5]: https://www.ietf.org/rfc/rfc1925.txt -[6]: https://www.fing.edu.uy/inco/cursos/intropln/material/p419-thompson.pdf -[7]: http://wiki.c2.com/?EightFallaciesOfDistributedComputing -[8]: http://w3.pppl.gov/~hammett/work/2009/AIM-239-ocr.pdf -[9]: https://dl.acm.org/citation.cfm?id=363723 -[10]: https://swtch.com/~rsc/regexp/regexp1.html -[11]: https://www.alamut.com/subj/ideologies/pessimism/califIdeo_I.html diff --git a/docs/resume.md b/docs/resume.md deleted file mode 100644 index 1a8bfd4..0000000 --- a/docs/resume.md +++ /dev/null @@ -1,43 +0,0 @@ -# Owen Jacobson - - - 416 554 2902 - -## Ada Support (2022-2023) - -As an engineering manager at Ada, I lead a team of engineers to build an internal platform for chat applications. Our goal was to enable growth into new markets, by making it possible to extend Ada's product in novel ways based on the needs of new customers. - -During my tenure the team set out on building an event processing system based on Kafka, intended to decouple the company's in-house chat frontend from the response generation services and to become the common interface for other customer service platforms, so that Ada could intervene to assist customers via email, phone, and other services our customers might already be using. - -## Heroku/Salesforce (2015-2022) - -In my time with Heroku (and with Salesforce, Heroku's parent organization), I've contributed to the delivery and operation of services that let developers bring their ideas to life on the internet, both as a developer and as a manager. I've been involved in maintaining and expanding existing features, exploring and developing new products, and in cultivating my peers and my team as people and as developers. - -* As an engineering manager (2018 to 2022), I've been responsible for building and supporting an effective, unified team across multiple continents. Moving into management was motivated by a desire to act as a force multiplier, which I've brought to life through coaching, process management, facilitating ongoing discussions about the direction and health of the team, and through actively being involved in my reports' progress as developers. - - Each of the teams I've worked on has been responsible for both developing and operating a mature product, delivered at scale via the internet, to a diverse range of customers. My team has served everyone from single developers working on hobby projects all the way up to billion-dollar enterprises who selected Heroku as their platform of choice for the enterprise. - - Those teams have been comprised of everything from unique, hard-to-replace domain experts to interns on their first outing. In order to organize and lead, I take a disciplined approach to communication, emphasizing clarity and empathy. I provide as much flexibility around scheduling as the organization can spare, to enable my teams to work when they're at their best. And, as my team's ambassador to the organization, I gather up the disparate and sometimes highly-speculative streams of work in flight to present as a coherent roadmap against organizational goals. - - I've also been responsible for the huge range of work that Salesforce expects from line management, including performance management and coaching, compensation planning, hiring and interviewing, balancing on-call schedules against burnout and retention risks, and skilling up the team to handle the parts of all of these processes that can be delegated, while keeping their time free to do the things they're good at as much as is possible. - -* As a lead developer (2015-2018), I worked on the [Heroku build system](https://devcenter.heroku.com/articles/git), which ingests code from end users and deploys that code to applications running on the Heroku platform. As part of that work, we implemented a number of features to control abuse, support language-specific features and needs, and to develop [new ways to deploy code](https://devcenter.heroku.com/articles/build-docker-images-heroku-yml) to Heroku. - -## FreshBooks (2009-2014) - -During the five years I was with the company, it grew from a 20-person one-room organization to a healthy, growing two-hundred-person technology company. As an early employee, I had my hand in many, many projects and helped the development team absorb the massive cultural changes that come with growth, while also building a SaaS product that let others realize their dreams. Some highlights: - -* As the team's database administrator, I was responsible for balancing concerns about reliability and availability against the need to deliver new services and functional improvements for customers. Alongside the operations team, I handled capacity planning, reliability, outage planning, and performance monitoring. Alongside the development team, I was responsible for designing processes tooling and providing advice on the most effective ways to use MySQL to accomplish their goals. - -* As an ops toolsmith, I worked extensively on deployment automation and standardizing process for internal services. I created a standard development VM to ensure developers had an environment consistent with reality, I automated packaging and rollout to testing servers, I explored options around platform-as-a-service products to look for fit, and more. As part of this work, I built training materials and ran sessions to teach other developers how to think like a sysadmin, covering Linux, Puppet, virtualization, and other topics. - -## Riptown Media (2006-2009) - -Riptown Media was an software development company tasked with building and operating a suite of gambling systems for a single client. I was brought on board as a Java developer, and rapidly expanded my role to encompass other fields. - -* As the primary developer for poker-room back office and anti-fraud tools, I worked with the customer support and business intelligence teams to better understand their daily needs and frustrations, so that I could turn those into meaningful improvements to their tools and processes. These improvements, in turn, lead to measurable changes in the frequency and length of customer support calls, in fraud rates, and in the percieved value of internal customer intelligence. - -* As a lead developer, my team put together the server half of an in-house casino gaming platform. We worked in tight collaboration with the client team, in-house and third-party testers, and interaction designers, and delivered our first game in under six months. Our platform was meant to reduce our reliance on third-party “white label” games vendors; internally, it was a success. Our game received zero customer-reported defects during its initial run. - -## For Fun - -I regularly mentor people new to programming, teaching them how to craft working systems. This is less about teaching people to write code and more about teaching them why we care about source control, how to think about configuration, how to and why to automate testing, and how to think about software systems and data flow at a higher level. I strongly believe that software development needs a formal apprenticeship program, and mentoring has done a lot to validate that belief. diff --git a/docs/satisfactory/Satisfactory - Phase 2 Final Report.pdf b/docs/satisfactory/Satisfactory - Phase 2 Final Report.pdf deleted file mode 100644 index 26f519c..0000000 Binary files a/docs/satisfactory/Satisfactory - Phase 2 Final Report.pdf and /dev/null differ diff --git a/docs/satisfactory/way-of-building.md b/docs/satisfactory/way-of-building.md deleted file mode 100644 index eddf279..0000000 --- a/docs/satisfactory/way-of-building.md +++ /dev/null @@ -1,232 +0,0 @@ -# A More Satisfactory Way of Building - -_Correction: This essay originally attributed the AWESOME Sink to Update 4._ - -In 2020, [Aphyr] wrote _[A Satisfactory Way of Building]_, exploring the the idea of pattern languages through the forces and consequences imposed by Satisfactory. In the intervening two years, the developers have released two major updates to the game, adding entirely new systems, adding new ways of responding to the forces and tensions in the game, and tweaking (or in some cases, completely reworking) the existing systems and tools. - -[Aphyr]: https://aphyr.com/ -[A Satisfactory Way of Building]: https://aphyr.com/posts/351-a-satisfactory-way-of-building - -Changing contexts and forces lead to changing solutions, and while much of _Building_ remains good advice today, some of it deserves revision. Those changes also create new forces, which give rise to new patterns not contemplated in Aphyr's original piece. - -## What's New - -The original essay was written against the background of Update 3, which included rail transport, Hypertube people-movers, fluids, and nuclear power, as well as extensive dry-goods automation. The two updates ([Update 4], [Update 5]) since have added features that fall into a few categories. - -[Update 4]: https://satisfactory.fandom.com/wiki/Patch_0.4.0.0 -[Update 5]: https://satisfactory.fandom.com/wiki/Patch_0.5.0.0 - -First, we have the features that require significant changes to the pattern language or entirely new patterns: - -* **Zooping** allows the player to build linear runs of up to ten foundations, walls, railings, or ramps in a straight line horizontally, vertically, or sometimes diagnoally. - -* **Floor holes** allow for vertical logistics at any point in the factory, without disturbing buildings that may be nearby. - -* **Drones** provide low-volume, long-range transport not significantly affected by terrain. - -* **Signals**, **Collisions**, and **Rail Timetables** dramatically change the behaviour and constraints on railway transport. - -* **Trucks and Tractors** are now reliable and able to move significantly more cargo, making them a viable transport option. - -* The **Hover Pack**, unlocked very late in the game, allows the player to move and hold position in mid-air indefinitely, so long as there's power. - -Second, we have significant changes to the game that require some revision to the pattern language, but fit within its existing boundaries reasonably well: - -* **Power plants** no longer scale their production (and thus fuel demand) with load. - -* **Power switches** allow the player to designate and control power flow within their factory without needing to understand and track what is supposed to connect to what. - -* **Nuclear Waste reprocessing** gives the player the option to run nuclear reactors indefinitely, or to re-use nuclear waste for additional power, depending on their needs and goals. - -* **Soft clearance** allows belts, buildings, foundations, and other structures to intersect with greater freedom, making some highly compact designs possible (if you can tolerate clipping) and increasing the range of artistic expression available (even if you can't). - -Third and finally, we have changes that may be significant, but which require no additional consideration given the range of Aphyr's proposed pattern language: - -* Lights, -* Power storage, -* The Zipline tool, -* New production recipes and machines, -* Resource wells and pressurizers, -* Gasses, -* A huge range of new building pieces and materials, -* Signs, - -and numerous other changes and improvements to the game. Aphyr's original pattern langauge is general, and accommodates these changes well. - -## What Holds Up - -Aphyr's analysis of the game is extensive, and _most_ of it has survived the changes to the game admirably. The fundamental forces of unlocked technology, player time, and lag still control the overall gameplay loop, and lead to the same goals. It is not foreseeable that a future update might change these forces, either: the developers have [gone on record][never-adding] about the kinds of features they will not add to the game, and their reasoning there confirms that they have a specific vision for the game, which Aphyr perceives clearly. - -[never-adding]: https://www.youtube.com/watch?v=J4LlorYbVV0 - -Following from this, the pattern terminology developed under [Factories](https://aphyr.com/posts/351-a-satisfactory-way-of-building#factories) is still essentially sound - -* **Factories Far Apart**, **Factories With Goals**, and **Shoulders of Giants** are still, collectively, an effective set of tools to make factories legible and expansible and to manage lag. With more to make, you will have more factories to build. - -* **Factories Near Raw Materials** is still an effective strategy for managing some of the bulkiest logistics connections in the game. The addition of Nitrogen, and recipes which require Nitrogen, require more decisions about which raw materials to build near, but do not fundamentally change the forces that apply to those decisions. - -The [Factory Connections](https://aphyr.com/posts/351-a-satisfactory-way-of-building#factory-connections) section is likewise very general, and easily applicable even with the changes to the game. **Backpressure** is probably the _only_ viable way to automate resource allocation in the absence of complex signalling and logic tools - -**Immiscible Materials** remains extremely good advice (to the point that both the Satisfactory Discord and the official subreddit advocate strongly against "sushi belt," or mixed-material-belt, designs, on the premise that they will always deadlock eventually). - -The [Special Sites](https://aphyr.com/posts/351-a-satisfactory-way-of-building#special-sites) portion likewise holds up well. In the intervening two years, **Shopping Malls** and **Home Base** have become standard fare for factory-building games. Building these sites near a **Transit Hub** is an essential improvement, and players continue to feel immense pressure within the game to connect special sites to the rest of the factory via accessible, reliable transit systems. - -The _logical_ [Factory Structure](https://aphyr.com/posts/351-a-satisfactory-way-of-building#factory-structure) remains much as it was in 2020, with a **Stage Cascade** laid out in **Floors** and **Zones** to provide logical and legibile groups of machines. Where buildings must be built near or on top of resource extraction, **Basement Miners** remains the only sensible solution, as well. The physical structure, however, is subject to different forces. - -In contrast, much of the gross-scale physcial [Factory Shape](https://aphyr.com/posts/351-a-satisfactory-way-of-building#factory-shape) patterns still hold. **Factory Site**, **Scenic View**, **Access to Light**, **Monumental Aspect**, **Floor Size**, **Odd Dimensions**, and **Floor Height** all remain important lenses for siting and laying out new factories. Similarly, the patterns developed under [Sense of Place](https://aphyr.com/posts/351-a-satisfactory-way-of-building#sense-of-place) still provide a solid basis for incorporating the game's scenery into the constructions you impose on it. - -Finally, while the [Refineries](https://aphyr.com/posts/351-a-satisfactory-way-of-building#refineries) section needs no changes, the developers seem to have taken **Fuck Refineries** as a personal challenge. The game now includes a second machine type that shares the property of producing more than one output: this term could be replaced with **Fuck Refineries and Blenders**. - ------ - -Now, we come to the parts of Aphyr's pattern language that mostly fit the game as it is today, but need some amendment to address the new ways the player has to engage with the game's challenges. - -## Tra(i)nsport and Logistics - -In 2020, Aphyr's essay concluded that the only viable way to connect factories was via train line: - -> Tractors and trucks are fun, but frustrating at scale. Truck-dense regions become subject to collisions which can disable trucks altogether. When power fails (and it will, invariably, fail), trucks cannot refuel, consume their available fuel, and become lost, somewhere miles in the wilderness. Tracking down dozens of trucks and refueling each by hand is a time-consuming process. Reorganizing stops requires laborious re-recording of each affected truck route. - -> Belts are a good alternative, but they have other weaknesses. Belts cannot send as many items as train tracks in a given area. Belts transmit neither power nor people (efficiently); those connections must be built separately. Most critically, a dedicated belt line must be created for each resource flow: sending two resources requires two belts. You can almost never re-use existing belts for new flows–as your network grows, you spend an increasing amount of time re-laying new belts between far-flung factories. - -> Instead, use trains as the principal means of connecting factories. Train networks provide power, material flow, and transportation in one network. Tracks are independent of the flows across them; once established, train networks rarely need to be changed, only gradually expanded to new factory sites. Trains are fast: a single freight station can easily saturate two belts. Use trains liberally: almost all factories should send their outputs out via trains, and receive low-volume materials (vis Shoulders of Giants) through trains as well. - -In 2020, that was a reasonable conclusion: belts, as he says, require an increasing amount of work to manage the more products you want to transport and cannot be adjusted to fit changing workloads, and trucks were unreliable and highly throughput-limited. - -Two updates later, trains remain a _very reliable_ way to connect factories together, and should still provide the backbone of the factory's logistics network. However, **Train Network** now needs two caveats: - -* For low volume parts moving between far-flung parts of the factory, use drones. The need for sulphur as part of the fuel process, and the intense power demands created by drone ports, limit the use of these, but in return they're extremely flexible, and require no effort to create routes and trackways for. - - Unlike train stations, drone ports should be built at the _top_ of a factory, feeding downwards into the building. The alternatives are either ugly clipping, with drones flying straight through factory structures, or a vertical vent through which drones can take off and land, making that area unavailable for use in the factory. - - One drone route can share multiple products if **Load Shedding** is used immediately at the destination to prevent clogs. However, this can create unlimited demand and interfere with **Backpressure**, and should be used sparingly, where sulfur is at a large premium. - -* For nearby buildings with high throughput, _consider_ using trucks or tractors instead of trains. Trucks and tractors effectively must run along the ground, as building roads is impractical at scale. Trucks also require fuel instead of power, but a truck network consumes a small fraction of the overall power that the same cargo delivered by rail would, and will make more efficient use of fuel per item moved in many cases. - - The addition of a second conveyor port to truck stops, truck route looping, and the improvements to collision recovery address all but one substantial problems with trucks as they were in Update 3. The remaining issue is that the truck network can break if truck stops are denied power. Therefore, each truck station should be paired with an **Uninterruptible Power Supply**, consisting of sufficient Power Storages to run the truck network. - - Truck stations should follow the **Station Per Component**, **Station Buffer**, and **Dedicated Trains (And Trucks)**. The same forces apply to Trucks in this regard. - -Aphyr also proposes a number of [Track Layout](https://aphyr.com/posts/351-a-satisfactory-way-of-building#track-layout) rules. These require some changes. When the original essay was written, trains could pass through one another freely, and could stack up in stations and intersections. Aphyr implicitly relied on this; his examples all depict factories connected by a single rail line, shared between inbound and outbound use. - -Update 5 made sweeping changes to trains, including the addition of collisions, train station scheduling options, and signals to control access to regions of track. A few new new terms are needed to address these new forces: - -### Dependent Blocks - -Rails, by their shape, naturally want to run in long, mostly-straight lines, broken up only by intersections and construction boundaries. Grids, alternate routes, and other complex topologies are difficult to construct, and train pathfinding prefers the shortest path that avoids other stations, so they often go unused anyways. - -This inherent form ensures that most railway segments are strictly dependent on a single segment, further along the rail in the direction of travel. In order for a train to leave the segment, the path to that eventual outlet must eventually become clear of other trains. This can only happen if all the trains between here and there are travelling in the same direction, and if they are prevented from interfering with each other and derailing. - -These block dependencies form a directed graph, consisting mostly of linearly-connected subgraphs, which in turn dictate traffic flow through the network. Intersections and manifolds break up this regularity, and add complex dependencies locally, but do little to change the large-scale structure of the network. - -### Double Rails - -**Train Network** tells us that each factory should use trains to send and receive materials. Train collisions will cause traffic problems or even outright stalls if trains collide, and **Dependent Blocks** tells us that any length of track can only be used in at most one direction at a time. - -To avoid deadlocks on main routes, and to minimize the time spent waiting for trains to clear the track, use **Double Rails** for all rail connections. (And drive on the right. The game's art assets look terrible and are often illegible with left-handed train layouts.) - -### Intersection Blocks - -Path signals allow trains to reserve a path through an occupied block if no other train reserves a conflicting path, allowing multiple trains to pass safely through a block at the same time. - -At junctions, use path signals for the approach, to allow trains to enter the junction only if they can leave it, and to allow trains to share the junction when passing one another in opposite directions. - -### Elevated Interchanges - -**Intersection Blocks** tells us that path signals make more effective use of track time at junctions. However, for extremely busy junctions, path signals alone may not be sufficient to ensure the smooth flow of trains. - -Using [elevated interchanges] allows trains to move through the junction without conflict even when crossing other lanes. However, this requires substantially more space, and more construction effort, than a flat intersection, and should only be used where needed. - -Interchanges can also make convenient turnaround points if trains may need to return from the direction they arrive from, for example in order to revisit a factory after leaving it. - -[elevated interchanges]: https://www.youtube.com/watch?v=Ux7WC3wzTP8 - -(Roundabouts, which Aphyr recommends, have [known issues with deadlocks](https://www.reddit.com/r/factorio/comments/6sbvrl/deadlock_free_roundabout/), and should no longer be used.) - -### Station Blocks - -**Station Manifold** implies that the tracks leaving each station will share a single exit, and that the tracks arriving at each station will share a single entrance. To allow multiple trains to visit stations at the same time, we must place block signals at the shared exit and entrance, and before and after each station, to keep parked trains from blocking traffic into and out of the station. - -### Waits at Receiving Stations - -**Station Blocks** implies that only one train may be waiting in a station at any given time. **Station Per Component** tells us that that station only processes a single product, so the station must either be loading or unloading that material, never both. **Dedicated Trains** means that stations are served by trains that only serve that specific component type, and often a specific destination. - -This means that stations that supply a component are likely to be shared by multiple trains, but stations that receive a component are probably only served by a single train. To limit contention at the supply, trains should be configured to load quickly, even if they do not receive a full load, so that they vacate the supply station quickly and make room for the next train. They should also be configured to unload only when the receiving station can accept their full cargo, to minimize the frequency with which they visit the supply stations. - -Because of [game bugs](https://questions.satisfactorygame.com/post/61a3e1b0831c852052362a1a), this is best paired with a safety timeout. I recommend the setting "Freight Wagon is fully loaded/unloaded OR wait 3600 seconds" for receiving stations, so that a train which doesn't unload when it should will automatically be released after at most an hour. Shorter waits may be appropriate if the train will always unload frequently when the factory is under normal load. - -### Train Queues - -**Station Blocks** separate traffic from trains that are loading or unloading. **Wait To Unload** keeps trains parked at their receiving stations as often as possible. However, if a supply station has high demand, then trains may be waiting for an earlier arrival to finish loading. Rather than allowing them to wait in the inbound station manifold, where they would block other stations from receiving trains, build an additional length of track with its own signals, leading into the platform, to accommodate waiting trains at the station. - -Longer queues can wind back and forth or even spiral up and down in order to conserve space. (Unfortunately, Factorio-style railway stackers do not work and cannot be used to conserve space by having trains wait in parallel.) - -### Two Platform Stations - -Train stations have a [maximum throughput], determined mostly by the time spent loading and unloading trains. With one significant but rare exception, the maximum is more than 1.5 belts and less than two belts per freight platform. Using two platforms per station, and planning for a throughput of 1.5 belts per platform, allows a train station to deliver up to three belts of product, which is more than adequate for the huge majority of factory complexes and flexible enough to support lower throughputs easily. - -Rarely, certain components that can't easily be made on-site will need more than three belts. This is particularly common with ingots, which are needed in bulk in many factories and which often can't easily be refined on-site without access to water. These high-demand situations should be managed with multiple stations, rather than by building specialized large trains and platforms: exceptional trains distort the layout of the factories they serve. - -[theoretical maximum throughput]: https://www.reddit.com/r/SatisfactoryGame/comments/taku85/15_mark_v_belts_per_freight_platform/ - -## Verticality - -The addition of **Zooping** fundamentally changes the process of constructing factories. By allowing the placement of whole runs of structural pieces, it makes building floors, walls, and supporting structures much easier. Because of the addition of a dedicated vertical zooping mode, and because vertical construction takes twice as many foundations to cover the same distance as a horizontal structure, zooping has a disproportionate impact on the ease with which a factory can be expanded _upwards_. - -Aphyr observed in **Zone** that "…building vertically is time-consuming…," but this is no longer the case. Building verticaly is nearly as straightforward as building horizontally. - -This is a critical change, and has an extreme impact on the design of factories. **Floors** and **Zones** remain reasonable design primitives, and **Orientation** towards directions of likely growth and away from obstructions is still the foundational constraint on factory layout, but there is vastly more vertical space available than horizontal, and the constraint "you can't expand into the ground" holds throughout the majority of the map. With Zooping, the default direction in which to expand a factory or add a new zone is _upwards_. Lateral expansion is much less important, and space to the sides may now be better reserved for transit infrastructure or separation between nearby factories. - -There's a synergy between this and another change. The addition of floor holes allows conveyor lifts, pipes, and hypertubes to move vertically through foundations, rather than requiring an area clear of foundations and machinery. [Filling In The Core](https://aphyr.com/posts/351-a-satisfactory-way-of-building#filling-in-the-core), once an essential and inevitable fixed point in the game's design language, is now an organizational choice to be made. Cores remain aesthetically pleasing and space-efficient, but share design space with a significant alternative: - -### Edge Lifts - -As an alternative to a core, route vertical connections through floor holes at or near the edge of each floor, in the **Comfortable Margins** around the zone itself. Edge lifts can be located flexibly around the margin, as required by the design of the zone and any integration with the zones above and below it. - -## Power Management - -Power switches and power storages interact with Aphyr's pattern language in a few interesting ways, and deserve some treatment of their own. - -Power storages fight neatly within the **Backup Grid** guideline, and provide a viable alternative to restart coal or gas-based power production. Providing enough battery power to restart nuclear power is less practical, but by that stage of the game, geothermal generators are available and provide ample power to restart fuel production. - -**Single Power Link** proposes that power be connected to a factory, zone, or floor by a single wire that can be severed. Switches are a much more legible alternative to constructing and dismantling power lines, and do not require that the player carry cable on their person in order to turn factories back on. This is a small change on its own, but the convenience of switches makes other patterns more approachable: - -### Main Switch - -**Single Power Link** tells us that power networks should form a tree, rooted at a train station. To make this root connection connection more legible, connect the power network through a power switch, and label it appropriately to make it immediately clear that pulling the switch will power off the whole factory. - -The switch's indicator light provides an easy, at-a-distance indication of whether the factory is running. It may be worth placing the main switch on an exterior platform, so that it's visible from afar. - -### Floor and Zone Switches - -In factories with large numbers of zones, the ability to scale down production during periods of low power supply can be important. Zones can be connected to the factory's **Power Chase**, or to a floor-wide power bus, using switches, to make it possible to selectively power down specific areas. - -In factories with multiple zones per floor, floors should also be isolated from the **Power Chase** by switches, so that whole floors can be powered off quickly. - -## Flight - -Aphyr comments, in an aside, that "the jetpack is a way of life." This is something that _has_ changed dramatically: within the powered areas of the factory, the preferred way to travel is now the **Hoverpack**, which requires power instead of fuel and allows free movement in all three axes. This change has mostly relegated the Jetpack to exploration. - -The Hoverpack gives the player freedom to choose a perspective where they can see the essential systems clearly, the ability to move over or under obstructions easily, and extended reach to manipulate and inspect the factory as needed. - -The presence of the Hoverpack makes the use of vertical transport infrastructure - ladders, stairs, ramps, and hypertubes, much less important. However, hoverpacks require power, and cannot operate during a power outage or away from powered areas of the factory. In addition, the hoverpack conflicts with other PPE, including hazmat suits, making it unusable in radioactive or poisonous environments. - -This creates three new factory design tropes: - -### Temporary Power Lines - -When building a factory with a hoverpack, it's often useful to be able to get a high perspective. Rather than building temporary structure, place powerlines with the intention of removing them once the factory is more complete, to provide lift. - -Temporary power lines should be placed so that they can be quickly identified and removed once the factory's power network is complete, and so that buildings are not inadvertantly connected to them. - -### Fall Arrest Areas - -You can't fly when there's no power. Even if you rely completely on Hoverpacks, place key infrastructure at floor level, and connect the floors of your factory with ladders or stairs for emergency use. - -If you build out over the void, or if higher floors are open to the elements, surround them with railings or enclose them with glass, to prevent unplanned falls. - -More extreme hazards may require the construction of a catch structure at a lower level to catch falling players before they suffer fatal injuries or fall out of the map. - -### Walkable Hazards - -Zones and floors that deal with hazardous materials need special consideration for hoverpack access. Dangerous areas must be provided with pedestrian access, and should be provided with **On-site PPE**. \ No newline at end of file -- cgit v1.2.3