summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Rough in client and server side of web-pushKit La Touche2025-07-23
|
* Fix some minor weirdness when Pilcrow is (unwisely) used as a library.ojacobson2025-07-23
|\ | | | | | | | | | | | | | | | | | | | | | | Pilcrow isn't meant to be used as a library, and the only public interface the `pilcrow` lib crate exposes is the CLI entry point. However, we will likely be publishing Pilcrow via crates.io (among other options) one day, and so it will _be usable_ as a library if someone's desperate enough to try. To that end, let's try to be good citizens. This change fixes two issues: * The docs contained links to internal items, which are not actually included in the library documentation. The links are simply removed; the uses of those items were already private anyways. * The CLI `Error` type is no longer part of the public interface, using `impl Trait` (`impl std::error::Error`) shenanigans to hide the error type from callers. (To be clear, this would be _extremely_ rude in code intended for library use.) This frees us up to change the structure of the error type - or to replace it entirely - without making the world's most pedantic semver change in the process. Merges lib-crate-weirdness into main.
| * Remove `pilcrow::cli::Error` from the lib crate's public interface.Owen Jacobson2025-07-22
| | | | | | | | This might be the pettiest rude change I've ever made to a Rust program. If I saw this - or did this - in code _intend_ to be used as a library, I'd be appalled.
| * Stop linking to private documentation items in public docs.Owen Jacobson2025-07-22
|/ | | | The Pilcrow crate library docs are something of a wart; Pilcrow isn't meant to be used as a library, and the only public interface it exposes is the CLI entry point. However, we will likely be publishing Pilcrow via crates.io (among other options), and so it will _be usable_ as a library if you're desperate enough to try. The docs should at least be coherent.
* Add a `--umask` option to determine what permissions new files/databases get.ojacobson2025-07-23
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new `--umask` option takes one of three values: * `--umask masked`, the default, takes the inherited umask and forces o+rwx on. * `--umask inherit` takes the inherited umask as-is. * `--umask OCTAL` sets the umask to exactly `OCTAL` and is broadly equivalent to `umask OCTAL && pilcrow --umask inherit`. This fell out of a conversation with @wlonk, who is working on notifications. Since notifications may require [VAPID] keys, the server will need a way to store those keys. That would generally be "in the pilcrow database," which lead me to the observation that Pilcrow creates that database as world-readable by default. "World-readable" and "encryption/signing keys" are not things that belong in the same sentence. [VAPID]: https://datatracker.ietf.org/doc/html/rfc8292 The most "obvious" solution would be to set the permissions used for the sqlite database when it's created. That's harder than it sounds: sqlite has no built-in facility for doing this. The closest thing that exists today is the [`modeof`] query parameter, which copies the permissions (and ownership) from some other file. We also can't reliably set the permissions ourselves, as sqlite may - depending on build options and configuration - [create multiple files][wal]. [`modeof`]: https://www.sqlite.org/uri.html [wal]: https://www.sqlite.org/wal.html Using `umask` is a whole-process solution to this. As Pilcrow doesn't attempt to create other files, there's little issue with doing it this way, but this is a design risk for future work if it creates files that are _intended_ to be readable by more than just the Pilcrow daemon user. Merges options-umask into main.
| * Add a `--umask` option to determine what permissions new files/databases get.Owen Jacobson2025-07-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new `--umask` option takes one of three values: * `--umask masked`, the default, takes the inherited umask and forces o+rwx on. * `--umask inherit` takes the inherited umask as-is. * `--umask OCTAL` sets the umask to exactly `OCTAL` and is broadly equivalent to `umask OCTAL && pilcrow --umask inherit`. This fell out of a conversation with @wlonk, who is working on notifications. Since notifications may require [VAPID] keys, the server will need a way to store those keys. That would generally be "in the pilcrow database," which lead me to the observation that Pilcrow creates that database as world-readable by default. "World-readable" and "encryption/signing keys" are not things that belong in the same sentence. [VAPID]: https://datatracker.ietf.org/doc/html/rfc8292 The most "obvious" solution would be to set the permissions used for the sqlite database when it's created. That's harder than it sounds: sqlite has no built-in facility for doing this. The closest thing that exists today is the [`modeof`] query parameter, which copies the permissions (and ownership) from some other file. We also can't reliably set the permissions ourselves, as sqlite may - depending on build options and configuration - [create multiple files][wal]. [`modeof`]: https://www.sqlite.org/uri.html [wal]: https://www.sqlite.org/wal.html Using `umask` is a whole-process solution to this. As Pilcrow doesn't attempt to create other files, there's little issue with doing it this way, but this is a design risk for future work if it creates files that are _intended_ to be readable by more than just the Pilcrow daemon user.
* | Remove remnant `html` class from swatch textinputs.Owen Jacobson2025-07-18
|/ | | | This was a leftover from the idea that different swatches might have different input notations - and they do, but we turned out not to need to style them differently. And, in any event, this class was applied (only) to inputs that _aren't HTML_, because of 01ed82ac4f89810161fbc3aa1cb8e4691fb8938b.
* Prevent race conditions between `cargo run` and `npx vite build` in `tools/run`.Owen Jacobson2025-07-15
| | | | | | | | | | | | | | | | | | | To reproduce: * Make any change (even just `touch`) to a file in `ui`. * Run `tools/run`. There's a decent chance that the script will fail, with esoteric and variable errors: ``` failed to load config from .../pilcrow/vite.config.js_api(buil... error when starting dev server: Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@sveltejs/kit' imported from .../pilcrow/vite.config.js.timestamp-1752608239248-52b6b8965ad28.mjs ``` is one such exmple. These errors are due to the `npm ci` carried out by `cargo` at build time (see `build.rs`), which deletes and re-creates `node_modules`. Vite and Svelte do not like having `node_modules` deleted out from under them. As `cargo` will rerun `build.rs` any time `ui` changes, this means that `tools/run` effectively requires a complete build _first_, before it can be run.
* Create swatches for Svelte components.ojacobson2025-07-10
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A swatch is a live, and ideally editable, example of an element of the service. They serve as: * Documentation: what is this element, how do you use it, what does it do? * Demonstration: what does this element look like? * Manual test scaffolding: when I change this element like _so_, what happens? Swatches are collectively available under `/.swatch/` on a running instance. They do not require setup or login for simplicity's sake and because they don't _do_ anything that requires either of those things. Swatches are manually curated. First, we lack the technical infrastructure needed to do this based on static analysis, and second, manual curation lets us include affordances like "interesting values," that would be tricky to express as part of the type or schema for the component. The tradeoff, however, is that they will fall out of step with the components if not reviewed regularly. Swatches are _possible_ because we've gone to efforts to avoid global data access or direct side effects (including API requests) in our components, delegating that upwards to `+page`s and `+layout`s. However, the isolation is imperfect. For example, the swatch for `Conversation`, which renders the conversation sidebar entries, causes actual attempts to boot the app as browsers pre-fetch the links on mouseover, and clicking them will take the user to the "real" application because they really are links. Merges swatch into main.
| * Do not support users entering bare HTML in swatches.Owen Jacobson2025-07-09
| | | | | | | | | | | | | | | | You can inject Javascript into a swatch that uses `{@html <expr>}` fairly easily. `<script>foo()</script>` doesn't appear to work, but `<img src="x" onerror="foo()">` does, for example. That code then runs with the same access to cookies, and the same access to local data, as the Pilcrow client. This change removes that capability, by replacing the two swatches that exposed it with more limited examples. I love the generality and flexibility of generic HTML entry here, and I think it might have been useful for swatching components that are generic DOM containers (which both `Message` and `MessageRun` are today), but swatches are a user interface and are exposed to _all_ users. A user who is unfamiliar with HTML and Javascript, but who is persuaded to open a swatch and enter some code into it (think about an attacker who tells their victim "hey check out this funny thing that happens," preying on curiousity, while providing a lightly-obfuscated payload) can then impersonate that user, exfiltrate anything saved locally, or potentially install persistent code using JS' various background-processing APIs. Gnarly stuff. We're not up to mitigating that in place. Anyone who knows JS can likely learn to build the client from source, and can experiment with arbitrary input that way, taking responsibility for the results in the process, while anyone who doesn't is unlikely to be persuaded to set up an entire Node toolchain just for an exploit.
| *-----------------. Implement swatches for the existing component inventory.Owen Jacobson2025-07-08
| |\ \ \ \ \ \ \ \ \ \
| | | | | | | | | | | * Create swatch for the `Message` component.Owen Jacobson2025-07-08
| | | | | | | | | | | |
| | | | | | | | | | * | Create swatch for the `MessageInput` component.Owen Jacobson2025-07-08
| | | | | | | | | | |/
| | | | | | | | | * / Create swatch for the `MessageRun` component.Owen Jacobson2025-07-08
| | | | | | | | | |/
| | | | | | | | * / Create swatch for the `LogOut` component.Owen Jacobson2025-07-08
| | | | | | | | |/
| | | | | | | * / Create swatch for the `LogIn` component.Owen Jacobson2025-07-08
| | | | | | | |/
| | | | | | * / Create swatch for the `Invites` component.Owen Jacobson2025-07-08
| | | | | | |/
| | | | | * / Create swatch for the `Invite` component.Owen Jacobson2025-07-08
| | | | | |/
| | | | * / Create swatch for the `CreateConversationForm` component.Owen Jacobson2025-07-08
| | | | |/
| | | * / Create a swatch for the `ConversationList` component.Owen Jacobson2025-07-08
| | | |/
| | * / Create a swatch for the `Conversation` component.Owen Jacobson2025-07-08
| | |/
| * / Add a swatch for the `ChangePassword` component.Owen Jacobson2025-07-08
| |/
| * Event capture and display tools.Owen Jacobson2025-07-08
| | | | | | | | | | | | This is meant to be used in swatches, to display the events and callbacks generated by a component as part of the swatch. The usage pattern is described in the comments (in both places). Naturally, this has its own swatch.
| * Create "derivers," as an exception-free option for working with structured ↵Owen Jacobson2025-07-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | data in swatches. This is meant to be used alongside `$derive`, for inputs with complex structure. For example: ```js let jsonInput = $state('{}'); let json = $derived(deriver.json(jsonInput)); // … <textarea bind:value={jsonInput}></textarea> ``` This allows textual editing of the data, while preventing exceptions due to syntax or logical errors in partially-edited data from breaking Svelte's derive process (see comments). Note that these exceptions are not considered [unexpected errors] by SvelteKit, because they do not arise "while handling a request;" they are considered errors by Svelte, but Svelte doesn't appear to provide any affordances for handling errors in this context, so we have to bring our own. [unexpected errors]: https://svelte.dev/docs/kit/errors#Unexpected-errors
| * Set up a skeleton for swatches.Owen Jacobson2025-07-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A swatch is a live, and ideally editable, example of an element of the service. They serve as: * Documentation: what is this element, how do you use it, what does it do? * Demonstration: what does this element look like? * Manual test scaffolding: when I change this element like _so_, what happens? Swatches are collectively available under `/.swatch/` on a running instance, and are set up in a separate [group] from the rest of the UI. They do not require setup or login for simplicity's sake and because they don't _do_ anything that requires either of those things. [group]: https://svelte.dev/docs/kit/advanced-routing#Advanced-layouts-(group) Swatches are manually curated, for a couple of reasons: * We lack the technical infrastructure needed to do this based on static analysis; and * Manual curation lets us include affordances like "recommended values," that would be tricky to express as part of the type or schema for the component. The tradeoff, however, is that swatches may fall out of step with the components they depic, if not reviewed regularly. I hope that, by making them part of the development process, this risk will be mitigated through regular use.
* | Remove container divs for `MessageInput` and `CreateConversationForm`.ojacobson2025-07-09
|\ \ | | | | | | | | | | | | | | | | | | | | | The styles for the `MessageInput` and `CreateConversationForm` components assumed that a container div would be present, and the components did not render as intended without that div. Therefore: the divs were "part of" the component in most of the ways that matter, not part of the context in which the component is used. It turns out that those divs aren't necessary or interesting anyways - they were targets for layout, but the same layout can be achieved without them. This change removes the divs entirely. Merges component-div-nesting into main.
| * | A few semantically-thin wrapper divs.Owen Jacobson2025-07-08
| | | | | | | | | | | | This is an extension of the previous commit: we don't need these divs _at all_ to achieve the layout we want, and we aren't attaching behaviour or semantics to them, so, out they go.
| * | Move container divs for components into those components.Owen Jacobson2025-07-08
| | | | | | | | | | | | | | | | | | The styles for the `MessageInput` and `CreateConversationForm` components assume that the container div will be present, and the components will not render as intended without them. Therefore: they are "part of" the component in most of the ways that matter, not part of the context in which the component is used. Moving the divs into the component will make it easier to reuse these components (for example, in swatches). The diff for this looks worse than it is because of indentation changes.
* | | Stop sending `{}` to the `/api/auth/login` endpoint when the login form ↵ojacobson2025-07-09
|\ \ \ | |_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | hasn't been touched. Steps to reproduce: **Note**: You will need to watch the traffic in a DOM inspector; this has no user-observable symptoms because there's presently no error reporting for the login form. 1. In a new private tab, visit the `/login` page of a Pilcrow instance. 2. **Without touching the username or password fields**, click `sign in`. The client _should_ send a request to `/api/auth/login` with the following payload: ```json { "name": "", "password": "" } ``` However, it instead sends an empty payload, leading to a 422 Unprocessable Content response as the request is missing required fields. Subsequent requests, or any request after the user enters data in the input fields, are correctly serialized. Merges login-form-nulls into main.
| * | Set non-`undefined` initial values for the login form.Owen Jacobson2025-07-08
| | | | | | | | | | | | The default state of a `$state()` with no arguments is `undefined`, which was then leaking out of this component if the user clicks `sign in` without changing the values. Axiom, our HTTP client library, suppresses fields with `undefined` values in JSON payloads (sensibly enough), leading to empty requests.
| * | Bug: the login form generates incorrect requests (once per pageview).Owen Jacobson2025-07-08
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Steps to reproduce: **Note**: You will need to watch the traffic in a DOM inspector; this has no user-observable symptoms because there's presently no error reporting for the login form. 1. In a new private tab, visit the `/login` page of a Pilcrow instance. 2. **Without touching the username or password fields**, click `sign in`. The client _should_ send a request to `/api/auth/login` with the following payload: ```json { "name": "", "password": "" } ``` However, it instead sends an empty payload, leading to a 422 Unprocessable Content response as the request is missing required fields. Subsequent requests, or any request after the user enters data in the input fields, are correctly serialized.
* / Remove the (entirely unused and unusable) `body` property from `Message`.Owen Jacobson2025-07-08
|/
* Rename "channels" to "conversations."ojacobson2025-07-04
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The term "channel" for a conversational container has a long and storied history, but is mostly evocative of IRC and of other, ah, "nerd-centric" services. It does show up in more widespread contexts: Discord and Slack both refer to their primary conversational containers as "channels," for example. However, I think it's unnecessary jargon, and I'd like to do away with it. To that end, this change pervasively changes one term to the other wherever it appears, with the following exceptions: * A `channel` concept (unrelated to conversations) is also provided by an external library; we can't and shouldn't try to rename that. * The code to deal with the `pilcrow:channelData` and `pilcrow:lastActiveChannel` local storage properties is still present, to migrate existing data to new keys. It will be removed in a later change. This is a **breaking API change**. As we are not yet managing any API compatibility promises, this is formally not an issue, but it is something to be aware of practically. The major API changes are: * Paths beginning with `/api/channels` are now under `/api/conversations`, without other modifications. * Fields labelled with `channel…` terms are now labelled with `conversation…` terms. For example, a `message` `sent` event is now sent to a `conversation`, not a `channel`. This is also a **breaking UI change**. Specifically, any saved paths for `/ch/CHANNELID` will now lead to a 404. The corresponding paths are `/c/CONVERSATIONID`. While I've made an effort to migrate the location of stored data, I have not tried to provide adapters to fix this specific issue, because the disruption is short-lived and very easily addressed by opening a channel in the client UI. This change is obnoxiously large and difficult to review, for which I apologize. If this shows up in `git annotate`, please forgive me. These kinds of renamings are hard to carry out without a major disruption, especially when the concept ("channel" in this case) is used so pervasively throughout the system. I think it's worth making this change that pervasively so that we don't have an indefinitely-long tail of "well, it's a conversation in the docs, but the table is called `channel` for historical reasons" type issues. Merges conversations-not-channels into main.
| * Rename "channel" to "conversation" throughout the client.Owen Jacobson2025-07-03
| | | | | | | | Existing client state, stored in local storage, is migrated to new keys (that mention "conversation" instead of "channel" where appropriate) the first time the client loads.
| * Move the `/ch` channel view to `/c` (for conversation).Owen Jacobson2025-07-03
| |
| * Replace `channel` with `conversation` throughout the API.Owen Jacobson2025-07-03
| | | | | | | | This is a **breaking change** for essentially all clients. Thankfully, there's presently just the one, so we don't need to go to much effort to accommoate that; the client is modified in this commit to adapt, users can reload their client, and life will go on.
| * Rename "channel" to "conversation" within the server.Owen Jacobson2025-07-03
| | | | | | | | | | | | I've split this from the schema and API changes because, frankly, it's huge. Annoyingly so. There are no semantic changes in this, it's all symbol changes, but there are a _lot_ of them because the term "channel" leaks all over everything in a service whose primary role is managing messages sent to channels (now, conversations). I found a buggy test while working on this! It's not fixed in this commit, because it felt mean to hide a real change in the middle of this much chaff.
| * Rename "channel" to "conversation" in the database.Owen Jacobson2025-07-03
|/ | | | | | I've - somewhat arbitrarily - started renaming column aliases, as well, though the corresponding Rust data model, API fields and nouns, and client code still references them as "channel" (or as derived terms). As with so many schema changes, this entails a complete rebuild of a substantial portion of the schema. sqlite3 still doesn't have very many `alter table` primitives, for renaming columns in particular.
* Stop accepting new messages on deleted channels.ojacobson2025-07-04
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a channel is deleted, Pilcrow should no longer allow new messages to be sent to it. This is part of the intended lifecycle of a channel, and either clients or the server may rely on it implicitly. Steps to reproduce: 1. Create a channel: ```bash % curl \ 'http://[::1]:64209/api/channels' \ --header 'cookie: identity=REDACTED' \ --header 'content-type: application/json' \ --data '{"name": "oh no"}' {"at":"2025-07-01T04:38:47.483235Z","id":"C2s1c6r1sc9921y3","name":"oh no"} ``` Note its ID (here, `C2s1c6r1sc9921y3`). 2. Delete it: ```bash % curl \ 'http://[::1]:64209/api/channels/C2s1c6r1sc9921y3' \ --header 'cookie: identity=REDACTED' \ -XDELETE {"id":"C2s1c6r1sc9921y3"} ``` Substitute the ID appropriately. 3. Send it a message: ```bash % curl -v \ 'http://[::1]:64209/api/channels/C2s1c6r1sc9921y3' \ --header 'cookie: identity=REDACTED' \ --header 'content-type: application/json' \ --data '{"body": "oh very no"}' < HTTP/1.1 202 Accepted < […] {"at":"2025-07-01T04:39:23.658387Z","channel":"C2s1c6r1sc9921y3","sender":"L3twnpw918cftfkh","id":"Mhx9k6w3wxnk1f4t","body":"oh very no"} ``` Substitute the ID appropriately here, as well. What **does** happen is, as above, that the message is sent to the channel. It really is, too - you can see it in the ensuing event stream, or in boot. What **should** happen is that the API should reject the third request with an appropriate error - probably a 404. With this change, that's what happens: ```bash % curl -v \ 'http://[::1]:64209/api/channels/C2s1c6r1sc9921y3' \ --header 'cookie: identity=REDACTED' \ --header 'content-type: application/json' \ --data '{"body": "oh very no"}' < HTTP/1.1 404 Not Found < […] channel C2s1c6r1sc9921y3 deleted ``` We had a test case that was intended to catch this. Unfortunately, I got a bit sloppy when writing it; instead of testing "can you send to a channel after deleting it," it was testing "can you send to a randomly-selected channel ID after deleting a channel" - which you can't, but not because of anything to do with deleting channels. The test case is now fixed, and does actually detect this issue. This defect was not exercised by the client (and in practice probably affected nobody so far), so there are no client changes included. A client that tries this specific operation will get a 404, as above; this may happen if a user tries to send to a channel that is in the process of expiring at that moment, for example. --- This change also proposes some changes to the conventions used for histories and for database access. It does not carry those changes through, so it introduces some inconsistencies, though I think they're minor. * For histories: `Channel` histories now have an `as_of` method that takes an instant or a sequence number, and produces a snapshot of the channel at that point in history. It's used here to figure out what the state of the target channel is at the moment we're sending messages to it, but I think this might be more generally useful, as well. * For database access: inserting a message now takes a `channel::Channel` (snapshot) instead of a `channel::History` as evidence that the caller has actually resolved the ID to an appropriate entity. This works in tandem with the `as_of` snapshot method, and I'm hoping this might help reduce our reliance on passing histories around to database methods if it generalizes well. Merges send-to-deleted-channel into main.
| * Prevent sending messages to deleted channels.Owen Jacobson2025-07-03
| | | | | | | | I've opted to make it clear in the error message which scenario - deleted vs. non-existant - a channel falls into. This isn't particularly consistent with the rest of the API, so we might need to review this decision later, but it's at least relatively harmless if it's mistaken. (Formally, they're both 404s, so clients that go by error code won't notice.)
| * Sending messages to a deleted channel should send to the deleted channel's ↵Owen Jacobson2025-07-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ID, not to a fictitious ID. The existing test scenario: * Create a channel (with ID C1). * Delete channel C1. * Roll the dice to invent a channel ID C2. * Send a message to channel C2. * Observe that sending fails. This was not verifying anything about the deleted channel C1 - it was basically reproducing the `nonexistent_channel` test scenario with the most marginal of garnishes on it. This is probably copy-paste damage from when this test was originally written. Sending did fail, so this test scenario passed, but it passed effectively by accident. The new test scenario: * Create a channel (with ID C1). * Delete channel C1. * Send a message to channel C1. * Observe that sending fails. Concerningly, sending does _not_ fail in this scenario (i.e., the test _does_ fail), so the broken test case was masking a real bug.
* | Organize the developer docs into a "Pilcrow for Developers" book.Owen Jacobson2025-07-01
|/ | | | | | The audience for this is developers looking to make changes to Pilcrow, either on the server, on the included client, or via its data model. Most of the material here is drawn from existing documents, but organized somewhat more coherently. I've left some space for client documentation, though no such documents exist yet.
* Send back the current state as events, not snapshots, during client boot.ojacobson2025-07-01
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are a couple of contributing reasons for this. * Each client's author - even ourselves - is best positioned to know how best to convert history into state to meet the needs of that specific client. There is (probably) no universal solution. You can already see this with the built-in client, where unread tracking gets stapled onto snapshots locally and maintained as events roll in, and I would expect this to happen more and more regularly over time. If we ever sprout other clients, I'd also expect their local state to be different. The API, on the other hand, must expose a surface that's universal to all clients. For boot, that was a very rote list-of-nouns data model. The other option is to expose a surface specific to one client and make other clients accommodate, which is contrary to the goals of this project. * The need to compute snapshots adds friction when adding or changing the behaviour of the API, even when those changes only tangentially touch `/api/boot`. For example, my work on adding messages to multiple conversations got hung up in trying to figure out how to represent that at boot time, plus how to represent that in the event stream. * The rationale for sending back a computed snapshot of the state was to avoid having the client replay events from the beginning of time, and to limit the amount of data sent back. This didn't pan out - most snapshots in practice consisted of the same data you'd get from the event stream anyways, almost with a 1:1 correspondence (with a `sent` or `created` event standing in for a `messages`, `channels`, or `users` entry). Exceptions - deleted messages and channels - were rare, and are ephemeral. * Generating the snapshots requires loading the entire history into memory anyways. We're not saving any server-side IO by computing snapshots, but we are spending server-side compute time to generate them for clients that are then going to throw them away, as above. This change resolves these tensions by delegating state management _entirely_ to the client, removing the server-side state snapshots. The server communicates in events only. ## Alternatives I joked to @wlonk that the "2.0-bis" version of this change always returns `resume_point` 0 and an empty events list. That would be correct, and compatible with the client logic in this change, and would actually work. In fact, we could get rid of the event part of `/api/boot` entirely, and require clients to consume the event stream from the beginning every time they reconnect. The main reason I _don't_ want to do this has to do with reconnects. Right now - both with snapshots, before this change, and with events, after - the client can cleanly delineate "historical" events (to be applied while the state is not presented to the user) and "current" events (to be presented to the user immediately). The `application/event-stream` protocol has no way to make that distinction out of the box, and while we can hack something in, all the approaches I can think of are nasty. Merges boot-events into main.
| * Remove the snapshot fields from `/api/boot`.Owen Jacobson2025-06-20
| | | | | | | | Clients now _must_ construct their state from the event stream; it is no longer possible for them to delegate that work to the server.
| * Boot the client by consuming events.Owen Jacobson2025-06-20
| | | | | | | | We use the same event processing glue that the client has for keeping up with live events, which means that a significant chunk of state management code goes away entirely.
| * Include historical events in the boot response.Owen Jacobson2025-06-20
| | | | | | | | The returned events are all events up to and including the `resume_point` in the same response. If combined with the events from `/api/events?resume_point=x`, using the same `resume_point`, the client will have a complete event history, less any events from histories that have been purged.
| * Support querying event sequences via iterators or streams.Owen Jacobson2025-06-20
| | | | | | | | | | | | | | | | | | | | | | | | | | These filters are meant to be used with, respectively, `Iterator::filter_map` and `StreamExt::filter_map`. The two operations are conceptually the same - they pass an item from the underlying sequence to a function that returns an option, drops the values for which the function returns `None`, and yields the value inside of `Some` in the resulting sequence. However, `Iterator::filter_map` takes a function from the iterator elements to `Option<T>`. `StreamExt::filter_map` takes a function from the iterator elements to _a `Future` whose output is `Option<T>`_. As such, you can't easily use functions designed for one use case, for the other. You need an adapter - conventionally, `futures::ready`, if you have a non-async function and need an async one. This provides two sets of sequence filters: * `crate::test::fixtures::event` contains functions which return `Option` directly, and which are intended for use with `Iterator::filter_map`. * `crate::test::fixtures::event::stream` contains lifted versions that return a `Future`, and which are intended for use with `StreamExt::filter_map`. The lifting is done purely manually. I spent a lot of time writing clever-er versions before deciding on this; those were fun to write, but hell to read and not meaningfully better, and this is test support code, so we want it to be dumb and obvious. Complexity for the sake of intellectual satisfaction is a huge antifeature in this context.
| * Hoist heartbeat configuration out to the web handler.Owen Jacobson2025-06-20
| | | | | | | | | | | | The _snapshot_ is specifically a snapshot of app state. The purpose of the response struct is to annotate the snapshot with information that isn't from the app, but rather from the request or the web layer. The heartbeat timeout isn't ever used by the app layer in any way; it's used by the Axum handler for `/api/events`, instead. I straight-up missed this when I wrote the original heartbeat changes.
* | Remove misleading and long-since stale "panic" notes.Owen Jacobson2025-06-23
| | | | | | | | These were invalidated by eff129bc1f29bcb1b2b9d10c6b49ab886edc83d6, back in September.
* | Remove a stray reference to users being called "logins."Owen Jacobson2025-06-23
| | | | | | | | Missed in 9f7f82dbd9adee8ae18ae7ff2600b3e1dc8fadbc.