Retain deleted messages and channels temporarily, to preserve events for replay.

Previously, when a channel (message) was deleted, `hi` would send events to all _connected_ clients to inform them of the deletion, then delete all memory of the channel (message). Any disconnected client, on reconnecting, would not receive the deletion event, and would de-synch with the service. The creation events were also immediately retconned out of the event stream, as well.

With this change, `hi` keeps a record of deleted channels (messages). When replaying events, these records are used to replay the deletion event. After 7 days, the retained data is deleted, both to keep storage under control and to conform to users' expectations that deleted means gone.

To match users' likely intuitions about what deletion does, deleting a channel (message) _does_ immediately delete some of its associated data. Channels' names are blanked, and messages' bodies are also blanked. When the event stream is replayed, the original channel.created (message.sent) event is "tombstoned", with an additional `deleted_at` field to inform clients. The included client does not use this field, at least yet.

The migration is, once again, screamingingly complicated due to sqlite's limited ALTER TABLE … ALTER COLUMN support.

This change also contains capabilities that would allow the API to return 410 Gone for deleted channels or messages, instead of 404. I did experiment with this, but it's tricky to do pervasively, especially since most app-level interfaces return an `Option<Channel>` or `Option<Message>`. Redesigning these to return either `Ok(Channel)` (`Ok(Message)`) or `Err(Error::NotFound)` or `Err(Error::Deleted)` is more work than I wanted to take on for this change, and the utility of 410 Gone responses is not obvious to me. We have other, more pressing API design warts to address.

2 files changed, 31 insertions, 15 deletions

diff --git a/docs/api/channels-messages.md b/docs/api/channels-messages.md
index d032ac1..99a525e 100644
--- a/docs/api/channels-messages.md
+++ b/docs/api/channels-messages.md
@@ -8,7 +8,7 @@ stateDiagram-v2
 	[*] --> Active : POST /api/channels
 	Active --> Deleted : DELETE /api/channels/C1234
 	Active --> Deleted : Expiry
-	Deleted --> [*]
+	Deleted --> [*] : Purge
 ```
 
 ```mermaid
@@ -19,14 +19,19 @@ stateDiagram-v2
 	[*] --> Sent : POST /api/channels/C1234
 	Sent --> Deleted : DELETE /api/messages/Mabcd
 	Sent --> Deleted : Expiry
-	Deleted --> [*]
+	Deleted --> [*] : Purge
 ```
 
 Messages allow logins to communicate with one another. Channels are the conversations to which those messages are sent.
 
 Every channel has a unique name, chosen when the channel is created.
 
-Both channels and messages expire after a time, to prevent the service from consuming unlimited amounts of storage. Messages expire 90 days after being sent. Channels expire 90 days after the last message sent to them, or after creation if no messages are sent in that time.
+
+## Expiry and purging
+
+Both channels and messages expire after a time. Messages expire 90 days after being sent. Channels expire 90 days after the last message sent to them, or after creation if no messages are sent in that time.
+
+Deleted channels and messages, including those that have expired, are temporarily retained by the service, to allow clients that are not connected to receive the corresponding deletion [events](./events.md). To limit storage growth, deleted channels and messages are purged from the service seven days after they were deleted.
 
 
 ## `POST /api/channels`
@@ -103,6 +108,7 @@ This endpoint will respond with a status of `202 Accepted` when successful. The
 This endpoint will respond with a status of `404 Not Found` if the channel ID is not valid.
 
 
+
 ## `DELETE /api/channels/:id`
 
 Deletes a channel (and all messages in it).
diff --git a/docs/api/events.md b/docs/api/events.md
index 9fe9ca9..da2f4b6 100644
--- a/docs/api/events.md
+++ b/docs/api/events.md
@@ -133,11 +133,16 @@ Sent whenever a new channel is created.
 
 These events have the `event` field set to `"created"`. They include the following additional fields:
 
-| Field  | Type      | Description |
-|:-------|:----------|:--|
-| `at`   | timestamp | The moment the channel was created. |
-| `id`   | string    | A unique identifier for the newly-created channel. This can be used to associate the channel with other events, or to make API calls targeting the channel. |
-| `name` | string    | The channel's name. |
+| Field        | Type                | Description |
+|:-------------|:--------------------|:--|
+| `at`         | timestamp           | The moment the channel was created. |
+| `id`         | string              | A unique identifier for the newly-created channel. This can be used to associate the channel with other events, or to make API calls targeting the channel. |
+| `name`       | string              | The channel's name. |
+| `deleted_at` | timestamp, optional | If set, the moment the channel was deleted. |
+
+When a channel is deleted or expires, the `"created"` event is replaced with a tombstone `"created"` event, so that the original channel cannot be trivially recovered from the event stream. Tombstone events have a `deleted_at` field, and a `name` of `""`.
+
+Once a deleted channel is [purged](./channels-messages.md#expiry-and-purging), these tombstone events are removed from the event stream.
 
 ### Channel deleted
 
@@ -184,13 +189,18 @@ Sent whenever a message is sent by a client.
 
 These events have the `event` field set to `"sent"`. They include the following additional fields:
 
-| Field     | Type      | Description |
-|:----------|:----------|:--|
-| `at`      | timestamp | The moment the message was sent. |
-| `channel` | string    | The ID of the channel the message was sent to. |
-| `sender`  | string    | The ID of the login that sent the message. |
-| `id`      | string    | A unique identifier for the message. This can be used to associate the message with other events, or to make API calls targeting the message. |
-| `body`    | string | The text of the message. |
+| Field        | Type                | Description |
+|:-------------|:--------------------|:--|
+| `at`         | timestamp           | The moment the message was sent. |
+| `channel`    | string              | The ID of the channel the message was sent to. |
+| `sender`     | string              | The ID of the login that sent the message. |
+| `id`         | string              | A unique identifier for the message. This can be used to associate the message with other events, or to make API calls targeting the message. |
+| `body`       | string              | The text of the message. |
+| `deleted_at` | timestamp, optional | If set, the moment the message was deleted. |
+
+When a message is deleted or expires, the `"sent"` event is replaced with a tombstone `"sent"` event, so that the original message cannot be trivially recovered from the event stream. Tombstone events have a `deleted_at` field, and a `body` of `""`.
+
+Once a deleted message is [purged](./channels-messages.md#expiry-and-purging), these tombstone events are removed from the event stream.
 
 ### Message deleted