| Commit message (Collapse) | Author | Age |
|---|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
This will make it much easier to slot in new event types (login events!).
|
| |
|
|
| |
This structure didn't accomplish anything and made certain refactorings harder.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
The client now takes an initial snapshot from the response to `/api/boot`, then picks up the event stream at the immediately-successive event to the moment the snapshot was taken.
This commit removes the following unused endpoints:
* `/api/channels` (GET)
* `/api/channels/:channel/messages` (GET)
The information therein is now part of the boot response. We can always add 'em back, but I wanted to clear the deck for designing something more capable, for dealing with client needs.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
It is deliberate that the expire() functions do not use them. To avoid races, the transactions must be committed before events get sent, in both cases, which makes them structurally pretty different.
|
| |
|
|
|
|
| |
This separates the code that figures out what happened to an entity from the code that represents it to a user, and makes it easier to compute a snapshot at a point in time (for things like bootstrap). It also makes the internal logic a bit easier to follow, since it's easier to tell whether you're working with a point in time or with the whole recorded history.
This hefty.
|
| | |
|
| |
|
|
| |
This helped me discover an organizational scheme I like more.
|
| |
|
|
| |
This is primarily renames and repackagings.
|
| |
|
|
| |
(This is part of a larger reorganization.)
|
| |
|
|
| |
sequence.
|
| |
|
|
| |
Per-channel event sequences were a cute idea, but it made reasoning about event resumption much, much harder (case in point: recovering the order of events in a partially-ordered collection is quadratic, since it's basically graph sort). The minor overhead of a global sequence number is likely tolerable, and this simplifies both the API and the internals.
|
| |
|
|
|
|
|
|
| |
expires.
When tokens are revoked (logout or expiry), the server now publishes an internal event via the new `logins` event broadcaster. These events are used to guard the `/api/events` stream. When a token revocation event arrives for the token used to subscribe to the stream, the stream is cut short, disconnecting the client.
In service of this, tokens now have IDs, which are non-confidential values that can be used to discuss tokens without their secrets being passed around unnecessarily. These IDs are not (at this time) exposed to clients, but they could be.
|
| | |
|
| |
|
|
|
|
|
|
| |
Trying to reliably do expiry mid-request was causing some anomalies:
* Creating a channel with a dup name would fail, then succeed after listing channels.
It was very hard to reason about which operations needed to trigger expiry, to fix this "correctly," so now expiry runs on every request.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
It now includes events for all channels. Clients are responsible for filtering.
The schema for channel events has changed; it now includes a channel name and ID, in the same format as the sender's name and ID. They also now include a `"type"` field, whose only valid value (as of this writing) is `"message"`.
This is groundwork for delivering message deletion (expiry) events to clients, and notifying clients of channel lifecycle events.
|
| |
|
|
| |
I had no idea `std` included a `matches!` macro, and I feel we're better off using it.
|
| | |
|
| |
|
|
| |
This'll catch style issues, mostly.
|
| |
|
|
| |
vector-of-sequence-numbers stream resume.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
streams.
The timestamp-based approach had some formal problems. In particular, it assumed that time always went forwards, which isn't necessarily the case:
* Alice calls `/api/channels/Cfoo` to send a message.
* The server assigns time T to the request.
* The server stalls somewhere in send() for a while, before storing and broadcasting the message. If it helps, imagine blocking on `tx.begin().await?` for a while.
* In this interval, Bob calls `/api/events?channel=Cfoo`, receives historical messages up to time U (after T), and disconnects.
* The server resumes Alice's request and finishes it.
* Bob reconnects, setting his Last-Event-Id header to timestamp U.
In this scenario, Bob never sees Alice's message unless he starts over. It wasn't in the original stream, since it wasn't broadcast while Bob was subscribed, and it's not in the new stream, since Bob's resume point is after the timestamp on Alice's message.
The new approach avoids this. Each message is assigned a _sequence number_ when it's stored. Bob can be sure that his stream included every event, since the resume point is identified by sequence number even if the server processes them out of chronological order:
* Alice calls `/api/channels/Cfoo` to send a message.
* The server assigns time T to the request.
* The server stalls somewhere in send() for a while, before storing and broadcasting.
* In this interval, Bob calls `/api/events?channel=Cfoo`, receives historical messages up to sequence Cfoo=N, and disconnects.
* The server resumes Alice's request, assigns her message sequence M (after N), and finishes it.
* Bob resumes his subscription at Cfoo=N.
* Bob receives Alice's message at Cfoo=M.
There's a natural mutual exclusion on sequence numbers, enforced by sqlite, which ensures that no two messages have the same sequence number. Since sqlite promises that transactions are serializable by default (and enforces this with a whole-DB write lock), we can be confident that sequence numbers are monotonic, as well.
This scenario is, to put it mildly, contrived and unlikely - which is what motivated me to fix it. These kinds of bugs are fiendishly hard to identify, let alone reproduce or understand.
I wonder how costly cloning a map is going to turn out to be…
A note on database migrations: sqlite3 really, truly has no `alter table … alter column` statement. The only way to modify an existing column is to add the column to a new table. If `alter column` existed, I would create the new `sequence` column in `message` in a much less roundabout way. Fortunately, these migrations assume that they are being run _offline_, so operations like "replace the whole table" are reasonable.
|
| | |
|
| |
|
|
| |
This is intended to make it a bit more opaque to callers, and to free me up to experiment with the event ID format. It also makes event IDs tractable for testing.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This API structure fell out of a conversation with Kit. Described loosely:
kit: ok
kit: Here's what I'm picturing in a client
kit: list channels, make-new-channel, zero to one active channels, post-to-active.
kit: login/sign-up, logout
owen: you will likely also want "am I logged in" here
kit: sure, whoami
|
| |
|
|
|
|
|
|
|
|
| |
This is intended to manage storage growth. A community with broadly steady traffic will now reach a steady state (ish) where the amount of storage in use stays within a steady band.
The 90 day threshold is a spitball; this should be made configurable for the community's needs.
I've also hoisted expiry out into the `app` classes, to reduce the amount of non-database work repo types are doing. This should make it easier to make expiry configurable later on.
Includes incidental cleanup and style changes.
|
| | |
|
| | |
|
| |
|
|
| |
This is implemented by making the return values, in most cases, idiosyncratic ad-hoc types that then convert to the approprate error response. This also should make endpoints more testable, since the return values can now be inspected to check their properties without having to process or parse an HTTP response.
|
| | |
|
| | |
|
| |
|
|
| |
This provides a convenient place to _stick_ "not found" errors, though actually introducing them will come in a later commit.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Having them contained in the individual endpoint groups conveyed an unintended sense that their intended scope was _only_ that endpoint group. It also made most repo-related import paths _quite_ long. This splits up the repos as follows:
* "General applicability" repos - those that are only loosely connected to a single task, and are likely to be shared between tasks - go in crate::repo.
* Specialized repos - those tightly connected to a specific task - go in the module for that task, under crate::PATH::repo.
In both cases, each repo goes in its own submodule, to make it easier to use the module name as a namespace.
Which category a repo goes in is a judgment call. `crate::channel::repo::broadcast` (formerly `channel::repo::messages`) is used outside of `crate::channel`, for example, but its main purpose is to support channel message broadcasts. It could arguably live under `crate::event::repo::channel`, but the resulting namespace is less legible to me.
|
| |
|
|
| |
BoxedError conceals the exact nature of the error, which in turn prevents me from using sqlx::Error::RowNotFound to signal absences.
|
| |
|
|
| |
This bugged me aesthetically. At `app.channel().events(channel)`, the caller knows the channel ID; they don't need to be told. Having the same info come back out in the returned events felt bad.
|
| |
|
|
| |
This is groundwork for a JSON-based API, after a conversation with Kit.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
While reviewing [MDN], I noticed this note:
> SSE suffers from a limitation to the maximum number of open connections, which can be specially painful when opening various tabs as the limit is per browser and set to a very low number (6). […] This limit is per browser + domain, so that means that you can open 6 SSE connections across all of the tabs to www.example1.com and another 6 SSE connections to www.example2.com.
I tested it in Safari; this is true, and once six streams are open, _no_ more requests can be made - in any tab, even a fresh one.
Since the design _was_ that each channel had its own events endpoint, this is an obvious operations risk. Any client that tries to read multiple channels' streams will hit this limit quickly.
This change consolidates all channel events into a single endpoint: `/events`. This takes a list of channel IDs (as query parameters, one `channel=` param per channel), and streams back events from all listed channels. The previous `/:channel/events` endpoint has been removed. Clients can selectively request events for the channels they're interested in.
[MDN]: https://developer.mozilla.org/en-US/docs/Web/API/EventSource
|
| | |
|
| | |
|
