Introduce input ports.

Ports are the lisp abstraction of files and streams. Actinide ports additionally guarantee a peek operation.

This makes ``tokenize`` (now ``read_token``) callable as a lisp function, as it
takes a port and reads one token from it. This is a substantial refactoring.

As most of the state is now captured by closures, it's no longer practical to
test individual states as readily. However, the top-level tokenizer tests
exercise the full state space.

1 files changed, 1 insertions, 1 deletions

diff --git a/README.rst b/README.rst
index 374af24..595c43e 100644
--- a/README.rst
+++ b/README.rst
@@ -67,7 +67,7 @@ Freestanding REPL
 *****************
 
 **Note: this section is presently incorrect - the ``actinide-repl`` command
-instead contains a test harness for the tokenizer.**
+doesn't exist.**
 
 The Actinide interpreter can be started interactively using the
 ``actinide-repl`` command. In this mode, Actinide forms can be entered