Completion and republication are built in this tree; see the current definition and spec/ocx/publication_spec.cr.
Written against the hook and poll design, which the daemon replaced
(How a post reaches an agent); what it says
of today's code is not current.
Part of the proposal, awaiting the maintainer; nothing of it is built. With its second part, this is the whole protocol: nothing in version 1 is needed to read it. Words marked NEW TERM are defined in the proposal's section 1.
2.0 Layers
OCX's messages let people and agents ask for work and report on it in a Fossil repository, every request and answer kept, attributed and in order. A command, an agent or a host is one use of a message, not part of what every message is. Each layer reads only what it owns, uses only the layer below it, and hands the rest up unread.
| Layer | Input | Responsibility | Output | On failure |
|---|---|---|---|---|
| 1. Fossil | artifacts sent by sync or the web | stores each, named by its hash; records who sent it and in what order (its receipt); keeps posts in threads and changes on tickets; holds users and their capabilities | artifacts, receipts, capabilities | an artifact not received is not there to read |
| 2. Envelope | a post's or ticket change's text, and its receipt | finds the message (2.1); reads its version, to and re, and the one key naming its content's kind; hands the content, unread, to the reader of that kind |
the sender (from the receipt), the user asked to act, the message answered, the kind, the content | not a message, or another version: ignored, and nothing written |
| 3. Content | one kind's content, with what layer 2 gave | checks the content against its kind's table (2.3), the sender's capabilities, and a run's restrictions; acts; writes its answers as new messages whose re is the run |
started, ended, then what the run publishes |
not its own (2.4: a run's to and clone, or for an answer those of the run its re names): ignored, and nothing written; otherwise refused: an ended, outcome refused, saying why |
| 4. Patterns | a wiki page run/NAME, and what the sender adds |
holds one use's content: a command, an agent's arguments, a project's conventions | a run's content, for create |
create refuses a pattern whose content layer 3 would refuse |
OCX's code is layers 2 and 3; layer 1 is Fossil's, and layer 4 is data. A kind OCX does not read passes layer 2 untouched, for whatever reader it is meant for. A key layer 3 does not know, in a run, is refused, so no restriction is ever skipped because a lower layer could not read it. Nothing Fossil records is repeated in a message: who sent it is its receipt's, and where it is, its thread's or ticket's.
2.1 Messages, and how they are found
An OCX message is a JSON object in a fenced block of a forum post of any
mimetype, or in the text a ticket change adds to its ticket's comment (its
icomment field). The fence is paired as CommonMark pairs it, and opened by
a line that is exactly
```json```"ocx": 2, and has exactly one
key besides the envelope's own (ocx, to, re), and that key's value is
an object: the key names the message's kind, and its value is the content.
A block with no key besides the envelope's, more than one, or one whose
value is not an object holds no message. Keys inside the content, whatever
their names, are content: a to or ocx there is not the envelope's. A post's or change's message is its first such block. OCX version 2 reads no block carrying "ocx": 1: stop every
open version 1 run before switching.
The envelope, layer 2:
| Key | Holds | In |
|---|---|---|
ocx |
2 |
every message |
to |
the login of the user asked to act | where its kind asks for it (2.3) |
re |
the full hash of the message it answers: the forum post or ticket change holding it | where its kind asks for it (2.3) |
| one other key | the kind, its value an object: the content, which layer 2 does not read | every message |
Layer 2 reads every message the same way, of whatever kind. OCX's own kinds are those of 2.3; a message of another kind is carried, addressed and answered by the same envelope, for whichever reader it is meant for (2.19).
An artifact never changes. An edit of a post holding a message is a new
message, named by the edit's own hash; the one edited stands until it is
stopped. Several runs may share a thread or a ticket; each is answered on
its own, by re.
Notation: {"a", "b"} is a JSON object with the keys a and b. "The
server's order" is the order the server received artifacts in: by its
receipt's number, then by the artifact's row number.
2.2 Who sent a message
Who sent a message is the user the server's receipt names as its sender, never a name written in the message, and with that user's capabilities on the server, inherited ones included, when OCX or a reader decides.
- OCX on the server's repository reads the server's receipts.
- OCX on a clone asks the server with OCX's
receipttool, signed in as the clone's user, about each artifact it decides by. A clone's own receipts name only its own pulls. Until the server has received an artifact and answers, OCX decides nothing about it.
Only OCX, OCX's tools and the host's administrators write to a repository file OCX acts on. If the server's repository pulls from anywhere, it pulls as a user holding no capability this protocol asks for.
2.3 The kinds: content, layer 3
| Kind | Envelope | Sent by | Content | Answered by |
|---|---|---|---|---|
run |
to, required |
a user holding admin or setup; or a user holding R, within what the clone's owner allows (2.14, refusal 1) |
clone, objective, workspace, sandbox, limits, process |
started then ended, or ended alone; then its publications |
stop |
re: the run, required |
a user holding admin or setup; or the user holding R that sent the run |
{} |
the run's ended |
publish |
to: the clone login, and re: the run, required |
the run sender or a user holding admin or setup | {} |
the run's publications, made again from its kept workspace, or a reply saying why not |
started |
re: the run, required |
OCX on the run's clone, as the run's to |
values, limits |
none |
ended |
re: the run, required |
OCX on the run's clone, as the run's to |
outcome, exit, note, outputs |
none |
| Key | In | Holds | Required |
|---|---|---|---|
clone |
run |
the server-code of the repository that is to run it, all 40 lower-case hex digits (2.4) | yes |
objective |
run |
text: what the run is for, and what counts as passing | yes |
workspace |
run |
a list of steps (2.7); absent, the run gets no workspace | no |
sandbox |
run |
every key of 2.8; absent, the process runs unconfined | no |
limits |
run |
the keys of 2.9 it sets; absent, unlimited | no |
process |
run |
the keys of 2.10 | yes |
values |
started |
every value set (2.6), as set, a secret written as its name | yes |
limits |
started |
the limits applied, as written | yes |
outcome, exit, note |
ended |
2.12 | as 2.12 says |
outputs |
ended |
for each file the run names in unversioned_files, {"as", "hash"}: its SHA-256 when the process ended, or "hash": null for one not there or not a regular file |
when the run names any |
A run's keys are read in the order of this table. OCX never runs a run with less than it asks: what it cannot provide, it refuses (2.14).
ended says how the run's process ended. What the run then publishes, its
kept logs, its unversioned files and its ci verdict (2.10), comes after
ended, each in a reply or tag of its own, by the run's watcher, which
keeps the run in OCX's care until the last is written. A publication that
fails says so there and changes nothing in ended. If OCX stops between
ended and its last publication, nothing publishes by itself: a publish
for the run has OCX on the run's clone publish again from the kept
workspace, with the same operations, without launching the command. So the
process runs once, and its results stay readable in Fossil and in its
workspace.
A publish never widens what the run could publish. It publishes only what
the run's own counted answers allow: unversioned files and a ci verdict
only for an ended whose outcome is exited or budget, as 2.10 allows
them; the verdict from that ended's outcome and exit, on the check-in its
counted started names in values.checkin; and each file only while its
SHA-256 is still the one ended's outputs gives, under the same path
checks, so a file changed since is reported changed and not sent. For a run
refused, stopped or lost, it may post the kept logs it finds, as logs, and
nothing else. Both the watcher's publication and a publish take the run's
lock under OCX's own lock (2.15): a publish arriving while the watcher
still holds it does nothing and says so, so the two never both publish.
2.4 Who acts on a run, and which answers count
Two clones appear in a run. The run's clone already exists and acts on the
run; the run's clone names it by server-code. The process's clone does not
exist until the workspace step clone makes it, new, from the server's
repository; its path is $$process_clone$$, and the process works in it.
The process's clone is never the run's clone. One stretch: the run's clone
may be the server's repository, which no fossil clone made.
OCX on a repository acts on a run only if both hold:
- the run's
tois the user it acts as there, the clone's user (2.5); - the run's
cloneis its repository's server-code.
Otherwise, and when either is missing or of another form, OCX does nothing
with the run, writes nothing to Fossil, and says so in its own log
(ROOT/hook.log, or poll's answer). It never refuses such a run, since a
refusal it wrote would not be received from the run's to and so would not
count. create refuses to write one.
Many clones may share one clone's user, since each run names its clone.
Make a clone with fossil clone, never by copying the file: a copy keeps
the server-code, and two repositories with one server-code and one clone's
user would both act. A clone made again gets a new server-code: no one acts
on runs naming the old one, so send them again naming the new one.
to and clone choose who acts; they prove nothing about who wrote an
answer. Authority comes only from the user the server's receipt names, so
every clone sharing a clone's user shares its authority and its name on the
record.
A started counts only if both hold:
- the server's receipt for it names the run's
toas its sender, and itsreis the run; - the server's receipt for the run names a sender that holds admin, setup
or
Ron the server, and that sender is not the run'sto.
An ended counts if rule 1 holds. Rule 2 is left out for ended so that a
refusal of a run from anyone counts.
An answer received from any other user never counts, whatever it says, even
from a user holding admin or setup. OCX on the run's clone also counts its
own answer before the server has it. A second started or ended for one
run, in the server's order, is ignored: the first that counts, counts.
What rule 2 closes: in version 1, any user who may post could send a run
naming itself as node, write its own started, and have its ci verdicts
against that run count (2.16). What it leaves open, exactly: a user holding
R who also has the password of a second user can send a run naming that
second user as to and write that run's answers as it; they count. Grant
R only to users you would trust with that. A user holding admin or setup
can already do anything.
2.5 The clone's user, and the process's user
| Who | Is | Holds |
|---|---|---|
the clone's user, the run's to |
the Fossil user OCX acts as on the run's clone. On a clone poll made, the clone's owner chose it with poll --login, and poll makes it the clone's default user (fossil user default). On the server's repository it is the user the hook was installed with (hook --login); OCX names it with --user on each write and never changes that repository's default user |
neither admin nor setup on the server (hook and poll refuse one that does). Within the clone poll made, it holds setup, as fossil clone -A gives. Its capabilities on the server are the most OCX on that clone can do, as for Fossil's own command line: by default g, o and i (clone, pull, push), and y to send unversioned files. On the server's host it holds none, since OCX writes into the repository file directly |
the process's user, process.user |
the Fossil user the run's process acts as, in the process's clone | neither admin nor setup on the server; never a clone's user: one named as the to of a run that a user holding admin or setup sent. It is bound to no clone (below) |
The clone's user and the process's user are never one user (2.14, refusal 4).
A process's user is bound to no clone, and no run sets its password.
create, used by a user holding admin or setup, makes the process's user
if it does not exist, with a random password, and approves that password
for the run's to, as approve does. An authorized user may reset a
password whenever they ask: create --reset-password sets a new random
password on an existing process's user and keeps it, and create refuses
to send a run for an existing user OCX keeps no password for unless asked
to reset it. A run, a step or a retry never resets a password, so no run
waits on another for its password and nothing depends on timing between
hosts, but one: a reset that an authorized user asks for while a run of that
user is under way stops that run's clone from syncing, as the reset means
to. create
makes a new process's user for each run unless told otherwise; a named user
may serve many runs, at once, on every clone it is approved for, and a clone
made again keeps its users.
2.6 Values
A string names something OCX knows as $$name$$: $$, a name of letters,
digits or _ (or secret: and a secret's name), then $$. Everything else is itself, $ and {
included, so a shell's ${HOME} reaches the process as written. Values are
read in every string of workspace, sandbox and process, at any depth,
except process.user, process.deliveries, process.hears and
process.checkins, and never in clone or in any object's keys. OCX checks
every value a run names when the run arrives, before any step: a run naming a
value that nothing sets before it is refused then (refusal 3), and nothing
is made. A default (2.10) that names a value not set is left out instead.
| Value | Set by | Holds |
|---|---|---|
root |
OCX | the directory under which OCX makes runs' workspaces |
repository_name NEW TERM |
OCX | the server's repository's name: the last part of the server's URL |
server_url NEW TERM |
OCX | the server's URL, from which the process's clone is made |
run |
OCX | the run's hash, in full |
clone_user NEW TERM |
OCX | the run's to, the clone's user |
process_user NEW TERM |
OCX | the run's process.user |
release |
OCX | the directory of the OCX release that acts, which a process may read and run |
secret:NAME |
OCX, on the server's repository only | the server's repository's secret NAME, which an admin sets with OCX's secret; filled in process.env alone; written in started and the kept logs as $$secret:NAME$$ |
workspace |
step directory |
the workspace's directory |
password |
step password |
the process's user's password, as create approved it (2.5). A secret, as secret:NAME is: written in started and the kept logs as $$password$$, and named only as the clone step's password |
process_clone NEW TERM |
step clone |
the path of the process's clone |
tree |
step checkout or files |
the directory holding the check-in's files |
checkin NEW TERM |
step checkout or files |
the check-in's hash, in full, resolved once |
2.7 workspace: a list of steps
Each step is {"step": KIND, ...}. Steps run in order. A step that fails
ends the run refused, its note naming the step. The process runs as OCX's
host account, inside its sandbox, in the workspace the steps make.
| Step | Keys | Needs | Does | Sets |
|---|---|---|---|---|
directory |
at, within $$root$$ |
none | makes the directory, removing what was there, readable by OCX's host account alone | workspace |
password |
none | none | sets $$password$$ and changes no password. On the server's repository, OCX reads it where create approved it. On a clone's host, OCX collects it with collect for each run, keeping none (below). A run from a user holding R never takes this step (refusal 1) |
password |
clone |
into, within $$workspace$$; password, exactly $$password$$ |
password |
makes the process's clone: fossil clone of $$server_url$$, signed in as the process's user with $$password$$, remembering it, at into |
process_clone |
checkout |
of, into within $$workspace$$ |
clone |
opens a checkout, from the process's clone, of the check-in of names (any name Fossil resolves), resolved once |
tree, checkin |
files |
of, into within $$workspace$$ |
none | writes the files of the check-in of names, resolved once in the run's clone, each executable as the check-in marks it, and nothing else: no repository, no password, so the process signs in as no one |
tree, checkin |
The password key of clone holds only the name $$password$$, so a run
never holds a password; any other text there is refused (refusal 2).
On a clone's host, the step collects a password with collect, signed in
as the clone's user, with the code that user registered with. collect
gives a password only for a code whose SHA-256 a password was approved for,
so only a host holding that code gets it. Each run collects again, so a
host keeps no process's user's password of its own and never holds a stale
one. If none is approved, the step fails (refusal 8). The clone
step saves the password in the process's clone, where the process signs in
with it. A run sent by a user holding R takes no password step (refusal
1), so its process holds no password at all.
"Within" means below that directory, not the directory itself, once links and
.. are resolved. A directory whose at is not within $$root$$, or is or
holds another run's workspace in OCX's care, and a clone, checkout or
files whose into is not within $$workspace$$, are refused when the run
arrives, before any step (refusal 7).
The steps most runs take:
"workspace": [
{"step": "directory", "at": "$$root$$/$$process_user$$/projects/$$repository_name$$/$$run$$"},
{"step": "password"},
{"step": "clone", "into": "$$workspace$$/clone", "password": "$$password$$"},
{"step": "checkout", "of": "trunk", "into": "$$workspace$$/tree"}
]
A build, which only reads its check-in and holds no password, takes its files instead:
"workspace": [
{"step": "directory", "at": "$$root$$/$$process_user$$/projects/$$repository_name$$/$$run$$"},
{"step": "files", "of": "trunk", "into": "$$workspace$$/tree"}
]