Versioning a graph

A graph moves. A number does not: it was produced by one shape of the graph, and if the graph can be edited afterwards the number stops meaning anything. Versioning is what keeps the two apart.

Where versions are kept

A store is where the versions of a lineage live. Which store you have is chosen once, at the edge, and nothing after that point knows which one it got.

import chatty_lab as cl

cl.Store.in_memory()                  # lives as long as the object does — for tests
cl.Store.at("./work")                 # the .chatty inside this directory
cl.Store.on("https://chatty-lab.com", # one lineage on a platform
            token=TOKEN, graph="3f47b2c8-…", workspace=TEAM)

Store.at makes the .chatty if it is not there yet. Store.on is happy with either the API root or the address of the site — it adds the /api. Its graph is the lineage id over there, which need not be the one it has here.

A store answers two questions about a lineage directly:

store.versions("3f47b2c8-…")   # [Version, …] — unordered
store.refs("3f47b2c8-…")       # [Ref(name='main', head='…'), …]

A workdir

A Workdir is one graph’s history and a branch of it — the copy somebody works in.

it = cl.Workdir(cl.Store.at("./work"), graph="3f47b2c8-…", branch="main")

it.graph      # '3f47b2c8-…'
it.branch     # 'main'
it.store      # the store it is a line of work in — the same object, not a copy

A handle, not a checkout

Nothing is written into the directory for you to edit. A caller in Python is already holding a dict, so what a version holds comes back as a value — it.content("HEAD~1").graph — and the .chatty beside it holds the history, not a copy of the files. The chatty command is the half that does keep a working copy on disk, because a terminal has no dict to hand you.

It used to be called Investigation

chatty_lab.Investigation still imports and is the same class, with a DeprecationWarning. An investigation is what somebody does; this is the working copy of a lineage, the way a working tree is git’s.

Committing

done = it.commit(graph, {agent_id: definition}, message="the reader is stricter")

done.version.n            # 2
done.version.digest.short # 'a1b2c3d4e5f6'
done.already              # False

The second argument is the agent definitions the graph points at, keyed by id. They are part of what gets named: editing a prompt inside an agent is editing the graph.

Commit the same content again and nothing is written — the answer comes back with already set, holding the version that was there already.

That is not a check catching a repeat

A version is its content. Its name is a digest of the canonical form of the graph and of every agent definition the graph points at, so arriving twice at the same thing arrives at one version. This is what makes two histories that grew apart able to take each other in at all.

Before you commit

status() answers would this write anything, and what would it grow out of:

where = it.status(graph, agents)

where.branch     # 'main'
where.head       # Version | None — None before the branch's first commit
where.holding    # Version | None

Three shapes, told apart by two fields:

holding	head	Means
is head	a version	Nothing to commit.
None	a version	Unsaved work — this content is not in the lineage.
None	None	The branch has not been born yet.
some other version	a version	The lineage already holds this content; it is simply not this branch’s head.

Reading the history

it.log()                  # everything, newest first
it.log("main~3..main")    # a range
it.log("stricter")        # the history back from that revision

Note

log() with no argument is everything in the lineage, not a walk back from this branch. Three variants of one idea are three branches, and a walk from one tip cannot see its own siblings.

A revision is a revspec: a branch name, HEAD, ~n back from either, a range with .., or a digest.

it.at("HEAD")             # Version
it.at("main~2")           # Version
held = it.content("HEAD~1")
held.graph                # the blob
held.agents               # the definitions it pinned, in reduced form
held.version              # which version this is

content() is chatty checkout without a directory. The definitions come back in the reduced form a version keeps, which is what commit wants — reducing an already-reduced one changes nothing, so a round trip is safe.

What a version is

id	this store’s id for it — not the identity
digest	what it is, and the only thing that means the same in two different stores
kind	graph, dataset or job
parent_ids	one ordinarily, two for a merge, none for the first
op	edit, merge, fork, import, …
op_params	whatever that verb recorded
message	what somebody said it was for
n	its place in the lineage as this store counts it — what makes it “v3” out loud
created_by, created_at	who and when

n is local. The digest is the identity.

The store that writes a version mints the ordinal against what it holds, so the same content can be v2 here and v3 there. Git is the same: a clone does not promise that its fourteenth commit is anybody else’s fourteenth. Never key anything on n across two stores — key it on digest.

Branching

strict = it.start("stricter", at="HEAD~2")     # name an old version, and work there
strict.commit(narrower, message="a reader that refuses more")

it.on("stricter")                              # the same history, another line of it

start() is the verb a commit cannot stand in for. Committing at content the lineage already holds is idempotent, so go back to v2 and start a variant there has no way to be said by committing. Branch names take letters, digits and ., _, /, - — enough for a revspec to tell a name from a range or a digest.

Comparing

d = it.diff("main", "stricter")

bool(d)             # False when nothing moved
d.unchanged         # how many elements are the same
d.not_comparable    # the one number worth reading first
for c in d.changes:
    c.id            # the node or edge
    c.findings      # ['added'], ['changed', 'downstream'], ['gone'], ['moved'], …
    c.fields        # which parts differ — only beside 'changed'
    c.because_of    # the upstream nodes whose keys moved: *why*, not only *that*

not_comparable is first for a reason: results either side of those elements are not results about the same thing, so a diff that looks small can still invalidate every comparison you were about to make.

d.about carries findings about the whole thing rather than an element of it. It is always empty for a graph, which is its nodes.

Merging

got = it.merge("stricter", message="bring it in")

got.landed          # does this branch now hold what came out?
got.already         # there was nothing on the other side that is not here
got.fast_forwarded  # the branch simply moved; no version was written
got.version         # None when nothing was written
got.clashes         # what somebody has to choose a side for — never partial

When there are clashes, nothing is written. Answer them and call again:

if got.clashes:
    for c in got.clashes:
        c.id, c.part          # 'reader', 'node'
        c.ours, c.theirs      # None on one side means that line deleted what the other edited
        c.fields              # which parts the two lines disagree about

    it.merge(
        "stricter",
        message="bring it in, keeping our reader",
        choose={"reader": "ours", "e_input__reader": "theirs"},
    )

choose is keyed by id across both halves of the graph. agents= on a merge is what the definitions are now, and wins over what either version pinned.

landed can be False even on success

A blend can land on content the lineage already holds. Then nothing is written, version is the one that already holds it — on the other line — and the branch has not moved. Idempotence by content doing its job.

When it refuses

Every exception below inherits ChattyError, so one name catches the lot.

Exception	Raised when
GraphError	The graph handed in is not one the engine could run.
CommitError	The version could not be decided.
BranchError	That name could not be given to a version.
RevisionError	That revspec does not point at anything here.
MergeError	These two lines cannot be brought together at all.
StoreError	The store could not answer.
SyncError	This line of work could not be carried to the other store.
PlatformError	The platform’s HTTP API refused — carries .status.

Where to go next

Sending it to the platform push, pull, and why the same version has a different uuid at each end.

The chatty command The same verbs over a directory you can edit.

Defining a graph Where the blob you are committing comes from.

End to end A whole loop: version, push, run, read the number, change it, do it again.