Callable Human Judgment
Callable Human Judgment is human-derived judgment represented so a system can request its application within a defined scope. An agent can draw on a person's represented way of weighing a decision without requiring that person to be present every time.
The question behind our work is how that judgment can be useful to a system while the person keeps a say in what represents them and how it is used.
PAVE
Where it applies
Take an editor deciding whether a story is ready. What matters might be the source behind one detail, or a gap that another editor would let pass. Incorporating Callable Human Judgment might include taking earlier human judgment used when editing a story, and applying its weighing into a later workflow whether it’s related or not. In this example the editor has made their judgment and experience available for machines to interact with, within a scope she defines and has agreed to.
Another example would be a system requesting access to human judgment at the point in its workflow where that judgment is relevant. The workflow might also require someone to approve publication. So, in this example Human-in-the-loop becomes relevant and describes the participation arrangement while Callable Human Judgment describes the represented contribution it can draw on. The two can coexist.
In all examples the person remains the authority on what represents their thinking while permission to consult their judgment and permission to act are kept separate. Relatedly, we believe human provenance is important and should remain traceable at the source level.
The editor is an illustrative example.