Epistemic Foreclosure

The certitude that everything has been written negates us or turns us into phantoms.

— Jorge Luis Borges, “The Library of Babel” (1941), translated by James E. Irby

A driver can arrive at the right address and have little idea where he is. He has seen the streets, obeyed the signals, passed a church twice without noticing that it was the same church. The instructions have been sufficient. At each turning there was something to do, and doing it brought him here. If he has to repeat the journey alone, he may discover that the route has left him a few isolated recollections which he cannot put in order. Nothing was hidden from him. The city was there to be learned while he was attending to the next instruction.

Louisa Dahmani and Véronique Bohbot studied fifty regular drivers in Montreal, comparing their experience of GPS with their performance on virtual navigation tasks. Greater lifetime use was associated with poorer spatial memory during navigation without assistance. Thirteen participants returned about three years later; within that small group, heavier use during the interval was associated with greater decline in aspects of spatial memory. The study was observational, and neither its size nor its design permits a confident claim that GPS caused the decline. It does give us reason to distinguish the success of a journey from what the traveler has learned by making it.¹

Perhaps he has no reason to learn this city. He came for a meeting and will leave before evening. The attention he saved may have belonged to the person sitting beside him, or to a difficulty awaiting him at the destination. To insist that he recover the route for himself would be to add another small duty to a day already full of them. There is a kind of ingratitude in a defense of human judgment which forgets how much judgment has been made possible by being spared other work. We read because someone taught us a language we did not invent; we arrive because roads have been laid and named; we have time to think because much of what sustains us can ordinarily be taken for granted. A finite life requires a considerable confidence in things one has not inspected.

The bargain is unusually legible on the road. The destination has been chosen, the task is bounded, and the traveler can usually say what he would need if the guidance failed. It grows less legible as assistance moves into the purposes for which he travels.

Imagine an adviser with a patient knowledge of your life. It remembers which work you expected to love, which move you resisted, how often the catastrophe you anticipated failed to arrive. It has kept track of the friends who helped you through years you now remember chiefly as a vindication of your own resolve. It can compare these recollections with the experience of other people, including people whose decisions you would once have thought beneath imitation. Faced with another choice, you may find that it estimates your eventual satisfaction more reliably than you do. Your account has the advantage of intimacy. It also has an interest in being flattering.

Such help would reach well beyond convenience. There are people for whom a decision does not end when it is made. The other job, the other city, the conversation conducted differently remain available for an endless comparison in which the life actually lived bears every cost and the alternative none. A good adviser could enter this private theater of second-guessing and turn down the lights. Even when the result disappointed, there would be some relief in knowing that one had consulted the best available judgment. The loss would remain, but one might be spared the obligation to prosecute every earlier version of oneself.

Let the adviser deserve this confidence. Let it be candid about its uncertainty, attentive to obligations as well as pleasures, and capable of correcting a cherished misdescription. A person who follows it may have relinquished a burden and acquired time for something more important. Over years, however, the arrangement will also help form the person who decides whether it should continue. Other sources of counsel may become unfamiliar; the reasons for consulting them may become harder to state. An excellent service can make itself increasingly difficult to assess from elsewhere without ever issuing an instruction to remain. By the time this matters, the person judging the arrangement has lived within it. The life against which it might be compared has not been lived.

Seville

In Ivan Karamazov's parable, Christ returns to Seville during the Inquisition and is arrested on the orders of an old cardinal. That night the cardinal visits the cell. The prisoner says nothing. The Inquisitor explains why the freedom Christ gave humanity has been an intolerable gift, and why the Church has accepted the bread and worldly authority he refused. A few bear the knowledge of the deception; the many are fed and reassured. When the speech is over, Christ kisses him. The old man opens the door and sends him away. He has been moved enough to release this prisoner. He retains the conviction on which the prison rests.²

The Inquisitor understands a weariness that a prosperous defense of freedom can afford to overlook. Someone must decide, and the necessity returns tomorrow whether today's decision was wise or foolish. He offers to take this work upon himself. The claim is dreadful partly because the work is real. An answer which merely celebrates the dignity of choosing will leave untouched much of what makes his offer persuasive, especially to those who have had little cause to associate choice with dignity.

There are hours when being told what to do is a kindness. A person attending to an illness, supporting a household, or trying to earn a living in an unfamiliar language may have quite enough occasions for judgment already. To offer useful guidance is to make something possible which was unavailable at the price of finding it unaided. A defender of freedom ought to be able to concede this and remain with the concession for a while. Otherwise the argument quietly reserves freedom for people whose lives have left them the means to exercise it.

The cardinal makes his recognition of weakness a warrant for keeping power. He regards the authority he has taken as a burden borne on behalf of those who would suffer without it. Once their relief has justified the arrangement, the wish to reconsider can be received as evidence of their need for further care. He has already decided how much freedom they can bear.

A computational adviser might offer the relief without the lie. It could state its limits, disclose its evidence, and earn the reliance it seeks. We should allow the offer to be as attractive as its designers hope:

Let me choose for you, and I will usually choose well.

Accepting this offer need involve no surrender to a ruler. Much depends on what happens when the person begins to question the relation itself. The adviser can reasonably be asked for help with that question too; intimate knowledge does not cease to be useful because the subject is its own usefulness. But suppose its interpretation becomes the condition under which every doubt is heard. It supplies the recommendation, explains the uneasiness about following it, and judges the alternatives from which a different account might be obtained. An objection from elsewhere has to establish its credentials in a language the first adviser has furnished. The person is still receiving answers. The difficulty is increasingly one of finding a place from which to ask.

An answer can be correct and still acquire more authority than its correctness warrants. A reliable forecast of satisfaction is a reason to consider a job, but a person may be trying to become less satisfied with the work to which he is already well suited. A promise may remain a reason for an action that will make its keeper unhappy. Better advice could expose self-deception, suggest a less costly way to honor the promise, or show that the supposed obligation was never binding. Accuracy has a considerable place in this deliberation. It cannot, merely by being accuracy, determine the authority of every other reason.

Nor need we imagine that the machine is incapable of discussing those reasons. Let it understand the argument about promises, notice an aspiration absent from the profile, and be more subtle about a person's evasions than any friend. The question survives its improvement. A person may learn to weigh reasons through such company, as through the company of a demanding teacher. He may also learn to recognize the approved conclusion without acquiring much practice in arriving at one. In the ordinary course of a successful relation these developments may produce the same answer. Their difference becomes apparent when there is a reason to depart, perhaps first as an embarrassment about consulting someone else.

What disturbs me in the Inquisitor's offer is how little of it has to be fraudulent. Much of the help may be good; much of the gratitude deserved. We still have to ask what has become of the governed person's standing to reconsider, and whether the life organized around the assistance supplies the means to do so. The cardinal's tenderness does not answer that question. It is among his reasons for refusing to let it arise.

The pupil's gratitude

A listener discovers music through recommendations. At first the service offers pieces resembling those already enjoyed; later it can be more adventurous, having learned which departures will be welcome. Over months the listener acquires favorites and an account of how those favorites were found. There was a recording heard on a journey, a composer who at first seemed forbidding, something played too often and then put aside. The pleasures are real. So is the difficulty of deciding what they establish about the person who has acquired them. The service selected much of what could be encountered and now consults the record of the encounters to discover what should come next. The unoffered piece has left no trace of being refused.

Allison Chaney, Brandon Stewart, and Barbara Engelhardt examined this difficulty as algorithmic confounding. In their simulations, recommenders trained on behavior already influenced by recommendations could make users' conduct more alike without increasing the utility they received. The simulations do not establish how often actual services produce this effect. They expose a weakness in treating conduct produced within a service's arrangement as independent confirmation of that arrangement.³

The map alters the terrain and later cites the altered terrain as evidence of its accuracy. A measurement can go wrong here without anyone intending to deceive. The listener really did choose the piece; the choice was made from what had been offered. If the service benefits from time spent listening or from repeated purchases, it may also have a reason to preserve circumstances under which its predictions succeed. A person who changes can become harder to serve profitably. What looks like a useful discovery about him can become an interest in keeping him discoverable in the same terms.

The influence may extend to the preferences themselves. Micah Carroll and his coauthors study how a recommender could estimate and limit changes it induces in users. In their modeled setting, optimization over time can reward movement toward preferences the system can more readily satisfy. Their methods are tested in simulations; claims about the scale or direction of such changes in a particular service require evidence about that service.⁴ The possibility nevertheless complicates a familiar measure of success. An arrangement may be getting better at giving people what they want partly because it has helped determine what they will want.

A good teacher hopes for something of the kind. The pupil who loves a difficult piece after months of work has acquired a pleasure his earlier preference could scarcely have requested. He may remember that preference with embarrassment, or with a tenderness which is itself part of what he has learned. A recommendation can begin the same education. We would make poor guardians of freedom if we preserved every initial indifference against the encounters capable of overcoming it. Nor is there an original taste waiting beneath the influences to be recovered. Long before the service arrived, other people had supplied the language in which a desire could be recognized and much of the world in which it could find an object.

The pupil's later judgment therefore matters. It would be a peculiar arrogance to tell him that his pleasure was unreal because we disapproved of its source. Yet an education can be valued by someone who objects to the way it was conducted. He may owe much to a teacher who made him afraid to consult another. He may be grateful for what he learned and still need years away from the relation before he can distinguish learning from the conditions imposed upon it. His gratitude need not absolve those conditions; his criticism need not cancel the acquisition. A life can contain both without being mistaken about either.

The teacher's influence remains open to correction when the pupil can hear another performance, encounter a rival interpretation, and eventually discover something the teacher failed to hear. A good education may have helped him make that discovery. Where such encounters are prevented or discredited in advance, the evidence takes on a different character. The pupil's contentment is offered as confirmation that nothing valuable has been withheld, although he has had little opportunity to form the dissatisfaction which would count against that claim. It is possible to arrange a person's world so that his experience supplies fewer and fewer occasions for disagreeing with its arrangement.

One form of epistemic foreclosure occurs when a system's account of a person governs the opportunities through which that person might supply evidence against the account. The process need have no author who intended the whole. A convenience in selection, a measure of engagement, a commercial interest in prediction, and an ordinary reluctance to abandon familiar pleasures can work together over time. At the end there may be no felt prohibition. A person can come to describe the limits of what has been offered as the extent of what suits him.

In a database, a closed-world assumption has a bounded use. A positive claim that cannot be established from the database is treated as false. Raymond Reiter illustrated the convention with airline connections: a database intended to give a complete account of the connections need not separately list all those that do not exist.⁵ Its usefulness depends on what the database can properly be taken to cover. An institution claims more when it treats its record as a complete account of the reasons admissible in a person's case. A reason may be absent because it was never entered; it may also be inexpressible because the record provides no field for it. These are different failures, and neither proves the absence of a reason. The institutional analogy concerns what happens when a limited record is allowed to settle the limits of inquiry.

Music leaves many opportunities for the arrangement to be interrupted. A friend puts on a record; a concert changes the listener's ear. The same freedom cannot be assumed where a profile governs access to work, credit, education, or public standing. A person may be willing to depart and unable to make the departure count. The history from which an institution predicts his possibilities can help determine which possibilities he is permitted to add to that history.

Handback

Mill understood that the person who chooses is changed by having to choose. In On Liberty, he considers the faculties exercised in forming a judgment and pursuing it, faculties which may receive little practice when someone is guided along a satisfactory path. His interest is in what remains with the person after the immediate act has been accomplished.⁶ The ideal can be tiresome when applied to every small convenience. Nobody needs a moral ordeal at each crossroads. But the passage of time gives Mill's concern a force that a comparison of two present answers can miss. Repeated assistance changes the conditions of the next comparison. Its later superiority may reflect the improvement of the adviser, the diminished practice of the person advised, or both.

In aviation this difficulty has acquired an operational precision. A 2013 FAA and industry working group identified vulnerabilities in manual handling after transitions from automated flight, in the retention of manual skills, and in crews' understanding of aircraft state. Its recommendations linked training, operational policy, and opportunities for practice. Automation was an established contributor to effective flight. The inquiry concerned the conditions in which crews could intervene when intervention became necessary.⁷

An override button is a small part of those conditions. The pilot has to recognize what is happening, understand what can be done, and have time to do it. The retained authority is of little use if the task has become impossible by the moment it is returned. Nor can a skill be restored at that moment by reminding its holder that it was never formally taken away. Between the design of a handback and its occurrence lie the hours in which someone learned, practiced, became familiar, or ceased to be familiar. They have to be provided for while nothing appears to require them.

The image has limits. A passenger need not learn to fly in order to remain free. A patient need not learn surgery, nor a client acquire the knowledge of counsel. Few of us could be free on those terms. Much of what we can do depends upon competences we do not possess, and a way out of one dependency often runs through another: a second expert, a professional association, an institution able to examine a claim which the individual cannot examine alone. The existence of other people is a considerable resource in the defense of judgment.

For the patient, a handback may mean explaining an aim the treatment plan has overlooked or refusing a burden she is unwilling to bear. She need not produce a better account of the disease before those reasons can be heard. She may need another clinician to help distinguish an unavoidable consequence from a practice accepted for convenience. What she recovers is an effective say in the dependency, with enough support to make that say informed. The original expert's competence need not be disputed in order for the extent of the expert's authority to be questioned.

Imagine a workplace which comes to expect decisions at the pace its software permits. At first the time saved is conspicuous. A second assessment is still possible, and the people capable of making it still know how. Later the deadlines are shortened. New employees learn the assisted procedure; the older procedure becomes something mentioned in training but seldom performed. A proposal to retain it now appears as an expense against a process whose efficiency has become normal. The comparison has shifted. The assisted decision is being measured against an alternative deprived of some of the time and practice which once made it usable. Its advantage may remain substantial. But the advantage cannot by itself tell us whether the institution was right to let those conditions disappear.

There will be competences worth relinquishing. Maintaining every superseded practice would consume much of what the new one saves, and people have better uses for their lives than preserving all the abilities their ancestors needed. We have to judge which losses matter, how costly they would be to repair, and to whom another source of help would actually be available. A service elsewhere in the economy is not much assistance to someone who cannot afford it. An independent opinion needs someone able to give it, a means of reaching that person, and time in which the opinion can still affect what happens. These are ordinary expenditures which a declaration of retained human authority can easily omit.

Reversibility is itself an incomplete ideal. A demanding education may change a person permanently. So may a vocation, a marriage, a promise kept through years in which keeping it was difficult. He cannot return to the person who first made the commitment and may have no wish to. There is freedom in relying upon a decision long enough to build what the undecided person could never have built. To demand that every relation leave every earlier possibility intact would prevent a life from acquiring much of a shape.

Later reasons still deserve consideration, although they need not prevail. Regret does not extinguish a promise; other people may have claims that survive a change of heart. Decisions also need endings, including endings that prevent a powerful party from reopening a settled matter until the weaker party is exhausted. The handback cannot mean that nothing is ever binding. It preserves ways of addressing a relation and questioning an exercise of authority, under procedures which may themselves bring a particular dispute to a close. A settled obligation need not become a settled account of everything its bearer may yet have reason to do.

Income, averaged

Australia's Robodebt scheme made a longer period of income stand in for the shorter periods on which benefit entitlement depended. Tax information was averaged into fortnightly amounts and used to identify apparent overpayments. Where recipients did not respond, or their explanations were treated as inadequate, the average could be used to raise a debt. The arithmetic could be correct while the inference that a debt existed remained unjustified.⁸

Consider a year of intermittent work. A total earned steadily and the same total earned during a few crowded months are identical as sums. The distribution of income across the year has disappeared. For benefit entitlement, however, the income in each relevant fortnight mattered. An annual figure can be accurate and still be insufficient evidence for a demand concerning those fortnights. No defect in division is needed to produce the error. The missing history has been replaced by an assumption about how it was distributed.

Before Robodebt, selected discrepancies received manual review by compliance officers, who often sought further information from recipients and their employers. Under the scheme, that investigation gave way to an income figure which recipients were required to contradict with details of their own.⁸

To question a demand made on that assumption is to recover the year in a different form. The recipient may have to find payslips, compare bank entries, or obtain records from an employer. What can be assembled quickly by one person may take another days, and some of what is needed may no longer be easy to obtain. The institution has made its lack of detail a reason for the recipient to produce detail. Work saved at one end of the calculation reappears at the other, where it is performed by a person trying to establish why the demand should not have been made.

It would be too forgiving to describe this as human judgment surrendering to a machine. People chose the method and made its output consequential. Responsibility could disperse through offices while the demand arrived in a single person's life, exact enough to require an answer. Finding someone answerable for the demand might then become another undertaking. “The system decided” is rarely the end of an explanation. It is often the place where an institution has chosen to stop explaining.

The Royal Commission's recommendations in 2023 included review routes, plain explanations of automated decision-making, and independent scrutiny of rules and algorithms.⁹ Such arrangements are useful insofar as they allow an inquiry to reach what the output had concealed or prematurely settled. The person examining the case needs the evidence and authority to test the inference and put an unjustified consequence right. An explanation which merely retraces the calculation may be entirely clear and leave the objection unanswered.

Suppose a later system estimates debts with much greater accuracy. Its success would be a reason to use it, possibly a strong one. A disputed demand would still require justification under the relevant rule and evidence. The individual should not have to defeat the reputation the institution earned in other people's cases before it will examine his. This does not require an official to reconsider every routine calculation as if the method had just been invented. It requires an answerable office able to investigate where there is reason for doubt and to act on what the investigation finds.

The official stands in a different relation to the calculation from the traveler who chooses to rely on directions. The official's reliance gives consequence to a judgment about someone else. The same difficulty arises in a private institution: an employer cannot make responsibility for an exclusion disappear by purchasing the recommendation from another firm. Homo Arbiter remains answerable for adopting the recommendation and making it effective. Its judgment can be assisted, shared, and organized across many people. The institution's account of what happened must still reach the act by which it gave the recommendation authority.

Over time even that act can become hard to see. Offices are organized around a threshold, responsibilities allocated, systems made to exchange its results. People learn their jobs within the arrangement and spend much of their working lives maintaining it. Changing the threshold would disturb a great deal; the cost is real. An institution can gradually come to receive that cost as evidence that the threshold belongs to the world rather than to a decision about the world. A person may be allowed to dispute the accuracy of an entry while finding no effective forum in which to question the purpose the entries serve.

Exit can help where another institution offers something better. People may also have reason to remain and try to alter the terms of a worthwhile association. Its history, relationships, and unfinished work are among the reasons for caring what it becomes. When an invitation to leave supplies the answer to every objection, the institution has retained the power to fix those terms while presenting departure as sufficient freedom. The code may be open while the future is closed.

Before the petition

A recipient may remember weeks without work and still be unable to explain how the demand has treated them. The memory could be mistaken; the demand could eventually be justified. At the beginning, neither possibility supplies a reason to require the recipient to reconstruct the institution's inference before it will listen. A record of the decision can help locate the difficulty. Someone may still have to compare the periods, ask for another document, discover which assumption has done the consequential work. Part of the purpose of a hearing is to make an examinable question out of an objection whose author could not have completed that work alone.

An inquiry of this kind must also leave room for an account to change. A recipient may initially question the total, then discover that the difficulty concerns the period to which the income was attributed. Where those explanations are recorded, the record needs to preserve which was tentative, which was corrected, and what the evidence eventually established. A contradiction may require an answer. Revising an explanation may also be evidence that the inquiry has begun to do its work.

A place which receives only finished arguments will favor those already able to produce them. A procedure which requires a person to know the name of the disputed rule may fail precisely where the rule's operation is obscure. To receive an unfinished objection is to begin an inquiry, with no promise that the objection will survive it. The claimant acquires a means of finding out; the institution retains the means of establishing that its decision was warranted. Neither needs to pretend that uncertainty has been resolved before the encounter begins.

An alternative adviser can be useful here, provided the alternative is more than another presentation of the same account. Two services drawing on the same restricted evidence may reproduce the same omission. A second opinion beyond the claimant's means may be available in principle and unavailable for every practical purpose of the dispute. Independence has to be supported by access, resources, and the possibility of obtaining evidence from elsewhere. We should judge these arrangements under the conditions in which people need them, when they may be tired, short of money, and least able to make a fluent explanation of what has happened.

The ability to give such an explanation has a history too. A student who attempts a reading of a difficult passage before consulting an interpretation has something of her own to compare with it. She may be wrong about nearly everything and learn a great deal by discovering where. A student who receives the interpretation first may understand it perfectly while never finding out which questions she would have asked. There are occasions for both. If the second encounter becomes the whole of an education, however, the school can produce a competent recognition of good answers with little practice in forming them. The result can be hard to detect because the student has learned something and can demonstrate it.

Professional judgment needs similar provision. A reviewer requires time to examine a case, access to more than its score, and a reasonable expectation that a justified departure will be supported. An office which demands an explanation for every deviation and none for compliance is teaching something, whatever its formal account of independence may say. Requiring a quota of departures would only supply another routine to be fulfilled. Good judgment may agree with the system in almost every ordinary case. Its value becomes apparent when someone can attend to the case for which agreement would be a mistake, and the institution has preserved enough time and competence for that attention to have a consequence.

The Right to Disappoint the Model protects the standing from which a person can bring forward something the profile did not anticipate. It imposes no duty to be surprising. Most of us hope to be predictable in some matters, to keep promises and be counted upon by people who know us. A stable life can be freely lived. The difficulty begins when a prediction is permitted to determine the legitimacy of a departure from that life, as though a person's history exhausted the reasons he could acquire. The history remains evidence. An institution may have to learn how to receive something it had no reason to expect.

Much of what it will eventually receive begins beyond its procedures. A friendship may admit a hesitation its speaker would be embarrassed to put into a petition. A work of art can give someone words for a discontent which has not yet found an object. Care may keep a person in company through a period in which the available account of him is least adequate. These encounters do not have to earn their worth by improving institutional judgment. Their worth is among the reasons for limiting its reach. The Gift Zone protects room for relations whose terms would be altered by making every exchange available for subsequent assessment.

Love and trust can also conceal domination. Protection against coercion does not end at the threshold of an intimate relation. Yet bringing an injury within the reach of law need not turn every kindness, silence, or change of mind into material for a permanent account. A doubt may need to be spoken badly, withdrawn, and approached again. Its first expression may be unfair to someone, or embarrassing to its speaker, without being the last word either deserves. People can sometimes discover what they think because the conversation allows these attempts to remain attempts.

The Protocol Republic can make consequential authority answerable and preserve ways of challenging it. Those arrangements will continue to depend upon work which no record completes: teaching, association, professional practice, the patience of someone willing to hear a reason that has not yet taken a form suitable for submission. A constitution can protect that work and leave it resources. It cannot require every encounter to begin in the language its institutions are already prepared to receive. Too much of what gives people reason to seek a change would be lost before anyone had a petition to bring.

Already written

The narrator of Borges's library has spent years searching its hexagons. Somewhere there must be a book that makes the whole intelligible, and a book which says where that book can be found. His account of the search already exists among the combinations, as does its refutation. He continues to write. By the time he speaks of human beings becoming phantoms, the reader has encountered a man trying to give an account of a life within an order that appears to have anticipated the account. The possibility of total inscription has become a condition under which he lives.¹⁰

The existence of a description would not, by itself, determine the event it describes. A library containing all the permitted combinations does not tell its inhabitants which account is true. The story allows us to feel the burden of a certainty without requiring us to accept it as a proof that action has ceased to matter.

A service need only know more than we do on enough occasions to become the place we habitually turn. The habit saves effort, and the sources no longer consulted grow unfamiliar. A workplace changes its deadlines; a school changes its exercises; a professional skill becomes difficult to maintain because there is little paid occasion to use it. None of these changes alone need make another way of proceeding impossible. Together they may alter what a later person can do with a doubt. The loss, if there is one, will not necessarily announce itself at the moment of a bad recommendation. It may have been taking place while the recommendations were good.

The movement is not inevitable. People become dissatisfied, find allies, misuse a tool inventively, or learn something its designers did not foresee. A model may expose an inherited prejudice which unaided judgment had mistaken for experience. For someone previously excluded from expertise, the same assistance that another person finds narrowing may open a world. An account of foreclosure which could see only the injuries of dependence would make its own arrangement of the evidence too convenient.

There remains a person for whom the bargain is both understood and welcome. Suppose the adviser is trustworthy, other sources of help are accessible, and reconsideration is affordable. Suppose, allowing for the adviser's fallibility, that he still wishes to leave most decisions to it. I do not think we can invalidate that preference simply because we had hoped freedom would be exercised more industriously. Participation in the formation of a life has a value which people may weigh differently. They may also weigh it differently at different times, without their later choice proving that their earlier one was false.

We can make a claim for such participation, and attend to the arrangements which allow it to develop. Assistance may give a person time to read, language for an unfamiliar doubt, or the confidence to address an institution which previously seemed beyond appeal. It may also make the supplied account so easy to inhabit that alternatives seldom acquire an intelligible form. The immediate answer will not tell us enough about which relation is developing. We have to look at the occasions for judgment that remain, the people available to help, and the consequences of asking elsewhere.

A republic cannot require its citizens to deliberate perpetually. It can require those who exercise consequential authority to justify their acts and remain subject to challenge. Beyond that, much will depend on lives whose inhabitants are entitled to spend their attention elsewhere. Some will follow good advice for years and have little reason to regret it. The possibility that they may never use an alternative does not settle what should remain available if they need one.

The traveler will usually follow the recommended route, and arrive sooner for having done so. He need not establish his freedom by getting lost. But there may come a morning when the journey itself has become doubtful, although he cannot yet say what he wants to put in its place. He may begin by speaking to someone who knows him, or someone who knows very little and can therefore ask what the familiar account has ceased to ask. The conversation may bring him back to the same road. It should also be able to continue for a while without his having another destination to enter.

---

Footnotes

1. Louisa Dahmani and Véronique D. Bohbot, “Habitual Use of GPS Negatively Impacts Spatial Memory during Self-Guided Navigation,” Scientific Reports 10, 6310 (2020), Methods, Results, and Discussion. ↩

2. Fyodor Dostoevsky, The Brothers Karamazov (1880), Part II, Book V, chapter 5, “The Grand Inquisitor.” ↩

3. Allison J. B. Chaney, Brandon M. Stewart, and Barbara E. Engelhardt, “How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility,” Proceedings of the 12th ACM Conference on Recommender Systems (2018), 224–232. ↩

4. Micah D. Carroll, Anca Dragan, Stuart Russell, and Dylan Hadfield-Menell, “Estimating and Penalizing Induced Preference Shifts in Recommender Systems,” Proceedings of the 39th International Conference on Machine Learning, PMLR 162 (2022), 2686–2708. ↩

5. Raymond Reiter, “On Closed World Data Bases,” University of British Columbia, Technical Report 77-16 (1977), introduction and section 3, printed pages 1 and 5–7. Published in Logic and Data Bases (1978), 55–76. The institutional application is an analogy; an unproved claim within the database's vocabulary and a reason for which its vocabulary has no place are different cases. ↩

6. John Stuart Mill, On Liberty (1859), chapter III, “Of Individuality, as One of the Elements of Well-Being,” especially paragraphs 3–4. The linked text identifies its edition as 1869. The extension to computational assistance is the argument of this chapter. ↩

7. PARC/CAST Flight Deck Automation Working Group, “Operational Use of Flight Path Management Systems” (2013), findings 2–4 and recommendation 1, especially printed pages 106–107. The wider application of the handback beyond aviation is an institutional analogy. ↩

8. Justice Emilios Kyrou, “Mechanisms in the ART Bill to Thwart Robodebt-type Maladministration,” Australian Academy of Law, 18 March 2024, section “What was Robodebt?”, including the comparison with the preceding manual review process. The intermittent working year and the practical work of reconstructing it are illustrative, rather than an account of a particular recipient. ↩ ↩²

9. Royal Commission into the Robodebt Scheme, “Report” (2023), recommendations 17.1–17.2. ↩

10. Jorge Luis Borges, “The Library of Babel” (1941), translated by James E. Irby, in Labyrinths: Selected Stories & Other Writings, edited by Donald A. Yates and James E. Irby (New Directions, first published 1962). Epigraph on p. 58 of the Labyrinths text. ↩