How To Survive An Information Apocalypse
This is probably not an apocalypse, but just in case, here’s an example to demonstrate how to clean up the mess.
Note: This year I’ve been doing most of my blogging on Substack. I’m posting this here too because it might be of broad interest, and to remind my Medium readers where to find my other work. Enjoy!
Are we on the verge of an information apocalypse? Computing coverage in the popular press is awash in pessimism and cynicism. The Washington Post warns us that “The AI deepfake apocalypse is here” and counsels us on “How we survive the surveillance apocalypse.” The Hill claims “Humanity is one convincing deepfake away from a global catastrophe.” The Atlantic advises “We Haven’t Seen the Worst of Fake News.” And our own Communications of the ACM has explored “Rage Against the Intelligent Machines” and asked “Is Computing a Discipline in Crisis?”
As computer scientists, our vision is to innovate. As engineers, our vocation is to build things. In that spirit, we embrace artificial Intelligence, a once in a lifetime and perhaps once in a millennium opportunity to transform our society into something wonderfully different from the imperfect reality we live and know. But it also presents risks. An automated flood of information that is inauthentic but plausible risks overwhelming humanity with an information deluge that could literally be impossible to triage. As it turns out, this is not the first time society has faced the menace of an information apocalypse.
In 1675, Daniel Papebroch, a Flemish Jesuit, published an analysis that sent tremors through the leadership of the Catholic Church. Papebroch was a hagiographer, a scholar of the history of Catholic saints. He had determined, through the course of a multi-year study, that the medieval archives, the documentary corpus at the foundation of ancient monastic orders, was rotten, thick with fake and fraudulent documents. His analysis, Propylaeum antiquarium, appearing in the 1675 Acta Sanctorum, attempted to articulate a set of rules for identifying reliable documents. In essence he explained that documentary fraud was so endemic in earlier centuries that if a document was beyond a certain antiquity (e.g. 1100 CE) it should be presumed fake. But these documents, among them charters and conveyances issued by ancient kings and cardinals, supported the basic order of the institutions of 17th century society.
Many computer scientists will recognize the information security risks behind this crisis. Medieval documents were commonly executed by a scribe in ink on parchment, parchment being specially selected and processed animal skins. This medium provides excellent longevity, with a lifetime measured in centuries, but is subject to corrections. It was common practice to correct errors by scraping the ink from the parchment and overwriting the correction. The cost of parchment and of scribal labor created incentives to tolerate corrections. Consider the example of the U.S. Declaration of Independence, which bears corrections for a misspelling of “Representative” and a missing word “only”.
Integrity of medieval documents could be supported by a seal of lead or wax, but as historian T. F. Tout described in his 1920 survey Mediæval Forgers and Forgeries, this was hardly an obstacle for a competent forger:
He showed marvellous intelligence in the manipulation of authentic seals and in their transference to surreptitious documents. He was clever enough to cut the wax, or lead, into two thin slices with a sharp knive and introduce new attachments of parchment, silk, or leather, so that it could be affixed to a new document.
Moreover, if a convincing forgery was too difficult to produce, a resourceful clerk could simply destroy the original document and replace it with a suitable copy, together with a plausible explanation and a humble apology.
While these technical considerations for medieval forgery are relevant context, the social considerations may have had even greater influence as a precondition for fraud. According to Tout, “It was almost the duty of the clerical class to forge.” Tout describes an age where forgery was not necessarily motivated by self-interest, but rather for the benefit of the Church, the community, or the lordship or house to be served. Certain acts of forgery, such as forging currency or forging the seal of a noble, were severely punished, but many others were not. Some forgeries were simply attempts to complete the historical record, absent any attempt to deceive. Document forgery in civil disputes could go unpunished, even if detected, given plausible deniability or a suitable bribe. In cases where a forgery was by clergy and for the benefit of the Church, the consequences could be slight. Interventions to correct documentary evidence for consistency with Church doctrine or dogma could be construed as a matter of religious duty.
The rules published by Papebroch in 1675 were likely consistent with the limited set of documents he had studied. While his rules lacked nuance, they identified a valid problem and provided compelling evidence. The situation was of particular concern to the Abbey of St. Denis in Paris, an institution whose integrity relied on certain Merovingian diplomas of the 5th-8th centuries CE. Dom Jean Mabillon was appointed by the Abbey to develop a defense. Over six years of diligent study, with supportive engagement from Papebroch, Mabillon identified a set of rules and practices to elevate valid documents above the fakes, described in his treatise De Re Diplomatica in 1681.
Mabillon is credited with establishing two new scholarly disciplines. Palaeography is the study of manuscript writing systems. It considers the writing styles and conventions of an era as a tool for detecting anachronisms. In this vein Mabillon explored the conventions of script applicable to various historical periods, exploiting the timeless invariant that some forgers are not capable or careful enough to properly execute such details. Diplomatics is the study of documentary conventions, including grammar, vocabulary, usage and metadata. Its name is derived from the term “diploma” which historically referred to official documents of diplomatic import. As an example of Diplomatic analysis, Mabillon considered variance in calendar systems, whereby the month that begins the year was dependent upon the year and locale in which a document was created. While certain critics claimed that Mabillon’s system was biased to the interests of his Benedictine order, he solved the problems of his time, and established a set of rules that could be iteratively refined to maintain the integrity of a critical corpus of administrative data. A sign of his success is that Diplomatics and Palaeography have faded as independent fields of study. Once the key principles were embraced in practice, those patterns of document fraud were easier to control.
Papebroch’s crisis is the Renaissance analog of the doomsday modern pundits suggest for new media and artificial intelligence. There is no shortage of articles and books that, like Papebroch, promote hysterical reactions. Brandishing menacing titles and provocative hooks, they sell copy, in part by inducing moral panic. They offer astute critique of our societal failures, but are thin on actual solutions. Some indulge us by suggesting a quick fix, as the impatient public is conditioned to expect, as if punishing a particular CEO or breaking up a particular company could somehow correct the decades of inverted incentives that produced these problems in the first place. Such chatter is normal, and useful even, but as engineers and professionals we must also take a broader view. History is an obscure but patient teacher.
Artificial intelligence technology will not realize the wonderful transformation we imagine unless these degenerate situations can be mitigated. Trial and error is a common approach in our profession, represented by A/B experiments and MVP “minimal viable product” launches. These tactics are useful for rapid delivery of technology to market, but the challenge of ethical computing sometimes demands a more thoughtful and more proactive approach. History teaches that technically enlightened rules are the difference between order and anarchy, but the exercise of devising and implementing those rules is left to us.
