People who rarely produced original writing are suddenly producing extensive design proposals, business plans, documentation, presentations, tickets, pull requests, blog articles, and meeting summaries, all generated by AI. It is all unreadable. AI is helping me write better and faster, but I’m fed up with reading almost anything written by AI. In this essay, I want to express my frustration, and describe how I’m finding value in AI for writing.

AI Is Terrible for Writing

A pattern I see is that people use AI to build something new, then they use AI to retrospectively summarize what they have already built into a design document. Reading a document like this isn’t just difficult—it is punishing. The design document is no longer a proposal to build consensus, bring people along with you, and refine ideas through slow, deliberate thinking—it is a summary made by a machine, in exhausting detail, without context or perspective. It is unreadable. Inhumane. The authors of these documents become impatient when people aren’t engaged, and it is hard not to be, because the thing they built is already working.

I also see more and more pull request summaries clearly written by machines for machines. They are rich in detail: this was changed to that, these things were split, those things were merged, this was left untouched, tests were added for this, and so on. But I’m left asking: Why are we doing this? What is the value? How risky or urgent is this work? Where do you want my input? What should I pay attention to? The same goes for tickets or meeting summaries made by AI. The writing, like more and more of the world, consists of statements outside of context.

I had someone write me a personal message about a sensitive topic that was clearly workshopped with AI in an effort to nuance the conversation and not offend me. But the message became impersonal, dispassionate, and disjointed. It had all of the parts, but it didn’t make sense as a whole. I wasn’t interested in reading it, or responding. I would much rather someone be themselves and write with their own voice, or with some passion, even if they can’t quite find the words, or risk offending me.

In relationships with others, there is vulnerability and risk. In communicating with your AI, there is none, and attempts to eliminate vulnerability and risk will eliminate the process of relationship itself. As Simon Sarris writes in Resist Summary, an essay I keep returning to:

I think one should write as much as they can with their own empiricism, their own senses, giving the reader their own characterization of life or events.
—Simon Sarris, Resist Summary

Perhaps the laziest and most offensive use of AI writing I have seen: I had someone use AI to summarize my comments on their proposal as their response to me! Thanks! I couldn’t have said it better myself!

I Don’t Have the Context You Do

Cynthia Dunlop recently shared survey results on reading AI-written articles entitled Report: How developers react to AI-scented blog posts. For the majority of readers, if they think an article is AI-assisted or AI-authored, they will stop reading (78%) and avoid the author in the future (71%). As Bryan Cantrill wrote about this research in his essay The revolt of the reader: “our brains pull an LLM-triggered ejection handle, bailing us out mid-sentence in an act of self-preservation.”

The strongest result in the survey was people overwhelmingly preferred the author’s own writing (98%), with all its flaws and idiosyncrasies, as compared to a soulless rewrite by an AI. Writing authentically is one of the best ways to connect with people. It reminds me of a comment Bjarne Stroustrup, the creator of C++, made during a panel discussion I attended over a decade ago on writing books:

When I read a book, I really feel that if I can hear the author’s voice ringing in my head—complete with weird accents and peculiarities of their speaking patterns—then I feel like something has succeeded. If it feels like dry, academic text, somebody has written yet another dry, academic text. And so, giving talks about the subject really helps. And, for many people, ... I can hear them when I read it, and that is good.
—Bjarne Stroustrup

When you are the one who prompted the AI, the written output is often quite useful. But do not mistake this for effective writing. When you read the output, you already have lots of context—you formulated the prompts, established the constraints, and provided artefacts like documentation, source code, logs, statistics, pictures, etc., that led to the output. You can skim the output and quickly decide what is relevant and valuable versus what is irrelevant or wrong. You are also part of the process—prompting the AI and seeing the results—a call and an answer. You bring a human perspective to the machine. When you send the same text to someone else, they have little to no context. They are not part of the process—there is no unfolding—so it is much harder to decide what is relevant versus what is not. They are forced to read exhaustively and consider every line, peering into the internals of a machine with the hope of establishing context. It is only natural to stop reading.

AI Can Be Great for Writing

I recently wrote an academic paper. I used AI extensively and it made the writing process faster and more enjoyable. But AI did not write a single line of the paper. I wrote the paper in LaTeX. The context I provided to the AI included the journal style guide and LaTeX template, all papers previously published in the journal, the source code for the software systems I was writing about, and the configuration, logs, and metrics for production deployments of the software.

After writing a paragraph, many times I would ask the AI to ensure what I had written was correct by referencing the source code or configuration, or examining production logs or metrics. For example, ensuring I had correctly described which columns in the database were indexed, or how the rows in the Parquet files were sorted. This allowed me to stay in the context of writing, often moving on to the next paragraph while the AI verified the previous one. It helped me spot important omissions and inaccuracies. Remarkably, reversing this process—asking the AI to write the paragraph from the context I provided, like describing how something works from the source code—was never valuable. Not once. It was consistently unpleasant to read, and often inaccurate.

Another thing that helped me stay in context was using AI to complete the citations. It is tedious to look up the details of a paper, website, or book, fill in the BibTeX format, and then reference it in the text. Instead, I would leave parenthetic instructions for the AI to reference what I wanted, and then keep on writing without breaking stride.

The AI was incredibly good—indeed ruthless—at finding spelling and grammar mistakes. It was great at suggesting simplifications to sentences that are too long or unclear. It was skilled at drawing technical diagrams in TikZ, again speeding up the work, and saving me from learning this syntax. The AI even found a very subtle mistake where I had used the incorrect notation in one paragraph, something four human reviewers who were experts in the system all missed.

What was the one thing the AI was incredibly good at writing? It wrote the abstract—the most terse, mechanical, inhuman part of the whole paper—indeed, the most abstracted part. I included it entirely unchanged. Perhaps this shouldn’t be a surprise, as AI is great at summary, abstraction, and re-presentation of original work.

Can AI Become Good at Writing?

It would be a mistake to underestimate the writing that can one day be achieved by AI. We are still in the very early days of model development, and the models most of us are using are trained to be factual, direct, and correct, rather than creative. However, Murat Demirbas’s excellent essay The Safest Job from AI may be Writing notes that the quality of writing from LLMs has plateaued and it may be difficult to improve since unlike programming, mathematics, and accounting, it is difficult to provide a clear formulation and verifiable output.

Since LLMs lack an active mental model of a specific human reader, they are just optimizing for the statistical probability of the next word over a vast dataset. They cannot empathize with the human reader, as they don’t have the human lived experience. And they have zero skin in the game.
—Murat Demirbas, The Safest Job from AI may be Writing

I am interested in two efforts to improve AI writing that I’ve recently come across. The first is ASD-STE100 Simplified Technical English, a standard for writing technical documentation that grew out of an aerospace working group in the 1980s. It has two parts: a controlled dictionary and writing rules about grammar and style. A version of ASD-STE100 has been created for use with AI models. ASD-STE100 isn’t appropriate for all AI-generated writing—it is specifically targeted at clear and easy-to-follow instruction manuals. I’m going to give it a try for installation instructions, runbooks, and similar documentation.

The second is Pangram, a model fine-tuned to detect AI writing. Bryan Cantrill in the essay The revolt of the reader notes:

To use an LLM to write is to void the social contract between writer and reader: we readers shouldn’t be expected to labor to understand a sentence that the writer themselves didn’t work to create.
—Bryan Cantrill, The revolt of the reader

I could not agree more. Bryan has started experimenting with Pangram to detect AI-generated writing that violates this contract and has gone so far as to mandate that all public writing at Oxide be reported by Pangram as human-authored. A tool like this could save us from the laziest and worst AI writing.

When There Are No Words

When we write, the meaning of our words is not always direct, factual, or concise. Examples include poetry, literature, song, mythology, humour, and theatre. Iain McGilchrist, who I believe is one of the most important writers of our generation, writes in the book The Matter with Things: “The more important something is, the more we have to struggle in the attempt to reduce it to language.” Ask an LLM what the reliable formula is for falling in love. In AI-generated writing, detailed information gives the illusion of meaning. But in mimicking human knowledge, intelligence, and understanding, an AI writing about the facts is very different from the quality of writing from a lived experience. The whole cannot be decomposed into parts without a substantial loss of meaning.

The direct approach destroys its object. Similarly, there are things that can be conveyed only indirectly. To recapture the depth of what is said, or in order to say anything truly new, we are always engaged in going beyond everyday language, which means being alert to what is being said implicitly.
—Iain McGilchrist, The Matter with Things

It might seem silly to say that writing in business, engineering, or science should involve uncertainty, not just facts and conclusions. But operations, incident response, product development, performance engineering, hunting for security vulnerabilities, performance reviews, mission statements, and many other endeavours, are all creative and involve uncertainty. The narrative is regularly as important as the facts and conclusions. It is also easy to anthropomorphize LLMs as good systems thinkers because they can synthesize such a wide range of information. However, returning to Simon Sarris: “narrative resists compression” and “the opposite of summary is attention to detail”.

Despite an emphasis on linear, first-principles thinking, the work of a good systems engineer, just like the work of a good anthropologist or psychologist, is difficult to put into words, and especially into summary. You can’t necessarily observe and write things down using a formula. Even writing things down loses some of the context. The observer is also part of that context. Systems work always has some sense of discovery, holding open the unknown, an unfolding of relationship with the system itself, and a willingness to be surprised. Normal language doesn’t always have the words.

The nature of the output, of summary, is speed things along. But the consequence is to avoid you having to build your own complex mental model of anything. I worry that without a complex model in one’s own mind, one may never notice complex relationships that are otherwise missed. A loss of attention to detail.
—Simon Sarris, Resist Summary

Conclusion

I don’t want to live in a world where you use AI to summarize something important into unreadable text, and then I use AI in an attempt to decipher it. I want to hear you, imperfections and all. I want your interpretation of aesthetics, beauty, quality, relationship, time. I want to know how you feel. I want you to cut through and tell me what really matters. I don’t want to read a summary of facts and connections separate from experience and relationship. As Iain McGilchrist describes, the way we attend to the world changes the nature of what comes into being, the only world we can know.

If you ask a machine to do something for you, you are at risk of not asking for what you want, but what it can do well. If what it can do well is summarize, and you rely upon the tool, you may tune your questions so that you get better answers. This looks like training a machine to do what you want, but its also training yourself to ask questions about the world in a certain way. What are the questions you are omitting?
—Simon Sarris, Resist Summary

Intentional writing will likely become more valuable. People who write, and write to think, to think deeply and carefully, or to create, to share, or to capture something important without explicitly expressing it will continue to write and produce original work. The people who never were writers will use AI to produce lots of text.

As we read more and more text generated by AI, we might not notice what AI has disrupted and changed, because we’ve become more shallow anyway. The search for certainty limits our humanity. Sometimes it is more important to not find the words. I don’t want to read what you didn’t write.