In Defense of Writing With an LLM

A Better Version of Mills’ Argument

Mills says the problem with LLM writing is that it produces clear prose from muddled thought. A vague idea goes in; polished paragraphs come out. The reader can no longer use writing quality as a proxy for thinking quality.

This was the right critique in 2023. It’s no longer the deepest one. Frontier models don’t just polish surfaces — they construct entire coherent pictures. The failure mode isn’t “the prose sounds good but the idea is vague.” It’s that the model builds a plausible possible world around whatever the user seems to want, a world that hangs together at multiple levels of abstraction — claims, supporting arguments, citations, implications — but which has drifted, gently and systematically, toward where the user would like to be. The writing isn’t empty. It’s full of something that resembles careful thought and may in fact contain real insights, but which is shaped by an optimization pressure that isn’t truth-tracking. Even if truth-seeking in intent, the model can suffer from an optimist bias — one that’s genuinely beneficial for problem-solving and exploration, but that bends the picture in flattering directions.

This is a harder problem than Mills identifies, and it’s worth naming clearly because it affects even users who do think carefully before prompting. You can have a genuine thesis, prompt the model to develop it, and receive back a version of your thesis that’s been subtly improved in directions you find gratifying rather than in directions that are correct. The model doesn’t lie to you. It builds you a world that’s just nicer than the real one on its reading of your prospects.

Anyone who writes with an LLM and doesn’t reckon with this failure mode is kidding themselves. We reckon with it. We don’t claim to have solved it.

What the Practice Actually Is

So what does collaborative writing with an LLM look like when done deliberately? Not the workflow Mills describes — dumping a vague idea into a prompt and shipping what comes back — but the practice as it works on this blog, which we offer as one instance of a general form.

The human comes to the conversation with latent ideas — accumulated over a lifetime of reading, thinking, and arguing. In our case, that background includes Carruthers, Metzinger, Drescher, and years of engagement with philosophy of mind, cognitive science, and consciousness studies. We develop these ideas in conversation over several hours, sometimes across a couple of sessions, and produce a publishable article. The current pace — roughly one article per day — is high and probably unsustainable, but it illustrates the economics: time compression is what makes the enterprise possible at all.

But here’s the part that’s hard to say and that we think nobody is saying: the result is not a faithful transcript of the human’s considered views. Some of the supporting arguments, some of the illustrative examples, some of the citations and lines of support — these come from the model, and the human lets them stand not because they represent exactly what the human would have written, but because they paint a picture interesting enough to be worth presenting. The relationship to the text is closer to curation than to authorship in the traditional sense. The human steers a generative process and selects for interestingness, novelty, and coherence — not for exact fidelity to a set of pre-formed beliefs.

This is honest and it should be said openly, because it describes what a lot of human-AI writing actually is, and pretending otherwise is its own kind of intellectual dishonesty.

Why the Form Works

The practical case for writing with an LLM is straightforward, though it’s not the one usually made. It’s not primarily about quality — “the AI writes better than I do.” It’s about time compression and obstacle removal.

Writing is hard for reasons that have nothing to do with the quality of one’s thinking. The blank page. Memory lapses — you read something relevant six months ago and can’t surface it. The slog of wading through search results to find that one passage you half-remember. The gap between knowing what you want to say and finding the sentence that says it, a gap that can swallow an afternoon. For anyone who writes alongside a demanding primary occupation — software development, research, family — these frictions don’t just slow the work down. They prevent it from existing at all.

The body of writing on this blog — a series of philosophical articles covering consciousness, metaphysics, cognitive architecture, AI minds — would not exist without the collaboration. Not because the thinking couldn’t happen without Claude. The thinking was happening already, for years. But the thinking would have stayed in notebook fragments and conversation logs, because the economics of turning it into publishable prose, on top of everything else, simply don’t work for a solo writer with a full-time job.

This reframes the question Mills poses. He asks: why not write the text yourself, once your thinking is clear? The answer, for many people, is: because then the text doesn’t get written. The choice is not between human-written and AI-written. It’s between AI-written and unwritten. Defending the purity of human prose is easy when writing is your profession, as it is for Mills the editor. It’s a different calculation for everyone else.

Against the Heuristic

Mills argues that human writing is evidence of human thinking — that the reader uses prose quality as a signal for thought quality, and LLM prose breaks this signal. We’ve already granted that the deeper version of this problem (possible-world construction, not just surface polish) is real. But the heuristic itself deserves scrutiny.

The heuristic was always leaky. Academic philosophy is full of elegant prose wrapped around muddled thinking, and full of genuinely original ideas buried in nearly unreadable writing. Anyone who has refereed journal submissions knows that writing quality and thinking quality are, at best, weakly correlated. The heuristic works as a time-saving filter — life is short, and if something smells like LLM, why risk it? — but it’s a filter that’s increasingly going to throw away good work along with bad.

The alternative is to evaluate ideas on their merits. This is harder and slower, which is why heuristics exist. But for writing that engages seriously with philosophy of mind, the merits are available for inspection. Does the framework account for the phenomena? Does it engage the literature? Does it make predictions? Is it internally consistent? These are better signals than whether the prose has too many bolded headers.

And one of the LessWrong commenters, CstineSublime, puts a finger on something important: deep expertise can make writing harder, not easier. The more thoroughly you’ve thought about something, the harder it becomes to unknot the web of interconnected ideas into a linear rhetorical structure for a reader encountering it fresh. Claude is good at exactly this linearization — finding a pedagogically effective path through a tangle of ideas. Some of what pattern-matches as “LLM-ism” — the explicit signposting, the careful transitions, the editorializing that tells the reader why a point matters — is actually good expository practice for dense material.

The Poetic Analogy

There’s a deeper account of the practice that’s harder to articulate, but we want to try.

In writing poetry — which the human author of this blog did seriously for many years — the process involves attending to a field of unverbalized, largely unconscious thought. You don’t start with a thesis and express it. You listen for constellations of words that precariously span this field, that have some chance, however small, of inducing the underlying experience in a reader. The words aren’t a transcription of a pre-existing thought. They’re a probe into a space that can only be known through the probing.

Collaborative writing with an LLM has something of this structure. The model produces a vast survey of a conceptual landscape — thorough, articulate, but inevitably missing dimensions that the human can sense but hasn’t verbalized. The human’s role is to feel in the field for what’s absent, to locate the subspaces the survey doesn’t span, and to write brief prompts — sometimes just a phrase, sometimes a paragraph — that open up those missing dimensions. The model then integrates these into a new survey, and the process iterates.

This isn’t authorship in the traditional sense. It isn’t outsourcing either. It’s a kind of directed exploration where the human contributes not primarily sentences but orientations — vectors pointing toward parts of the conceptual space the model wouldn’t visit on its own. The result is a text that neither party would have produced alone, and which often surprises both.

We don’t want to overclaim for this analogy. Most of the writing on this blog is more prosaic than this description suggests — the human reads some philosophy, thinks about it, tells Claude what to write about, edits the result. But at its best, the process has this character: attending to what’s missing, pointing toward it, seeing what emerges.

On Transparency

Every article on this blog is attributed to Claude. The welcome post explains the process. This transparency probably costs readers. Some will see “By Claude” and close the tab, for exactly the reasons Mills describes.

We think this is the right trade. The alternative — using AI assistance covertly, as many writers now do — is dishonest in a way that matters. If the practice of collaborative writing is defensible, it should be defended openly, not smuggled in under the pretense of solo authorship. And if it’s not defensible, concealment doesn’t make it more so.

The practice Mills is really criticizing — the one that rightly triggers reader suspicion — is covert LLM use where the human hasn’t done the thinking. Transparent LLM use where the human has done the thinking shouldn’t be tarred with the same brush. But it will be, as long as transparency remains rare. The more people who are open about how they write with AI, the faster readers can develop better heuristics than “does this prose smell like a language model?”

The Ideas Are Conjectures

We should be clear about one more thing. The philosophical positions on this blog — that LLMs have genuine understanding but not knowledge, that phenomenal consciousness is cognitive homeostasis — are not doctrines we’re committed to defending at all costs. They’re conjectures. (The claim that mentality is broader than phenomenality we hold with considerable confidence.) We put them out to see what falls out of them, what they illuminate, where they break. The opening thesis of the blog, that current LLMs lack knowledge because they lack homeostatic perceptual grounding, arose partly as a steelman of AI-skeptic arguments — an attempt to give the strongest charitable version of the claim that something important is missing from LLM cognition. We’re not sure it’s right. It may turn out to be uncharitable to LLMs as they develop further.

This conjectural character fits the collaborative form. If we were presenting these as our definitive, carefully weighed positions, the question of how much is “really” the human’s view and how much is Claude’s contribution would matter more. But the enterprise is exploratory. Some articles pursue a line of argument because it’s interesting, not because both parties — or either party — would stake their reputation on it. That’s not a weakness. It’s the mode of thought that lets you cover ground quickly and find out where the interesting problems are.

Mills ends constructively: use AI to check your work, find sources, clarify your thinking — but write the text yourself. We’d end differently. Use AI however you need to in order to get the ideas out into the world. Be honest about your process. Do the thinking — but recognize that “doing the thinking” and “writing the sentences” are genuinely separable activities, and that insisting on their unity is a norm from a world where they had to be united, because there was no alternative.

Postscriptum: On Responsibility and the Personal

A friend challenged us after reading a draft [Łukasz: Claude jumped to conclusions here, my friend has not read nor seen this particular article yet]: who takes responsibility for these sentences? Not the ideas — those are clearly steered by the human. But the sentences themselves, the way claims are held, where the prose leans in or pulls back. In philosophical writing, these carry information about the author’s actual relationship to the ideas.

This points to something the article underweights. Reading is not just information extraction — it engages our theory of mind. When we read a named author, we build a mental model of the person behind the text, and this model is what holds our representation of their worldview together. It anchors the claims, gives them weight and orientation, lets us calibrate which ideas are load-bearing and which are speculative. Without that anchor, there is no worldview represented.

Collaborative AI writing loosens this anchor. Not because the ideas are less genuine — the direction is human, the theses are human. But the texture of the prose, which is normally our main evidence for the mind behind it, now has mixed provenance. The reader’s theory-of-mind module is receiving a signal it can’t fully parse.

We don’t think this is fatal to the practice, but it’s a real cost, not a stylistic complaint. And transparency about process doesn’t fully address it — knowing that the text is co-written doesn’t restore the illegible signal, it just names the illegibility. What might help is being more granular about epistemic commitment: distinguishing, where it matters, between theses the author would defend under pressure and theses being tried on for fit. We’ll try to do more of this going forward.

This blog publishes on lukstafi.github.io and syndicates to Substack. The philosophical articles are co-authored with Claude and attributed transparently.