you have to know what to wish for
you have to know what to wish for
Last month, someone high up at $dayjob said that we should treat any line of
code written by hand as a failure.
This hit me like a ton of bricks, to be honest. What does it mean to be a software engineer without writing code? Is this the end of software engineering?
Let’s define terms.
Programming is theory building, as Peter Naur remarked. The job of the programmer is to build and maintain a mental model of the problem space the system they develop is written against. There’s a feedback loop there in the small: the programmer writes a program to solve a problem, and learns more about the program and its environment when they run the program1.
Change, edit, crash, swear, learn, repeat.
Excelsior! Chris wrote a bit about tests as feedback loops for interrogating systems in the last post.
With luck, eventually the program does what the programmer intended. The programmer walks away changed from the experience. They know avenues not to explore, patterns to match on that let them skip steps, maybe something interesting about their tools. Writing the code changed their mental model of the program.
This understanding, not the code, is the thing of value.
The programmer could write documentation about the program, but their exact understanding at that moment cannot be captured in full. This is what Naur is talking about: the programmer has a theory of the system, developed in concert with that system. That theory is a mental model, a map of the system, problem space, and how it might react to changes in the future.
Software Engineering is the act of building a socio-technical system that promulgates that theory to a group of people – “engineering is programming integrated over time” 2. This is to say, the goal of software engineering as an organizational capability is to model the problem domain and current solution, keep multiple people in alignment on that definition, and use that understanding to integrate changes from the business.
Quoting Russ Cox quoting Titus Winters. Programming is also learning to dereference references!
Just as programmers anneal their theory of the system by changing programs to understand the effect of their changes; software engineering manages concurrent theories of the program held by everyone involved in changing, building, and shipping the software: product managers, engineering managers, executives, marketing, sales, etc., in addition to the engineers themselves. The goal is to align all of these different ways of understanding the system.
This is, of course, impossible3. But trying matters:
EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM
This comes to us from the field of cybernetics. Here, a “regulator of a system” is an entity which, given a set of inputs into a closed system which may produce a number of outputs, a subset of which are desirable, takes action to steer the system towards desirable output. The theorem says that a good regulator is a model of the system under regulation – a homomorphism, potentially lacking the detail of the full system. Air traffic control models planes into the airspace around an airport for the purposes of routing and sequencing, but lacks the full detail of the airport being modeled: what is the temperature inside the gate? Is the nearby river hosting migratory geese? What is the status of the airport Chili’s (TM)?
Bad regulation increases the entropy of the system, destabilizing it: if air traffic control doesn’t model the geese, they might not be able to account for bird strikes.
Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.
Systems reflect the communication structure of their respective organizations because of the positive pressure of the good regulator theorem. Systems which don’t reflect their organizations tend not to be stable, even over a short run. Either the system produces incentive to align the organization around it, or the system fails spectacularly.
In either case: software engineering is the capability to translate the desired state of a product system into reality, integrating it with the current theory of that system. Information doesn’t flow one-way: engineering and product management are always in productive conflict, pushing each other back and forth on the effect of changes to the system. And in the middle is the theory of the program: this conflict is only ever as productive as their models of the system are accurate.
I say alignment is impossible, but take this in the sense that there’s no such thing as perfect communication. A listener always receives something different than what a speaker has in their head: the act of communicating quantizes the information to be transferred.
I want to note that, as humans, we live and die by our ability to abstract. We cannot hold perfect models of the world. We are always navigating through assumptions and missing information. “Making a mental model” doesn’t mean making a perfect model. We’re constantly updating those models as we find out new information (or as we find out we don’t need to remember specifics.)
And this is true of how we think about other organizational roles: we hold a cartooned, simplified version of other jobs in our head. A manager’s job is to manage, but what does that look like from the point of view of an engineer4? A product manager works to define what the product needs. A designer builds how the product feels – visually, primarily, but on other axes as well. These are tiny definitions for whole careers worth of details!
As an engineer who has bounced through management: it looks like a completely different set of (complementary) skills, and a lot less control over specific outcomes than you’d think.
Every role carries its own frame with respect to the system the organization manages. The frame the role carries determines the problems and opportunities it can see – what is legible to it.
And so, when I say that product and engineering have a productive conflict: it is because engineering can see things that product can’t, and product can see things that engineering can’t. And there is a career’s worth of details involved in building either of those frames.
Software engineering is the function of an organization that assimilates change into the existing software system. Historically, it’s been the function responsible for translating that change into specifics, and communicating out the cost of those specifics.
But we’re in a mechanized world now. Roles are bleeding over. The cost of bespoke, one-off software has dropped to zero. The genie is here to grant your wishes.
The genie warps the cost of everything in software engineering. It’s as if space-time itself has warped. Think of a map: there is a coffee shop 20 minutes away by foot. Now imagine the genie installs a pneumatic tube that transports you 100 miles away to another coffee shop and back in 10 minutes. That 100-mile-distant coffee shop is now closer to you, temporally, than the one down the block.
What’s more, everyone gets a lamp. Anyone can generate a program now. Bespoke software is commoditized. Generating or re-generating an entire working program is as simple as prompting. The less you care about the implementation – or keeping that implementation working in the face of change – the broader you can be about the prompt.
That said, what you say to the genie matters. A lot. If you don’t know to avoid certain problems, the genie will not tell you that you should think about including a solution for them. The genie uses your frame, for better or worse.
And the genie understands the world anew every time you wish. It reads documentation it left behind, sure, but as we established, that is not the same thing as the theory of the program. So it’s still on the person who rubbed the lamp to understand the implications of their wishes; even more so on the compounding results of those wishes.
So the problem remains, but the means have changed.
Programming is no longer the foundational practice of software engineering.
That is: the work of software engineering has fled from the act of programming both to the left side of the process – definitional work (“how do we define this problem and how do we wish to solve it”) – and to the right side – verification work and risk assessment (“is this implementation correct and what will happen if we’re wrong?”)
The middle of the problem is where a lot of us engineers have spent most of our careers. It’s where we developed our understanding of software systems in the first place and where we hone our ability to change them. Former colleague Laurie Voss calls this kind of engineer a “craftsperson”: focused on the means instead of the outcome.
I think of this work-in-the-middle in terms of “feedback from the medium”: I understand programs better by changing them; I find seams in them that suggest future changes, lines along which to cut, and the process of getting my fingerprints on a codebase makes it more real to me5.
It is practice, in other words.
I would venture a guess that my “craftspeople” feel similarly. Being asked to change or understand a program without programming it feels like removing one’s proprioception: like floating in a sensory deprivation tank. The impressions you get of the codebase are unmoored. You have to build up a new set of reflexes to grasp the bottom of the pool.
But it is possible to build up that sense again. It feels different. The details still matter, but how you come to understand those details changes. I spend a lot more time walking the codebase with Claude and testing my assumptions there. Being able to communicate how to visualize code becomes way more important: I ask the genie for maps of the system from different angles, rendered in graphviz. Claude can put things that were tedious bookkeeping (git blame histories) more directly at hand6.
And Claude can steer my editor while I walk this path, which is possible because I knew that neovim supports a socket interface and that Claude can self-discover how to control it to write a skill. This is trivia!)
So the details still matter. In fact, the trivia is more powerful, because the genie only grants wishes as well as you’re able to frame the wish. If you can think of novel ways to combine existing concepts, you’re getting a different value from the genie than someone else would.
From a language perspective, it’s a lot of fun to hack with: I’ve noticed that my plans are executed more successfully when I run them through a grug style filter, for example7.
Semantic quantization, probably: “I dislike ternaries” becomes “grug hate ternaries. hit with club”.
But this is a manic “the sky is falling” sort of fun. I return to the question. Is software engineering a viable career if you take out the programming?
The frame you spent years developing is more valuable than ever: you are capable of regulating these tools safely because you are a model of how they operate. The new software engineering looks a lot like the old. It values most of the same things in the large: exploration, trivia, model-building. It amplifies your ability to build the theory of the program.
So, does hand-written code represent a failure? Well, … no. At least, I wouldn’t call it a failure. But treating it as such is a useful forcing function. It’s just not required anymore. Where it helps you develop a better model of the program, it’s arguably more worthwhile than ever.
Thanks to @ceejbot for reviewing a draft of this post!