•
2 min read

Code is accelerating faster than language, why?

Recently coding ability has advanced much faster than prose generation e.g. expectations about GPT5 vs GPT4. There are both obvious and more subtle reasons for this. On the obvious side, there is the economic argument that with developers being early adopters they both provide clear feedback and financial reward in adoption creating a positive feedback loop. There is also the utility, frontier labs can themselves benefit greatly from enhancing the productivity of their own software projects.

A more subtle issue lies in the very nature of the LLM as a transformation function. When you start from a small prompt and transform it out to a larger amount of content, you are limited by input entropy. There are only so many meaningfully good outputs from a small input.

In coding, it is possible to generate large quantities of reasonable code from a small description of a problem. Coding is inherently formulaic and structured, there is often a lot of boilerplate (highly compressible information) and the utilisation of standardised repetitive paradigms can actually improve the quality of output rather than degrade it.

In contrast, prose has standards and expectations but ultimately rewards greater novelty and subversion of paradigms. That makes generation from a small input very problematic. Producing highly standardised outputs (listings, reports etc) is a simpler problem than producing truly engaging or delightful writing.