In Linkedin Group “Agile and Lean Software Development”, Allen Holub asked:
Many of us use LLMs as part of our development process. I'm interested in hearing how other people do that. Could you discuss your process in the comments?
My response:
I have been using intensively Windows/M365 Copilot, ChatGPT / Codex and Claude.ai in developments: 1. Entry level understanding of old new tech before further research and study. Old means the tech has been around for years, new means I am new to it. 2. Simple ad-hoc data transformation. 3. AI to craft simple helper functions and modules of common topics. 4. AI to craft a simple service broker, for example a broker handling the auth of Quickbooks for unattended data sync service. 5. Some tech details I have come across but I could barely remember, while I understand the concepts. Regarding the quality of code generated by AI, mostly working with minimum modification by me. The more complex the requirement is, the poorer the quality is: 1. More human workload needed to fix it. 2. The code generally is over complex in both design and implementation. 3. Rather verbose and too many low level API calls, while a few simple high level APIs can replace. I do understand why LLMs AI has such performance in "writing" codes: 1. LLMs largely depend on statistics and scanning existing codes. 2. LLMs by nature essentially do not understand how to code or understand the problem. It is more like simulating understanding and crafting. 3. Most working codes out there are mediocre: verbose and over complex. 4. No matter how well you have literal conversation with AI, it is mostly impossible for the AI to fully understand your business contexts. 5. ... Overall, Copilot is fairly helpful, while as a programmer you need to be very mindful about its inherent limitation of LLM. One of the key factors is how much maintenance efforts you are going to pay.
Over years, I have written a few non-trivial code generators, and the recent ones include:
The driving force is that I would deliver more business value through less work. And I am a lazy programmer who easily get tired of writing repetitive code even if I get paid to do so, in particular, HttpClient code in C# and AJAX calls in JavaScript.
I am weary of inheriting legacy codebases authored by diligent programmers but burdened with overly complex architecture and software designs. In many cases, the architects and developers did not fully appreciate the inherent capabilities of the underlying platform, nor did they apply fundamental computer science principles such as high cohesion and loose coupling effectively.
In one notable example, designed by an A-list consultant firm in town, an XML transformation pipeline performed five rounds of XML DOM schema validation, despite the fact that XML serialization alone could have solved the problem far more cleanly and efficiently. .NET (Framework) can generate assemblies at runtime or build time for both XML readers and writers directly from XML schemas, making much of that validation logic redundant. And the performance of that part of data transformation had become as least 20X faster with must less process power and memory consumption. This was not only example I had seen: using cutting edge technologies the dirty way, though not entirely wrong, since the legacy, fresh and dirty codes were “working”.
These experiences put me in a good position to discuss how AI can assist software development, and where it falls short—grounded in mid-term and long-term, and explore real-world systems, tools, and trade-offs, rather than abstract theory alone, even though I am not an AI expert.
These days, most programmers in commercial worlds have still been using 3GL like C++, Java, C#, Python, Dart, Swift and JavaScript etc. Therefore, in the following chapters, I will mostly focus on examples of 3GL, and general purpose programming languages, and C# .NET. If you are a Java or C++ programmer, you may find similar examples by your own researches.
The following is summaries of various technical resources that help lazy programmers.
Hints:
.NET Framework and .NET Core include a lot built-in framework modules that app programmers could utilize in most business applications:
Utilizing those built-in module wisely could make your app code short and simple, so easy to maintain.
.NET and Java provide runtime optimization. And generally clean code can help the runtime to do better optimization.
Hints:
Remarks:
These days, when googling “Application Generator”, the top results are basically all AI app generator or builder. I remember I have seen a lot advertisements of application generators in 1980s, generating business applications for DOS or Windows. For example, input a few business constraints with a little further configuration and optional customization, the generator could then generate a Northwind like application. Here’s a summary about app generators before LLMs AI becoming popular.
Rather than generating the whole app, “Model to Code” offers more flexible and generic solution for application programmers, to realize implementation from high level design with UML or alike and to maintain an “active model” where changes in the model instantly updated code, and vice versa
I have come across ModelMaker, and liked it for a while. ModelMaker was a UML‑style modeling and refactoring tool built specifically for Borland Delphi’s Object Pascal. It supported:
ModelMaker has faded away in history, and these days you have Visual Paradigm and alike supporting Java, C++ and C# etc generated from UML diagrams.
Rather than starting from UML diagrams, Windows Workflow Foundation offers a form of “Model‑to‑Code”, more exactly, a model‑driven execution system where the model is the program. For some reasons, MS has discontinued WF. Nevertheless, there are still alternatives around for such classic Model-Driven Development (MDD).
“The term 4GL was first used formally by James Martin in his 1981 book Application Development Without Programmers to refer to non-procedural, high-level specification languages. Some researchers state that 4GLs are a subset of domain-specific languages.”.
Application Development Without Programmers
Oh yeah, literally isn’t this the main intend that nowadays companies are pushing for AI code agents?
Nevertheless, I have happened to use dBase, Visual FoxPro, PowerBuilder and Omnis Studio in 80s and 90s. In my observations, 4GL may reach the bottlenecks when the business grows and changes:
According to Wikipedia, Domain-specific language is a computer language specialized to a specific application domain. This is in contrast to a general-purpose language (GPL), which is broadly applicable across domains. There are a wide variety of DSLs, ranging from widely used languages for common domains, such as HTML for web pages, down to languages used by only one or a few pieces of software, such as MUSH soft code.
In last 2 or 3 years, there have been bombardments of academic papers, marketing materials and criticism around AI Code Agents. I am at the consumer end of AI code agent, and here’s my observation: The output of traditional code generation is predictable and deterministic, and can be compiled and used immediately without modification, while AI generated code particularly though Python meta-programming mediated by AI is not so predictable and deterministic, and often contains language syntax errors and other logic errors.
I asked M365 Copilot about my observation, and here’s the analysis returned about “Traditional Code Generators v.s. AI Code Agent”. And you are very welcome to ask similar question to your favorite AI agents, or even better, ask real AI experts.
Keep the shortfalls of AI code agents in mind, you as a programmer are in better position to get the best from those agents, and deliver more business value through less work.
Probably you have heard: “Scrum: The Art of Doing Twice the Work in Half the Time”. In my observations, the practices by the name of Scrum have often resulted in doing twice the waste in half the time, since most work involved in value delivery is waste, due to various of ScrumBut practices and factors. Overall it is not too bad, if the waste is inevitable.
And if not being careful, mindful and cautious enough, you using AI code agent could result in delivering 10 times of the technical debts in 1/10 of the time, since the code generated could be severely over complex and lengthy.
Are you confident that AI is able to reduce the pure technical debts it created?
Are you sure you don’t need to care about technical debts in most real world business applications?
In my observations, AI code agents are generally good at doing Plus - adding tons of code politically correct regarding SOLID and design patterns etc., just like a junior developer armed with all the academic knowledge of software engineering but without real world experience in making trade-off according to business context. And currently those AI code agents have near zero capability of doing Minus – removing code and simplifying design.
There are reasons why I have listed many traditional codegen resources predated LLMs AI code agents. Before the AI code agents become a thing, I have often said the following statements or examples:
Remarks:
I have always got poor memory and could barely remember technical details except those I am using actively and repetitively, but I don’t enjoy doing repetitive technical works. Look at the bright side, the IT landscapes keep changing, basically reshuffled every 2 years since early 70s. Therefore, such disadvantage make me forget legacy and obsolete technologies, while I am good at understanding concepts.
Remarks:
Likewise, AI is your alcoholic friend. You might have experienced AI had often mixup old and new technical items. And when being questioned, AI will try to fix through further research or “imagination”. And you have probably come across studies about AI hallucination.
For example:
Old means that the tech has been around for years, new means that I am new to it. Hit rate: 95%.
Remarks:
Ask yourself some questions.
Who care if the code is long or short, clear or dirty? as long as it run? as long as we have fast hardware?
I have some online showcases, open sourced, which 1/5 legacy commercial. done it. some by A list consultant firm, done it, or evaluated it.
I am not sure when training the AI, how the labelling process works?
My example, with Google AI Studio
There are 2 types of learning: fact learning and concept learning.
According to “Traditional Code Generators v.s. AI Code Agent”, AI‑generated code (including Python‑mediated meta‑programming) is:
As a developer or a business person, are you so sure that you would put your business interest to the hand of AI code agent and also expect human developers to quickly fix the hidden problems or technical debts generated by AI?
One bit can kill.
Summary
Mocking how AI tries to sound correct. delusion.
Scrum: Doing Twice the work at half the time.
My take: Delivering twice the value through half the work.
AI: Doing thrice the work at no time.
Well trained developer + AI: Delivering twice the value through half the work at short time.
The bright future …
The glimbe future …
A List consultant firm showoff their abilities of creating complex architectures, utilizing 3rd party DI frameworks..
Systematic study is more important than ever.
You can not grow your systematic thinking and critical thinking through learning from the fast foot knowledge provided by AI.
With systematic thinking and critical thinking, you can maximize the usefulness of AI to your daily brainwork.
Even before AI coding has become a thing, as a full-stack developer, when leading, I spend around 1/3 and 1/4 of billable hours in coding including testing, and spend the rest in studying, thinking and talking to peers and business stakeholders. Typically the size of codebase is around 1/5 to 1/3 of those written by diligent programmers, for the same functional requirements and technical stack. Think about maintenance cost and runtime performance.
Now I have AI as diligent programmers, I have been thinking how to make best use of them, get the benefits, and avoid over complexity and bloating codebase.
Human programmers have desire of making things short and simple, for the sake of maintenance and runtime performance, AI code generators don’t care, since they can write large amount of probably working codes quickly.
All these are just my impressions, not so scientific backing up by statistics. However, I believe the industries have been gathering statistics as AI coding has still be rapidly evolving.
How much AI can improve overall productivity?
The following is my personal opinions without statistics support.
Definition senior developers:
Without a degree, self-taught software developer is often OK, while having a degree is not to obtain only knowledge, but also undergo systematic trainings for the ways of thinking and teamwork. Surely some can obtain such skills without a degree.
Due to the current infrastructure and architecture of AI training, AI can not really understand business contexts and technical contexts as human can.
Clean code + meaningful doc comment may help AI to help you.
Scrum: Doing Twice the Work at Half the Time. Reality: Doing twice the waste at half the time. Not too bad, since many so called works are essentially waste. Better: Delivering twice the Value through half the work.
How to determine value, and some works are not waste?
Through proper conversations and in-depth study.
How to identify waste and technical debts?
“Ask you senior developers” said Patrik Varga.
Indeed. Why? because senior developers are the first who suffer, the next is the customers/users, then the business. In my observation, over 95% of technical debt is the result of mismanagement at multiple levels, and this is matching what documented in various software engineering textbooks like “Facts and Fallacies of Software Engineering”, rather than the capability of software developers. This is one of the reasons I often say, technical debt is essentially management debt. Why single out senior developers? because many junior developers may enjoy resolving technical debt and creating technical debt. So they may be blind no matter what method you would use. Therefore,
Great Confident:
Onb bit can kill,
Fashion recycle? regarding app generators.