What's the point of the rewrite if it doesn't fix the underlying issues, though?
A rewrite being a good idea often hinges on the ability to simplify. After a decade or more, it's now apparent what the application should and shouldn't do, so one can build it with those learnings and shed all tech debt from how it grew organically.
Aka preserving all behavior is not what I would want from a rewrite. The point would be to make decisions on what behavior should be kept and what complexity can be removed. An AI can't do that. It can help with execution if the decisions are made, but they're made by being very intimate with the codebase and floating all cases and then talking with stakeholders.
Au contraire - LLMs are quite bad at large scale pattern fidelity. They'll even forget key details and constraints unless told over and over again. That's why AI-written code has the quality of a patch-on-patch-on-patch.
Fully agree. I tried to refactor parts of a large code base with Fable+ultracode and it just keeps accidentally merging distinct concepts and making up explanations/reasonings that the code base did not contain.
For example, the code base contains a physical controller. It’s closed loop in that it can react in realtime to changes. But it’s a slightly untypical implementation because this one can even look into the future through simulations. But Fable does not understand that. Instead, I need to remind it every 30 minutes that this is closed loop. It keeps wrongly claiming that the controller was open loop and then based upon that it will make up constraints that don’t actually exist.
Joel is right, but he's also wrong. I've been on the other side of a timid engineering culture that commerical rides roughshod over and its this depressing immeasurable decline. The company stagnates and slowly tailspins around an unmaintainable product until a competitor steals their lunch in a way that that further obscures cause and effect.
Estimates are considerably longer, QA is much harder, integration is full of buckets and rakes, some "senior" devs are afraid to touch stale core code, innovation is stifled, devs are frustrated, hiring is harder, attrition bites. The most frustrating thing is that its very hard to communicate the issues as everyone experiences a fragment of the pain and none of it lines up in a spreadsheet for anyone to appreciate the whole cost. Everything just sucks.
LLMs changing the economy of this sounds great, especially if removes the essential issue with the ground up rewrite, which is the "ground up" part.
The LLM might change the economy of this, but I doubt it.
I tend to believe that the engineering culture you describe will end up producing similar or, as Joel postulates, an even worse result, just dressed up in a modern stack.
If the technical leadership remains the very same one that enabled such a culture, I don't see them being able to suddenly produce a genuinely better software product only because an LLM is in a picture - especially considering how easy it is to convince an LLM that your idea is the best one.
I think the important lesson is to use clear eyes to evaluate what the rewrite buys you. I was on a team that rewrote a native code app in C#. We also had access to early cloud tech in the Azure stack, what is called queue now and then was called service bus.
These two technologies combined greatly simplified this specific product making it far easier to maintain. Performance on these services was not important so native code was carrying a lot of penalties without the benefits.
Having a well documented messenger like service bus with great SLAs removed several tools we had needed in the old implementation.
We were able to leverage the tests form the original product to define success and tmthus were able to solve a lot of the edge cases in the new code w before we even shipped.
However, the old code was perfectly fine code. If new technologies had not provided significant simplification of the service architecture, a rewrite would've been foolish. And without the very good previously existing tests, we would've run into a lot of issues as we released.
Somehow this article doesn't even mention the fact that AI makes software rewrites much, much faster than before and with higher confidence of backwards compatibility.
Nowadays, a good AI harness can fairly reliably rewrite a medium complexity piece of software to an appropriate modern tech stack with pretty strong confidence of exactly preserving its behavior. The AI can pick up legacy details and keep them exactly the same as before in ways that a human rewriter would usually not bother with. After rewriting each feature it can then exhaustively smoke test all the happy paths and edge cases and ensure the code behaves exactly the same as before, which is another thing that human rewrites basically never do.
AI <<can>> do a lot of things, but does it actually do that without an exhaustive test suite (which legacy software generally doesn't have, and it can never be 100%, anyway)?
Between context collapse and hallucinations, how likely is it that the end result isn't slightly polished slop that misses lots of crucial details?
every sentence stands on its own because it's the most insightful soundbite of wisdom every constructed.
·
Aphorisms for the collective upgrade of consciousness.
·
delivered one tweet at a time.
·
(this comment adds to the discussion ironically by demonstrating how ridiculous it is to have to derive signal from this format. Please do what you need on Linkedin but take some semblance of effort to honor this community. Or don't. sigh)
That's very true. People put up with the many limitations of off the shelf software because it's cheaper, not because it's better. Developing bespoke software solutions is now a lot cheaper than it used to be. So, there are a lot of cases where that now becomes the better option.
Doing in days what used to take months, is a bit of a game changer. Like with past cost reductions, people will underestimate the work and get it wrong. It helps if you know what you are doing rather than just vibe coding things.
But for rewrites, the sunk cost fallacy becomes a lot cheaper. So, that changes how you deal with stuff that clearly isn't living up to expectations. Unceremoniously replacing what wasn't that expensive to begin with might be the cheaper option relative to fixing it.
This certainly does. If we think from this angle, it really begs the question of what language/tech stack to use if a company wants to start a new project. On one hand, if company uses a very well tech stack, development and rewrites will be faster due to AI having way more examples to draw from. In certain cases, AI will handle some edge cases which are difficult to come by/replicate under strictest test procedures. Overall, that results in faster workflow. On the other hand, if this company choose a newer stack which may be better better than older popular frameworks, development time will increase (along with rewrite time)but the product might be better. we have to see how companies handle this in the future, given this is also affected by how cheap/expensive token consumption becomes. Using something pretrained vs training and then using an AI has cost implications when done in a large scale. It will be interesting to see what directions companies go to, faster workflows and delivery using AI or potentially a better product using more manually written proprietary code with lesser AI involvement.
I don't think that holds. Even internal docs for bespoke frameworks, with examples, are enough to steer AI. The main thing is that both the API and the docs are well written. Easier said than done, but you can ask AI how to write effective documentation for AI.
>if company uses a very well tech stack, development and rewrites will be faster due to AI having way more examples to draw from.
Eh maybe not.
Stuff that has a lot of deprecated features is honestly burdensome on AI. It keeps rediscovering the deprecated features as the understanding that they are deprecated fall outside of the context window.
What you need is something that either never deprecates syntax, or is <10 years old with minimal changes over that time.
This kind of data-free opining reminds me of the Mythical Man-Month. Yeah, in theory adding more people to a project will speed it up. And all people are replaceable so I can hire 100 bodies for cheap and we'll be done with this project ASAP.
Sounds great! Have you tried this? Did you see what went wrong? Otherwise this is just the same nonsense as always.
The point where I truly feel that AI is a game changer is that these kinds of posts keep appearing. Tautological outcries keep going on both sides, pro and con, endlessly repeating circular logic. There's no real substance or evidence, and rather than discussing how things were actually applied, it's just an echo chamber for whatever group you belong to.
In that sense, my homepage (https://www.makonea.com/en-US) doesn't even make it to the HN front page—it's mostly in SHOWDEAD. Does that mean it has less value than this post? I'm feeling a sense of doubt about myself.
What do your tests look like. Because rewriting by hand and rewriting via AI have the same load bearing on whether or not your tests cover your scenarios and your integrations well.
The amount of armchair quarterback commentary in the software business as concerns people waxing eloquent a out difficult things safe atop a perch of the same easy things achieved multiple times has always been obnoxious, offensive to the thermodynamics of the situation as situated by Landauer.
But this new "you're holding it wrong" series by people whose grasp of the system gets fuzzy somewhere in the v8 headers is a new land speed record for being vacuously correct and still an attractive nuisance for profit.
Yes, the trend towards encoding hard-won domain knowledge as property and fuzz testing and sometimes even proof system was underway before ChatGPT, and yes, the economics of this approach bend sharply under a post terrawright world.
But no, you haven't added anything except tinsel and chaff and some green css on mixpanel.
Just stop with this shit. If you knew shit about AI you'd be too busy printing cash to teach the rest of us about it.
I'm not sure there is any value in knowing shit about AI. I know quite a lot about enterprise organisation level AI, but really, you could just ask an AI and it'd guide you through the processes. Knowledge in general is going to become real cheap in the age of AI. I've been a data archtiect in the past, so I used Opus 4.8 as I would've used a consultant agency on how to do our data architecture for multiple standard systems which can't directly share data with eachother. After a couple of hours with it as a sparring partner, I had some pretty awesome powerpoint decision making slides, one for c-levels and one for it-management.
Since our owners also own an IT consultant agency, I ran the same process through with one of our regular consultants who is an actual awesome data architect. The output was strikingly similar, well except that I/we didn't need to make the slides. I then had him run over the actual slides, and all we changed was adding a { between some arrows to make the source of the arrows more clear.
We're still going to use real human consultants in the loop because they are readily and freely available, and because this is still new. I doubt we'd want to spend 100 consultant hours on something like this in 5 years though. I mean, we'd still do it for decisions where we'd want someone to blame.
A rewrite being a good idea often hinges on the ability to simplify. After a decade or more, it's now apparent what the application should and shouldn't do, so one can build it with those learnings and shed all tech debt from how it grew organically.
Aka preserving all behavior is not what I would want from a rewrite. The point would be to make decisions on what behavior should be kept and what complexity can be removed. An AI can't do that. It can help with execution if the decisions are made, but they're made by being very intimate with the codebase and floating all cases and then talking with stakeholders.
For example, the code base contains a physical controller. It’s closed loop in that it can react in realtime to changes. But it’s a slightly untypical implementation because this one can even look into the future through simulations. But Fable does not understand that. Instead, I need to remind it every 30 minutes that this is closed loop. It keeps wrongly claiming that the controller was open loop and then based upon that it will make up constraints that don’t actually exist.
AI won’t.
https://www.joelonsoftware.com/2000/04/06/things-you-should-...
Maybe the LLM will catch and reproduce all corner cases... maybe not...
Estimates are considerably longer, QA is much harder, integration is full of buckets and rakes, some "senior" devs are afraid to touch stale core code, innovation is stifled, devs are frustrated, hiring is harder, attrition bites. The most frustrating thing is that its very hard to communicate the issues as everyone experiences a fragment of the pain and none of it lines up in a spreadsheet for anyone to appreciate the whole cost. Everything just sucks.
LLMs changing the economy of this sounds great, especially if removes the essential issue with the ground up rewrite, which is the "ground up" part.
I tend to believe that the engineering culture you describe will end up producing similar or, as Joel postulates, an even worse result, just dressed up in a modern stack.
If the technical leadership remains the very same one that enabled such a culture, I don't see them being able to suddenly produce a genuinely better software product only because an LLM is in a picture - especially considering how easy it is to convince an LLM that your idea is the best one.
These two technologies combined greatly simplified this specific product making it far easier to maintain. Performance on these services was not important so native code was carrying a lot of penalties without the benefits.
Having a well documented messenger like service bus with great SLAs removed several tools we had needed in the old implementation.
We were able to leverage the tests form the original product to define success and tmthus were able to solve a lot of the edge cases in the new code w before we even shipped.
However, the old code was perfectly fine code. If new technologies had not provided significant simplification of the service architecture, a rewrite would've been foolish. And without the very good previously existing tests, we would've run into a lot of issues as we released.
Nowadays, a good AI harness can fairly reliably rewrite a medium complexity piece of software to an appropriate modern tech stack with pretty strong confidence of exactly preserving its behavior. The AI can pick up legacy details and keep them exactly the same as before in ways that a human rewriter would usually not bother with. After rewriting each feature it can then exhaustively smoke test all the happy paths and edge cases and ensure the code behaves exactly the same as before, which is another thing that human rewrites basically never do.
Between context collapse and hallucinations, how likely is it that the end result isn't slightly polished slop that misses lots of crucial details?
Doing in days what used to take months, is a bit of a game changer. Like with past cost reductions, people will underestimate the work and get it wrong. It helps if you know what you are doing rather than just vibe coding things.
But for rewrites, the sunk cost fallacy becomes a lot cheaper. So, that changes how you deal with stuff that clearly isn't living up to expectations. Unceremoniously replacing what wasn't that expensive to begin with might be the cheaper option relative to fixing it.
Eh maybe not.
Stuff that has a lot of deprecated features is honestly burdensome on AI. It keeps rediscovering the deprecated features as the understanding that they are deprecated fall outside of the context window.
What you need is something that either never deprecates syntax, or is <10 years old with minimal changes over that time.
Sounds great! Have you tried this? Did you see what went wrong? Otherwise this is just the same nonsense as always.
In that sense, my homepage (https://www.makonea.com/en-US) doesn't even make it to the HN front page—it's mostly in SHOWDEAD. Does that mean it has less value than this post? I'm feeling a sense of doubt about myself.
But this new "you're holding it wrong" series by people whose grasp of the system gets fuzzy somewhere in the v8 headers is a new land speed record for being vacuously correct and still an attractive nuisance for profit.
Yes, the trend towards encoding hard-won domain knowledge as property and fuzz testing and sometimes even proof system was underway before ChatGPT, and yes, the economics of this approach bend sharply under a post terrawright world.
But no, you haven't added anything except tinsel and chaff and some green css on mixpanel.
Just stop with this shit. If you knew shit about AI you'd be too busy printing cash to teach the rest of us about it.
Since our owners also own an IT consultant agency, I ran the same process through with one of our regular consultants who is an actual awesome data architect. The output was strikingly similar, well except that I/we didn't need to make the slides. I then had him run over the actual slides, and all we changed was adding a { between some arrows to make the source of the arrows more clear.
We're still going to use real human consultants in the loop because they are readily and freely available, and because this is still new. I doubt we'd want to spend 100 consultant hours on something like this in 5 years though. I mean, we'd still do it for decisions where we'd want someone to blame.