Oh well had a talk with a director at office. He says, instead of using AI to get more productive, people were using AI to get more lazy.
1)
What he means to say is, say you needed to get something done. You could ask AI to write you a Python script which does the job. Next time around you could use the same Python script. But that's not how people are using AI, they basically think of a prompt as the only source of input, and the output of the prompt as the job they want get done.
So instead of reusing the Python script, they basically re-prompt the same problem again and again.
While this gives an initial productivity boost, you now arrive at a new plateau.
2)
Second problem is ideally you must be using the Python script written once and improve it over time. An ever improving Python script over time should do most of your day job.
That's not happening. Instead since re-prompting is common, people are now executing a list of prompts to get complex work done, and then making it a workflow.
So ideally there should be a never ending productivity increase but when you sell a prompt as a product, people use it as a black box to get things done.
A lot of this has to do with lack of automation/programming mindset to begin with.
The way I'm using it is I have AI generate the tool (python script) and then it will use it for the task and for future tasks. As time goes on, the AI has more tools to call on which makes it (and me) more productive (higher quality work in less time)
You are just validating the point that code is spec.
For your proposed system to work one must have a deterministic way of sending said spec to a system(Compiler?) and getting the same output everytime.
Input/Output is just one thing, software does a lot of 'side effect' kind of work, and has security implications. You don't leave such things to luck. Things either work or don't.
Absolutely let’s not do away with the determinism entirely. But we can decouple generation of the code from its deterministic behavior. If you are adequately able to identify the boundaries of the system and run deterministic tests to validate those boundaries that should be sufficient enough. It’s not like human written code was often treated with even that much scrutiny in the before times. I would validate human written code in the exact same way.
>>If the behavior of the system is specified well enough, then the code itself is cheap and throwaway. Why have a static system that is brittle to external changes when you can just reconstruct the system on the fly?
You mean to say if the unit and functional tests cases are given the system must generate code for you? You might want to look at Prolog in that case.
>>Might be quite awhile before you can do this with large systems but we already see this on smaller contextual scales such as Claude Code itself
We have been able to do something like this reliably for like 50 years now.
Start ups mostly move fast skipping the necessary ceremony which large corps have to do mandatorily to prevent a billion dollar product from melting. Its possible for start ups because they don't have a billion dollar to start with.
Once you do have a billion dollar product protecting it requires spending time, money and people to keep running. Because building a new one is a lot more effort than protecting existing one from melting.
One part of the system moving fast doesn't change the speed of the system all that much.
The thing to note is, verifying if something got done is harder and takes time in the same ballpark as doing the work.
If people are serious about AI productivity, lets start by addressing how we can verify program correctness quickly. Everything else is just a Ferrari between two traffic red lights.
>>It's funny to me how still so many don't realize you don't get hired for the best positions for being a 10x programmer who excels at hackerrank
Competitive coding is oversold in this generation. You can log in to most of these sites and you will see thousands of solutions submitted to each problem. There is little incentive to reward situations where you solved some problem which a thousand other people have solved.
To that end its also a intellectual equivalent of video game addiction. There is some kind of illusion that you are indulging in a extremely valuable and productivity enterprise, but if you observe carefully nothing much productive actually gets done.
Only a while back excessive chess obsession had similar problems. People spending whole days doing things which make them feel special and intelligent, but to any observer at a distance its fairly obvious they are wasting time and getting nothing done.
I think the point he is trying to make is that you can't outsource your thinking to a automated process and also trust it to make the right decisions at the same time.
In places where a number, fraction, or a non binary outcome is involved there is an aspect of growing the code base with time and human knowledge/failure.
You could argue that speed of writing code isn't everything, many times being correct and stable likely is more important. For eg- A banking app, doesn't have be written and shipped fast. But it has to be done right. ECG machines, money, meat space safety automation all come under this.
Replace LLM with employee in your argument - what changes ? Unless everyone at your workplace owns the system they are working on - this is a very high bar and maybe 50% of devs I've worked with are capable of owning a piece of non trivial code, especially if they didn't write it.
Realiy is you don't solve these problems by to relying on everyone to be perfect - everyone slips up - to achieve results consistently you need process/systems to assure quality.
Safety critical system should be even better equipped to adopt this because they already have the systems to promote correct outputs.
The problem is those systems weren't built for LLMs specifically so the unexpected failure cases and the volume might not be a perfect fit - but then you work on adapting the quality control system.
>>replace LLM with employee in your argument - what changes ?
I mentioned this part in my comment. You cannot trust an automated process to a thing, and expect the same process to verify if it did it right. This is with regards to any automated process, not just code.
This is not the same as manufacturing, as in manufacturing you make the same part thousands of times. In code the automated process makes a specific customised thing only once, and it has to be right.
>>The problem is those systems weren't built for LLMs specifically so the unexpected failure cases ...
We are not talking of failures. There is a space between success and failure where the LLM can go into easily.
Its crazy how management has gone from productivity to a kind of policing, crowd control and abstract number management that has no sense.
One of things that's being discussed now is AI won't really help people ship products faster, because no one was serious about building them at the first place. In most large corps the managerial layers simply care about preserving their own jobs and comp.
Also similar things an be said about Product management too. Only a few months they would says developers are busy with other things, now that we can ship quite fast, whats your world changing idea? None
I like my manager but I think what many people don't seem to understand is that the manager tends to be the bottleneck. We were moving as fast as he was able to give requirements even without AI, and now we're being told to use as much AI as makes sense and now we just more product roadblocks faster. We may be shipping faster but amortized over all the roadblocks I'd say our velocity is like 10% faster.
>>given that the alternative would be spending 12 hours sewing fast ...
That is the best case scenario. Mostly women roll beedis(a kind of needle sized cigar) on which you get like 1 paise per 10 rolls or something like that. Or worse do assorted labor chores which can really sap one's soul real fast.
Even with all that women actually have it a lot better than men. Men literally die and are reborn every day in most parts of India.
Just drive 30 kms North of Bangalore, and you will see abject poverty scenes. People scavenging bovine dung for fuel, children with flies, no clothing. The ever present scene is always that of an elderly person with pencil thin legs wearing shorts he likely is wearing since a decade with nothing but boiled rice and salt water+turmeric to eat daily. 8 - 10 hr power cuts are the norm, that is if you can afford electricity at all. Most health care is either entirely absent, or you have to travel to the nearest metro and hope you don't die out of hunger getting treatment there. I could go on but that is life here.
£260 a month is actually great for some place like this.
1)
What he means to say is, say you needed to get something done. You could ask AI to write you a Python script which does the job. Next time around you could use the same Python script. But that's not how people are using AI, they basically think of a prompt as the only source of input, and the output of the prompt as the job they want get done.
So instead of reusing the Python script, they basically re-prompt the same problem again and again.
While this gives an initial productivity boost, you now arrive at a new plateau.
2)
Second problem is ideally you must be using the Python script written once and improve it over time. An ever improving Python script over time should do most of your day job.
That's not happening. Instead since re-prompting is common, people are now executing a list of prompts to get complex work done, and then making it a workflow.
So ideally there should be a never ending productivity increase but when you sell a prompt as a product, people use it as a black box to get things done.
A lot of this has to do with lack of automation/programming mindset to begin with.
reply