While alchemy was mostly para-religious wishful thinking, stone masonry has a lot in common with what I want to express: it‘s the tinkering that is accessible to everyone who can lay their hands onto the tools. But I still think the age of nuclear revolution is a better comparison due to a couple of reasons, most importantly the number of very fast feedback loops. While it might have taken years to even build a new idea from stone, and another couple of years to see if it’s stable over time, we see multi-layered systems of both fast and slow feedback loops in AI-driven software development: academic science, open source communities, huge companies, startups, customers, established code review and code quality tools and standards (e.g. static analysis), feedback from multiple AI-models, activities of regulatory bodies, etc. pp. - the more interactions there are between the elements and subsystems, the better a system becomes at doing the trial-and-error-style tinkering that leads to stable results. In this regard, we’re way ahead of the nuclear revolution, let alone stone masonry.
The inherently chaotic nature of system makes stable results very difficult. Combine that with the non deterministic nature of all the major production models. Then you have the fact that new models are coming out every few months, and we have no objective metrics for measuring software quality.
Oh and benchmarks for functional performance measurement tend to leak into training data.
Put all those together and I’d bet half of my retirement accounts that the we’re still in the reading chicken entrails phase 20 years from now.
