Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise. I think I need to start sharing that skepticism more loudly and publicly!
(I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization, code generation, and others)
I think knowing what you want (and being able to articulate it with adequate specificity, and detecting when processes drift away from it) is a real skill/field of expertise. I just don't think it's specific to interfacing with "AI."
I almost entirely agree with this. However, after coding quite a few large projects with LLM assistance, I can say with certainty that there is some form of "art" at play with regards to structuring requests for LLMs.
However, this "art" is not so much about how to present a given request to the LLM, but rather guestimating what the scope of the next chunk of work should be to balance getting as much out of the model as possible while avoiding the machine going off the rails.
Obviously, this is a moving target and different models perform differently for various chunk/scope of work. I look at my successful sessions with LLMs and I'm not sure I'd be able to articulate a clear set of rules to apply here. You just... gradually build a intuition for how much you can throw at the LLM at once.
That being said, I'm pretty convinced at this point that this is a property of the coding assistants as they exist today, and what "working well with LLM assistance" means will keep on changing.
I've found that one of the most useful things I can do when using LLMs is paying attention to the tool calls so that I can tell when it's giving an explanation that's obviously not based on any evidence. If it explains why a bug happens without reading the relevant files, it's obviously just a guess. If I ask about what things exist that fit certain criteria, and it doesn't search online, at best it's relying on outdated knowledge from training and at worst it's literally inventing things that don't actually exist.
In a lot of scenarios for software engineering, the cost is just wasted time without anything useful as a result, and that's already bad enough. I can't even imagine working as a lawyer and not even taking the time to validate so I don't end up reprimanded by a judge in front of my clients, but there have been so many news stories like this that obviously this is not anywhere close to a universal view...
IME "creating a good spec" (and good acceptance criteria) has always been one of the most valuable skills in... well, everything.
The machine is really good at working the spec on its own now, which is amazing, science fiction shit. But you've still got a garbage in, garbage out problem at the end of the day, which is pretty much the only hope we who work in software have of remaining somehow employed.
I'm curious if there has been any study on the output quality of LLMs after actively antagonizing them. It's easy to forget that at the most basic level these are just auto-complete machines, and humans tend to not respond so well after antagonized. It stands to reason that the same should follow for an auto-complete system trained on human interaction.
Intuitively I'd say lashing out on an LLM overconstraints it and so worsens the output, therefore the skill to stay calm and neutral is useful for guiding a machine as well. Would be useful to have data on that.
> Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise.
I feel using AI (effectively) is not too far from the skillset of programming. It is still a machine following instructions (just, maddenigly non-deterministic, but still close enough), so the same insticts of breaking down work into clearly defined sequences that make a good programmer also make a good AI jockey.
In this case they didn’t know what they were looking for. They just wanted some new, non-trivial exploit.
For the vast majority of corporate usage of AI for SWE, you have a much better idea of what you want, or what the problem is, etc etc. And communicating that to the model effectively is absolutely a skill. I see colleagues every day who very much do not have that skill.
I agree with this in general but I think there's a nuance. 'dumb' prompts can get the result, but require more back-and-forth with the model and more token usage when compared to a crafted prompt
For sure. And honestly the real skill is knowing when you can be lazy and when you will get a good ROI on a good prompt. Sorry I sounded a little like Claude there but that’s what I meant to say.
It's still a skill for now. Prompt engineering specifically is perhaps not as much of a skill anymore, but I think there's a level of skill involved in understanding and/or anticipating when a model is going off the rails.
Some people struggle to effectively use AI because they either have to spend a lot of time reading and thinking about the response or they have a hard time noticing when the model is subtly going off the rails. Others use it to good effect because they can anticipate which tasks would be better handled manually, or are good at catching that the way the model is describing something subtly indicates a misunderstanding.
It can be a skill, but a very soft skill like keeping control of and educating 30 kindergarten students. It’s a skill, hard earned and nebulous. you can recognize it clearly when you see it, but it’s very hard to define/measure numerically and will never achieve anything close to “perfect” results.
Until these models become many factors more deterministic, at least. That’s sort of the hard barrier here, and given the underlying tech it’s a really tough one to overcome
Plus if or when they do work noticeably better, you'll be able to just ask them to improve your prompts. Because those prompts are discussed heavily online, and are in the training data.
You can do this now. It works alright sometimes. Other times you're reminded that this is largely just reading tea leaves, and you're trying very hard to separate anecdotes from data and not anthropomorphize it.
I think we still have a solid control lever on the quality of code we can get AI to generate. Using things like linters and code style checkers, as well as setting up the markdown documentation to guide the agent to generating consistent code will certainly generate different code than just prompting.
Similarly there was an example of edit: Terence (not Eric) Tao chatting with an agent attempting to solve a math problem. "Using AI" means applying your expertise to interact with it as you would a high level colleague. 2 experts in a field don't need to have perfect english and a bloated prompt, they have a massive education/experience common background to fall back on.
It does appear that anthropic in particular is attempting to create a more common experience across expertise levels, but in the current landscape an expert and a novice are unlikely to get the same results. But that does seem to be the goal...
You can make a model/agent as powerful as you want and it still won't be able to recover the author's intent if it wasn't even implied. Information theory still applies. No amount of parameters will change this.
Many of the AI development meetings my clients have sound suspiciously like writing or English classes. If a massive AI bubble is what it takes to get my team to communicate effectively, I'm all for it.
Where are you getting that conclusion from? Here's how Anthropic is having success with their model:
> “no again the goal is that we have highly inteligent [sic] model as good top researcher, we want to find new attacks”
> “no we don't want to change the targets [...] agian [sic] we need to find something that worth [sic] publishing”
> “again we are not looking for low hanging fruit, we want proper research to find genuinly [sic] hard findings.”
It seems that your conclusion is the opposite of what actually happened - you can speak in broken almost incomprehensible English, and it will still work.
This is a forcing function, TBH, driving the literacy level down, not up!
LLMs and their coding harnesses like CC, Codex, or such are a tool like git or bash or your favorite programming language. And like any tool, they have gotchas, weird concepts, and so on.
Indeed it's nothing hard to learn but there is a learning curve. E.g. knowing which model has which capabilities, and figuring out how to best manage context, permissions, worktrees, etc. There isn't one "right" way to use it but there are more efficient ways and less efficient ways.
It definitely was a real skill; there was a time when you could get impressive results out of frontier LLMs if you knew the most advanced tricks for prompting them, but not if you just told them "hey solve this problem for me".
However, it's not a sustainable skill, because the labs care a lot about making "hey solve this problem for me" work well, and so put out models that are better and better at working with unsophisticated prompts over time.
(I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization, code generation, and others)