A study by Anthropic demonstrates that Large Language Models (LLMs) can generate significantly better code when provided with clear and detailed prompts. While LLMs can produce functional code, incorporating prompt engineering techniques leads to substantial performance improvements.
Large language models (LLMs) can write better code if prompted correctly, although it requires some software development experience to do so effectively, which limits the utility of AI code help for novices. Woolf explained, 'If code can indeed be improved simply through iterative prompting such as asking the LLM to 'make the code better' — even though it’s very silly — it would be a massive productivity increase.
And if that’s the case, what happens if you iterate on the code too much?' This is what happened: Anthropic's Claude was tasked with writing Python code to find the difference between the smallest and the largest numbers whose digits sum up to 30, given a list of one million random integers between 1 and 100,000. And the LLM, which Woolf characterized as something a novice programmer might write, took an average 657 milliseconds to run on an Apple M3 Pro Macbook Pro. And when asked to 'write better code,' Claude responded with optimized code that performed 2.7x faster. Asked again, Claude made further improvements, returning code that incorporates multithreading for a 5.1x performance improvement over the initial implementation, though it was at the cost of creating errors that require fixing. Woolf then repeated the experiment using 'prompt engineering,' which simply means providing the LLM with more detail about what's expected and how to proceed. This was done in part by modifying the Claude system prompt, available via API as a way to set the rules for LLMs, to do things like use certain code efficiency strategies. 'Although it's both counterintuitive and unfun, a small amount of guidance asking the LLM specifically what you want, and even giving a few examples of what you want, will objectively improve the output of LLMs more than the effort needed to construct said prompts,' observes Woolf in his write-up. 'In all, asking an LLM to 'write code better' does indeed make the code better, depending on your definition of better,' Woolf conclude
Large Language Models Prompt Engineering Code Generation AI Software Development
United Kingdom Latest News, United Kingdom Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Prompt Engineering Boosts LLM Code GenerationLLMs can write better code with the right prompts, but novice programmers may struggle to leverage this effectively. An experiment using Anthropic's Claude LLM showed that iterative prompting and prompt engineering can significantly improve code performance.
Read more »
LLMjacking: The Rise of AI-Fueled Supply Chain AttacksThe threat of large-scale supply chain attacks using generative AI is growing as criminals increasingly exploit stolen credentials to access and manipulate existing LLMs. While full AI-driven attacks are still limited, LLMs are proving highly effective in social engineering campaigns, raising concerns about spear phishing attacks originating from LLMs as early as 2025.
Read more »
Personality testing reveals behavioral bias in LLMsMost major large language models (LLMs) can quickly tell when they are being given a personality test and will tweak their responses to provide more socially desirable results-;a finding with implications for any study using LLMs as a stand-in for humans.
Read more »
Can AI Replace Analyst Jobs?This article explores the potential of AI to disrupt the role of financial analysts. It argues that large language models (LLMs) can outperform analysts in tasks like interpreting financial statements and predicting earnings due to their ability to process vast amounts of data and identify patterns. The article cites a study where LLMs trained to estimate earnings variance outperformed human analysts, highlighting the potential for AI to automate tasks traditionally performed by analysts.
Read more »
Microsoft won't let customers opt out of passkey pushEnrollment invitations will continue until security improves
Read more »
New immunotherapeutic platform improves mRNA therapy efficacy against cancerTherapeutic mRNAs offer great potential as a versatile and precise tool against cancer and other diseases.
Read more »