Generative AI and productivity: 34% gain for novice agents

Controlled experiments on generative AI and productivity reach a consistent conclusion: AI speeds up tasks that sit inside what it does well, the gain is larger for less experienced workers and results get worse when the task falls outside that frontier. In the most cited study, with 5,179 support agents at a software company, the AI assistant raised issues resolved per hour by 14% on average, and novice and lower-performing agents resolved 34% more issues per hour, while the most experienced barely gained (Brynjolfsson, Li and Raymond, NBER, 2023). With 758 BCG consultants, those using AI completed 12.2% more tasks and worked 25.1% faster on tasks inside the frontier, and on a task chosen to sit outside it they were 19% less likely to reach the correct answer (Dell’Acqua and colleagues, 2026). For commercial leadership, the practical reading comes down to three decisions: where to use it, who reviews and how to train the team to recognize the line.

In leadership classes, I see these numbers travel without the context that explains them. The “55% faster” reaches the board meeting without the detail that the sample was 95 freelance programmers doing a single standardized task. In this article I go back to the original papers, show what each experiment measured and what it left out, and translate the results for sales, marketing and customer support. How to measure the return of an AI project in your company is the subject of AI ROI; here the focus is on what research has already shown and what it has not.

What did the experiments on generative AI and productivity measure?

Most numbers about AI at work come from opinion surveys, in which people say how much time they think they saved. The studies on this list randomize who gets the tool, or use a staggered rollout, and compare the measured performance of those who used it with those who did not. That design supports causal claims, within the limits of each study’s task and sample. I also included a Danish study with administrative data, which asks whether the task-level gain shows up in earnings and hours worked.

Source	Year	Sample and method	Main finding
Brynjolfsson, Li and Raymond, NBER; published version in the Quarterly Journal of Economics	2023 and 2025	5,179 support agents (5,172 in the published version), staggered rollout of an AI assistant, 3 million conversations	Issues resolved per hour up 14% (15% in the published version); 34% among the least skilled and least experienced
Noy and Zhang, Science	2023	453 college-educated professionals, preregistered online experiment, writing tasks from their own occupation	Time down 40% and quality up 18%; the gap between workers narrows
Dell’Acqua and colleagues, Harvard Business School and BCG; published version in Organization Science	2023 and 2026	758 consultants, preregistered experiment with three groups (no AI, GPT-4, GPT-4 plus a prompting overview)	Inside the frontier, 12.2% more tasks and 25.1% faster; outside it, 19% less likely to be correct
Peng and colleagues, Microsoft Research, GitHub and MIT	2023	95 freelance programmers, randomized to use or not use a coding assistant	The AI group finished the task 55.8% faster
Becker and colleagues, METR	2025	16 experienced developers, 246 real tasks in open-source projects they had known for five years on average	With AI, tasks took 19% longer, against participants’ expectations
Humlum and Vestergaard, NBER	2025, revised 2026	Surveys of 25,000 workers at 7,000 workplaces in Denmark, linked to administrative records	Null effect on earnings and recorded hours two years after ChatGPT’s launch, ruling out effects above 2%

Customer support: what did the Brynjolfsson, Li and Raymond study measure?

The authors followed chat support agents at a Fortune 500 company that sells business process software. The tool read the conversation in real time and suggested responses, and the agent stayed responsible for the conversation, free to ignore the suggestion. Because the company rolled out the assistant to different groups on different dates, the researchers compared agents who already had the tool with those who did not yet. The main measure was issues resolved per hour, which combines handle time, simultaneous chats and resolution rate.

The 14% average hides the distribution that matters most. Agents in the bottom fifth of pre-AI performance resolved 34% more issues per hour, while those in the top fifth gained no productivity and saw a small, statistically significant drop in resolution rate and customer satisfaction. Agents with two months of tenure and access to the assistant performed as well as colleagues without AI who had more than six months. Customer requests to speak to a supervisor fell by almost 25%, and turnover among newer agents went down. During periods when the system was offline, agents who had used the assistant longer stayed above their own baseline, a sign that the tool spread the practices of the best agents to the newer ones. The version published in 2025, with 5,172 agents, reports a 15% average gain and describes, for the most experienced, small gains in speed with small declines in quality.

Writing tasks: what did Noy and Zhang find in Science?

Shakked Noy and Whitney Zhang, from MIT, gave marketers, grant writers, consultants, data analysts, HR professionals and managers two writing tasks typical of their occupation, such as a press release, a short report and a delicate email. Each task took 20 to 30 minutes, with a bonus for quality and blind grading by experienced professionals from the same field, and half of the participants got access to ChatGPT before the second task. Average time fell 40%, quality rose 18%, and those who did worst on the first task gained the most.

One detail from the preliminary version released by the authors matters for management. In it, 68% of participants with access to the tool said they submitted ChatGPT’s first output without editing, and the final texts did not score higher than the tool’s raw output. The authors list as a central limitation that the tasks were short, self-contained and lacking the context-specific knowledge that real work requires. For a commercial operation, the question becomes how much of your team’s work looks like a 25-minute email with no customer history.

BCG consultants: what is the jagged technological frontier?

The study by Fabrizio Dell’Acqua and colleagues, run with Boston Consulting Group and circulated as a Harvard Business School working paper in 2023, coined the term “jagged technological frontier”. AI improves performance on some tasks and worsens it on others of similar difficulty, within the same workflow, and an outside observer cannot predict which side each task falls on. On the 18 tasks inside the frontier, from product ideation to analysis, those using AI completed 12.2% more tasks, worked 25.1% faster and delivered higher quality. On the task outside the frontier, in which consultants had to cross performance spreadsheets with executive interviews to find the root cause of a company’s problem, those using AI were 19% less likely to reach the correct solution.

BCG’s publication on the experiment adds details I bring to the classroom. On the problem-solving task, the performance of those using GPT-4 was 23% worse, because many participants accepted the tool’s wrong answer. On the creative task, participants with lower baseline proficiency improved 43%, and those with the highest proficiency gained 17% there and lost 17% on the problem-solving task. The group that received about 30 minutes of guidance on GPT-4 use and its limits did worse on average on the problem-solving task than those who did not, and the authors suspect overconfidence. The more uniform output of AI also reduced the group’s diversity of ideas by 41%.

Programming: does the coding assistant speed people up or slow them down?

In the experiment by Sida Peng and colleagues, 95 programmers hired on a freelancing platform had to write an HTTP server in JavaScript as fast as possible. The 45 randomized to use the coding assistant finished in 71 minutes on average, against 161 minutes for the 50 in the control group, a 55.8% reduction with a confidence interval of 21% to 89%. Programmers with less experience tended to gain more, with a coefficient at the edge of conventional significance.

The 2025 METR study measured a different situation: 16 experienced developers worked on 246 real tasks in mature open-source projects, and each task was randomized to allow or forbid AI tools. Before starting, they expected a 24% reduction in time; afterward, they estimated a 20% reduction; measured time went up 19%. In February 2026, METR published an update with 57 developers and more than 800 tasks, in which the estimate pointed to time savings, with a confidence interval that includes zero, and the institute called the signal unreliable because many developers refused to work without AI. From this pair of studies, I take to the commercial operation the gap between what people feel and what the clock measures.

Who gains most and where does AI make results worse?

The pattern repeats in four of the five task studies: the gain concentrates among those starting out or below average, and the loss shows up among the most experienced or on tasks that require context and judgment.

Study	Task	Who gained most	Where AI hurt or did not help
Brynjolfsson, Li and Raymond	Chat support for software	Novice and lower-performing agents: +34% issues resolved per hour	Top-performing agents: no gain and a small drop in resolution and satisfaction
Noy and Zhang	20- to 30-minute texts from their own occupation	Participants with the lowest grade on the first task	Long tasks or tasks with company context were outside the study
Dell’Acqua and colleagues	Product ideation and business problem solving	Consultants with lower baseline proficiency: +43% on the creative task	Problem solving outside the frontier: 19% less likely to be correct
Peng and colleagues	HTTP server in JavaScript, standardized task	Less experienced programmers, with weak evidence	No statistical difference in completion rate
Becker and colleagues (METR)	Real tasks in projects the developer knows well	No group in the 2025 study	Experienced developers took 19% longer

What these data do not say

The studies measure tasks, and your company hires people who do dozens of tasks a week. A 40% gain on a 25-minute email does not turn into 40% more team capacity, because a salesperson also runs meetings, negotiates and updates the CRM. In the Danish study by Humlum and Vestergaard, across 11 AI-exposed occupations, including customer support and marketing, the authors found no effect on earnings or recorded hours through the end of 2024, and 85% of users said they reallocated the time saved to other tasks. The effect on jobs is the subject of AI and jobs.

The samples have specific profiles. The programmers in the 2023 study were freelancers, mostly aged 25 to 34 and from India and Pakistan, and the authors work for the company that develops the tool tested. BCG consultants are a selected population, and the support agents worked at a single software vendor, with a tool trained on the conversations of that company’s best agents. None of the studies measured B2B salespeople in negotiation, and any number about “salesperson productivity with AI” needs the same check on sample and method.

The tools have also changed. The 2023 experiments used models that are now outdated, and the frontier has moved, as the METR update suggests. The lesson of the BCG study still holds with better models: the line is jagged and invisible, and it shifts. Finally, almost all gains were measured in speed and in short-term grader scores, and effects on learning, diversity of approaches and company revenue remain understudied.

What do these studies mean for sales, marketing and customer support?

Sales: shorter ramp-up and risk in discovery

The most useful finding for sales comes from customer support: two months with AI matched six months of experience. For a manager hiring salespeople, this points to sales onboarding, where the assistant can carry the best reps’ practices, as long as someone has written them down in the playbook and the CRM. The warning comes from the BCG study: the task outside the frontier, crossing numbers with interviews to find the cause of a problem, resembles discovery in consultative selling. A junior salesperson who asks AI to read the customer’s pain may get a convincing and wrong answer, and has less experience to notice.

Marketing: production speed and loss of variety

The writing and product ideation studies describe content work well, and the time gain on short pieces has experimental support. The 41% reduction in diversity of ideas measured by BCG calls for attention from whoever handles positioning and segment campaigns, because a whole team asking the same model for the same task produces pieces that look alike, and alike to those of a competitor using the same tool. I recommend keeping the idea and angle stage with the team and using AI for production and variations.

Customer support: the strongest evidence, with attention to top performers

Customer support has the strongest evidence on the list, with a large sample, a real setting and measures of both productivity and quality. The same study shows that the assistant can get in the way of the most skilled agents, and leadership needs to track resolution and satisfaction for that group separately from the average. The uses that already work in triage and response are in AI in customer service.

For HR and L&D: the training these studies call for

In the BCG study, 30 minutes of prompting guidance did not protect consultants on the task outside the frontier and, on average, made results worse. The training the data suggests teaches people to recognize the frontier in their own area’s tasks, to check the answer against the source and to decide when not to use the tool. The Danish study points the same way from the adoption side: in workplaces that combined encouragement, enterprise tools and training, 93% of workers had used chatbots at work, 28% used them daily and 19% reported saving more than one hour a day. Give priority to newer hires, who gain the most and have less experience to spot errors.

What leadership does with these numbers

1. Map tasks, not job titles. List each team’s recurring tasks and mark which look like the ones the studies measured, such as short text and first drafts, and which require diagnosis with data and account context.
2. Test the frontier before rolling out. Use the routine in the box above for each candidate task, with real cases from your operation, and record the result along with the model used.
3. Set review by experience level. Novices gain more and make errors without noticing; experienced people gain less and can lose quality by following suggestions. Write down who reviews each profile, with what sampling, and put it in your AI use policy.
4. Track quality by group. Separate the resolution, conversion or satisfaction indicators of your most experienced people, the group in which AI lowered quality in customer support.
5. Measure time with system data. METR’s developers believed in a 20% gain while losing 19%; compare before and after with the dates in the CRM and the help desk.
6. Decide where the freed time goes. Without a decision, the time dissolves into other tasks, as the Danish study recorded; decide whether it becomes more accounts per person, more customer meetings or a postponed hire.

The experiments show real and uneven gains, and the teams that capture them are the ones that know where the tool gets it right and where it is confidently wrong. That is the work of the in-company applied AI training, built on the company’s own processes and data, with leadership and operations groups from sales, marketing and customer support. It delivers a prompt library by area, tested against each task’s frontier, and an agent architecture covering what each agent reads, what it does on its own and who approves the rest.

Frequently asked questions

How much does generative AI increase productivity?

In the most cited controlled experiments, the gain ranged from 14% in issues resolved per hour in customer support to a 55.8% time reduction on a standardized programming task, with 40% less time on writing tasks and 25.1% faster work on consulting tasks. These numbers apply to specific tasks, and a study of experienced developers on real projects measured the opposite effect, with 19% more time.

Who gains the most productivity from AI at work?

In four of the five task experiments, the least experienced or lowest-performing professionals gained the most. In customer support, novice and lower-performing agents resolved 34% more issues per hour; among BCG consultants, those with the lowest baseline proficiency improved 43% on the creative task. The most experienced gained little and, in some cases, lost quality.

What is the jagged technological frontier?

It is the term coined by Harvard and BCG researchers to describe how AI improves performance on some tasks and worsens it on others of similar difficulty, within the same job. In the experiment, consultants who used AI on a problem-solving task outside that frontier were 19% less likely to reach the correct answer.

Do these studies apply to B2B sales teams?

None of the experiments measured B2B salespeople in negotiation, so extrapolation calls for care. The support and writing studies are close to presales, follow-up and content tasks, and the BCG study warns about discovery, which is at the center of consultative selling. Test your own task with real cases before assuming a gain measured in another context.

How do I prepare my team to use AI without making results worse?

Train the team to recognize the frontier in their own tasks, check answers against the source and decide when not to use the tool, in addition to writing good requests. Prioritize newer professionals and define who reviews each profile’s work. In the BCG experiment, a short prompting overview did not protect consultants and made results worse on the problem-solving task.

Why doesn’t the experimental gain show up in company results?

An experiment measures an isolated task, and saved time only becomes a result when leadership decides how to use it. In a study of 25,000 workers in Denmark, the authors found no effect on earnings or hours two years after ChatGPT, and 85% of users said they reallocated the time to other tasks.