AI and the workplace
How employees and CEOs alike can plan for the future.
As a CEO or business leader in 2024, it's difficult not to feel anxiety about artificial intelligence. Employees are experimenting with powerful new tools like ChatGPT, while companies like Microsoft and Google are racing to integrate AI into their core products. The pressure to develop an "AI strategy" is intense, even if no one fully knows what that entails. And there’s maybe even a nagging question of “is this AI thing even real, or just a passing trend?”
Yet one thing that's crystal clear - at least to me - is that AI already has a measurable impact on work and productivity. Midway through last year, Asana published a "State of AI at Work" report based on surveys of over 4,000 knowledge workers. 25-30% of respondents used AI for administrative tasks or data analysis - but 50-60% of employees were eager to give it a try. And those numbers have only grown since.
Today, I want to look at some studies and some anecdotal evidence of how much of an impact AI is having on workers, and how we can all try and keep up with the ever-advancing state of technology.
AI and employees
Over the past twelve months, several studies have corroborated the idea that AI is making workers more productive.
Last September, Boston Consulting Group and Harvard Business School made a big splash with their paper measuring GPT-4's impact on consultants. They found that consultants using AI "finished 12.2% more tasks on average, completed tasks 25.1% more quickly, and produced 40% higher quality results than those without." That's a pretty big impact, though, of course, it came with some caveats: when workers became too reliant on LLMs, they stopped checking the outputs and made more mistakes than they would have otherwise.
We've seen similar results from Microsoft: a November survey of early Copilot users found that 70% felt more productive, with participants being 29% faster when measured across searching, writing, and summarizing. Interestingly, these numbers are consistent with a 2022 study of GitHub Copilot users; the study concerned software developers using a code-completion model and was done long before the release of ChatGPT. In that case, 88% of developers reported feeling more productive, with those using AI being 55% faster.
Of course, it's not all upside. There are clear indicators that despite the productivity gains, AI still stands to create a significant amount of job displacement. A study from OpenAI and the University of Pennsylvania found up to 80% of U.S. employees could have their work affected by AI, whether through new products, new workflows, or new job displacement risks.
This isn't theoretical - it's happening today. Last year, language-learning app Duolingo steadily fired contract writers and translators, replacing them with AI. The CEO of IBM announced that the company plans to replace nearly 8,000 jobs with AI over the coming years. And Tyler Perry recently announced he's pausing an $800 million studio expansion - thanks to the advances he's seeing in generative AI.
From the employee perspective, AI is a double-edged sword. But how are CEOs approaching the technology?
AI and CEOs
One thing that influenced my thinking about this piece was talking to several readers who run their own businesses.
Katrina McKinnon, owner of the copywriting firm Copysmiths, is one example. Despite the existential threat AI poses to her industry, she's trying to find ways to augment her team's work with the technology. "As content marketers, we've fiddled around with loads of generative AI tools in the writing, image creation and data analysis space," she said. "None yet deliver on our wildest fantasies." Despite being up to date on the latest tools and technologies, McKinnon still finds the current state of AI underwhelming.
To be fair, it's hard to be an early adopter here. CEOs are under pressure to act, but moving too hastily risks embarrassing gaffes and wasted effort. Carla Sanchez Silva, founder of marketing firm Proyecto Colectivo, mentioned a number of those fears as well. She was worried about issues like "AI making stuff up, the idea of OpenAI peeking at company data for their training, and the whole mess around who owns the rights to AI-created images." Taken together, there are significant challenges that CEOs need to account for before diving headfirst into new AI tools.
While small businesses may be able to play fast and loose, larger organizations need to worry about data privacy rules and laws - what data employees should (or are even legally allowed to) share with third-party AI vendors. There are also plenty of open questions when it comes to intellectual property an AI. It's still unclear who owns the copyright on the generated images, or whether images generated by AI can be found to violate copyright. And of course, hallucinations are still a major issue and potential source of liability.
At the same time, leaders can't afford to sit on the sidelines. Employees need clear guidelines to understand AI’s role in their jobs, ideally with training to harness AI’s potential capabilities fully. Many employees are currently in the dark about how they should be using AI at work or are being told not to use AI at all - which is likely a mistake.
, a fellow Substack author and gym owner, has jumped in headfirst with AI and has been enjoying the results. “While AI hasn't exactly revolutionized our business, I've managed to leverage the incredible tools for research. I've been able to craft quick templates for business proposals and agreements, both of which I'm comfortable collaborating with an AI on because I have a lot of experience with both.”Yet, whether you're a CEO or an employee, riding this AI wave might seem like a daunting challenge. But there are a few things that you can do to avoid being pulled under.
AI and you
First, learn the limitations of the current technology. If you can, get comfortable with at least one LLM (large language model), like ChatGPT, Gemini, or Claude. Ideally, use the most advanced version because of how unpredictable their limitations are. Wharton professor Ethan Mollick (co-author on that BCG/HBS study) has called this "The Jagged Frontier":
AI is weird. No one actually knows the full range of capabilities of the most advanced Large Language Models, like GPT-4. No one really knows the best ways to use them, or the conditions under which they fail. There is no instruction manual. On some tasks AI is immensely powerful, and on others it fails completely or subtly. And, unless you use AI a lot, you won’t know which is which.
With that in mind, it also helps to encourage AI experimentation within your company. Do this as safely as possible: if privacy or other regulatory hurdles make this too risky, then avoid it. That said, getting the most value from AI is a deeply individualistic process - each of us has different strengths and weaknesses, meaning we're going to benefit from AI assistance in different ways.
Second, consider AI a tool to augment your work and workforce, not replace it. It's a mistake to think of AI assistants as experts. Instead, I suggest thinking of tools like ChatGPT and its ilk as interns - interns who are available 24/7 and are always ready for another task. That means asking for narrow tasks without context, checking for hallucinations or other errors, and assuming I’ll have to edit the final version regardless of the AI's output.
Sánchez Silva echoed that idea: “I'm pretty cautious about the whole AI thing, especially when it comes to bias. That's why I often double-check everything with different tools at different times. It might sound like a hassle and kind of beats the point of using AI in the first place, but in the end, I think it's worth the extra effort to get reliable results.”
Lastly, it's important to understand how early we still are. The current capabilities of AI chatbots are akin to an iPhone 3G: it's clearly going to be impactful, but there's a long way to go before it's a mature, ubiquitous technology. In the case of the iPhone, early adopters were able to capture low-hanging fruit (remember dedicated flashlight and calculator apps?) but truly mobile-native businesses such as Instagram and Uber took nearly a decade to materialize.
I've sometimes found it hard to keep up with the constant stream of AI products, services, and content. But there was an underlying idea that helped motivate me:
If you can put the reps in now, you will be way ahead of most people (and if you're not in tech, most of your coworkers). It's easy to forget how early we still are. If you've played with ChatGPT at all in the past year, you're in the top 10% of AI early adopters. If you know there's a difference between GPT-3.5 and GPT-4, you're in the top 1%.
And as AI inevitably becomes more ubiquitous - as it makes its way into our phones, our apps, our smart devices, our workplaces, and our lives - having that experience now is (probably) going to pay big dividends in the near future.
Good stuff, Charlie.
One huge question for me is how much these tools will affect global productivity once they're rolled out. I've been having some interesting conversations with some of my readers along these lines, and it seems like a real challenge to come up with a credible answer. To me, it seems like most people who are using LLMs and generative AI at work (almost always LLMs from what I have heard and seen) are the ones who would benefit the most from using it, so causation =/= correlation there.
I do get a sense that something akin to Sal Khan's AI tutor in every kid's hands would be dramatic and revolutionary, although those are productivity gains with a much longer tail. Any particular thoughts along these lines?
I know this is an enormous question, so feel free to do what Daniel Nest and I would do at this juncture and make some dad jokes!