A recent post from Wharton professor Ethan Mollick shared an impressive new study. In a nutshell, it found that BCG consultants who used GPT-4 were up to 43% more effective at tasks vs. employees who didn’t.
I like the thinking of using ChatGPT as a mental unblocker. Where’s writers block would exist “ I know where I want to go but not how to get there”, we can easily use different ChatGPT ideas to guide our thinking. Perhaps none will be used, but going there when I’m stuck is really useful.
This reflects my usage quite well, and I like the intern/coach description better than the centaur/cyborg.
If you had asked me a year ago, "Do you want a tool that can carry about any menial task you want, and also help you think on a higher level about complex problems? It won't be perfect, but it will get a lot of things right." The answer would have been a clear yes. Most of the criticism I've seen has been centered around the fact that LLMs get things wrong sometimes. Of course they do, but they sure get enough right to amaze me on a daily basis.
I'm not a blind fan; AI is surely going to exacerbate inequality for some time to come. But the tool itself is absolutely impressive, and most certainly useful in a broad range of applications.
Ethan's article (and the study's findings) was excellent, but this is a great framework to have as well!
I can see that I'm mostly using ChatGPT as a coach these days, because I guess I'm too much of a control freak to run with whatever it produces. Also, sometimes the idea of having to review and edit seems more daunting than simply writing things yourself.
It's technically nothing more than examples of his own Custom Instructions for ChatGPT (which have been around for months), but I found the way he frames and contextualizes them very helpful. And it appears getting ChatGPT to *know* you right off the bat is more helpful than you'd anticipate.
The Interns and Coaches analogy is way more relatable. These consulting firms of course produce studies that benefit their expensive integrations of GPT-4. The Centaur and Cyborg analogy is truly dumb.
But whatever, having Interns and Coaches can make us more productive and more innovative? Could you found a better startup with these tools? If time on site is going down to your average user of ChatGPT, what could it mean about the actual value of the tool?
I think the idea that we should all become prompt-engineering natives is a bit ridiculous, what about you? Pi can overwhelm me with emoji but am I a better person or less lonely for it? I'd love to write a piece about the "Myth of the Inflection Point". If real utility had been achieved, ChatGPT's traffic metrics would be still going up by the millions.
What are we training GenZ and Alpha cohorts with these products? That augmenting ourselves with LLMs is desirable? We are making GenZ feel FOMO in relation to A.I. adoption as the emerge socially handicapped from mobile (and the pandemic) into the workforce.
The Jagged Frontier study is great but I (respectfully) disagree with your interpretation. The speed +25% | quality +40% | +12% completion gains were observed by the ~ half (385 out of 758 total participants) who were assigned so-called Inside Tasks. The -19% decrease was observed among the group assigned so-called Outside Tasks. Both experiments included control and GPT (with and without prompt training). Some think this weakens the study. I do not. But it's different than confusing the distribution plot that is circulating (which plots only the Inside half) with the overall result.
So the difference in results (+40% versus -19%) is primarily attributed not to complacency by the worse performers but simply that, per ex ante hypothesis and design apparently, they were given tasks designed to stump the AI.
I like the thinking of using ChatGPT as a mental unblocker. Where’s writers block would exist “ I know where I want to go but not how to get there”, we can easily use different ChatGPT ideas to guide our thinking. Perhaps none will be used, but going there when I’m stuck is really useful.
This reflects my usage quite well, and I like the intern/coach description better than the centaur/cyborg.
If you had asked me a year ago, "Do you want a tool that can carry about any menial task you want, and also help you think on a higher level about complex problems? It won't be perfect, but it will get a lot of things right." The answer would have been a clear yes. Most of the criticism I've seen has been centered around the fact that LLMs get things wrong sometimes. Of course they do, but they sure get enough right to amaze me on a daily basis.
I'm not a blind fan; AI is surely going to exacerbate inequality for some time to come. But the tool itself is absolutely impressive, and most certainly useful in a broad range of applications.
Ethan's article (and the study's findings) was excellent, but this is a great framework to have as well!
I can see that I'm mostly using ChatGPT as a coach these days, because I guess I'm too much of a control freak to run with whatever it produces. Also, sometimes the idea of having to review and edit seems more daunting than simply writing things yourself.
Since you're using ChatGPT in such an interactive manner, you may find value in this recent article from Dan Shipper: https://every.to/chain-of-thought/using-chatgpt-custom-instructions-for-fun-and-profit
It's technically nothing more than examples of his own Custom Instructions for ChatGPT (which have been around for months), but I found the way he frames and contextualizes them very helpful. And it appears getting ChatGPT to *know* you right off the bat is more helpful than you'd anticipate.
The Interns and Coaches analogy is way more relatable. These consulting firms of course produce studies that benefit their expensive integrations of GPT-4. The Centaur and Cyborg analogy is truly dumb.
But whatever, having Interns and Coaches can make us more productive and more innovative? Could you found a better startup with these tools? If time on site is going down to your average user of ChatGPT, what could it mean about the actual value of the tool?
I think the idea that we should all become prompt-engineering natives is a bit ridiculous, what about you? Pi can overwhelm me with emoji but am I a better person or less lonely for it? I'd love to write a piece about the "Myth of the Inflection Point". If real utility had been achieved, ChatGPT's traffic metrics would be still going up by the millions.
What are we training GenZ and Alpha cohorts with these products? That augmenting ourselves with LLMs is desirable? We are making GenZ feel FOMO in relation to A.I. adoption as the emerge socially handicapped from mobile (and the pandemic) into the workforce.
The Jagged Frontier study is great but I (respectfully) disagree with your interpretation. The speed +25% | quality +40% | +12% completion gains were observed by the ~ half (385 out of 758 total participants) who were assigned so-called Inside Tasks. The -19% decrease was observed among the group assigned so-called Outside Tasks. Both experiments included control and GPT (with and without prompt training). Some think this weakens the study. I do not. But it's different than confusing the distribution plot that is circulating (which plots only the Inside half) with the overall result.
So the difference in results (+40% versus -19%) is primarily attributed not to complacency by the worse performers but simply that, per ex ante hypothesis and design apparently, they were given tasks designed to stump the AI.