A recent post from Wharton professor Ethan Mollick shared an impressive new study. In a nutshell, it found that BCG consultants who used GPT-4 were up to 43% more effective at tasks vs. employees who didn’t.
I like the thinking of using ChatGPT as a mental unblocker. Where’s writers block would exist “ I know where I want to go but not how to get there”, we can easily use different ChatGPT ideas to guide our thinking. Perhaps none will be used, but going there when I’m stuck is really useful.
Yes! One of the most useful things by far that I do with ChatGPT is solving the "blank canvas problem." When I don't know where to start, I just put my goal/idea into it and see what it comes back with. Usually there's some nugget that I can start working with at that point.
This reflects my usage quite well, and I like the intern/coach description better than the centaur/cyborg.
If you had asked me a year ago, "Do you want a tool that can carry about any menial task you want, and also help you think on a higher level about complex problems? It won't be perfect, but it will get a lot of things right." The answer would have been a clear yes. Most of the criticism I've seen has been centered around the fact that LLMs get things wrong sometimes. Of course they do, but they sure get enough right to amaze me on a daily basis.
I'm not a blind fan; AI is surely going to exacerbate inequality for some time to come. But the tool itself is absolutely impressive, and most certainly useful in a broad range of applications.
Thanks for confirming that you've adopted this strategy too. I'm always flipping between "this is obvious and everyone already knows it" and "am I doing something weird/unique here?"
I've said this before, but when people complain about ChatGPT not being useful/being too censored/getting dumber, I'm always curious to actually see their chat history. I suspect the usage patterns are really, really diverse.
> I'm always flipping between "this is obvious and everyone already knows it"
I've been making it a point to ask people outside of tech if they're starting to see any use of AI in their work, and in their life. There are a lot of people who still haven't heard of it, and the people who are seeing it in use have very little understanding of how it works.
Most people seem to think it's a tool for really quickly searching the internet. If they're aware it's not doing that, it's entirely a black box. I'm going to try to host a community event where people from all walks of life can share their experiences with AI, and their questions. Winter in a small fishing town is a great time for that kind of event!
Ethan's article (and the study's findings) was excellent, but this is a great framework to have as well!
I can see that I'm mostly using ChatGPT as a coach these days, because I guess I'm too much of a control freak to run with whatever it produces. Also, sometimes the idea of having to review and edit seems more daunting than simply writing things yourself.
It's technically nothing more than examples of his own Custom Instructions for ChatGPT (which have been around for months), but I found the way he frames and contextualizes them very helpful. And it appears getting ChatGPT to *know* you right off the bat is more helpful than you'd anticipate.
This is on my to-read list! I've been trying to figure out a good, opinionated way to use Custom Instructions, hopefully this can point me in the right direction.
> Sometimes the idea of having to review and edit seems more daunting than simply writing things yourself.
I suppose I'm lucky in that I (usually) find editing far less daunting than writing things myself. Shaping an existing ball of clay takes way less mental energy than creating the clay to begin with. AI's actually given me a bunch of shortcuts for solving this "blank canvas problem," including just rambling into a voice note and then piping it through Whisper (transcription) and GPT-4 (light editing).
That's exactly what Dan's article does: Give you a strong baseline for how to actually make Custom Instructions useful to you (wtih examples that make it very concrete). Definitely a worthwhile read.
As for the editing vs. writing: For me it's a curious dichotomy.
When I work with clients, I'll take an editing job over a copywriting job any day. I'm pretty good at reworking copy for readability, clarity, etc. on someone else's behalf.
But when it comes to my own writing, I get very particular and have a hard time letting go. Which is honestly more of a mental block on my end than a limitation of the current tech. I'm sure that if I gave it a chance, ChatGPT would speed up my writing process dramatically. I just need to get there.
The Interns and Coaches analogy is way more relatable. These consulting firms of course produce studies that benefit their expensive integrations of GPT-4. The Centaur and Cyborg analogy is truly dumb.
But whatever, having Interns and Coaches can make us more productive and more innovative? Could you found a better startup with these tools? If time on site is going down to your average user of ChatGPT, what could it mean about the actual value of the tool?
I think the idea that we should all become prompt-engineering natives is a bit ridiculous, what about you? Pi can overwhelm me with emoji but am I a better person or less lonely for it? I'd love to write a piece about the "Myth of the Inflection Point". If real utility had been achieved, ChatGPT's traffic metrics would be still going up by the millions.
What are we training GenZ and Alpha cohorts with these products? That augmenting ourselves with LLMs is desirable? We are making GenZ feel FOMO in relation to A.I. adoption as the emerge socially handicapped from mobile (and the pandemic) into the workforce.
The Jagged Frontier study is great but I (respectfully) disagree with your interpretation. The speed +25% | quality +40% | +12% completion gains were observed by the ~ half (385 out of 758 total participants) who were assigned so-called Inside Tasks. The -19% decrease was observed among the group assigned so-called Outside Tasks. Both experiments included control and GPT (with and without prompt training). Some think this weakens the study. I do not. But it's different than confusing the distribution plot that is circulating (which plots only the Inside half) with the overall result.
So the difference in results (+40% versus -19%) is primarily attributed not to complacency by the worse performers but simply that, per ex ante hypothesis and design apparently, they were given tasks designed to stump the AI.
I like the thinking of using ChatGPT as a mental unblocker. Where’s writers block would exist “ I know where I want to go but not how to get there”, we can easily use different ChatGPT ideas to guide our thinking. Perhaps none will be used, but going there when I’m stuck is really useful.
Yes! One of the most useful things by far that I do with ChatGPT is solving the "blank canvas problem." When I don't know where to start, I just put my goal/idea into it and see what it comes back with. Usually there's some nugget that I can start working with at that point.
This reflects my usage quite well, and I like the intern/coach description better than the centaur/cyborg.
If you had asked me a year ago, "Do you want a tool that can carry about any menial task you want, and also help you think on a higher level about complex problems? It won't be perfect, but it will get a lot of things right." The answer would have been a clear yes. Most of the criticism I've seen has been centered around the fact that LLMs get things wrong sometimes. Of course they do, but they sure get enough right to amaze me on a daily basis.
I'm not a blind fan; AI is surely going to exacerbate inequality for some time to come. But the tool itself is absolutely impressive, and most certainly useful in a broad range of applications.
Thanks for confirming that you've adopted this strategy too. I'm always flipping between "this is obvious and everyone already knows it" and "am I doing something weird/unique here?"
I've said this before, but when people complain about ChatGPT not being useful/being too censored/getting dumber, I'm always curious to actually see their chat history. I suspect the usage patterns are really, really diverse.
> I'm always flipping between "this is obvious and everyone already knows it"
I've been making it a point to ask people outside of tech if they're starting to see any use of AI in their work, and in their life. There are a lot of people who still haven't heard of it, and the people who are seeing it in use have very little understanding of how it works.
Most people seem to think it's a tool for really quickly searching the internet. If they're aware it's not doing that, it's entirely a black box. I'm going to try to host a community event where people from all walks of life can share their experiences with AI, and their questions. Winter in a small fishing town is a great time for that kind of event!
Ethan's article (and the study's findings) was excellent, but this is a great framework to have as well!
I can see that I'm mostly using ChatGPT as a coach these days, because I guess I'm too much of a control freak to run with whatever it produces. Also, sometimes the idea of having to review and edit seems more daunting than simply writing things yourself.
Since you're using ChatGPT in such an interactive manner, you may find value in this recent article from Dan Shipper: https://every.to/chain-of-thought/using-chatgpt-custom-instructions-for-fun-and-profit
It's technically nothing more than examples of his own Custom Instructions for ChatGPT (which have been around for months), but I found the way he frames and contextualizes them very helpful. And it appears getting ChatGPT to *know* you right off the bat is more helpful than you'd anticipate.
This is on my to-read list! I've been trying to figure out a good, opinionated way to use Custom Instructions, hopefully this can point me in the right direction.
> Sometimes the idea of having to review and edit seems more daunting than simply writing things yourself.
I suppose I'm lucky in that I (usually) find editing far less daunting than writing things myself. Shaping an existing ball of clay takes way less mental energy than creating the clay to begin with. AI's actually given me a bunch of shortcuts for solving this "blank canvas problem," including just rambling into a voice note and then piping it through Whisper (transcription) and GPT-4 (light editing).
That's exactly what Dan's article does: Give you a strong baseline for how to actually make Custom Instructions useful to you (wtih examples that make it very concrete). Definitely a worthwhile read.
As for the editing vs. writing: For me it's a curious dichotomy.
When I work with clients, I'll take an editing job over a copywriting job any day. I'm pretty good at reworking copy for readability, clarity, etc. on someone else's behalf.
But when it comes to my own writing, I get very particular and have a hard time letting go. Which is honestly more of a mental block on my end than a limitation of the current tech. I'm sure that if I gave it a chance, ChatGPT would speed up my writing process dramatically. I just need to get there.
The Interns and Coaches analogy is way more relatable. These consulting firms of course produce studies that benefit their expensive integrations of GPT-4. The Centaur and Cyborg analogy is truly dumb.
But whatever, having Interns and Coaches can make us more productive and more innovative? Could you found a better startup with these tools? If time on site is going down to your average user of ChatGPT, what could it mean about the actual value of the tool?
I think the idea that we should all become prompt-engineering natives is a bit ridiculous, what about you? Pi can overwhelm me with emoji but am I a better person or less lonely for it? I'd love to write a piece about the "Myth of the Inflection Point". If real utility had been achieved, ChatGPT's traffic metrics would be still going up by the millions.
What are we training GenZ and Alpha cohorts with these products? That augmenting ourselves with LLMs is desirable? We are making GenZ feel FOMO in relation to A.I. adoption as the emerge socially handicapped from mobile (and the pandemic) into the workforce.
The Jagged Frontier study is great but I (respectfully) disagree with your interpretation. The speed +25% | quality +40% | +12% completion gains were observed by the ~ half (385 out of 758 total participants) who were assigned so-called Inside Tasks. The -19% decrease was observed among the group assigned so-called Outside Tasks. Both experiments included control and GPT (with and without prompt training). Some think this weakens the study. I do not. But it's different than confusing the distribution plot that is circulating (which plots only the Inside half) with the overall result.
So the difference in results (+40% versus -19%) is primarily attributed not to complacency by the worse performers but simply that, per ex ante hypothesis and design apparently, they were given tasks designed to stump the AI.