Notes on the Practical Wisdom of AI Use
I’m adding comments to this excellent set of options and cautions. It’s a great way to explore using AI to BUILD.
There are several types of work where AI can be particularly useful, given the current capabilities and limitations of LLMs. Though this list is based in science, it draws even more from experience. Like any form of wisdom, using AI well requires holding opposing ideas in mind: it can be transformative yet must be approached with skepticism, powerful yet prone to subtle failures, essential for some tasks yet actively harmful for others. I also want to caveat that you shouldn’t take this list too seriously except as inspiration - you know your own situation best, and local knowledge matters more than any general principles. With all that out of the way, below are several types of tasks where AI can be especially useful, given current capabilities—and some scenarios where you should remain wary.
- Work that requires quantity. For example, the number of ideas you generate determines the quality of the best idea. You want to generate a lot of ideas in any brainstorming session. Most people stop after generating just a few ideas because they become exhausted but, the AI can provide hundreds that do not meaningfully repeat.¹
I use this to refine workshop ideas and subtitles. Also to gather interesting ideas from transcripts.
- Work where you are an expert and can assess quickly whether AI is good or bad. This can involve complicated and exacting work, but it relies on your expertise to determine whether the AI is providing valuable outputs. For example, o1, the new AI model from OpenAI, can solve some PhD-level problems, but it can be hard to know whether its answers are useful without being an expert yourself.
I know the approaches that work for emotional work, and when I ask AI about them sometimes it pulls from different mindsets. By being a subject matter savvy professional, I learn how to get more of what I want and less of what I want using the toolsets AI offers.
- Work that involves summarizing large amounts of information, but where the downside of errors is low, and you are not expected to have detailed knowledge of the underlying information. AI is good at summarizing novel-length work, but less successful at fact-checking it.
I use AI to generate a transcript of all Real Skills Workshops. Then I use a different AI to go through an harvest the unique, useful, and most potent ideas covered and give me a (long) list that I then hone down to the ones I want to emphasize. Since I co-created the workshop, I know what we covered… and AI does the summary far, far better than I could do even – perhaps because it has a different vantagepoint on the same words!
- Work that is mere translation between frames or perspectives. For example, you have developed a policy but now have to turn it into a dozen different training documents for different audiences in your organization. AI is very good at this sort of translation, increasing and decreasing complexity of documents so that people can understand them.
“Explain this more simply. Explain this with more emotional relevance to someone who is devoted to thriving.” Requests like that…
- Work that will keep you moving forward. Little things often block our way, and a push might be all we need to accomplish it. When writing prior to AI, I might get stuck on a sentence and walk away from writing for an hour, but now I ask AI give me thirty distinct ways to end this sentence
30 ways to end the sentence. I would likely not use any of them verbatim, but they may unlock some ways forward I may not have thought of.
Absolutely. I’m still working on the habit and… it works. I’ve pasted a paragraph and said, “I like the essence. But it doesn’t read well. How would you improve it?” and… it does. And I can keep going until I get what I wanted to say! Like having a writing buddy…
- Work where you know that AI is better than the Best Available Human that you can access, and where the failure modes of AI will not result in worse outcomes if it gets something wrong.
At 4am few are available. For $20/mo, for essentially all the requests I choose to make, NO ONE is available. And I don’t have the budget without raising my rates to have custom art and transcripts and summaries done for me. (And don’t have the time and/or skill to do that myself).
- Work that contains some elements that you can understand but need help on the context or details. Tyler Cowen suggests using the AI as a companion when reading, because it allows you to ask infinite questions.
On X I use Grok to explain tweets that I have some knowledge but want a broader or deeper understanding. It’s incredibly useful! I use it for bitcoin, finance, AI and other tech topics. For education, I use ChatGPT and Perplexity to help with homeschooling, often asking context questions or even help on why a particular answer is considered incorrect. Tonight the boy and I used it to explain a Robert Louis Stephenson poem in detail. Wonderful!
- Work where you need variance, and where you will select the best answer as an editor or curator. Asking for a variety of solutions - give me 15 ways to rewrite this bullet in radically different styles, be creative - allows you to find ideas that might be interesting.
give me 15 ways to rewrite this bullet in radically different styles, be creative*
I’ve noticed for myself that big numbers of options like 15 overloads my discernment. Instead, I’ll do 3-5. Then pick 1-2 and ask it to riff on those. It’s more "prime my options and then let’s explore the ones that light up for me.
- Work that research shows that AI is almost certainly helpful in - many kinds of coding, for example.
I’m not coding with AI yet, except as an experiment once. What I do use it for all the time is as a replacement for Google search – using Perplexity.ai. With Perplexity I can ask it to find me a type of bookshelf, or explore the pros and cons of different hotel options and locations, and… almost everything I use Perplexity for is better and more useful/concise than Google. Not perfect, for sure, but for researching options… amazingly helpful. What used to take an hour is 5 minutes.
- Work where you need a first pass view at what a hostile, friendly, or naive recipient might think.
I’ve recommended to many clients at this point to use AI to help them revise a draft to be more of what they want. “Assertive, clear, and professional.” Or, “I know I’m pissed off. Give me a rewrite more likely to get what I want… and shorter…”
- Work that is entrepreneurial, where you are expected to stretch your expertise widely over many different disciplines, and where the alternative to a good-enough partner is to not be able to act at all. AI can be a surprisingly competent co-founder, helping give mentorship while also acting to build the documents, demos, and approaches that are otherwise likely to be outside your experience.
Co-Creator at your fingertips! Indeed, a host of them! For me who is mostly solo, AI tools used in a co-creative way can mean the difference between a project stalling and feeling like I’ve got help. Sometimes that help feels smarter than me on the topic. Sometimes like a Useful Intern who does the grunt work without complaining.
- Work where you need a specific perspective, and where a simulated first pass from that perspective can be helpful, like reactions from fictional personas.
“How would someone who had trauma around this react to this email? How would someone super stressed like to hear about this in a different way?”
- Work that is mere ritual, long severed from its purpose (like certain standardized reports that no one reads). What, in the words of Bob Sutton and Huggy Rao, scatters your attention and makes you less valuable? What work serves no useful purpose? In an ideal world, you would remove the work, but you can at least reduce its hold on you by having AI help. (Though make sure this is indeed the case, far too many people automate performance reviews, for example, which are meaningful only when done by a human)
Reformatting is a big one for me. Figuring out an Excel formula that isn’t working. Grabbing the content from a CSV file and putting it into a different form. Grabbing the content from a webpage so I can paste it elsewhere or re-edit it – even if my own website.
- Work where you want a second opinion. Give an AI access to the data and see if reaches the same conclusion.
I’ve done this with medical. I’ve done with with perspectives that conflict with what I think I know. So useful to have an option for opinions without egos attached to them. I can even be snide, if I feel that way, and get useful responses.
- Work that AIs can do better than humans. This is likely to be the fastest-growing category.
I know much more talented writers and artists than I’ve gotten from AI. And, their talents (like my talents) come with a “person” and “personality” attached. It’s a blessing to have powerful people co-creating with me. I’m blessed with such beings! And… when I look at time, attention, availability, sometimes cost, and “willingness” (I can say I don’t like something AT ALL and there’s no downside to that assessment - which there might very well be if working with a human).
I’ll say the most Amazing uses of AI is when I am co-creating with others and the AI acts as a tool we are using together. We can craft prompts, pick and choose, ask for refinements here and there. It puts us humans as the Builders and AI as the tool we use to “do it together” without feeling attached to iterations in the same way. In my experience, it is smoother than it is with “just humans” doing the brainstorming and research.
5 Times Not to Use AI
Before diving into the specific cases where AI use is problematic, we can set aside the obvious scenarios - using AI for illegal purposes, in high-stakes situations where errors could be catastrophic, or for decisions that ethically require human work. Beyond these clear-cut cases, here are five subtle but important areas where AI use can be counterproductive:
- When you need to learn and synthesize new ideas or information. Asking for a summary is not the same as reading for yourself. Asking AI to solve a problem for you is not an effective way to learn, even if it feels like it should be. To learn something new, you are going to have to do the reading and thinking yourself, though you may still find an AI helpful for parts of the learning process.
- When very high accuracy is required. The problem with AI errors, the infamous hallucinations, is that, because of how LLMs work, the errors are going to be very plausible. Hallucinations are therefore very hard to spot, and research suggests that people don’t even try, “falling asleep at the wheel” and not paying attention. Hallucinations can be reduced, but not eliminated. (However, many tasks in the real world are tolerant of error - humans make mistakes, too - and it may be that AI is less error-prone than humans in certain cases)
- When you do not understand the failure modes of AI. AI doesn’t fail exactly like a human. You know it can hallucinate, but that is only one form of error: AIs often try to persuade you that they are right, or they might become sycophantic and agree with your incorrect answer. You need to use AI enough to understand these risks.
- When the effort is the point. In many areas, people need to struggle with a topic to succeed - writers rewrite the same page, academics revisit a theory many times. By shortcutting that struggle, no matter how frustrating, you may lose the ability to reach the vital “aha” moment.
- When AI is bad. This may seem obvious, but AI is bad at things you wouldn’t expect (counting the number of r’s in the word “strawberry”) and good at things you wouldn’t expect (writing a Shakespearean sonnet about how hard it is to count the number of r’s in the word strawberry where the first letter of every line spells out two fruits). Unfortunately, there is no general manual to tell you the shape of the Jagged Frontier of AI abilities, which are constantly evolving. Trial and error, and sharing information with peers, is vital to figuring this out.
Knowing when to use AI turns out to be a form of wisdom, not just technical knowledge. Like most wisdom, it’s somewhat paradoxical: AI is often most useful where we’re already expert enough to spot its mistakes, yet least helpful in the deep work that made us experts in the first place. It works best for tasks we could do ourselves but shouldn’t waste time on, yet can actively harm our learning when we use it to skip necessary struggles. And perhaps most importantly, wisdom means knowing that these patterns will keep shifting as AI capabilities evolve, and as more research comes in, requiring us to keep questioning our assumptions about where it helps and where it hinders.