I think this study does not say what most people are taking it to say.
> our research questions - when and how knowledge workers perceive the enaction of critical thinking when using GenAI (RQ1), and when and why do knowledge workers perceive increased/decreased effort for critical thinking due to GenAI (RQ2)
This is about the application of critical thinking to AI outputs and in AI workflows. Are people cognitively lazy when some other entity hands them plausible sounding answers?
The answer of course is yes. If some entity gives you a good enough result, probably you aren’t going to spend much time improving it unless there is a good reason to do so. Likewise you probably aren’t going to spend a lot of time researching something that AI tells you if it sounds plausible. This is certainly a weakness, but it’s a general weakness in human cognition, and has little to do with AI in and of itself.
In my reading, what this study does not say, and does not set out to answer, is whether or not the use of AI makes people generally less able or likely to engage in critical thinking as a result of use of AI.
On your last point I tend to think it will. Tools replaced our ancestor's ability to make things by hand. Transportation / elevators reduced the average fitness level to walk long distances or climb stairs. Pocket calculators made the general population less able to do complex math. Spelling/grammar checks have reduced knowing how to spell or form complete proper sentences. Keyboards and email are making handwriting a passing skill. Video is reducing our need / desire to read or absorb long form content.
The highest percentage of humans will take the easiest path provided. And while most of the above we just consider improvements to daily life, efficiencies, it has also fundamentally changed on average what we are capable of and what skills we learn (especially during formative years). If I dropped most of us here into a pre-technology wilderness we'd be dead in short order.
However, most of the above, it can be argued, are just tools that don't impact our actual thought processes; thinking remained our skill. Now the tools are starting to "think", or at least appear like they do on a level indistinguishable to the average person. If the box in my hand can tell me what 4367 x 2231 is and the capital of Guam, why then wouldn't I rely on it when it starts writing up full content for me? Because the average human adapts to the lowest required skill set I do worry that providing a device in our hands that "thinks" is going to reduce our learned ability to rationally process and check what it puts out, just like I've lost the ability to check if my calculator is lying to me. And not to get all dystopian here... but what if then, what that tool is telling me is true, is, for whatever reason, not.
(and yes, I ran this through a spell checker because I'm a part of the problem above... and it found words I thought I could still spell, and I'm 55)
It’s paramount to not conflate the two. LLM answers are almost always the latter with no guarantee of being the former. That is a tremendous flaw with real consequences.
> it’s a general weakness in human cognition, and has little to do with AI in and of itself.
A distinction without merit. Like blaming people for addictive behaviour while simultaneously exploiting human psychology to sell them the very same addiction.
This “technically correct” approach to arguing where the fault lies precludes productive discussion on fixing the problem. You cannot in good faith shout from the rooftops LLMs are going to solve all our problems and then excuse its numerous failures with “but we added a note you should always verify the answer. It’s down in the cellar, with no lights and stairs, in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’”.
It has become norm for people to “settle” disputes by quoting or showing a screenshot of an LLM answer. Often wrong. It is irrelevant to argue people should do better; they don’t, and that’s the reality you have to address.
The study is basically just... a bunch of self-reports on how people feel about AI usage in their daily job. That's it. The "critical thinking" part is completely self-reported.
It's not "how good a person is at critical thinking." It's "how much a person feels they engages in critical thinking." Keyword: feels.
It's like all the nutrition studies where they ask people to self-report what they eat.
There is a growing number of companys and agencies looking at banning AI, whatchamcallit, due to the increasing number of issues around eronious stuff causing liability, of the scary kind.
What is bieng passed off as journalism does nothing to give confidence in "AI" or its minders.
And behind the sceens it is easy to imagine that actual conciensous humans biengs are out competed by there "enhanced" co-workers, and just walking out to seek better compensation, or at least find work in a profesional environment, sorry, sorry, an environment that suits there "legacy" skill set.
It really has been a sight watching the loudest anti-AI people flog this around, then turn to rage when you clarify the range of the actual conclusion.
Generative AI got a lot more useful when I started seeing it abstractly. I put data in, it finds correlations in its neural network, and produces different data. I have to be intentional in interpreting that data to figure out what to do with it.
Once I started thinking in terms of "this word will kick up a hornet's nest in that branch of the network and make the output useless. Let's find another word that gets closer to what I'm aiming for," the outputs got so much better. And getting to the good outputs took fewer iterations.
> Generative AI got a lot more useful when I started seeing it abstractly. I put data in, it finds correlations in its neural network, and produces different data. I have to be intentional in interpreting that data to figure out what to do with it.
In my opinion this is a mischaracterization just as you stated others have "raged" [0] about. The simple question for you is: how do you know how to interpret? When precision and/or depth has no critical bearing I agree with your sentiment. However shades of grey appear in the simplest of prompts, often, quick. People who do not already have the skill to "interpret" the data, as you stated, can (and probably will) assume it is correct. That end user is also not constantly reminded of the age of the underlying data the model was trained on, nor are they aware how an LLM foundationally works or if it is reasoning or not - amongst many other unknowns.
Yes, while I feel as though the Microsoft report can have an air of "yes, that's the condition we expect" you're also not considering other, very important, inputs to that trivial response. Read the paper in the context of middle and high school students and now how does the "rage" feel? Are you a parent on a school board seeing this happen first hand?
Not everyone has the analytical pedigree of people like yourself and the easy access to LLMs is pissing people off as they watch a generation being robbed via the easy (and oft wrong) button.
[0] "It really has been a sight watching the loudest anti-AI people flog this around, then turn to rage when you clarify the range of the actual conclusion."
I beg to differ. AI made it possible for human to pursue critical thinking. Overwhelmed by the basic facts and routine works, limited by bandwidth and 8 hours a day, we hardly have the luxury to think above and beyond. That's when you hire consulting firms to stuff the content, the ocean of information, the type of work now potentially suitable for AI.
It is time for human to move up the game and focus on the critical thinking, even only the critical thinking while the AI is still unable to perform the critical thinking. Eventually there is the hope that AI would be able to handle the critical thinking, but it remains a hope at the current state of art.
I did not link to it directly because the PDFs title - "The Impact of Generative AI on Critical Thinking: Self-Reported
Reductions in Cognitive Effort and Confidence Effects From a
Survey of Knowledge Workers" - was way too long for the HN title length limit.
Abstract (from the PDF):
> The rise of Generative AI (GenAI) in knowledge workflows raises questions about its impact on critical thinking skills and practices.
> We survey 319 knowledge workers to investigate 1) when and how they perceive the enaction of critical thinking when using GenAI, and 2) when and why GenAI affects their effort to do so. Participants shared 936 first-hand examples of using GenAI in work tasks.
> Quantitatively, when considering both task- and user-specific factors, a user’s task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted and the effort of doing so in GenAI-assisted tasks.
> Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking. Qualitatively, GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship. Our insights reveal new design challenges and opportunities for developing GenAI tools for knowledge work.
> Analysing 936 real-world GenAI tool use examples our participants shared, we find that knowledge workers engage in critical thinking primarily to ensure the quality of their work, e.g. by verifying outputs against
external sources.
The researchers appear to be saying that a lot of people perceive LLM outputs to be superior to whatever they could generate themselves, so users accept the LLM's logical premises uncritically and move on. Sort of a "drive by" use of chatbots. The researchers are not wrong, but intuitively I don't think this is a fair critique of most LLM tasks, or even most users.
One of the most compelling LLM use cases is individualized learning or tutoring, and LLM's can support a much deeper dive into arcane scientific or cutting-edge topics than traditional methods. I don't see anything here that suggests the researchers balanced these scenarios against one another.
> One of the most compelling LLM use cases is individualized learning or tutoring, and LLM's can support a much deeper dive into arcane scientific or cutting-edge topics
When you deep dive with llm, everything needs to be verified. LLM could hallucinate or provide incorrect information and you won't even know unless you look it up. Last thing a learner needs is incorrect information presented as facts while learning.
The key thing about this paper is the invalidity of the measures. They are essentially looking at the correlation between measures of frequency of AI usage and measures of critical thinking skills.
Yet, in the questions themselves, questions like “I always trust the output of AI” (paraphrasing) are in the measures of frequency—and questions like “I question the intentions of the information provided by AI” (which is not a reasonable question, if you use AI regularly) is in the measures of critical thinking.
Sorry I don’t have time now to share the actual text. Take a look yourself though, at the end of the paper.
If a study comes out which demonstrates to high confidence that some extremely horrible outcome is certain with this technology, would any changes be made? Would we tell OpenAI to close shop? I don't think, with the way our society and politics is set up, we would be able to deal with this. There's too much money, ego, and general inertia at play to change course.
That's too broad though, because there been specific issues we've (humanity) managed to come together to fix, like the "ozone hole" stuff being "solved" with regulations, protocols and public policy, while there are other issues we haven't quite been able to do the same with, all under the umbrella of "climate change".
I mean, that question is really a question for yourself. You cannot control others, so assume others will do whatever is quickest, easiest and/or least resistance, outcomes be damned.
Of course nothing would immediately do a 180, but if there would be research put out that your brain slowly stops working to more you use LLMs, then you can at least make your own choice from there on out.
Interesting, in my experience LLMs hallucinate so much on stuffs I know about that I instinctively challenge most of their assumptions and outputs, and found out that this kind of dialectic exchange bring the most out of the "relationship" so to speak, co creating something greater than the isolation of us two.
I haven't used LLMs a lot and have just experimented with them in terms of coding.
But about a year ago, I had a job to clean up a bunch of, let's call them, reference architectures. I mostly didn't mess with the actual architecture part or went directly to the original authors.
But there wasn't a lot of context setting and background for a lot of them. None of it was rocket science; I could have written myself. But the LLM I used, Bard at the time, gave me a pretty good v 0.9 for some introductory paragraphs. Nothing revelatory but probably saved me an hour or two per architecture even including my time to fixup the results. Generally, nothing was absolutely wrong but some I felt was less relevant and other stuff I felt was missing.
I wonder a lot why people use generative AI in the first place.
I’ve found in my work that I don’t really have anything to ask an AI to help me with that the generative AIs available to me are particularly good at. I distrust them, but I also haven’t really found a use for them. Which isn’t to say that such generative AIs don’t exist; I just don’t have access to those particular ones due to the nature of my work, and so haven’t tried them.
I don’t really have a use for them personally either. I think the closest I come is voice assistants, which I use so sparingly I wouldn’t miss if they were gone.
Readit News
> our research questions - when and how knowledge workers perceive the enaction of critical thinking when using GenAI (RQ1), and when and why do knowledge workers perceive increased/decreased effort for critical thinking due to GenAI (RQ2)
This is about the application of critical thinking to AI outputs and in AI workflows. Are people cognitively lazy when some other entity hands them plausible sounding answers?
The answer of course is yes. If some entity gives you a good enough result, probably you aren’t going to spend much time improving it unless there is a good reason to do so. Likewise you probably aren’t going to spend a lot of time researching something that AI tells you if it sounds plausible. This is certainly a weakness, but it’s a general weakness in human cognition, and has little to do with AI in and of itself.
In my reading, what this study does not say, and does not set out to answer, is whether or not the use of AI makes people generally less able or likely to engage in critical thinking as a result of use of AI.
The highest percentage of humans will take the easiest path provided. And while most of the above we just consider improvements to daily life, efficiencies, it has also fundamentally changed on average what we are capable of and what skills we learn (especially during formative years). If I dropped most of us here into a pre-technology wilderness we'd be dead in short order.
However, most of the above, it can be argued, are just tools that don't impact our actual thought processes; thinking remained our skill. Now the tools are starting to "think", or at least appear like they do on a level indistinguishable to the average person. If the box in my hand can tell me what 4367 x 2231 is and the capital of Guam, why then wouldn't I rely on it when it starts writing up full content for me? Because the average human adapts to the lowest required skill set I do worry that providing a device in our hands that "thinks" is going to reduce our learned ability to rationally process and check what it puts out, just like I've lost the ability to check if my calculator is lying to me. And not to get all dystopian here... but what if then, what that tool is telling me is true, is, for whatever reason, not.
(and yes, I ran this through a spell checker because I'm a part of the problem above... and it found words I thought I could still spell, and I'm 55)
It’s paramount to not conflate the two. LLM answers are almost always the latter with no guarantee of being the former. That is a tremendous flaw with real consequences.
> it’s a general weakness in human cognition, and has little to do with AI in and of itself.
A distinction without merit. Like blaming people for addictive behaviour while simultaneously exploiting human psychology to sell them the very same addiction.
https://www.youtube.com/watch?v=EJT0NMYHeGw
This “technically correct” approach to arguing where the fault lies precludes productive discussion on fixing the problem. You cannot in good faith shout from the rooftops LLMs are going to solve all our problems and then excuse its numerous failures with “but we added a note you should always verify the answer. It’s down in the cellar, with no lights and stairs, in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’”.
It has become norm for people to “settle” disputes by quoting or showing a screenshot of an LLM answer. Often wrong. It is irrelevant to argue people should do better; they don’t, and that’s the reality you have to address.
Deleted Comment
It's not "how good a person is at critical thinking." It's "how much a person feels they engages in critical thinking." Keyword: feels.
It's like all the nutrition studies where they ask people to self-report what they eat.
What is bieng passed off as journalism does nothing to give confidence in "AI" or its minders.
And behind the sceens it is easy to imagine that actual conciensous humans biengs are out competed by there "enhanced" co-workers, and just walking out to seek better compensation, or at least find work in a profesional environment, sorry, sorry, an environment that suits there "legacy" skill set.
Dead Comment
Deleted Comment
Generative AI got a lot more useful when I started seeing it abstractly. I put data in, it finds correlations in its neural network, and produces different data. I have to be intentional in interpreting that data to figure out what to do with it.
Once I started thinking in terms of "this word will kick up a hornet's nest in that branch of the network and make the output useless. Let's find another word that gets closer to what I'm aiming for," the outputs got so much better. And getting to the good outputs took fewer iterations.
In my opinion this is a mischaracterization just as you stated others have "raged" [0] about. The simple question for you is: how do you know how to interpret? When precision and/or depth has no critical bearing I agree with your sentiment. However shades of grey appear in the simplest of prompts, often, quick. People who do not already have the skill to "interpret" the data, as you stated, can (and probably will) assume it is correct. That end user is also not constantly reminded of the age of the underlying data the model was trained on, nor are they aware how an LLM foundationally works or if it is reasoning or not - amongst many other unknowns.
Yes, while I feel as though the Microsoft report can have an air of "yes, that's the condition we expect" you're also not considering other, very important, inputs to that trivial response. Read the paper in the context of middle and high school students and now how does the "rage" feel? Are you a parent on a school board seeing this happen first hand?
Not everyone has the analytical pedigree of people like yourself and the easy access to LLMs is pissing people off as they watch a generation being robbed via the easy (and oft wrong) button.
[0] "It really has been a sight watching the loudest anti-AI people flog this around, then turn to rage when you clarify the range of the actual conclusion."
It is time for human to move up the game and focus on the critical thinking, even only the critical thinking while the AI is still unable to perform the critical thinking. Eventually there is the hope that AI would be able to handle the critical thinking, but it remains a hope at the current state of art.
For the record, I’m not going to read the article to verify your statement.
I did not link to it directly because the PDFs title - "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers" - was way too long for the HN title length limit.
Abstract (from the PDF):
> The rise of Generative AI (GenAI) in knowledge workflows raises questions about its impact on critical thinking skills and practices.
> We survey 319 knowledge workers to investigate 1) when and how they perceive the enaction of critical thinking when using GenAI, and 2) when and why GenAI affects their effort to do so. Participants shared 936 first-hand examples of using GenAI in work tasks.
> Quantitatively, when considering both task- and user-specific factors, a user’s task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted and the effort of doing so in GenAI-assisted tasks.
> Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking. Qualitatively, GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship. Our insights reveal new design challenges and opportunities for developing GenAI tools for knowledge work.
The researchers appear to be saying that a lot of people perceive LLM outputs to be superior to whatever they could generate themselves, so users accept the LLM's logical premises uncritically and move on. Sort of a "drive by" use of chatbots. The researchers are not wrong, but intuitively I don't think this is a fair critique of most LLM tasks, or even most users.
One of the most compelling LLM use cases is individualized learning or tutoring, and LLM's can support a much deeper dive into arcane scientific or cutting-edge topics than traditional methods. I don't see anything here that suggests the researchers balanced these scenarios against one another.
When you deep dive with llm, everything needs to be verified. LLM could hallucinate or provide incorrect information and you won't even know unless you look it up. Last thing a learner needs is incorrect information presented as facts while learning.
Yet, in the questions themselves, questions like “I always trust the output of AI” (paraphrasing) are in the measures of frequency—and questions like “I question the intentions of the information provided by AI” (which is not a reasonable question, if you use AI regularly) is in the measures of critical thinking.
Sorry I don’t have time now to share the actual text. Take a look yourself though, at the end of the paper.
Of course nothing would immediately do a 180, but if there would be research put out that your brain slowly stops working to more you use LLMs, then you can at least make your own choice from there on out.
Relevant 2018 essay by Nicky Case «How To Become A Centaur»: https://jods.mitpress.mit.edu/pub/issue3-case/release/6
But about a year ago, I had a job to clean up a bunch of, let's call them, reference architectures. I mostly didn't mess with the actual architecture part or went directly to the original authors.
But there wasn't a lot of context setting and background for a lot of them. None of it was rocket science; I could have written myself. But the LLM I used, Bard at the time, gave me a pretty good v 0.9 for some introductory paragraphs. Nothing revelatory but probably saved me an hour or two per architecture even including my time to fixup the results. Generally, nothing was absolutely wrong but some I felt was less relevant and other stuff I felt was missing.
In my experience most people don’t do that. Therein lies the problem.
I’ve found in my work that I don’t really have anything to ask an AI to help me with that the generative AIs available to me are particularly good at. I distrust them, but I also haven’t really found a use for them. Which isn’t to say that such generative AIs don’t exist; I just don’t have access to those particular ones due to the nature of my work, and so haven’t tried them.
I don’t really have a use for them personally either. I think the closest I come is voice assistants, which I use so sparingly I wouldn’t miss if they were gone.
Deleted Comment
(By Scott Alexander)