On the one hand, this is sorely needed: AI detection software will inevitably be mostly snake oil.
Academia and education desperately wants this software to work! As a result, selling them something that doesn't work is going to be very profitable.
The most obvious problem with this class of software is how easy it would be to defeat if the students could access it themselves: generate some text, run it through the detector, then fiddle with it (by manually tweaking it or by prompting the AI to "reword this to be less perfect") until it passes.
Which means these tools need to not be openly available... which makes them much harder to honestly test and evaluate, making it even easier to sell something that doesn't actually work.
But... I don't think this site is particularly convincing right now. It has spelling mistakes (which at least help demonstrate AI probably didn't write it) and the key "How AI Detection Software Works" page has a "Coming Soon" notice.
The "examples" page is pretty unconvincing right now too - and that's the page I expect to get the most attention: https://itwasntai.com/examples
It looks to me like this is still very much under development, and is not yet ready for wider distribution.
its too easy to be negative about things in hype cycles and retroactively look back and go "see! i was right! this was a terrible idea!" but.. this is a terrible idea
to ai detection fans: show us on an information theory basis how you will smuggle in enough bits, avoiding user obfuscation, please. i will change my mind and support you the moment you prove this can be done, otherwise i am default extremely skeptical
Funny enough, one fo the first things I did with ChatGPT was teach it how to write with charcater keyboard transpositions and subtle typos so people wouldn't think varioius text was AI generated. Works pretty well.
(Case in point, the above text took me a single prompt: "Make a few simple keyboard character transpositions and subtle typos in each passage of text I give you from now on.")
But why should educational instutions care? Education is a business, students are the customers (or in countries with state-funded education, the government). If AI helps people graduate faster, that's more money to the institutions, less effort to the students, and nice statistics for the governments.
At least in my country most degrees aren't worth much anyway, they just open you doors to internships where you really learn stuff. AI isn't going to make the situation any worse.
Because educational institutions aren’t in the business of selling education: they give that away for free. Seriously, walk into any university campus and into a lecture hall, sit down and take notes. No one will stop you! No one will check ID. You can even talk to the professor and 99% of the time they’ll give you access to the course materials online at well.
What students are paying for is accreditation. It’s not just their name that goes on the piece of paper, it’s the school’s name. Cheating undermines that business entirely. If a school looks the other way long enough there will be cheating scandals in the news and the school’s reputation will be damaged.
and never get to learn about linear regression, bias and variance, cost function and gradient descent, regularisation and optimisation - all the good things taught by Andrew Ng in the amazing course he run 12 years ago just before creating Coursera.
Governments paying for education don't just want graduations, they want an educated workforce, because there are benefits from having that.
Instructors generally do not treat education like a business. On some level the institutions themselves often are business-like, but on the classroom level I don't think that's the case.
They care because this is massively disruptive to the way they teach at the moment. They have decades of practices in place for how they evaluate students which don't work if students can have AI do the work for them.
They can chose to reinvent everything about how they operate, or they can pay money to a company that promises to make that problem go away for them.
It's not surprising that many of them are trying the latter option first.
IMO one of the way that most schools are going to end up being able to detect plagiarism is going to be a custom word processor (or something similar) that can track all of the edits made into a document. Basically, have the students type an essay where all of the keystrokes are recorded by the program, and so it can be detected by the program whether someone is copy and pasting whole essays, or if someone is actually typing and revising the essay until it is submitted. Essays that are just turned in in general are probably going to be a thing of the past.
Maybe, but I doubt it. Spyware-based systems are doomed to failure as other commenters note. There's nothing you can do to prove the text came from a human. Faking inputs is extremely easy. People will sell a $20 USB dongle that does appropriate keyboard/mouse things. Worst case, people can simply type in the AI generated essay by hand and/or crib from it directly.
Schools are going to have to look at why take home work is prescribed, and if it should be part of a grading system at all. My hunch is that it probably shouldn't be, and even though it's a big change it's probably something they can navigate.
It's a cat-and-mouse game for sure. At the first level, any dongle that simply types the AI response through a fake HID device will be easy to detect. No real essay writer just types an entire document in one go, with no edits. They move paragraphs around, expand some, delete others, etc.
So this dongle will have to convincingly start with a worse version that's too short (or too long!). It'll have to pipe the GPT output through another process to mangle it, then "un" mangle it like a human would as they revise and update.
If trained on the user's own previous writings, it can convincingly align the AI's response with the voice and tone of the cheater.
Then the spyware will have to do a cryptographic verification of the keyboard ("Students are required to purchase a TI-498 keyboard. $150 at the bookstore") to prevent the dongles. There will be a black market in mod chips for the TI-498 that allow external input into the traces on the keyboard backplane. TI will release a better model that is full of epoxy and a 5G connection that reports tampering...
... Yeah, I also predict more in-person learning :)
Which would be a huge benefit for the overall quality of education. A lot of student can write a passable essay in a word processor with spell check and tutors... but those same students sometimes have absolutely no idea what they've written. Group assignments has taught me this many times over.
My wife started teaching a class at the local university. She had a bunch of positives on the anti-plagiarism software used by the university. She ran a bunch of papers by me and man, analyzing the results are an art within it's self. People will unconsciously remember and write down phrases and smaller sentences they have read all the time. A little highlight here and there just has to be accepted. Then there are the papers that almost the entire thing is highlighted. It's the ones in between that are tricky as hell. A lot could have gone either way and it's a judgement call on the teacher whether to send it to the administration for review. I expect AI will just make it more difficult or hand writing is going to be the new hot subject taught to new levels in elementary...
To me it seems like academic papers force people to back up every statement with a quote and agree with assigned readings. This style of writing leads to unoriginal results.
LLMs are useful for a variety of things. What you're describing would only be useful for students cheating on assignments. I doubt that it will attract the many millions of dollars spent on training GPT-4.
But more importantly, LLMs are always available over the Internet. If students need to use a physical device to cheat, that's already a big step forward, since it increases the chance of detection — a key factor in deterring misbehavior.
When I was in college we had a number of group projects and I thought the whole time that it would make a ton of sense for the professor to set up a class repo (I'm a old person so they would be a CVS repo at the time) and be able to see exactly what each person had contributed to the project. Even for single person projects it would have made it so much easier to detect cheaters. I also think it might light a fire under some of the less shameless slackers.
I hope schools do this now. Not only for detecting cheaters but to get the kids used to working in a more real world environment.
I think you overestimate the competence of the majority of professors. They can’t require version control if they don’t understand what it is or how to use it.
As a parent whose student has worked with multiple essay entry editors/forms, they're almost all terrible with most students having to revert to writing the essay outside the system or risk losing their work multiple times. And this was with a simple editor - not more complex connections to even more sophisticated systems.
The budget available for educational technology is not sufficient to maintain the operation of the software, let alone sufficient to pay technical staff adequate to assess and select reliable systems.
Yeah, then you get a bunch of non-cheating students who are intelligent and just annoyed with the text editor will use cheating tools to insert their essay they wrote in a proper word processor - further poisoning the dataset.
But then you can have ChatGPT write your essay on a phone/tablet and you just slowly re-write it.
I think schools will need to change the way they go about testing student understanding of topics. Personally I'm excited for what this might look like and it is a great opportunity for hackers to really innovate the educational field.
Or they could move to a more British style, with in-person essays, proctored by human observers (not that there aren't old-fashioned ways to cheat on those too, but they're well-known).
Yes, when I had to take university entry exams in Brazil, all parts of the exam were in person, including writing the essay, with a mandatory topic only disclosed when the exam starts. Preventing ai cheating might become more difficult for educational projects that are long form, like writing a dissertation, or big coding challenges. Although, for coding, one thing that I have consistenly seen work, is to just ask students to do a walk through of the code. People that just copy someone elses work are generally lazy, and don't really study what they copied, and it becomes easy to see who put in the work.
If it's so easy to just copy and paste an essay from an AI generator that is of such high quality that it cannot be detected, then why are we still making students learn such an obviously obsolete skill? Why penalize students for using technology?
Surely, there are still things that are difficult to do even with the help of AI. Teach your students to use these tools, and then raise the bar. For example, ask your art students to make complex compositions or animations that can't be handled by Midjourney without significant effort.
The reason it's done is to teach students how to think. By writing down their thoughts they are forced to think about a topic. It's the same reason small children are still taught arithmetic although we have calculators.
That's the theory, anyway. In practice students learn that "really thinking for themselves" in essays is usually not rewarded while paraphrasing some reading assignments with some sprinkled quotations works much better and is less work than thinking about topics they don't care about.
Maybe the AI stuff will lead to practice better approximating the theoretical goal.
> If it's so easy to just copy and paste an essay from an AI generator that is of such high quality that it cannot be detected, then why are we still making students learn such an obviously obsolete skill? Why penalize students for using technology?
That's like asking, why do we have students do PE (physical education) when professional athletes exist? Clearly, having students play basketball is obsolete, because the NBA exists. Essay-writing is PE for thinking.
>why are we still making students learn such an obviously obsolete skill?
Just because a machine can generate an essay of questionable quality with a fair chance of containing hallucinations making it unusable for many fields of human endeavor doesn't mean that writing is no longer a useful pedagogical tool. Learning to write is a part of learning to think.
I also had a similar idea on how to determine that a piece of writing is genuine. It would be to make students use a word processor that contains a full audit trail of all changes, timestamped. The software would then use a trained AI to look for patterns that deviate from normal composition activities. This could catch a lot of the current fraud. Until someone creates AI bots to get around it...
I don't know if this is the way things should go, but it seems like a decent prediction about how they probably will. In fact, many law school exams are already administered using "blue book" software that functions as rudimentary word processors that lock down the computer's other functions for the duration of the exam. Perhaps other disciplines use this software too.
In the exam context, this software probably already solves the AI problem. Locking down the computer would not, of course, be a solution for other kinds of assignments, but I'll bet it won't be long until schools are using software like you described that are just do a lot of snooping instead of locking down the computer.
Unfortunately, the existing software is very clunky and not very reliable. And it doesn't seem like anybody has a strong incentive to improve it. (The schools license the software, and the schools understandably don't care all that much whether the software is nice to use.)
The cat and mouse iteration will be using ChatGPT integrated with Webdriver to slowly type the essay, writing a prompt that says "make occasional mistakes", etc.
Wouldn't it still be easier to type out the entire AI generated assignment than to come up with an assignment and then type out the assignment you came up with yourself?
Obviously typing from start to finish with few edits is also a "failed" result in such a program. Someone actually writing an essay should be creating structure, taking notes, rearranging paragraphs etc.
Then again you have a good point. Often you blat out an essay and then edit it. Same thing goes with typing in an AI generated template.
Maybe you connect to school chat AI and then it probes you for knowledge. Same AI watches you write essay type bits and helps you out if you get something wrong. Teacher will get report how well you did and how present you were.
Ironically, the best detector for plagiarism would be a 15 minute conversation asking the student about their research and opinions on the topics written, kind of like interviewing someone who claims redis expertise on their resume.
This is essentially the Oxbridge tutorial system where you have an hour(ish) meeting once(ish) a week with one or two other students and your tutor and talk over what you've learned, are set an assignment or three and have to answer any questions about the last week's assignments. A slightly more scalable version is the seminar system where you have up to 10 students and a tutor and you do roughly the same thing. It only works if participation is mandatory, missing seminars / tutorials is penalised and tutors are given leeway to adjust grades based on semina performance or flag students who do really well on their essays but appear incapable of explaining or defending what they've written.
Yes, that is true. ChatGPT refuses to work with VPN and regularly checks for your IP using Cloudflare. Most of these new AI-driven startups are using Cloudflare. It is funny that startups collect/scrap data from all over the web but don't want to scrap their chatbot response without paid API. I guess it is life...
I've been reviewing answers to questionnaires we send out to potential software engineering candidates. Sometimes candidates seem to write 90% of the submission themselves, and then use ChatGPT for the last couple of questions (which are more general, like "Outline your thoughts on documentation in software projects"). I joked to a colleague that I'd come up with a fool-proof ChatGPT detector in one line of Python:
How many essays are written and graded every year? Even with a 0.0001% FPR, how many students would be facing the severe punishments for plagiarism like auto-failing a course or even expulsion? Ironically enough, using AI to make such decisions seems like one of those unethical use cases.
Readit News
Academia and education desperately wants this software to work! As a result, selling them something that doesn't work is going to be very profitable.
The most obvious problem with this class of software is how easy it would be to defeat if the students could access it themselves: generate some text, run it through the detector, then fiddle with it (by manually tweaking it or by prompting the AI to "reword this to be less perfect") until it passes.
Which means these tools need to not be openly available... which makes them much harder to honestly test and evaluate, making it even easier to sell something that doesn't actually work.
But... I don't think this site is particularly convincing right now. It has spelling mistakes (which at least help demonstrate AI probably didn't write it) and the key "How AI Detection Software Works" page has a "Coming Soon" notice.
The "examples" page is pretty unconvincing right now too - and that's the page I expect to get the most attention: https://itwasntai.com/examples
It looks to me like this is still very much under development, and is not yet ready for wider distribution.
its too easy to be negative about things in hype cycles and retroactively look back and go "see! i was right! this was a terrible idea!" but.. this is a terrible idea
to ai detection fans: show us on an information theory basis how you will smuggle in enough bits, avoiding user obfuscation, please. i will change my mind and support you the moment you prove this can be done, otherwise i am default extremely skeptical
Nowadays if you want to be convincing you got to maek some spelling misakes. Something that looks like predictive keyboard errors, or typing errors.
(Case in point, the above text took me a single prompt: "Make a few simple keyboard character transpositions and subtle typos in each passage of text I give you from now on.")
Probably so. The problem, of course, is that the inability to detect AI authorship leads to the increase of general distrust of everything in society.
Deleted Comment
At least in my country most degrees aren't worth much anyway, they just open you doors to internships where you really learn stuff. AI isn't going to make the situation any worse.
What students are paying for is accreditation. It’s not just their name that goes on the piece of paper, it’s the school’s name. Cheating undermines that business entirely. If a school looks the other way long enough there will be cheating scandals in the news and the school’s reputation will be damaged.
LLM University - https://docs.cohere.com/docs/llmu
and never get to learn about linear regression, bias and variance, cost function and gradient descent, regularisation and optimisation - all the good things taught by Andrew Ng in the amazing course he run 12 years ago just before creating Coursera.
Is that a good thing?
Instructors generally do not treat education like a business. On some level the institutions themselves often are business-like, but on the classroom level I don't think that's the case.
They can chose to reinvent everything about how they operate, or they can pay money to a company that promises to make that problem go away for them.
It's not surprising that many of them are trying the latter option first.
Schools are going to have to look at why take home work is prescribed, and if it should be part of a grading system at all. My hunch is that it probably shouldn't be, and even though it's a big change it's probably something they can navigate.
I predict more in-person learning interactions.
So this dongle will have to convincingly start with a worse version that's too short (or too long!). It'll have to pipe the GPT output through another process to mangle it, then "un" mangle it like a human would as they revise and update.
If trained on the user's own previous writings, it can convincingly align the AI's response with the voice and tone of the cheater.
Then the spyware will have to do a cryptographic verification of the keyboard ("Students are required to purchase a TI-498 keyboard. $150 at the bookstore") to prevent the dongles. There will be a black market in mod chips for the TI-498 that allow external input into the traces on the keyboard backplane. TI will release a better model that is full of epoxy and a 5G connection that reports tampering...
... Yeah, I also predict more in-person learning :)
Which would be a huge benefit for the overall quality of education. A lot of student can write a passable essay in a word processor with spell check and tutors... but those same students sometimes have absolutely no idea what they've written. Group assignments has taught me this many times over.
But more importantly, LLMs are always available over the Internet. If students need to use a physical device to cheat, that's already a big step forward, since it increases the chance of detection — a key factor in deterring misbehavior.
I hope schools do this now. Not only for detecting cheaters but to get the kids used to working in a more real world environment.
The budget available for educational technology is not sufficient to maintain the operation of the software, let alone sufficient to pay technical staff adequate to assess and select reliable systems.
I think schools will need to change the way they go about testing student understanding of topics. Personally I'm excited for what this might look like and it is a great opportunity for hackers to really innovate the educational field.
If it's so easy to just copy and paste an essay from an AI generator that is of such high quality that it cannot be detected, then why are we still making students learn such an obviously obsolete skill? Why penalize students for using technology?
Surely, there are still things that are difficult to do even with the help of AI. Teach your students to use these tools, and then raise the bar. For example, ask your art students to make complex compositions or animations that can't be handled by Midjourney without significant effort.
That's the theory, anyway. In practice students learn that "really thinking for themselves" in essays is usually not rewarded while paraphrasing some reading assignments with some sprinkled quotations works much better and is less work than thinking about topics they don't care about.
Maybe the AI stuff will lead to practice better approximating the theoretical goal.
That's like asking, why do we have students do PE (physical education) when professional athletes exist? Clearly, having students play basketball is obsolete, because the NBA exists. Essay-writing is PE for thinking.
Just because a machine can generate an essay of questionable quality with a fair chance of containing hallucinations making it unusable for many fields of human endeavor doesn't mean that writing is no longer a useful pedagogical tool. Learning to write is a part of learning to think.
In the exam context, this software probably already solves the AI problem. Locking down the computer would not, of course, be a solution for other kinds of assignments, but I'll bet it won't be long until schools are using software like you described that are just do a lot of snooping instead of locking down the computer.
Unfortunately, the existing software is very clunky and not very reliable. And it doesn't seem like anybody has a strong incentive to improve it. (The schools license the software, and the schools understandably don't care all that much whether the software is nice to use.)
Then again you have a good point. Often you blat out an essay and then edit it. Same thing goes with typing in an AI generated template.
Deleted Comment
https://stealthoptional.com/news/us-constitution-flagged-as-...
As long as that kind of egregious mistake is possible, we should look at such tools with suspicion.