> Did an Institutional Review Board consider this study?
> We submitted an application detailing our research methods to the Princeton University Institutional Review Board, which determined that our study does not constitute human subjects research.
From the social experiment[as reported by OP's link]:
> I look forward to your reply without undue delay and at most within 45 days of this email, as required by Section 1798.130 of the California Civil Code.
I'm pretty sure most people would find that to be a thinly veiled threat of a lawsuit. I'd like to know if the review board considered the text of the email that the "researchers"[1] intended to send and the fact that they were likely to send it to individuals instead of solely publicly traded companies.
If I had to guess, the wording is in the study's FAQ is carefully chosen: "an application detailing our research methods" doesn't necessarily mean "an application with the verbatim text of the emails we planned to send, including our thinly veiled legal threat at the end."
Not trying to turn this thread into a generic flameware against "academic" research methods, but this whole things seems oddly reminiscent of the "let's try to insert malicious code into Linux" fiasco [1]. I'm conceptually fine with generic passive tools like web crawlers to conduct research, but since when did the internet become a place where nonconsensual interactive research became fine?
> Not trying to turn this thread into a generic flameware against "academic" research methods, but this whole things seems oddly reminiscent of the "let's try to insert malicious code into Linux" fiasco [1]. I'm conceptually fine with generic passive tools like web crawlers to conduct research, but since when did the internet become a place where nonconsensual interactive research became fine?
In a very real sense, every landing page A/B test is nonconsensual interactive research.
Or at least, if there is line between them, however blurry, I can't find it.
I am skeptical of the idea that such a line should be drawn according to who is doing the experimentation, I don't think that a manipulative act becomes okay just because it is being done by an academic for research purposes, nor do I think that it becomes okay just because it is being done by a layman with a profit motive (or a political one, for that matter).
The problem, like in that previous case, is that "human subject research" is a pretty narrowly defined category. It is mostly meant to cover testing out drugs on human subjects, and stuff like that. Notably, there is plenty of unethical research that doesn't qualify. So when an IRB gets a proposal that amounts to "I'm going to send some emails/interact with some folks online" their reply is likely to be along them lines "not our problem", and it becomes the responsibility of the research to assess the ethics of what they're doing.
The false legal threat is particularly galling, but this absolutely should have gone through IRB even without it. Someone should have had to at least consider the impact on recipients of the messages before they were sent.
IRB review is typically required even for just simple research surveys.
How is a reminder of the law a legal threat? More specifically when you feel like you're not impacted by this law, it's as far from a legal threat as could be.
I don’t think it’s intended as a veiled threat of a lawsuit so much as a statement of the compliance requirements. Unfortunately it seems they misunderstood the scope which makes the it inaccurate. But if the statement was true and accurate I would just take it as a helpful reminder of the timeframe.
> But if the statement was true and accurate I would just take it as a helpful reminder of the timeframe.
No. Absolutely not. A helpful reminder of the timeframe would be "the deadline for our study is ..., please try to send your response by then if you wish to be included."
Quoting legal code is not at all a helpful reminder of a timeframe, but is a direct implication of legal ramifications for failure to comply.
> if the statement was true and accurate I would just take it as a helpful reminder of the timeframe.
People don't go through the trouble of digging up the particular section number of the specific statute of the specific jurisdiction in question for the mere sake of a generic "helpful reminder of the timeframe" required by law.
The context is important: if you deal with user support (especially in the context of privacy) then someone quoting law at you is a huge red flag for an impending nightmare. I’ve dealt with irate users who actually did go as far as to file lawsuits and the email from this “study” activated my fight or flight response because of how much it (unintentionally?) mirrors the way angry litigious internet users communicate. The only worse phrase to read is “free speech”.
I would guess as an attorney you're more used to that style of communication than a random blogger or small entity. Citing law has very different signaling purpose and effect in different contexts.
It is interesting in the study web page (https://privacystudy.cs.princeton.edu/) that they consistently mention contacting "websites" instead of "people." As if a website is some autonomous thing that can communicate with a researcher.
I wouldn't be sleeping well if I were involved in this study. There is no way an IRB could determine that this is not human subjects research if you're emailing people and asking them anything.
If I want to ask random people about the weather and publish the results in a journal, that qualifies as human subjects research and IRB protocols must be followed. Emailing people asking about privacy policies is definitely human subjects research. Either they misrepresented the study in their IRB application, or the IRB didn't do due diligence reviewing the application.
In my understanding (from a french cultural context), asking people questions as part of a field study is not human subjects research. Ethical questions arise when you ask people to take specific actions in order to measure their reactions, not when you're asking about the status quo.
Lying about who you are and pretending that you are allowed a certain thing… I mean legally it’s not fraud but it sure feels like it!
Imagine someone showing up to your office building pretending to have an interview , walking into the office waiting room, then walking out saying “oh, just an experiment!”
“It’s just an email” the ease of the mode of communication here is not super relevant to the action, right?
The email ended with a "looking forward to your response within 45 days". Immediately in-house council was under the impression it was someone trying to entrap the website owners into a lawsuit for not being compliant.
There was a Twitter thread of a number of in-house forwarding to external council which costs time and money. This study incurred monetary damages.
Guidelines depend on jurisdiction. Research conducted in the United States may have different requirements than other jurisdictions.
The underlying ethos is that researchers should respect the people who are participating in their studies. People should know that you are conducting a study, the aims of the study, and choose whether or not they want to participate.
> I wouldn't be sleeping well if I were involved in this study. There is no way an IRB could determine that this is not human subjects research if you're emailing people and asking them anything.
Do you have a citation for this? What I'm seeing from random Googling is that you have to be obtaining information about the person for it to count.
If I were researching, say, price trends in some commodity and I called up several companies' sales lines and asked for their current price that looks like it would not be human subject research despite the fact that I'm talking to a human to get each company's price.
If I were researching pay trends at those companies and called up their sales lines and asked the people who answered how much they were paid it would be human subject research.
They're sending requests to websites to see how they behave, not to request information.
If you called up companies' sales lines to ask for prices, that's just gathering facts.
If you called up companies' sales lines to see what happens when you ask them for prices for things they don't sell, or to see if they're willing to accept a bribe, or to see if they respond with different prices when you lie to them about who you are, you're researching human behavior.
In this case, they are testing what procedures, if any, companies have in place for handling CCPA and GDPR law, by posing as nonexistent customers and making potentially bogus and misleading requests under the terms of those laws.
This is like performing research on retail refund practices by going into a bunch of shops and seeing how they handle being asked for a refund for an item you didn't buy from there in the first place.
There's a more ethical way to do that study, though, which is to actually buy something from the store first, then go back and try to refund it.
Similarly, there's a more ethical way to discover how websites handle CCPA/GDPR requests, which is to use the website first, and determine in the course of that what possible information about you the website should have; then, within the terms of your rights under CCPA/GDPR, to contact them and make reasonable and legitimate requests to see if/how they are able to handle them.
Here are questions sent to individuals in the study: Would you process a CCPA data access request from me even though I am not a resident of California? Do you process CCPA data access requests via email, a website, or telephone? If via a website, what is the URL I should go to? What personal information do I have to submit for you to verify and process a CCPA data access request? What information do you provide in response to a CCPA data access request?
The word "you" appears in every question.
To quote my IRB training (citiprogram.org):
"Most research in the social and behavioral sciences involves gathering information about individuals. However, some research that involves interactions with people does not meet the regulatory definition of research with human subjects because the focus of the investigation is not the opinions, characteristics, or behavior of the individual. In other words, the information being elicited is not about the individual ("whom"), but rather is about "what." For example, if a researcher calls the director of a shelter for battered women and asks her for the average length of stay of the women who use the shelter, that inquiry would not meet the definition of research with human subjects because the information requested is not "about" the director. If the researcher interviewed the director about her training, experience, and how she defines the problem of battering, then the inquiry becomes about her - and therefore "about whom."
The current example is similar, in my opinion, to "how she defines the problem of battering" which the IRB training identifies as human subjects research. The people receiving the researchers' email in the current study are being ask to define the way they interpret and comply with a legal statute.
I can accept that some people don't see the information requested as being "about whom" and therefore is not human subjects research. But the fact that people who have received this email have panicked indicates that the recipients, at least, felt that the questions were more than merely recording impersonal data about their websites.
This is incorrect. It's only human subjects research if the researcher is obtaining data about a human. This is the "about whom" requirement. A classic example is calling a business and asking someone about the products and prices they offer. That's not human subjects research.
If you say "I am a researcher studying X, can you please answer the following questions" then you might be studying a "what", depending on the specific questions.
When you lie about who you are, what your purposes are, and use scary legal language in an attempt to elicit a response, that is absolutely human research. You may be able do those things ethically as scientist but you absolutely need IRB review becausr it is definitely human research.
My guess is that the IRB in this case was not informed of the deceptive nature of some of the emails as lieing is absolutely a red flag that you are doing human research and not just information gathering. Indeed, evaluating such lies for potential harm is an important part of why we have IRBs for psychological and sociological research.
Note that the title doesn't provide the full meat.
It reads: "CCPA Scam November 2021"
Story update notes: "This is a human subject research study conducted Princeton University"
I was attempting to submit my own instance of this when I discovered ColinWright's. My suggested title was going to be "CCPA Scam ... is a human subject research study conducted by Princeton University".
Panicking small web operators without consent being the issue.
I've gotten 4 of these mails to 4 of my domains (including my personal domain used just for email, and a one-page documentation site for an open source library)... 2 about CCPA and 2 about GDPR. They also gave me a lot of anxiety for no reason. Looking at the responses on Twitter, a lot of websites spent real money consulting lawyers before responding to these mails due to the thinly veiled threat of legal repercussions at the end of each one.
Why? If that's indeed the law, then it's up to the website owner to comply. Whether it's Princeton or a private individual writing the email doesn't matter.
On what basis? Why can't we reasonably expect these sites to follow the laws? Just that they have in past survived being unethical and not following them does mean they have some sort of claim when they scramble to fix their failures.
We’re some students from Princeton trying to understand how businesses are responding to CCPA and GRPR requests. Could you help us with our study? How would you answer these questions?
…
The point is disclosure. It’s unethical to do otherwise, especially given that is about the use of data. I’d love for there to be more data published about the impacts of these policies, but please don’t use the tactics of creeps in the process.
It is likely that quite a few people would lie if they knew they were going to be observed/studied or reported on. However, I'm sure they could've made the actual email less threatening and more friendly/ethical without revealing research intent. (or the intent to research this specfic aspect)
Possibly. And the IRB review process may allow for non disclosure under circumstances of that sort.
The problem is not merely that the IRB allowed non disclosure. It's that the IRB also granted an exemption from full review as human subject research. If the researcher expected that human behavior might change based on secrecy vs. disclosure then it is fundamentally not passive data collection.
But debating the secrecy issue or limits if what constitutes a human subject are all besides the point: the research protocol had an adverse impact on humans involved with the study. Not matter any other considerations, that makes the research defacto one that should have had full IRB review. Evaluating the potential for adverse impact is literally one of the foundational reasons for the existence of IRBs. The presence of an adverse impact is defacto proof that an exemption should not have been granted and that a full review should have been done to determine how the protocols could be tweaked to mitigate the issue.
Yes, there are reasons that studies sometimes lie to their participants. These lies are something that has to be justified to an IRB and the study has to be designed to carefully minimize harm. Lieing to unwilling particiants only raises that bar. In this case, the bare minimum ethical way to conduct this study would have been a careful manual review of every unwilling participant that was going to be recieving deceptive communication to ensure they actually fell under the laws in question.
The compound of deception, legal intimidation, and scatter shot automated selection of unwilling participants is a particularly egregious ethical failure.
Seems like a career academic with no experience in the real world playing around like this is some kind of game. I'm sure they meant no harm, because they don't consider anyone "participating" to be anything more than a potential subject in their agenda to get a good review on their paper.
That letter and their social media posts are nothing more than a facade to maximize return with no consideration of impact.
Mayer has a JD and is licensed in CA (I don't know about NJ), has worked for at least one US Senate office, and has been so involved in actual practical privacy work that ad companies pressured the president of Stanford to expel him for his legitimate work on DNT.
The person who designed and ran this study is not Mayer. Mayer runs the lab, but this is a subordinate's baby.
From the study's website: "Please contact the lead researcher for this study, Ross Teixeira (rapt@princeton.edu), if you have any questions, believe you received an email in error, or would like to opt out of any future communication related to the study. The additional members of the study team are Professor Jonathan Mayer at the Princeton University Center for Information Technology Policy, who is the Principal Investigator, and Professor Gunes Acar at the Radboud University Digital Security Group."
I received exactly the same email with a different sender ("Anna Roland", a resident of San Francisco, California) and was also left quite paranoid by it. The email had an combative tone and felt like a legal threat.
Readit News
> Did an Institutional Review Board consider this study?
> We submitted an application detailing our research methods to the Princeton University Institutional Review Board, which determined that our study does not constitute human subjects research.
From the social experiment[as reported by OP's link]:
> I look forward to your reply without undue delay and at most within 45 days of this email, as required by Section 1798.130 of the California Civil Code.
I'm pretty sure most people would find that to be a thinly veiled threat of a lawsuit. I'd like to know if the review board considered the text of the email that the "researchers"[1] intended to send and the fact that they were likely to send it to individuals instead of solely publicly traded companies.
[0] https://archive.md/cSDGT
[1] Seems to fall more under experimental psychology to me https://en.wikipedia.org/wiki/Experimental_psychology?wprov=...
Not trying to turn this thread into a generic flameware against "academic" research methods, but this whole things seems oddly reminiscent of the "let's try to insert malicious code into Linux" fiasco [1]. I'm conceptually fine with generic passive tools like web crawlers to conduct research, but since when did the internet become a place where nonconsensual interactive research became fine?
[1] https://www.bleepingcomputer.com/news/security/linux-bans-un...
In a very real sense, every landing page A/B test is nonconsensual interactive research.
Or at least, if there is line between them, however blurry, I can't find it.
I am skeptical of the idea that such a line should be drawn according to who is doing the experimentation, I don't think that a manipulative act becomes okay just because it is being done by an academic for research purposes, nor do I think that it becomes okay just because it is being done by a layman with a profit motive (or a political one, for that matter).
IRB review is typically required even for just simple research surveys.
No. Absolutely not. A helpful reminder of the timeframe would be "the deadline for our study is ..., please try to send your response by then if you wish to be included."
Quoting legal code is not at all a helpful reminder of a timeframe, but is a direct implication of legal ramifications for failure to comply.
People don't go through the trouble of digging up the particular section number of the specific statute of the specific jurisdiction in question for the mere sake of a generic "helpful reminder of the timeframe" required by law.
And similarly for the "without undue delay" part.
It's either a veiled threat or a serious error. Either way, this study needed more oversight.
I wouldn't be sleeping well if I were involved in this study. There is no way an IRB could determine that this is not human subjects research if you're emailing people and asking them anything.
If I want to ask random people about the weather and publish the results in a journal, that qualifies as human subjects research and IRB protocols must be followed. Emailing people asking about privacy policies is definitely human subjects research. Either they misrepresented the study in their IRB application, or the IRB didn't do due diligence reviewing the application.
Imagine someone showing up to your office building pretending to have an interview , walking into the office waiting room, then walking out saying “oh, just an experiment!”
“It’s just an email” the ease of the mode of communication here is not super relevant to the action, right?
There was a Twitter thread of a number of in-house forwarding to external council which costs time and money. This study incurred monetary damages.
"Answer my questions within 45 days or I will sue you." That seems to read like a demand for a specific action.
The underlying ethos is that researchers should respect the people who are participating in their studies. People should know that you are conducting a study, the aims of the study, and choose whether or not they want to participate.
Do you have a citation for this? What I'm seeing from random Googling is that you have to be obtaining information about the person for it to count.
If I were researching, say, price trends in some commodity and I called up several companies' sales lines and asked for their current price that looks like it would not be human subject research despite the fact that I'm talking to a human to get each company's price.
If I were researching pay trends at those companies and called up their sales lines and asked the people who answered how much they were paid it would be human subject research.
If you called up companies' sales lines to ask for prices, that's just gathering facts.
If you called up companies' sales lines to see what happens when you ask them for prices for things they don't sell, or to see if they're willing to accept a bribe, or to see if they respond with different prices when you lie to them about who you are, you're researching human behavior.
In this case, they are testing what procedures, if any, companies have in place for handling CCPA and GDPR law, by posing as nonexistent customers and making potentially bogus and misleading requests under the terms of those laws.
This is like performing research on retail refund practices by going into a bunch of shops and seeing how they handle being asked for a refund for an item you didn't buy from there in the first place.
There's a more ethical way to do that study, though, which is to actually buy something from the store first, then go back and try to refund it.
Similarly, there's a more ethical way to discover how websites handle CCPA/GDPR requests, which is to use the website first, and determine in the course of that what possible information about you the website should have; then, within the terms of your rights under CCPA/GDPR, to contact them and make reasonable and legitimate requests to see if/how they are able to handle them.
The word "you" appears in every question.
To quote my IRB training (citiprogram.org):
"Most research in the social and behavioral sciences involves gathering information about individuals. However, some research that involves interactions with people does not meet the regulatory definition of research with human subjects because the focus of the investigation is not the opinions, characteristics, or behavior of the individual. In other words, the information being elicited is not about the individual ("whom"), but rather is about "what." For example, if a researcher calls the director of a shelter for battered women and asks her for the average length of stay of the women who use the shelter, that inquiry would not meet the definition of research with human subjects because the information requested is not "about" the director. If the researcher interviewed the director about her training, experience, and how she defines the problem of battering, then the inquiry becomes about her - and therefore "about whom."
The current example is similar, in my opinion, to "how she defines the problem of battering" which the IRB training identifies as human subjects research. The people receiving the researchers' email in the current study are being ask to define the way they interpret and comply with a legal statute.
I can accept that some people don't see the information requested as being "about whom" and therefore is not human subjects research. But the fact that people who have received this email have panicked indicates that the recipients, at least, felt that the questions were more than merely recording impersonal data about their websites.
When you lie about who you are, what your purposes are, and use scary legal language in an attempt to elicit a response, that is absolutely human research. You may be able do those things ethically as scientist but you absolutely need IRB review becausr it is definitely human research.
My guess is that the IRB in this case was not informed of the deceptive nature of some of the emails as lieing is absolutely a red flag that you are doing human research and not just information gathering. Indeed, evaluating such lies for potential harm is an important part of why we have IRBs for psychological and sociological research.
Like the semantic distinction doesnt matter because nobody gives a fuck about Princeton’s organizational policy.
It reads: "CCPA Scam November 2021"
Story update notes: "This is a human subject research study conducted Princeton University"
I was attempting to submit my own instance of this when I discovered ColinWright's. My suggested title was going to be "CCPA Scam ... is a human subject research study conducted by Princeton University".
Panicking small web operators without consent being the issue.
I was concerned that might be too much adaptation. Apparently not.
We’re some students from Princeton trying to understand how businesses are responding to CCPA and GRPR requests. Could you help us with our study? How would you answer these questions?
…
The point is disclosure. It’s unethical to do otherwise, especially given that is about the use of data. I’d love for there to be more data published about the impacts of these policies, but please don’t use the tactics of creeps in the process.
Possibly. And the IRB review process may allow for non disclosure under circumstances of that sort.
The problem is not merely that the IRB allowed non disclosure. It's that the IRB also granted an exemption from full review as human subject research. If the researcher expected that human behavior might change based on secrecy vs. disclosure then it is fundamentally not passive data collection.
But debating the secrecy issue or limits if what constitutes a human subject are all besides the point: the research protocol had an adverse impact on humans involved with the study. Not matter any other considerations, that makes the research defacto one that should have had full IRB review. Evaluating the potential for adverse impact is literally one of the foundational reasons for the existence of IRBs. The presence of an adverse impact is defacto proof that an exemption should not have been granted and that a full review should have been done to determine how the protocols could be tweaked to mitigate the issue.
Even assuming your premise is true (it's not), you think the solution to not have people lie is....to lie to them?
The compound of deception, legal intimidation, and scatter shot automated selection of unwilling participants is a particularly egregious ethical failure.
That letter and their social media posts are nothing more than a facade to maximize return with no consideration of impact.
Total negligence.
From the study's website: "Please contact the lead researcher for this study, Ross Teixeira (rapt@princeton.edu), if you have any questions, believe you received an email in error, or would like to opt out of any future communication related to the study. The additional members of the study team are Professor Jonathan Mayer at the Princeton University Center for Information Technology Policy, who is the Principal Investigator, and Professor Gunes Acar at the Radboud University Digital Security Group."
https://news.ycombinator.com/item?id=29539266
I was concerned enough about this that I updated our project privacy policy with pre-emptive wording about CCPA (now reverted):
https://web.archive.org/web/20211218125309/https://textpatte...
I'm mildly annoyed about the time I wasted on this, but I guess that in itself is anecdata for this study.