Is A/B Testing Fundamentally Unethical?
Facebook created a controversy recently by sharing the study results of an experiment it did in the news feed of around 700,000 users. The basis of Facebook's study was to prove whether emotions are contagious on the social network.
For the study, Facebook altered the number of positive and negative posts that the sample population saw and measured what kind of emotions the users posted after seeing the post. In short, it was a version of an A/B test—a randomized experiment between two variants that's done to draw a hypothesis.
The nature of the test and its results recently made public raised a crucial concern over the integrity of A/B tests happening all over the web. Are these tests really ethical?
There were some problems with Facebook's approach to A/B testing:
- The testing was apparently done without the consent of participants. People sign the terms of service agreement during account creation, but it is actually a weak form of consent because people don’t usually read lengthy terms of service before accepting.
- Facebook didn’t explicitly inform those under study. So the people studied were unaware that their behaviors were being tested.
- There was obviously no way for the users under test to opt out of the study.
Combining these three points with the speculation that Facebook only did an internal review before deciding to go ahead with the tests adds fuel to the fire that not all was morally right with these A/B tests.
On the contrary, keeping the larger picture in mind, such experiments are happening all over the web without our knowing. Most companies rely on the data from such experimentation to provide a better user experience, which leads to better engagement and raises the potential for earning more revenue.
As clarified by a Facebook researcher, the impact of the A/B tests on users was minimal. The study didn’t even conclusively determine that showing more negative posts actually led people to share more negative news. So, it is likely that the impact of this A/B test was overblown.
Most companies—even outside the realm of IT—use advertisements that touch us emotionally and influence buying decisions. Could these be considered unethical?
Whenever we talk about a complex topic like ethics, not everything is black and white. The grey area here lies between doing A/B testing for earning revenue and the potential impact the tests may have on user behavior. The grey area lies in testing to see what a user will like versus testing to see if a user will be unhappy.
Facebook COO Sheryl Sandberg said that the company communicated badly on this issue and that psychology experiments on users were just product testing. Arguably, the evident problem was not really bad communication but rather the lack of communication when the tests were being conducted. A/B testing may not be fundamentally unethical and is here to stay, but there is a need to bring in structure and accountability when such tests are taking place.
Do you agree?
