Ben Simo Presents The Power of an Individual Tester at STARWEST 2014
Amphibious, time-traveling, content-driven cyborg. That’s Ben Simo’s “official” title. Both a tester and developer, Simo has more than two decades of experience in software, and he wanted to make that clear before he began his keynote about the struggles of HealthCare.gov.
As someone looking for smart health insurance for his granddaughter, Simo had to use HealthCare.gov when it first launched in order to get the best possible policy. What he found wasn’t a list of prices and boxes to check. Instead, he was greeted with error screens, broken pages, consistent glitches, and a service that just wasn’t prepared for the volume of unique visitors that arrived.
It was an incredible mess, and one that veteran testers and software developers couldn’t help but cringe at. When the administration behind the website was asked for advice on how to log on and get signed up, they simply said to keep trying. Keep logging in and eventually, this thing is going to work.
If you somehow got past the error messages and broken pages, you were greeted with confusing messaging and poor account creation. Complex password rules, messy username instructions, and an encumbrance of pages that very often timed out followed. Timing out, of course, forced Simo to start the process all over again.
Simo wouldn’t stand for it. He knew he needed the best insurance for his granddaughter, so he put on his X-Ray Specs, donned his tester hat, and began to explore his options. He took notes, screenshots, and gathered data so that he could discover consistency heuristics, usability heuristics, and recognize the problem at hand.
Here are some of the initial problems. If you chose a username that someone was already using, the system gave you that information. That means if you were looking to hack an account, you now knew that yes, that account exists. If you used an email already taken on the site, it wouldn’t tell you that directly, but if you dug a bit deeper into the script, a message to the effect of “email in use” could be seen. Once again, that’s a security risk that shouldn’t be allowed in a final product.
Confirmation emails not being sent. Pages timing out. Cookie data too great for the page to be able to complete a request. Data sent over insecure HTTP. Email addresses revealed over the subscription management system. Returned stack traces that, once again, could reveal sensitive data. Simo found all of this without any sort of special access.
Simo started to Tweet and blog all of his findings, detailing the many failures of HealthCare.gov in order to get answers. One of his answers was “stay away from HealthCare.gov for at least a month.” His reports attracted significant attention from places like The Verge and Time, so the security flaws were getting out there to the public. Simo was now known as the “sort-of-skilled hacker” who figured out the website’s vulnerabilities, and people were rightly concerned about the security of their personal data.
People were told that hackers couldn’t get much, as the site stores little data. That, as Simo found out, just wasn’t true.
Names, birth dates, income information, Social Security numbers, addresses, family relationships, disabilities, ethnicities, deductions—all of this was asked for and yes, it was stored on the site. For not being a data collector, HealthCare.gov sure did store a lot of data.
Really, HealthCare.gov just wasn’t created how it was advertised. It was said to be like Amazon or Kayak, but instead, this website has an initial bottleneck where you had to create your account before you could shop. Instead, Simo suggested they should have gone with a funnel approach, where users could shop beforehand and then create an account once they found an ideal plan. That’s how Amazon works, but not how HealthCare.gov was designed.
What’s the website’s current status? It’s still not up to snuff. Usernames are now email addresses, which still isn’t fully secure. You can now submit reports, which is a step in the right direction, but that should have been a feature offered much earlier. It’s almost 2015 and the hope is that things will get better, but as Simo explained in his detailed keynote, HealthCare.gov is not nearly as secure and functional as testers, developers, and everyone else would have expected.
