In order to do this, I enlisted a small but dedicated cadre of similarly interested folks. Their help is greatly appreciated, but I will not be naming any of them. While I consider what I did to be a legitimate exploration of an essentially undocumented process, Google may not feel that way about it and would be within their rights to suspend my account as a result of this. I would prefer that any such response fall on me, personally, rather than on any of the otherwise innocent individuals who gave me some assistance in pulling this off.
My goal was to determine a few things, namely:
1) How hard is it to get an account with a legitimate name suspended if that's what you want to do?
2) What actually happens when an account is reported? What evidence needs to be provided?
3) What safeguards are in place to insure that the reporting system isn't abused?
In order to find these things out, I decided to take the following steps documenting the process throughout:
1) Create a bogus account with the same first and last name as my own account. This is hardly unusual. I am the original Gary Walker on Google+, but I am not (as a quick people search will indicate) the only Gary Walker on Google+.
2) Fill out a simple profile for the account with no actual PII. Simple enough. Accounts require some fields to be filled out, but they don't have to have anything useful or realistic in them.
3) Have the account reported, initially as a "Fake Profile" and then if no action resulted, for "Impersonation."
4) Attempt to have the account re-instated.
So without further ado, here is the my dive into the black box of Google's profile reporting and suspension process.
Part 1: The Setup
I wanted to create a profile that was completely devoid of any kind of indication of ill-will or incriminating activity. With this in mind I created a new, non-Google email account and had an accomplice send me an invite to Google+. We are all familiar with the requisite hoops that have to be jumped through in order to do this. They were overcome in exactly the fashion you would expect.
Once the account was created, I filled out this profile:
Hi,
We've received a report indicating that your profile on Google is in
violation of our policies. We've temporarily blocked your profile and
removed it from search results. In order to remove the block on your
profile, we need to verify that your identity matches the one described on
the profile. Please fill out the form at
https://www.google.com/support/accounts/bin/request.py?contact_type=identity_verification
While your profile is blocked, only you will be able to see it, but we
won't delete any information you've included. Once we've verified your
identity, we'll make your profile visible again. Please note that Google
needs this document only for verification purposes. We will not retain or
share any of the information given. Feel free to blur or block out any
government identifying numbers. We only need the name and photo on the
identification provided.
Sincerely,
Name Removed
The Google Profiles Support Team
That was it. All it took was the provision of a (very) fake ID scan to have an account suspended.
I decided to fill out the reinstatement form which, as the email indicated requires a photo ID scan to be submitted. I figured, let's find out just how pointless this process is, so I sent them this:
You will notice that in addition to all of the obvious problems with the earlier "scan" this one has replaced a legitimate picture of me with the mug shot of Arizona multiple murderer, Jared Loughner. I've also changed the last digit of the ID number to an "8."
I submitted that ID about 30 minutes after the account was suspended on August 5th. The account was re-instated at some point after that without any notification to me. Seriously, Google? Seriously? I sent you a famously fake ID with a spree killer's picture and you turned the thing back on? I honestly don't know what to say about that. Are you asking for the scans for fun? Do you look at them at all? I am very disappointed.
Time to revisit my goals:
1) How hard is it to get an account with a legitimate name suspended if it is your intention to do so?
Not very. Not at all. I was able to get an account with no particular infringing information or activity suspended by providing a GIMP'ed version of the McLovin ID from Superbad. I was able to get the same account reactivated with a picture of a mass murderer. In the grand interwebs vernacular, What. The. Fuck? That said, I have to credit Google with the fact that it did require me to take such an action in order to get the account suspended. This leads me to believe that people using pseudonyms which don't step on the various community standards who have been suspended may well have either a) been suspended due to reasons other than or in addition to their names or b) have been reported by other users.
2) What happens when a profile is reported?
3) What safeguards are in place to prevent the system from being abused?
Depends on what it's reported for. I was very pleased to see that there is a difference in the reporting mechanism for "Impersonation" vs "Fake Profile." Unfortunately, that difference is what is known in the industry as "Security Theater." It does not, actually, hew to any kind of reasonable standard for validation. Are there "safeguards" against abuse in place? Yes. Of a sort. Are they adequate? No. Absolutely not. Is there any way to change that? I leave this as a question for Google to answer.
Google is trying to do the right thing here. They're trying to protect the validity of accounts, to prevent abuse, etc. I recognize and applaud them for making the effort. Alongside that recognition and applause, however, I have to point out that whatever process made the debacle I just described possible needs significant improvement.
I would make one final observation. Nothing that I have seen indicates that there is an automated process which suspends profiles whose names are "normal." Nothing. I do not say that there is no ban-bot. I say that if your name complies with the community standards, the ban-bot will not get you.
Where's the beef?
ReplyDeleteIt wouldn't be shocking to learn of "special" characters or mixed character sets triggering it.
ReplyDelete" people using pseudonyms ... who have been suspended may well have either a) been suspended due to reasons other than or in addition to their names or b) have been reported by other users."
It does seem likely that it's due to reports (the "Fake Profile") report. I wonder if your tests would have gone differently if you used an obvious pseudonym and an avatar profile picture...
I am certain that every Google employee reading your tale will be simply thrilled that the people responsible for this control 25% of their bonus this year.
ReplyDelete@Winter - you mean, as opposed to a WASPonym?
Winter, that would be why I specified "...people using pseudonyms which don't step on the various community standards...".
ReplyDeleteI'm sure obvious pseudonyms or names with verboten characters etc. are detected automatically. I'm similarly pretty sure that they aren't suspended automatically.
The problem is not that the people doing this are stupid. They probably aren't. The problem is that they are being asked to enforce a policy which can't be enforced in any sane fashion.
Without an Interpol database of all internationally acceptable forms of photoID this could never work as a foolproof method of verifying anyone's identity. If I recall myspace used to demand users posed holding up a card with their username if they claimed someone had stolen their pictures or impersonated them... Which was tricky if they were stolen pictures from their youth and they looked completely different...
ReplyDeleteI suspect the very act of having to submit something at least vaguely reminiscent of a photo ID is by itself a decent safeguard against abuse, in both directions.
ReplyDeleteHow? The same way a captcha can be: you need to make an effort. Most griefers that want to take down somebody's account would find it too much work to even make an obvious fake like yours. I wouldn't be surprised that the second or third of tenth "impersonation" or other report from the same account will be examined with rather more vigilance than the first.
And a spam account holder will, I guess, be unlikely to go to the trouble of submitting any ID - chances are they may never notice the alert email.
I will say that two of my friends were suspended even though they were using their real (ie drivers license) names. We can't tell if it was a ban-bot or they were reported, but both were asked to submit id to prove their identity. Sai was denied, despite showing his actual ID. Pip refused to submit his ID because he felt it was improper of the Goog to require ID from certain users but not others.
ReplyDeleteWhat's frustrating is some support staff seem to be using common sense when responding to appeals, and others aren't. Your comment at the bottom sums it up perfectly "The problem is that they are being asked to enforce a policy which can't be enforced in any sane fashion. "
Thanks, Doc.
ReplyDeleteWow, so it's vastly easier for a fake person to prove themselves, they can just dummy up a really bad fake id. A person who's legit and doesn't want to lie, but whose common use name isn't on a piece of plastic, is punished.
ReplyDeleteI gotta say, I was better trained to spot a fake ID when I was the ID checker for the beer hall at the WOW Hall (http://wowhall.org/) as a volunteer than these yahoos (can you use that term for a googler? :).
ReplyDeleteYou know with the quality control (or lack thereof) involved, it makes me wonder, do you think they outsourced this, with the anticipation it was no big deal, and now they're having issues getting the messaging through to some digital sweatshop somewhere?
ReplyDeleteShava: I expect this is being done by contractors, like the people who scan books for Google Books. I don't know that, but that's how I would do it if I were them. I similarly suspect that the contractors don't have any clear guidelines as to what is and is not acceptable...and how could they with 25 million users from God knows how many different countries? They are being charged with enforcing an unenforceable policy. They deserve our sympathy rather than our ridicule, IMO.
ReplyDeleteWhatever positive reasons Google might have for their policy, it has been clear for weeks that the administration of said policy is all over the place.
ReplyDeleteMore importantly, in a trial, your first response to users (many of whom have already been using your products under the names they are used to which have suddenly become no longer acceptable) should not be to suspend them. At the very least, they should be provided warning and options to make a change before resorting to suspension. This was a promised update two weeks ago from Brad Horowitz, but it has not occurred.
I find the fact that they continue to suspend users in a trial while the administration is all over the place to be disgusting company practice. Simple customer service and PR failure on their part.
When this issue came to light 2-3 weeks ago, the immediate response should have been to stop all suspensions until Google could internally figure their shit out.
I wonder if I submit that same ID with my picture on it, if they would reinstate my profile....
ReplyDeleteIt sounds like you solved everyone's problem for them - just create a fake ID with your username and get your account turned back on!
ReplyDeleteAnd, to be clear and release you from liability, no, you did not actually say or advocate any of that. But I bet it would work anyway.
Hmm, interesting .. Now it explains why there are so many Lady Gaga`s on + that haven`t been closed down..
ReplyDeletejoreth: Correction. I've solved everybody's problem except Google's. :)
ReplyDeleteHmm "your photo ID" and "valid ID"... I'm not seeing anything about government issued ID there. I think we need to set up a Blog ID provider. It's not fake, it's a validly issued photo ID, it's just not a government issued one, and it will not serve as proof of age. Hell, I've got a few convention ids that could work as a template.
ReplyDeleteBrilliantly executed sir. I know this is serious business, but it gave me just the laugh i needed today. Hats Off!
ReplyDeleteGryMor: Good call. I think we need one that puts the user's name on the McLoughner ID automatically. See how long that goes before the big G starts caring.
ReplyDeleteI have to wonder how easy it would be to get someone who is really popular suspended using these tactics. Also, I can't find you - Gary Walker - on G+ - there are too many of you!
ReplyDeleteIt's dangerous to Google+ Alone! Take this.
ReplyDeleteDanger Kirk
I appreciate that people are looking into this. That outta the way, it seems like a very good compromise to me. On one hand, the very trivial abuse will be curbed (spammers that don't have the care or time to fight back) and multiple complainers will likely be rate limited, and real complaints will feel like they're heard quickly, and the odd mistaken/incorrect complaint is easy and quick to fix.
ReplyDeleteOn the other hand, if there's a major problem with back-and-forth battling, THE THING YOU SUBMITTED HAS YOU AGREE THAT "I declare, under the penalty of perjury, that the information in this notification is accurate". That should make the court case far easier. Can you imagine trying to sue google now for not taking down that infringing user (ie. now that you knowingly submitted false identification and signed a digital contract declaring it was not false).
Seems like a really good CYA implementation that even has some benefits for the day-to-day use of it.
To the users that don't want to submit their ID to prove who they are... well, that's the whole point, right? And if you don't care about your profile that much, then just create a new one or lie to get yours back - but if you do the latter, don't expect google to CYA for you.
In reply to Jim Stevens:
ReplyDeleteHi Jim,
You can't perjure yourself unless you are sworn in, either in court or in a discovery procedure for examples.
I suspect that none of the folks filling out that paperwork have been sworn in before they filled it out and there probably isn't any official witness seal on the documents they send in either, so the "official-iating, turror-estatic" verbiage of under penalty of perjury is totally worthless. its just another "Appeal to anonymous authority" (w/apologies to Lois Mcmaster Bujold)
Priceless! Well done article and it shines some light on my story. I had my developer account of 3 years suspended a few weeks ago for no reason.
ReplyDeleteStore here: http://moonbeamdevelopment.com/?name=google-android-developer-suspension-notice
When I worked for *redacted but not google*, we weren't required to do anything with photo ID's sent to us other than match the name. Once the name was matched the data was thrown away.
ReplyDeleteHaving said that, the amount of data Google has probably rivals that of *redacted* so they should be matching the address on the ID card if it's a DL. That would at least reduce "impersonation" by simply matching the drivers license address with the profile address, if it has one. If there's no address or direct contact info in the profile, then they really have no grounds to block it since it doesn't violate the privacy of the person being impersonated.
So what happened here isn't really of any surprise.
Did you make both profiles from your home? I'm fairly certain they IP stamp profile access. I know they check email access because they have notified me when I accessed my email from an unusual IP.
ReplyDeleteYou do realize that this is a beta product in development and compiling info from users to improve a product and also wary of people (and corps employing ...gods I guess the term would be moles and sabotuers though that seems a little melodramatic) with less than good intentions, does seem to require a minimal amount of caution and suspicion.
ReplyDeleteAlso Gary this: "I expect this is being done by contractors, like the people who scan books for Google Books. I don't know that, but that's how I would do it if I were them. I similarly suspect that the contractors don't have any clear guidelines as to what is and is not acceptable" --expecting, not knowing, and suspecting doesn't seem to fortify your position
I guess I probably sound like a shill because I have a gmail account, though as a shill I guess I should use more than just Google Earth to look at your Mom's house to see if she needs a piece...
good article, shared it (of course on G+ ;))
ReplyDeleteSo, I had my account suspended for using a pseudonym. I actually had noted my nickname very much in the same way that people name their nicknames on business cards or grave stones (eg. Robert "Bud" Walker). I think the combination of using special characters (quotes -- I don't consider those special and was just conforming to Chicago Manual of Style) and a dictionary word (in my case "Normal"). I've detailed the whole thing on my blog: http://normal1.typepad.com/blog/2011/08/google-blocked-my-name-because-i-was-being-honest-for-the-first-time-ever.html
ReplyDeleteI was actually moving everything over to Google+ from Facebook, including all my friends, my pictures, and everything. Being blocked was frustrating because I was finally getting social currency in my amusing animated gif cat posts... har.
I was reinstated within 7 hours because:
1. someone I know has a friend who works at Google and he called them and told them that a. that was my real nickname, and b. I convince my friends to do a lot of crazy shit, not the least of which is joining a beta social networking site.
2. several of my friends started forwarding around that blog post and encouraging others to share it.
Basically, the kindness and geekiness of my friends got me reinstated.
I'd say all of this process is automated, there's no human on the other end. But it could be worse. At least Google aren't PayPal, who: (1) automatically freeze your account and maddeningly ask for one document after another if you do any one of a thousand flagged actions eg try to change your country address and type of account; (2) only process through software, customer support do not intervene with The Machine in any way and only humor you; (3) tell you to upload document scans when their upload facility doesn't work so you have to fax them (who still has a fax?); (4) will not correspond by email, all emails are from a bot.
ReplyDeleteThanks Gary,
ReplyDeletethe problem of Identity on the Internet has been neglected far too long. For an approach attempting to put the user back into the driving seat check out the "Identity Commandments" by The Jericho Forum.
http://www.opengroup.org/jericho/Jericho%20Forum%20Identity%20Commandments%20v1.0.pdf
By doing this aren't you committing fraud?
ReplyDeleteI mean, while the goal is constructive and illustrative of some issues with Google+, haven't you opened yourself to at least misdemeanor prosecution, at least in Hawaii (for the fraudulent d/l's) and your own state, very possibly there could be federal charges for interstate communications.
While the likelihood of prosecution is extremely small, the fact that you did it should prove at least a bar for the bulk of casual actors - insofar as any subsequent activity by a fake profile then could be argued to be based on the fraud, it would protect Google from prosecution/liability.
Seems like a decent, *adequate* system for their purposes. The point being: to discourage casual fraud, and if it's committed, make the task challenging enough that it provides legal exit for Google from liability.
I have maintained an account of facebook for several years under the name of a 1950s sit-com character. There are at least 6 other accounts with the same name and 4 are obvious phonies.
ReplyDeleteThere is no reason for them to care unless it is wide-spread enough to cut into the bottom line. Until these assclowns can ensure our privacy there is no reason to provide them the information they want.
It seems to me that there's already a simple process, at least in the US, that Google could use to handle this. They should just make users print a document affirming their identity and have the user get it notarized. That way a legally-trusted third party has already done the verification for Google, in person.
ReplyDeleteThe whole idea with G+'s ID verification isn't to add genuine "security" any more than locking my front door adds any genuine security to my house. Anyone who really wants to get in can get in. And I don't mean highly trained burglars - literally anyone can get in. What G+'s verification does is increase the "activation energy" required to impersonate someone.
ReplyDeleteWhen I lock my front door all that I'm doing is making it a little harder for someone to get in my house. This discourages what I'll call the casual opportunistic criminal - the person who won't go to a lot of trouble, but if there is a ready opportunity he'll take it. By far this class of opportunists makes up the greatest statistical threat to my house, so locking my front door is effective in discouraging most would-be burglars.
This is exactly the same thing G+ is doing. Creating a fake of a government-issued ID is a crime in most states. G+ is increasing the activation energy required for someone to impersonate you. Anyone can still do it, just like anyone can get in my locked house. But it takes a certain amount of initiative, and increases the risk to the perpatrator and this will keep most out.
Google isn't stupid. I'm sure they know that their ID system is about as really effective as a locked front door in keeping out someone who really wants to impersonate you. I suggest they have done a good job at creating a system to keep out the largest group of would-be impersonators.
Keep in mind that Google also needs to discourage the people who falsly report impersonators. I'm guessing that you scratched just the first tier response and that there is a next tier. It's is hardly likely that if someone was seriously impersonating you that if you saw that person's account reinstated that you would sit idly by and say "oh, they must have submitted valid ID of me, so that's ok now". I'm sure you'll report it again and/or make more noise which will elevate the whole issue to the next tier where the ID submitted on both sides is more seriously scrutinized. At the tier you reached, I'm sure it's just a casual look by someone real - if it was even looked at by someone real. OCR could have done it all without a human ever looking at it.
this is not about security at all. it's about money and data mining everyone. google only cares about having accurate data compiled. that's where the money is, they don't give a shit about anything else. anything positive that comes from google is a by-product.
ReplyDeletei have a great story about a girl who trusted a large corporate brand to understand the online community that their social platform business targeted. surprisingly, yet now days typically, they sold everyone out in their bid to be "boss" and then stripped her of her hard won online name and reputation among family, friends and those in the game industry who had come to know (her). is this a fairy tale ending where the prince comes and restores her to her former glory as the crowd roars? or is this the new end of times, and she fades into the dull, colorless world slightly wounded, going by an online name that means nothing to anyone, especially her.
This comment has been removed by the author.
ReplyDelete"I declare, under penalty of perjury, that the information in this notification is accurate"
ReplyDeleteYou see to have left that bit out of your article. Oh and did you just admit to deliberately breaking a contract that you agreed to.
"I decided to make it even more sketchy by changing the dummy account's profile pic to my own "Keep Calm and Post Cats" one."
Oh how scientific of you. I wonder how they possibly though the account was trying to pretend to be you with it having the same name and same profile picture. I got the feeling from reading this article that you're not all that bright.
Regarding "Oh and did you just admit to deliberately breaking a contract that you agreed to.":
ReplyDeleteWell, actually, I clearly show that in the picture. I didn't feel it to be worth mentioning as it is a "shrink-wrap" type of adhesion contract, the validity of which is not a settled question in US jurisprudence at this time, but which I do not recognize. You are quite correct that I violated such a contract if it is valid and Google is more than welcome to attempt to get damages for that...an act which would generate even more publicity about this farce than already exists.
I recall there is at least one south-east Asian country where people have no family name… I mean Americans despite their supposedly multi-cultural background are already brandmarked to be culturally insensitive clods, but why does Google, a global supposedly multi-cultural megacorporation has to be just as short-sighted?
ReplyDeleteAlso what about Celebrities? Who the f*ck is Stefanie Germanotta, or Rogers Nelson (TAFKAP)?
Aahh, you mean Lady Gaga and Prince? DUH.
Wow, the ID trick is an ooooold Security-Theater tactic. I worked at a grocery store customer-service counter in 1998, and when people came to make suspicious returns*, we would require them to provide a DL that we made a copy of. We did nothing with this copy. We may have thrown it away. It was to deter people from scamming and make them paranoid that we would provide the copy to the police.
ReplyDelete* Usually people returning very expensive, relatively small items, especially if you seem to recognize them as having made similar returns in the past. A simple scam is to purchase something small and expensive, and then to come back and steal it, and then return the stolen one. And, back then, if the cashier wasn't smart enough to mark the receipt, rinse and repeat as often as you can (now barcoded receipts and POS systems that track returns mean this kind of scam is limited).
I saw this most often with Polaroid film. My kid won't know what that is.
/tangent
So Google want a photo ID (minus the identifying official stuff) to prove ownership of the profile.
ReplyDelete... and yet when you sign up they don't ask for a stock photo of yourself to check this ID against.
What if you live in a country like the UK where there is no official government ID, no requirement to own a passport or driving license or other government provided ID with photo on?
Nice one Google, I see you thought this through properly.
Leads me to think, what's to stop you from reporting all the Gary Walker profiles and taking the all down? Highlander: There can be only one
ReplyDeleteThis begs the question: what happens with repeated reporting/suspensions of an account? Does it become more difficult to verify? Is there some form of escalation on the back end we don't see?
ReplyDeleteFurther research required.
This just proves having computers do most of the work does not work at all. That is why I think unemployment is high on top of the greedy companies. But with computers scanning stuff like this or resumes for job openings , nothing will be like it was. Good write up.
ReplyDeleteHaha Rainyday Superstar, I was going to suggest you submit a fake ID, but you're already here...
ReplyDeleteWhat I don't get is why they banned you, but I went on long tirades about it and changed my name 6 times without any consequence whatsoever. "Oh, don't touch him, he's making a big deal out of it!"... idk.
LOL! Too bad they did not Google image search your photo ID. That would have been the first clue.
ReplyDeletecoolbettycakes, I'd be interested to know this woman's story. Woman game developers are given insufficient publicity as it is.
ReplyDeleteBluejay
My account got blocked whern I tried to use my normal online name. what drove me potty was that there was no notification email or other message, but they blocked me out of my email as well - and sincei use a smartphone to access email, I ended up resetting the phone to factory settings s i assumed the problem was the phone not the account - only when I got to a pc did it say that it wanted me to reset everything.
ReplyDeleteI still don't understand locking out the email, but iwthout *saying* you are locking out the email.