Who’s afraid of Mechanical Turk?

Monday, February 20th, 2012

First of all, what on earth IS Mechanical Turk?  It’s a wonderful tool developed by Amazon.com, and I think one of the very best, and least used, tools out there for causes.  To understand the tool, it helps to understand the origins of the name.  The Turk was a hoax; an elaborate ‘machine’ which played decent chess against human opponents.  In fact, a very diminutive and talented player was hidden inside.

Amazon’s tool allows you to have legions of people execute simple, repetitive tasks that computers can’t (yet) do well.  People classify images, translate snippets of text, answer surveys, find bugs.  You structure your task as a simple set of instructions, and a simple web form for the results, and then you decide on an appropriate fee to pay the “Turker” for his or her services.

You could clean up address information, do prospect research, look for partner organizations, search for grants, test campaign language, test campaign images, and all sorts of other stuff, with an appropriately designed Mechanical Turk task.  One drawback is that you can’t target specific demographics, so you won’t know how closely respondents correspond to your community of supporters.

At HelpAttack!, we used Mechanical Turk a few times last year to gather data related to the social media communities of certain causes.  We had data from the IRS with name and revenue information, but we didn’t know which causes were active online, and how big their communities were.  Automating and parsing search results, or crawling the web, yielded poor results and would have taken a lot of effort.  Our instructions to Turkers basically said:  ”Google this nonprofit name, identify their main website, and see if a social media profiles is linked from the website.  Write down the URL and the total number of Likes or followers and enter them in the form.”

More recently, we used it for user testing.  User testing for internet startups is like flossing.  You know you should do it, your dentist says you don’t do it enough, and it’s easy to rationalize reasons not to do it right now.  We asked Turkers to visit a public pledge (“join me”) page on HelpAttack!, and we asked them how they thought our service works, why they would use it, and also why they would decide not to use it.  Getting 30 seconds worth of feedback from 20 people cost $5.  If you want to try this, check out Feedback Army, which makes the process easier.  Here are the results:

“How does it work?”

  1. You sign up for facebook or twitter, you give money to support disaster victims
  2. I believe that by using Facebook and Twitter and making a donation your friends will see it and it will encourage them to as well. Its the use of the social media to make a difference I believe
  3. This service would work to support causes and the American Red Cross
  4. You support ARC via Facebook and Twitter.
  5. We give our donation to the sites that except it for the Red Cross.
  6. You give authorization to the app to tweet or post to facebook on your behalf
  7. Contribute with each Facebook or Twitter update.
  8. You pledge money based on the number of facebook updates on the red cross page.
  9. YOu can pledge a specific amount to charity for every tweet or facebook update you enter.
  10. Not sure
  11. You tweet or use a facebook post to say you donated.
  12. Each time a person updates their twitter, a set amount of money is added to the account that will be charged to that person later, as a donation.
  13. I’m assuming it uses social networks to drum up money for charity.
  14. That Twitter or FB account is linked to a donation site associated with the Red Cross
  15. People pledge to give a few cents each time they make a tweet or facebook update and the money is donated to the American Red Cross.
  16. Frinds donate money
  17. No not really, but it might.
  18. It asks for an amount you’d like to donate and keeps track of every tweet or update so you can donate that amount per tweet/update.
  19. You pledge money via your facebook statuses/tweets.
  20. You donate money

Wow!  The good news is that most people got close – we’ve removed a bunch of words from that page over time, so it’s important to make sure it still makes sense.  (16) thinks that their friends donate the money, which may be true but not the primary focus. (17) either didn’t get it or didn’t finish the sentence.  (10) wasn’t sure.

Why would you decide to use it? (What do you like on the page?)

  1. “Bacause it’s easy; I have both Facebook & Twitter accounts”
  2. To help those less fortunate
  3. I would use it if someone I know was homeless or in need of blood or had another anomaly.
  4. Looks friendly.
  5. It would be faster for the contribution to get there.
  6. I think the message is important.
  7. See a person I might know at the top, and other people top left.
  8. It seems easy to use.
  9. It is easy, and the money goes to a god cause.
  10. To help people
  11. To tell people they should donate.
  12. The page shows clearly that it is related to Red Cross, a well-known organization. It also shows other people who are using the service. It also shows that you can set a limit.
  13. I like charitable donations in general, so I would probably give it a try.
  14. Seems like a legitimate site.
  15. People use and enjoy twitter and facebook, so it’s a fun way to help and be reminded.
  16. Contribute to a good cause
  17. I like it because you can see how much was raised.
  18. kinda of a cool way to raise awareness and donate at the same time.
  19. I think it makes it easy and it looks like a really interesting unique idea to raise money for relief funds.
  20. I like the social features and the orange buttons

This isn’t the best question, because it assumes the person will like at least one thing about the page, and the fact we are paying them (even a little bit) might bias the answers. Still, the reasons listed correlate fairly well with the answers we imagined people would give.

Why would you decide not to use it? (What don’t you like?)

  1. I’ll share my basic personal info as soon as I donate using my accounts
  2. Dont want to seem as if Im doing it just to make myself seem better than those who dont give to the cause
  3. I would not use it if the money did not go to the right people.
  4. None.
  5. If I did not it would be the lack of funs.
  6. If the tweets were more than i signed up for, as in spam my followers, I would stop using it.
  7. Don’t know how much, or what it might cost me to do it.
  8. I prefer to make fixed donation to charity and get the match from my employer.
  9. It could be really easy to donate more than you want if you don’t pay attention.
  10. The page was too cluttered, too confusing.
  11. If it automatically shows how much you donated, I wouldn’t want to use it because I don’t like showing off.
  12. It lacks information about how will the payments be made (logistics).
  13. I don’t know if it’s trustworthy in 30 seconds. I’d have to do some research. It looks a little clustered and confusing.
  14. It don’t use twitter or FB
  15. Me, I don’t send out tweets or use my facebook status to communicate, so it wouldn’t work. But I did, I guess the problem is it relies on the honor system. Or does it?
  16. Need more information or direct donation
  17. I dislike it because it tells you how much people pledged.
  18. having to create yet ANOTHER account for something. it’s not a big deal though.
  19. I’d definitely use it. I think it’s a brilliant idea. =)
  20. I don’t have spare money

(4), (5), (20), and (19) aren’t so valuable: Either they didn’t find a problem, or we can’t address it.  The rest are extremely fascinating, particularly those concerning how the supporter will be seen by other people, and Turkers reacting to seeing specific amounts given by others.  (12) is something we hear a lot, and probably the first real change we can make to the page based on this feedback.