And on the last day…

I thought that on the final day of voting it would be worth recapping the “experience” which has been our listing as a PayPal X Developers Challenge finalist. For us there are statistics which, to us, indicate that something has not been quite right with the way the contest has been run. These are;

  • PortaPayments was featured in the PayPal FOWA Presentation.
  • PortaPayments was recommended in one of the largest on-line Android blogs (Android Guys), and that article was re-tweeted and re-used on other sites.
  • As of today PortaPayments has clocked up over 800 downloads and has over 350 regular users.
  • We’ve been approached by companies and individuals who have ideas for partnerships and expansion.
  • We’ve received overwhealmingly positive feedback (4 stars in Android Market, several comments that it’s a great idea).
  • PortaPayments managed to get less than half of one percent of the votes (yup < 0.5%) in a contest with 54 other entries

Yup, that last one is the shocker, especially as the PayPal X developers challenge is described on it’s post-login home page as;

the contest that rewards innovative apps using PayPal’s new Adaptive APIs!

Personally I expected an app PayPal chose to feature, has gained a fair number active users in a short space of time, and has had expressions of interest from 3rd parties to do a little better than receiving less than a third of the expected average number votes for an entry  (i.e. the number of votes you would expect if all the votes were shared evenly between all the projects).

The reason?, well, it could be that it’s a bad idea, but the feedback we’ve received and the interest by 3rd parties makes us think that probably isn’t the case.

It could be that people who like it had trouble voting. This one holds some water because we’ve been told by multiple people they’ve encountered problems signing up to x.com and finding our entry, or that they simply didn’t want to open another account with another site just to vote. This is highlighed by the fact that less than 5% of the people who visited our entry-specific “How to vote” page actually completed the voting process.

Oh, and in case you’re wondering, at the time of writing the actual number of votes is 25. Yup, only 25 people completed PayPals voting process for PortaPayments.

But that should be a universal issue for all entries and not one that will penalise just us.

Or, what we believe is more likely, is that there are fundamental problems with the competition.

I’m going to highlight some of the top issues we have with the contest to show the kind of things we’ve been up against;

The rules have been ignored. We identified several applications which we believe don’t comply with the rules. These range from entries being allowed in despite missing clearly stated deadlines through to not having fully working applications and not meeting the requirements for the entry video. This particularly hurt us because even though the rules clearly said entrants had to have a fully working application there are 3 other “Scan & Pay” based entrants who do not have a fully working scanning application, and so votes for the Scan & Pay idea have been spread between our fully working app and three others that don’t fully work.

“Incentivized voting”. PayPal have admitted it’s been going on, yet appear to have done nothing about it which has most likely caused other entrants to “incentivize” their voters just to stay competitive. Up until recently the total vote count was visible, but there were a couple of days where the vote count jumped by around 20% or so, and at no point has the total vote count or the number of entries gone down, which, to us, indicates nothing has been done even where PayPal knew voters were being incentivized.

Possible misleading statements over the voting process. PayPal sent out an email saying “Please note that an x.com account is required for a vote to be valid” and “The issue you’ll run into is that the votes are only counted if the user starts at x.com, which makes it tough to direct link to your app.”, but I’ve been contacted by some entrants saying that’s not the case and that it’s possible to by-pass the sign up and be assigned a “Guest” account ID in order to vote.

Vote counting issues. The contest has been strictly a “one-user one-vote” contest, yet the total vote count statistics (which sometimes are hidden for some reason) have, on one occasion we’ve seen, showed around 300 more votes than users. PayPal have said this was a “caching issue” with the user count, but given the other problems we’re not entirely confident of this answer.

Now if this were just a few hundred dollars, or a pat-on-the-back contest then it most likely wouldn’t have cause so much anger amongst its’ entrants, but it isn’t. We’re talking about prizes of 50,000 and 25,000 dollars in cash and the same again in service fees, which is more than enough to take most of these ideas to the next level of development.

So folks, if you’re thinking of entering a contest in the future, even one with a big prize, think carefully, because you might find that all you end up doing is giving PR to the company running it, and getting little more than a whole heap of stress when you realise that the competition is being run like a dodgy election from a third world country.

9 thoughts on “And on the last day…

Add yours

  1. 8 is enough?

    First, forget everyone else, the fact that you were included in the Paypal FOWA presentation and the AndroidGuys’ blog resulted in 8 votes or less. You wished that Paypal’s implicit recommendation and the blog’s social media had resulted in more but it didn’t. So maybe your app isn’t as innovative or interesting as you think. Just 8 votes from the developer community with all that free PR. How do you explain that?

    Surely you don’t think the developer community is being bought. The “cheating” is coming from people who are just being paid to sign up and vote.

    You got 8 votes based on the merits. Was the winner supposed to get 10?

    1. Can I suggest you actually read the post before commenting. You’ve made several comments which the post explains, and the post contains an updated vote count which you seem to have missed.

  2. I read the post and other pages. No where do you explain the expected percent of votes received from the first two main points of your plea.

    And, now, as a user, having to click through three other posts or pages to dig deep and find that maybe you are up to 25 votes, still doesn’t explain the lack of traction you got from your 1) Prime placement w/Paypal, 2) PR mention, and 3) presumably happy avid customer base.

    Would you address my main comment – that comparing yourself to others doesn’t explain the lack of votes you got individually.

    1. If you read this post you’ll see that I have stated how I calculated an expected number of votes for an average project, and as most projects haven’t had the benefit of the prime placement and PR mention I’d expect us to be at least around average.

      The vote count in the title of the post you mention was correct at the time of writing. The post also clearly shows the body was updated later with more recent statistics.

      I’m afraid I’m going to have to reject your further posts until your posts start indicating you’ve actually read the information you’re commenting on and usnderstand there is a period of time involved here, and all of the posts were not just thrown out at the same time.

  3. Al

    I will chime in here in your defense since I selfishly read your blog and have not contributed yet in the comments.

    We too had a completed and submitted app long ago and followed the rules (we were in fact the first Dev Conference in November with an app).

    Your app is great, don’t let anyone tell you otherwise. While my vote count was a bit higher but still way below the “average”. I think my greater vote count, as anemic as it is, had to do with my crafting of a 2 step method of explaining how to vote (took me two hours to craft the email) and because I took the time to personally handhold some people through the process. It was not easy.

    If it walks like a duck quacks like a duck then it is a duck. Someone cheated. I would say that some companies in the contest miscalculated and used Mechanical Turk to vote and screwed up by getting too many votes. The lopsidedness of the results will make that chrystal clear. That was the “incentive” we were told about. There were around 4000 fraudulent votes when we got the “incentive” email iirc. I imagine one more company jumped on the bandwagon based on a sudden 1000 vote spike. Knowing my vote count at both times and the final vote tally I would imagine that the average vote count was probably somewhere in the 30-60 range.

    I have not seen anyone comments stating “Ihave 320” or “550” anywhere so I think it is a good gess. Maybe it is sour grapes, I doubt it highly.

  4. We have also been complaining to PayPal about how the competition has been run. There were many problems, but the worst from our point of view was how we were repeatedly told to have voters go through x.com to vote. This was an extremely painful process, which many, many people aborted.

    Despite this, a few entries were only using direct voting links. Ie. links directly to the page where people could vote for them. This bypassed the whole x.com thing, with the registration, the clicks, the logins and the searches, and required only a PayPal login. You got a guest screen name. This is an order of a magnitude easier than going through the officially sanctioned method — at best it required only one click. Indeed, these were not supposed to have been counted as votes, but I have very high suspicions that they were, based on the total vote count.

    PayPal has not responded to our complaints about this and have not released vote counts for the entries. We are left in the dark, angry and speculating.

    If other criteria were used to select finalists, we have not heard what they might have been. We do feel we were one of the more ambitious projects, and were really pushing the limits of the APIs. To be dropped in this fashion, with so many irregularities and oddities, is beyond disappointing.

    FWIW, we got 124 votes. Our ‘conversion rate’ was a bit better than yours, probably mainly because of Finnish tenacity, and maybe our guidance, but we’re not sure. Getting people to vote through that heavy process used up quite a bit of social capital! I think we got some random developer community votes, but I doubt many were given to any of the entries. The registration count went from 600 to over 7000 in a matter of days, and I hardly think loads of developers suddenly had the urge to sign up… It’s much more likely those figures came directly from voters linked to the entries.

    Of course actually deciding any competition as important as this with a public vote is probably not a good idea.

  5. Kristoffer already summed up our (Scred) thoughts on the issue, but on my behalf as well, it just makes me mad that PayPal did not address the issue of direct links to individual voting pages. That was a blatant violation of the guidelines given and none of those votes (as per rules laid out by PayPal in multiple emails) should count. Many of the people doing direct linking had bit.ly links so the evidence is all there.

    How come PayPal can’t clarify that those votes that did not come via x.com were not included in the results? I mean, every indication is that they *were* indeed included.

    Community votes usually suck, but this one sucked especially bad. Voting process was extremely painful if done by the book, and yet a loophole was left open to rig the vote, and not addressed in any way.

    I’ve always been a big believer in PayPal and considered much of the criticism they receive as overblown. And yes, we’ll continue to work with their APIs. But come on, is it even possible to run a voting process that is less transparent and less trustworthy than this?

  6. Oh, and what are the eight other winners? Twittersphere has confirmed Mixcloud and Appbacker, but other than that – nothing. That’s just plain weird. (Or maybe I’m mistaken to think the world revolves around Twitter.)

Leave a Reply

Your email address will not be published. Required fields are marked *

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑