•   almost 6 years ago

Will the judges scorecards ever get released?

I'm going to assume no because I don't really believe this contest was ever on the up-and-up.

1) Contest submission extension granted for no reason (servers were always up and functional)

2) Bronze Winner steals random pictures off the internet but isn't disqualified. Multiple rule violations.

3)The winner videos are more marketing than actual functionality. We were asked to demonstrate functionality, not to create a commercial.

4) Bronze winner is literally a picture and a text box. Where's the innovation?

5) I'm not convinced any app was actual run by anyone on the ChallengePost side. Testflight shows my app may have been run once for one minute since submission to ChallengePost.

6) Videos over 5 minutes are allowed even thought the official contest rules say they're not (response: it's just a suggestion).

7) Completely ambiguous judging criteria never fully outlined.

I recommend to anyone that ever thinks about participating in a contest like this in the future, to understand that the rules are arbitrary and will change for no reason at any time. Don't bother.

  • 7 comments

  • Manager   •   almost 6 years ago

    Hi bbcz,

    Thanks for writing. Currently, there is no plan to make the judges’ scores public.

    We're sorry that you have concerns about how the contest was run and hopefully we can allay them. We take pride in doing everything we can to make sure that contests like the Energy Apps for Ontario Challenge are fair, and a good experience for participants.  For the sake of completeness, I’ve addressed your additional questions in order below:

    1) The decision to extend the submission period was made because of the roughly two week period during which time the test lab was inoperable. We extended the submission period to make up for this downtime and ensure that those affected were not disadvantaged in the contest.

    2) We did reach out to the contestant in question and have confirmed that the images used by EnergyTipper meet the licensing requirements of the challenge in that they are either images that do not require a license, or images that require attribution (which is shown in white at the bottom of each image for which it is required).

    3) The rules state that the demonstration video must “clearly demonstrate[s] the Application’s functionality and features via video or screencast (by walking through the Application).” Including a promotional element in addition to that demonstration was allowed and not grounds for disqualification.

    4) The Energy Apps for Ontario Challenge set out to foster proof-of-concept application development using the Green Button API. Submitters went about this task using in a variety of methods and created many solution types. There are inevitably different opinions about which apps were the best or most innovative. The winning apps are based on the aggregate scores of the seven judges.

    5) All submitters were required to provide ChallengePost with a way to test their application. Per the Official Rules, “The internal panel and expert judging panel are not required to test the Application and may choose to judge based solely on the text description and video provided in the Submission. MaRS and the Administrator reserve the right to substitute or modify the internal judging panel and the Judges.”

    If an application advanced in the judging process to become  a potential winner, it was tested fully to ensure that it functioned as depicted as video and text description. Applications that did not advance the potential winner state may not have been tested. This is why the Official Rules require the video portion of the submission to “clearly demonstrate[s] the Application’s functionality and features via video or screencast (by walking through the Application.” This requirement is stated in Section 6 of the Official Rules, and the Requirements and How to Enter sections of the challenge homepage.

    6) You are correct that the rules state that the videos “Should be no longer than five (5) minutes.” This is designed to be a guideline and discourage longer videos (the rules don't say the video must be less than 5 minutes, so having a longer video didn't lead to disqualification). We don't believe that having a longer video to demonstrate and pitch an app was an advantage in this contest or any other that we've helped manage. Only one of the five winners had a video longer than 5 minutes and three were closer to 3 minutes.

    7) The challenge homepage and section 9 of the rules outline and define each judging criteria that judges were asked to score on. Each criteria score was equally weighted. 

    Finally, we take the official rules, and our role as the Challenge Administrator, very seriously. Any changes made to the rules (in this case the change in submission deadline) are broadcast to all participants via an Update (http://energyappsontario.challengepost.com/updates/1263-submission-deadline-extension) which is emailed to the registrant directly and posted on the challenge site. Additionally, we are diligent about applying the rules fairly across the board for each participant. It is of the upmost importance that all participants are treated equally and fairly in accordance with the rules.

    I hope that these answers were helpful to you and others. If you have any further questions, please feel free to reach out to us directly at support@challengepost.com.

    All the best,
    Serena

  •   •   almost 6 years ago

    The fact is, this contest was completely misrepresented from the start. Everyone who submitted could have easily created a 5 minute video that lies about what the app can do and make it sizzle. That's a marketing challenge, not an App Challenge targetted towards the Ontario development community.

    Not releasing the scorecards is a complete FARCE for a competition that is sponsored by ontario government.

    1) You talk about the test lab being unavailable. Again, I don't understand what you mean, because I've hit that test site from day 1 every single day and never had an outage. If the lab was truely down, why are there no comments about it in the discussion board? That's a pretty significant issue that would warrant discussion.

    3) The sample video that is linked to is a 2 minute demo that only outlines application functionality. Not a 5 minute 'marketecture' demonstration. This led most of the entries down the wrong path.

    4) RELEASE THE SCORES. I can live with losing, I can't live with losing arbitrarily in a public competition sponsored by the taxpayers.

    5) Do you understand what a cop out that is? The judges rate the apps by the videos which means the video is the only thing that matters. The hundreds of thousands of lines of code are just something to verify once they've already made up their minds!

    6) Guidelines do not belong in the OFFICIAL RULES. You say 5 minutes, it's a RULE. period. Worming you way out of it because submissions didn't follow it is unfair.

    7) You can both NOT release the scores and then tell me how the scores were calculated. Your argument is only valid once they are released.

    RELEASE THE SCORES. This will validate all the hard work that the 27 submissions performed for this contest.

    7) Here's an idea: Don't change the rules mid contest. Take the time upfront and ensure that they're correct at the outset. Picking January 3rd as the submission deadline around the holiday seasons? Poor decision, but a decision none-the-less that you need to stick to. If you took the administrator job seriously, then you failed miserably.

    Here's how I feel. I committed to the contest based on the rules that were presented, based on the dates that were presented. Then, once you lured some developers you change and don't enforce rules haphazardly, but we're hooked. We can't leave because we've already made time investments.

    This contest was disingenuous and shady and that should be the only take away for everyone involved.

  • Manager   •   almost 6 years ago

    Hi bbcz,

    Thank you for your feedback. Your concerns have been heard and we apologize that you didn’t have a good experience with this competition.

    All the best,
    Serena

  •   •   almost 6 years ago

    Why does the contest feel the need to hide behind private scorecards?

  •   •   almost 6 years ago

    Hello bbcz,

    A rule that might help your cause is found in C. Application Requirements

    (iii) Applications must be either newly created by the Contestant or, if the Application existed prior to the Competition Submission Period, *** must have integrated the Ontario Green Button Connect My Data standard API *** and new features and functionality after the start of the Competition Submission Period.

    I did look at every submission and thought some really good ideas came out of the challenge, however, it did seem like only some of the submissions were actually working with the Ontario Green Button Connect My Data standard API .

    I think it would be a good idea for all participants who did spend some time on making the API work to get together to demonstrate what they were able to do with the API, highlighting access and processing issues and their solutions.

    I am not sure if MaRS can provide a room for this purpose. The first step would be to find out who would want to attend.

    Cheers,
    David

  •   •   almost 6 years ago

    Hi David,

    Honestly, I'm done with thinking about this crooked competition in any way. Its obvious to anyone who looks at the actions of the administrators to not enforce rules and not make the judging results public. They asked developers to work on their api for months and can't be bothered to actually run of them. It's just all so bad it's an embarrassment to mars, challengepost and the ontario gov't.

    Theres nothing left for me to do except warn anyone in the future to never deal with a gov't run challenge. They'll do whatever they want whenever they want.

  •   •   almost 6 years ago

    Hello bbcz,

    It sounds like you did spend alot time working with the API, I actually found the API quite challenging. Many people could benefit from your work, you could be way ahead in terms of actually getting and processing the data.

    I do feel there would be a great benefit in collecting these ideas and knowledge about interacting with the API and determining how best to proceed this knowledge going forward.

    Lets talk!

    Cheers,
    David

Comments are closed.