March 15, 2024, 01:38:55 AM

Author Topic: Proposed Forecasting Contest  (Read 5105 times)

phw115wvwx

  • Guest
Proposed Forecasting Contest
« on: April 24, 2013, 02:15:47 AM »
As promised, I have created a forecasting contest after a lot of work.  Any member here is welcome to participate, and I will appreciate any feedback before we make it official and begin.  So, here is what I propose:


TWCT/TWCC Forecasting Contest Rules

Contest Objective:  During a two-week period, contestants will create ten weather forecasts for a particular city.  Each forecast will be valid for a calendar day (midnight to midnight local standard time for that city as defined in the NWS climate summary).  The forecasts will only be for Monday through Friday, and no forecasts will be made for the weekends.  Each forecast must have the following four elements: 

1.  The high temperature in whole degrees Fahrenheit

2.  The low temperature in whole degrees Fahrenheit

3.  The probability of precipitation (POP) in a percentage ranging from 0% to 100% that is in 10% increments (0%, 10%, etc.)

4.  The liquid precipitation amount (liquid equivalent if frozen precipitation occurs) in hundredths of an inch (0.00", 0.01", etc.)

City Requirements:  Any city used in this contest must be in the United States, have a functioning ASOS weather station, and an official NWS daily climate summary so that accurate and complete verifications can be done.  Patrick and Andy will handle all official verification and scoring, so any questions or concerns must be sent to them as all of their decisions will be final.

Forecasting Deadline:  All forecasts for a particular day are strictly due at 12:00:00 AM EDT (or EST when DST is not in effect) for the start of that day.  Thus, if you are forecasting for tomorrow, your forecast is due at midnight EDT (or EST) tonight.  The forum time will be used to verify that all forecasts meet the deadline.  Contestants will make a post with a forecast of the required four elements that are listed in the Contest Objective section.  If you need to make corrections before the deadline, simply delete your original post and make your corrections in a new post so that all contestant entries are made with unedited posts to alleviate any tampering concerns.  If your forecast is incomplete, is submitted after the deadline, or is not submitted at all, it will be considered as "missing" and will be subject to the penalty described in the Missing Forecast section.

Dishonest Forecasting:  If a contestant intentionally copies a forecast from another contestant that was previously submitted for the same day, or if a contestant intentionally makes a forecast that is completely unrealistic, the forecast will be considered as "missing" and will be subject to the penalty described in the Missing Forecast section.  All of the TWCT/TWCC forum rules also apply here, and any repeat offenders will be disqualified from this contest entirely and will be issued forum warnings.  This contest is simply for fun and educational purposes, so please respect all contestants regardless of performance.

Forecast Scoring:  All forecasts will be scored by accruing a total amount of error points for any imperfections against the final verification values from the four categories shown in this section.  A perfect forecast will earn zero error points.  Since weather forecasting is so difficult to where even the best forecasters can bust, mercy rules have been inserted so that the worst outcome for a single forecast in any of the four categories listed here is 10 error points:

High Temperature - 1 error point for every degree Fahrenheit off, and any errors more than 10°F will result in the maximum of 10 error points

Low Temperature - 1 error point for every degree Fahrenheit off, and any errors more than 10°F will result in the maximum of 10 error points

POP - For verification purposes, 0% is used when 0.00" or a trace (T) of precipitation occurs, and 100% is used when at least 0.01" of precipitation occurs.  Using the Brier Score concept that statistically gauges the skill in forecasting an event with two distinct outcomes, the forecast POP is subtracted from the verification POP, and the result is squared.  Scaling is then performed to make a possible range of 0-10 error points as shown here:

Forecast POP off by 0% - 0 error points
Forecast POP off by 10% - 0.1 error points
Forecast POP off by 20% - 0.4 error points
Forecast POP off by 30% - 0.9 error points
Forecast POP off by 40% - 1.6 error points
Forecast POP off by 50% - 2.5 error points
Forecast POP off by 60% - 3.6 error points
Forecast POP off by 70% - 4.9 error points
Forecast POP off by 80% - 6.4 error points
Forecast POP off by 90% - 8.1 error points
Forecast POP off by 100% - 10 error points

Liquid Precipitation Amount - 0.1 error points for every 0.01" off, and any errors more than 1.00" will result in the maximum of 10 error points

Missing Forecast:  Any forecast that is deemed as "missing" for any of the reasons given in these rules will accrue the maximum amount of error points from each of the four categories (40 points) plus an additional penalty of 10 error points.  Thus, any missed day will result in a total of 50 error points.  Meteorologists have to meet strict deadlines as part of their real jobs, so even submitting a forecast that may not be your best effort is far better than missing the deadline completely.

Winner Determination:  After the two-week period is completed, the contestant with the fewest total error points from all ten forecasts will be declared the winner for that city.  If there is a tie, the contestant with the fewest total error points from the second week (the final five forecasts) for that city will be declared the winner.  The winner will be given the privilege of selecting the next city for this contest, but this city must be in a different time zone than the last city that was completed and must also comply with the requirements stated in the City Requirements section.  This stipulation ensures that a wide variety of meteorological conditions will be encountered during this contest.  Once a new city is chosen, all scores will be reset to zero so that everyone can begin with a clean slate.  This transition period between cities is also the time that new contestants may join and existing contestants may leave.


So, what do you all think?  How many of you would be interested in participating?  We can tweak anything stated above to make it more accommodating for everyone.  Once we are all happy with everything, I will pick a random city along with the dates to begin this contest.  I hope you all like this proposed idea. B)

Offline Metarvo

  • Hero Member
  • *
  • Posts: 574
  • Gender: Male
  • Doesn't record TWC
    • View Profile
  • Cable Provider: Dish Network
  • HD Channel #: 214
  • WxStar Version: Satellite
Re: Proposed Forecasting Contest
« Reply #1 on: April 24, 2013, 08:56:28 AM »
It sounds interesting.  I might have to try it.  :)

Offline toxictwister00

  • Ultimate Member
  • *
  • Posts: 6921
  • Gender: Male
  • Settle It In SMASH!
    • View Profile
  • Cable Provider: Xfinity
  • HD Channel #: 832
  • SD Channel #: 32
  • SD WxStar ID #: 22204
  • WxScan Ch. #: 212
  • WxStar Version: IntelliStar
Re: Proposed Forecasting Contest
« Reply #2 on: April 24, 2013, 02:21:15 PM »
It sounds interesting.  I might have to try it.  :)

+1

I agree with everything he just said. :P


My Video Gaming YouTube Channel
NintenGamers Nation
https://www.youtube.com/channel/UCAi4m_Snvp3b4Vn13_Ir3rA

Offline TWCCraig

  • SKYWARN Spotter
  • Ultimate Member
  • *
  • Posts: 1675
  • Gender: Male
    • View Profile
  • Cable Provider: Cablevision
  • HD Channel #: 62
  • HD WxStar ID #: 31372
  • WxStar Version: IntelliStar
Re: Proposed Forecasting Contest
« Reply #3 on: April 24, 2013, 02:27:30 PM »
I'm in. Sounds interesting. Should choose a city that's difficult to forecast, ones with complex microclimates. Ex. A coastal city (temps could possibly be effected by sea breeze)
Youtube.com/Theweatherchannelman <- Uploaded the most Intellistar 2 videos!
My Weather Station

Long Island, August-September 2012 tornadoes, Hurricane Sandy, Blizzard of 2013, how many places on Earth do you know can get all 3 of those events within a 6 month period?

Offline Mr. Rainman

  • Ultimate Member
  • *
  • Posts: 1394
  • Gender: Male
    • View Profile
  • WxStar Version: IntelliStar
Re: Proposed Forecasting Contest
« Reply #4 on: April 24, 2013, 02:38:40 PM »
I'm in. If you give me a town near a mountain, though, I am in trouble.

I'm in. Sounds interesting. Should choose a city that's difficult to forecast, ones with complex microclimates. Ex. A coastal city (temps could possibly be effected by sea breeze)

I agree, as long as we have a balance of fairly easy regions to forecast for as well to accommodate first-timers.
Tiddlywinks.

Offline Mike M

  • Weatherscan Contributor
  • Ultimate Member
  • ********
  • Posts: 18957
  • Gender: Male
  • TWC Fan 1999-2008
    • View Profile
  • WxStar Version: IntelliStar
Re: Proposed Forecasting Contest
« Reply #5 on: April 24, 2013, 09:00:51 PM »
Awesome idea! I'm all for it. :yes:

phw115wvwx

  • Guest
Re: Proposed Forecasting Contest
« Reply #6 on: April 27, 2013, 12:16:59 AM »
I thought someone might have a small issue with the fixed deadline being in EDT (EST) or the scoring system itself.  Since you all like it as given, I'll work on a spreadsheet to handle the scoring and share it with Andy sometime next week, and I'll create a new topic for the first round with the rules pinned on their own to the Forecasting board.

Speaking of the first round, I have made my random city choice from the Mountain Time Zone:  Casper, Wyoming (METAR ID:  KCPR).  We can begin the two-week period on Monday, May 6 (first forecast goes out Sunday evening on May 5 before the deadline) if that's okay with everyone's schedules.  It would allow you a week to get ready and become familiar with my scoring system where risk vs. reward decisions are commonplace.  You may use anything that you wish for this entire contest, but try to beat the NWS and TWC forecasts if you can. ;)