30 September 2006
| Observing Location | NY Hall of Science, Corona Queens, NY | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Observational Period | 0845-0900 EDT | ||||||||||||||
| Atmospheric Conditions |
It is supposed to get cloudy and rainy today. It was clear this morning but cirrus clouds were starting to full in by 1030 EDT. By 1300 EDT it was mostly overcast with rain clouds, the wind had picked up, and the air felt moist and chilly. When I arrived home around 1500 EDT it was spitting rain but didn't amount to anything until late evening.
| ||||||||||||||
| Instruments | Brunton 8x21 compact binocular w/gold welder's glass - Charlie
Expecting poor weather and having my pack full of tools and a camera I only brought my compact binocular with me. It was clear enough that I should have had the Canons to see the smaller spots. | ||||||||||||||
| Observing Party | Charlie Ridgway |
| ||||||||||||||||||||||||||||||||||||||||||||
Back sometime in November of 2005 we had a discussion one night about the reliability of Clear Sky Clock. We use it daily but have no idea how much faith we should put in it. I decided to track the accuracy of its forecasts during the month of December 2005.
This was a very subjective process based on my feelings about how well the observed conditions (cloud cover and transparency) correlated with the CSC forecast. Each night I would go outside after sunset and see what I could see and compare that to what CSC told me I should be seeing. I classified my assessments into one of five categories:
- observed conditions were much worse than predicted;
- the prediction wasn't even close
e.g., CSC predicted good conditions but it was totally overcast- observed conditions were worse than predicted;
- conditions were not as good as had been predicted but not too far off
e.g., CSC said it would be clear through curfew but clouds moved in late in the observing session- observed conditions were as predicted;
- the observed weather correlated well with what was predicted
e.g., CSC said it would be a lousy night and it was raining- observed conditions were better than predicted; and
- I found that conditions weren't as bad as the forecast said
CKC said it would be clear but transparency would be average but we saw the whole Little Dipper- observed conditions were much better than predicted.
CSC missed the boat on this one
e.g., I got burned by staying home because CSC said there would be a muddy sky but I saw lots of stars from my light-polluted park
On days when conditions were good I was generally observing and made my assessment from my observing location, usually TotL. If conditions were poor or predicted to be so, or if I had a scheduling conflict which precluded observing, I made my assessment from wherever was convenient, generally The Bronx, NY.
The month having run its course and seeing the results I wondered if those results would hold thought the year or if CSC might be more accurate at certain times of the year than at others. Not wanting to be tied to this project for a full year I decided to make another month long assessment during each season for the next year at three month intervals to wit,
- De05
- Mar06
- Jun06
- Sep06
Having come to the end of the evaluation period I am ready to draw what conclusions I can based on the accumulated data.
The Raw Data
| Forecasts Accuracy | |||||
|---|---|---|---|---|---|
| Date | Much Worse | Worse | As Predicted | Better | Much Better |
| Dec05 | 1 3.4% | 6 20.7% | 17 58.6% | 4 13.8% | 1 3.4% |
| Mar06 | 2 6.9% | 8 27.6% | 15 51.7% | 2 6.9% | 2 6.9% |
| Jun06 | 2 7.7% | 5 19.2% | 17 65.4% | 2 7.7% | 0 0.0% |
| Sep06 | 2 6.9% | 6 20.7% | 17 58.6% | 4 13.8% | 0 0.0% |
| TOTAL | 7 6.2% | 25 22.1% | 66 58.4% | 12 10.6% | 3 2.7% |
Overall Accuracy
As would be expected, the results of the assessment made a nice bell curve as depicted in the area graph in the above chart. During the assessment period CSC was on the money 58% of the time as indicated by the central peak.| Overall Accuracy | |
|---|---|
| Good Forecast | 58.4% |
| Bad Forecast | 41.8% |
More of the area lies to the left of the central peak indicating that on the 42% of the occasions when CSC was wrong conditions were more likely to have been worse than predicted than they were to have been better than predicted.
| Bad Forecasts (41.8% of total) | |||
|---|---|---|---|
| Much Worse | Worse | Better | Much Better |
| 14.9% | 52.3% | 25.5% | 6.40% |
So the chance that the forecast will be correct is about 60:40, nearly like flipping a coin. If it is wrong conditions will more than likely be worse than the forecast says. So you will be luckiest at having good nights when CSC predicts <10% cloud cover and above average visibility.
Seasonable Variability
My premise in looking into the possibility of seasonal variability was that the algorithms upon which CSC is based may have more trouble predicting weather during certain seasons than others and should therefore be taken with a smaller grain of salt during times when correlation is expected to be poor.Data gathered over one year does not make a good sample size to make a generalization on seasonal variability. A proper study would have to be made across multiple years. I can only say that during the year that I was making observations CSC exhibited the best forecast accuracy during March, June and September and the worst performance during the month of December. Without repeating this study over several years it is impossible to say if this represents a trend or if they just happened to have a bad month that month. The swing was not great in any case.
Notes
Not every day of every month was assessed. Missing assessments generally occurred on isolated days when I was not able to get a look at CSC, but in June I forgot that I was supposed to be making assessments and missed the first six days of the month| Month | Assessments | Percent |
|---|---|---|
| Dec05 | 29 of 31 | 93.5% |
| Mar06 | 29 of 31 | 93.5% |
| Jun06 | 26 of 30 | 88.7% |
| Sep06 | 29 of 30 | 96.6% |
| Total | 112 of 122 | 92.6% |
Assessments were made at the start of the normal observation period (shortly after sunset) and, if bad, were not repeated later in the nigh to see if conditions had changed unless I happened to be outside for another purpose. If conditions were good and I was out observing my assessment period spanned the entire night from sunset until curfew.
Summary of Observing Time for the Year to Date:
| Day | Night | Total | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Hours | Days | Avg | Hours | Days | Avg | Hours | Days | Avg | |
| Jan | 3.25 | 4 | 0.81 | 17.50 | 6 | 2.92 | 20.75 | 9 | 2.31 |
| Feb | 1.00 | 1 | 1.00 | 10.25 | 6 | 1.71 | 11.25 | 6 | 1.88 |
| Mar | 2.00 | 1 | 2.00 | 12.00 | 7 | 1.71 | 14.00 | 8 | 1.75 |
| Apr | 12.5 | 12 | 1.04 | 40.75 | 14 | 2.91 | 53.25 | 16 | 3.33 |
| May | 3.00 | 4 | 0.75 | 27.00 | 10 | 2.70 | 30.00 | 10 | 3.00 |
| Jun | 7.00 | 7 | 1.00 | 18.50 | 12 | 1.54 | 25.50 | 13 | 1.96 |
| Jul | 10.50 | 14 | 0.75 | 43.50 | 15 | 2.90 | 54.00 | 19 | 2.84 |
| Aug | 7.25 | 13 | 0.56 | 47.50 | 16 | 2.97 | 54.75 | 20 | 2.74 |
| Sep | 8.75 | 20 | 0.44 | 34.75 | 10 | 3.48 | 43.50 | 20 | 2.18 |
| TOTAL | 55.25 | 76 | 0.83 | 251.75 | 96 | 2.52 | 307.00 | 121 | 2.61 |
<< Home