Update on solar cycle 24

David Thomson says:
April 7, 2011 at 6:35 pm
Go back and compare a sunspot count of 64 in the past solar cycle to a count of 64 today and see what I mean.
Is this what you mean:
http://www.specola.ch/drawings/1998/loc-d19980124.JPG where SSN = 66
http://www.specola.ch/drawings/2011/loc-d20110331.JPG where SSN = 66

rbateman says:
April 7, 2011 at 8:58 pm
Ok, let’s compare SC23 and SC24 from their start dates
note the sharp increase in SSN near the end of 1997, similar to the recent jump.

Geoff Sharp says:
April 8, 2011 at 2:21 am
The SIDC did look to over count March with many specks counted along with outlier specks in a group being split out as separate groups. Since Jan 2010 the SIDC result has been generally above the NOAA count. Both of these counts are of course higher than Wolf’s original method.
No, the NOAA/SIDC ratio for March was 0.63. The long-term average ratio is 0.64. Wolf’s original method is not used since 1893 [and shouldn’t be used as it is not reproducible – this has been recognized by all serious observers since the 1890s]. The data since then has been scaled down by a factor of 0.6 to put the modern count on the same scale as Wolf’s. The modern count since 1945 has been 20% higher than it should be compared with 1893-1944. This has nothing to do with Wolf’s original method, but is due to weighting by size of the spots where large spots can count up to five times a small spot. Only the Locarno station still does such weighting, but since Locarno is used as reference station by SIDC, the weighting carries over into all other counts.
tallbloke says:
April 8, 2011 at 12:19 am
SC24 currently looks more like SC4 (Dalton Minimum) than SC14.
The SSN for SC5 [you don’t mean SC4] is so uncertain that no meaningful comparison can be made. Wolf initially himself thought SC5 was a rather large cycle [based on the very meager solar observations at the time], but when he got a compilation of aurorae from Sweden published around 1880, he decided since there were rather few aurorae around and just after 1800 that he better reduce his sunspot number for SC5 and 6 [he almost cut them almost in half hence creating the Dalton Minimum].

Norman Page says:
April 8, 2011 at 7:28 am
Leif – your sector boundary explanation sounds plausible – but do you have any data to support it? Solar wind velocity change at the appropriate time for example.
Of course, I never say anything without data to support it. E.g. here: http://hirweb.nict.go.jp/sedoss/solact3
Geoff Sharp says:
April 8, 2011 at 7:49 am
Yep, heard it all before.
And still you haven’t learned anything
It adds nothing to your argument.
But it subtracts everything from yours, leaving nothing left.
It would be nice if you would stick to what is historically correct, and to what is factually correct for recent data.

Geoff Sharp says:
April 8, 2011 at 7:46 am
No, NOAA and I both get 0.69. No point arguing, the graph shows it all.
My bad. I had averaged the ratios rather than ratioed the averages. Apparently makes a small difference.
The rest of your response basically supports my argument, the Waldmeier factor making up the major component. It is time you accepted this publicly (your paper already says this)
No, the Waldmeier factor means that all old values must be increased by 20% before being compared to modern data. ‘Accepted’ ? Who discovered the Waldmeier factor and been telling everybody and his brother about is for several years now?
There is no doubt SC5 was a low cycle. Who do you think has the best account? Wolf or Hoyt and Schatten?
Of course SC5 was low. The point is that the values are still so uncertain that direct numerical comparison does not make much sense. To wit the difference between Wolf and H&S. We can’t really tell from today which is better. Even if I could, people would still cherry pick what they like, supporting whatever agenda they have.

Pamela Gray says:
April 8, 2011 at 7:29 am
Your writing style puts a grin on my face. Thanks.

Geoff Sharp says:
April 8, 2011 at 9:03 am
they both show a low cycle.
They show that we don’t know SC5 very well.
but glad to see you admit the adjustment needs to happen. The current counting method differs by a large margin compared to how Wolf counted and reconstructed.
Admit? I’m the one pushing this for years. Wolf’s method is inferior to the Wolfer count and irrelevant for any data before 1849 [when Wolf started]. When Wolfs [all pre-Waldmeier] [and H&S] are corrected [adjusted upwards] we have a consistent sunspot number series. In addition SIDC should correct their data 2001-2010 upwards by the 12.5% they have been undercounting. It may be that [because of my needling them] that their undercounting is coming to an end. We shall see. In any event, a return to Wolf’s method would be a huge step backwards, and would have no meaning for any data that Wolf did not observe himself, e.g. before 1849.. We have [eh, most of us, at least] learned a bit about how to count in a meaningful way since then.

rbateman says:
April 8, 2011 at 8:11 pm
Scopes like that give the first impression of being razor sharp, but you have to put them through the paces.
A dozen observers have put them through the paces over the past 150+ years.
The edges of green leaves and the moon, bright stars are what makes the chromatics stand out. And as we get older, it gets harder to see the chromatics.
The optics is among the finest possible. Fraunhofer made the finest achromatic optics. http://www.u.arizona.edu/~kennelly/finaldraft.htm
“Within ten years he was producing the world’s finest achromatic lenses and prisms”
from http://www.nextag.com/Spectrum-of-Belief-Joseph-1228969908/specs-html
Did you try them out projecting spots?
No, as the projection method was never used for counting spots, only direct visual observation. The handheld small telescope does not lend itself to projection [it not on a mounting].

Ah, Sunspots are like age spots, who cares?
Now when an old man farts that’s a smelly event. How bad does your 10.7cm flux smell??

Carla says:
April 9, 2011 at 2:12 pm
You might find plotting the so called space age high cosmic ray years interesting.
From the Moscow neutron monitor I input 2005 April 7 to 2011 April 7.
Moscow had severe instrumental problem over this minimum, especially at the very peak of its record at the end of April 2009. That increase was artificial and has nothing to do with reality, here is a comparison Moscow-Oulu http:/www.leif.org/research/Moscow-Oulu-2009.png

Geoff Sharp says:
April 12, 2011 at 2:05 am
Why bother with a confusing graph when NOAA themselves publish the difference to SIDC. SC24 shows the SIDC counting higher than NOAA…end of story. The same pattern is observed if we go back further.
The ‘confusing’ graph is NOAA’s and SIDC’s own data. This is real data. Clearly you don’t like what they show, but that is understandable in view of your agenda.
http://www.landscheidt.info/images/sidc_noaa1.png
As the long-term ratio is 0.66, use that instead of 0.60 and show the result.
But the 30th March provides an example.
Both Catania and Locarno have seeing conditions of 2.
There is extra detail in the Catania drawing which provides more specks and the detail in the penumbra/umbra area
Not at all, the other way around [if there is any difference]: http://www.leif.org/research/Catania-Locarno-March-30.png
The overall raw count for this day:
The SIDC official discounted figure for the day is 68 which is very close to Locarno x .6 showing the Waldmeier factor in full force. NOAA x .6 is 54.6. The LSC value for the same day was 50.
You should not use 0.6 for NOAA, when the real k-factor is closer to 0.66. 0.66*91= 60. The day you so carefully cherry picked had a very atypical ratio SIDC/NOAA as you can see here: [pink square]
http://www.leif.org/research/SIDC-NOAA-Recent-Ratios.png
Agreed that Catania has better seeing on average than Locarno, but the telescope factor along with the Waldmeier factor are clearly evident.
I just showed you that on the day you picked Locarno saw more detail than Catania. The Waldmeier factor is irrelevant because that apples to all days since 1945 and is simply built into the k-factor.
As I said, once the telescope is large enough its aperture doesn’t matter as seeing sets the limit. Study the upper right hand panel of this figure that shows the k-factor as a function of aperture size: http://www.leif.org/research/Wolf-80mm-Telescope.png .
No dependence on telescope above, say 60mm. One might actually argue that the k-factor is smallest [smaller k: see more spots] around 80-100mm and that a larger telescope makes it worse. This is the reason most sunspot observers use small telescopes of about this size.
The elephant in the room is that NOAA do not use the inflated Waldmeier counting method but still come reasonably close to the SIDC value when measured over the longer term
Nonsense, NOAA adjusts the individual [many] amateur observers using the SIDC numbers to ensure that NOAA stays close to the nominal 0.6. They are not quite successful, but close enough for Government work.
the SIDC are not undercounting when comparing with NOAA.
That you repeat this falsehood, does not make it true. On your own plot, use the 0.66 scaling factor instead of 0.60, and show us. Here is the real data:
http://www.leif.org/research/SIDC-NOAA.png
The time of undercounting by SIDC is marked with red, open triangles. The mean of those points is 0.601. Partly due to my needling them, SIDC is beginning to improve on that as the green triangles show. Their long term ratio is 0.659. The green ratio before the undercounting period began in 2001 was 0.662, and recently [green triangles after the red one] is has been 0.639, so they have improved. but are not quite there yet. I think we have gone around the bush too many times on this. Man up and accept the data so we don’t need to do this again, and again, and again… [seems we still have to…]
Wolf did not have the benefit of the extra observers, or the clearer skies and mainly used his 37mm telescope with a filter.
He had several assistants and an extensive network of external observers. He himself found that he could compensate for the smaller scope by multiplying his counts by 1.5 to align it with his standard telescope, the 80mm. Should we not trust him on that? Who can best judge this? Wolf or us? I’ll go with Wolf on that.
There is no way you can claim the two counting methods can be compared, and this is before applying the big jump imposed by the Waldmeier factor.
I’m not making that claim, Wolfer from 16 years of simultaneous observations [Wolfer using the 80mm and Wolf using the 37mm] is making that claim based on thousands of observations. In any event, Wolfer’s is the better way [every competent observer agrees] and the best we can do with the old Wolf data is to correct its deficiencies by comparing with the geomagnetic record, which actually works pretty well [Wolf did that himself with the data before 1849].

Geoff Sharp says:
April 12, 2011 at 7:36 pm
When comparing the Catania and Locarno drawings for 30th March it would be more honest to look at the whole drawings. Catania counts more specks and is a better representation of the umbral areas.
I added Mount Wilson with its large 42 cm image. Aperture 12 inches = 305mm, focal length 150 foot = 45720mm. Seeing was also 2 at MWO [although they use a reversed scale – but since 2 is near the middle that doesn’t matter]. As you can see it makes no difference to the sunspot image. Even Bill Livingston’s 82 cm image does not make any difference.
http://www.leif.org/research/Catania-Locarno-March-30.png
http://www.leif.org/research/Catania-Locarno-2-March-30.png

Geoff Sharp says:
April 13, 2011 at 12:30 am
you have absolutely no evidence that the SIDC is under counting compared to NOAA. The long term graph since 2001 shows a rough 0.6 factor which is where it should be. Prior to that there were some irregular periods.
SIDC is undercounting since 2001 compared to NOAA as the k-factor before that was 0.662 (0.677) and after 2001 0.591 (0.608). You can compute k three ways: ratio between average count, average of ratios on every months [or whatever period you select] (those are the numbers in parenthesis, or [the best, correct, and recommended by Wolf] only compare days where both observers have non-zero spot count. Taking the mean of the two methods gives k=0.6695 before 2001 and 0.5995 after. A difference of 12%. Comparing SIDC with SONNE, AAVSO, AKS, BAA, GFOES, GSRSI, OAA, RWG, TOS, and VVS [see: http://www.leif.org/research/Eddy-Symp-Poster-1.pdf slides 24-25] show that relative to all those [hundreds of] observers, SIDC undercounted by the same 12% [meaning that it was not NOAA that had changed]. So, it is conclusive that SIDC undercounted as I’ve said. Keep denying that reflects badly on you.
forget to mention the speck count is higher via the Catania image.
On the images you selected both observatories have the same speck count [16] and very nearly the same total spot count [38 and 36 – not weighted]. So, no difference on that carefully cherry picked image pairs.
(1) The telescope does matter. The 37mm can see less than the atmosphere allows on a good day. The 150mm Catania telescope will pick up minor specks that Locarno can’t resolve (more proof to come).
As I’ve said so many times [when does it sink in?], the 37cm is too small. The telescope has to be bigger than ~60mm. Wolf compensated for the smaller telescope by multiplying by 1.5. BTW, both Catania and Locarno have the same aperture, 15 cm. The stop down at Locarno actually improves the count, rather than making it smaller [Sergio, pers. comm], as about 80mm is optimal for sunspot counting. Any difference between observatories is due to seeing [and/or observer experience].
(2 )You have shown no evidence that the SIDC is under counting NOAA.
Just shown above.
(3) I never said anyone was playing tricks.
Oh, yes. “Wolf under threat”. You have hinted many times that you think there are deliberate mal-counting.
(4) Wolf’s data was not able to be cross checked by Wolfer during the higher speck activity of a grand minimum.
It was not Wolf’s data. He wasn’t even born. The data Wolf used was auroral counts scaled to sunspot numbers. Making any discussion of threshold meaningless.
(5) The Waldmeier jump is extremely unfortunate and is still part of the modern record (SIDC only, but how does NOAA come to a similar count without the factor?). This extreme jump is largely taken out in the LSC.
NOAA come to a similar count because they deliberately try to align themselves with SIDC [a bit tough since SIDC is a moving target]. The NOAA numbers are a continuation of the American Sunspot Numbers [AAVSO, see http://www.leif.org/EOS/Hossfield-2002JAVSO-31-48.pdf ] and were originally scaled by the infamous ~0.6 factor because existing nomograms used by the US Navy were calibrated to the Zurich scale. Those nomograms are no longer used, so NOAA does not scale any longer to Zurich, but still uses SIDC as a benchmark when calculating individual k-values for the observers. That guarantees that they don’t stray too far away.
(6) The confusion does need to be sorted out. Meanwhile we have the LSC for comparing against the Dalton.
The LSC is useless for that purpose, because it is not calibrated and because the Dalton numbers are not based on sunspots, but on auroral counts. Wolf himself reduced the sunspot-based count [by other observers] by up to 60% when he got [around 1880] a catalog of Swedish aurorae compiled by Rubenson. There is a good explanation of this in Encycl. Britannica, 11th editon (1910). vol 2, pages 927-934.
(7) This conversation started out discussing that SIDC had over counted NOAA during March. This is beyond question. (o.69)
One swallow does not a summer make. You should know that one cannot base general behavior on an isolated case. For April [so far] the ratio is down to 0.586. For February it was 0.550. For January it was 0.600.
Can we now put this to rest?

Leif Svalgaard says:
April 13, 2011 at 8:46 am
Geoff Sharp says:
April 13, 2011 at 12:30 am
the odd-man-out: http://www.leif.org/research/SIDC-Undercounting.png

Leif Svalgaard says:
April 13, 2011 at 8:47 am
BTW, take care when using NOAA or SWPC values. Their monthly values in
http://www.swpc.noaa.gov/ftpdir/weekly/RecentIndices.txt have transcription errors,
And also June 2006 which is listed as 37.7 but should be 24.4. There are undoubtedly others. Interestingly these errors were found because NOAA disagreed with the ‘rest of the World’. Recalculating the monthly means from the daily values corrected the errors and re-esatblished agreement with the “rest of the World”, showing the power of such comparisons.

Geoff Sharp says:
April 13, 2011 at 6:39 pm
You previously stated the long term average was 0.66, now you are saying this only applies only before 2001?
Leif Svalgaard says:
April 8, 2011 at 6:52 am
“The long-term average ratio is 0.64.” Over all the data.
I consider the SIDC undercounting after 2001 an exception and yes, the averages before and after 2001 must be treated separately. I hope that was clear enough, but from now on that should be clear [one can only hope].
You will need to describe how you scaled the amateur records to match with the SIDC, no doubt there is devil in the detail.
No devil. This is standard regression. I can describe the process thusly: To scale one observer to another ou plot the values of the observers against each other. Hre is the German organization SONNE vs SIDC:
http://www.leif.org/research/SIDC-SONNE-Regression-0.png
Several things to be aware of. If the data points fall along or close to a straight line, you can do a linear regression to find the relationship. For the plot it is SIDC = 0.9755 * SONNE. There is actually a tiny linear offset, but it is [as is evident] so small (+0.8) that it can be neglected, but for every regression made, one has to check this, and if necessary include that in the formula. I used here the period 1996-2010 because I have fairly complete data for many organizations [SONNE, . Minor gaps do not matter, as the regression is done only on days [or months, as I actually use] where both series have data.
One can now do this for all the organizations SONNE, AAVSO, BAA, GEFOES, OAA, RWG, TOS, VVS and calculate SIDC from each using the regression formula for that observatory. That gives you 8 lists [plus SIDC] of scaled, sunspot numbers. How well does the calculation do? That is given by the Coefficient of Determination, R-squared, which is 0.9898 for SIDC from SONNE. This means that 98.98% of the wiggles match in position and relative size. We express this by saying the the formula ‘explains’ 98.98% of the variance. This is a VERY significant correlation. Much better than you normally get in science, where R^2 above 0.80 is considered very good.
The next step is to plot all the scaled lists on the same graph. That allows several things to be gleaned: How well do the series match each others. Are there any that are obvious outliers and may have some problems? etc. Here is the result for the 8+SIDC lists: http://www.leif.org/research/Sunspot-Org-Comparison.png
You note that all the black curves [one for each observatory] lie very close to each other, meaning that they all have the same calibration towards each other. All this can be [and was] checked and verified by standard statistical methods. But, note that the red curve [SIDC] does not fit so well: it is generally above the black ones before ~2000-2001, but below after that. This can be made clearer by also plotting a 12-month running mean [the dashed curves]. This we could not know beforehand. But with that knowledge, we can repeat the regression for all stations using the data split into pre-2001 and post-2001 parts. For SONNE that gives this plot: http://www.leif.org/research/SIDC-SONNE-Regression.png and similar ones for all the other stations. We have now a choice to make: since for all stations the split seems to be around 2001 we can assume that either something happened at all stations at that time to upset their calibration, or that none of their calibrations changed, but SIDC’s did. I prefer the latter choice, but there are weird people out there that often claim silly things…
The last step is now to repeat the calculation of the scaled SIDC for each observatories. Here we again have two choices: should we assume that SIDC before 2001 was good, but after 2001 was too low, or the other way around. For various reasons, I prefer the first assumption, so we recalculate using the regression equation for the 1996-2000 period throughout. We can also include NOAA in this and do the same for it. The result is the Figure you have seen before: http://www.leif.org/research/SIDC-Undercounting.png, where the blue curve is the average of the ‘gang of eight’, the green curve is NOAA, and the red is SIDC. It is now without any doubt that SIDC is too low after 2001, thus ‘undercounting’.
Interesting that NOAA did not match SIDC closely prior to 2001(your new graph shows NOAA matching SIDC pre 2001?) but the amateurs did.
NOAA matches SIDC very closely [as all the other stations did] prior to 2001 as you can see at a glance. One call also compute various statistics on that as verification. Repeat: all observers agree with SIDC before 2001.
At the moment you are saying the amateurs are the world benchmark.
No, Locarno [Sergio Cortesi] is. SIDC play their cards very close to the vest about why they have a problem around 2001. You see the politics here: it is always difficult to get funding and if it gets out too widely or with too much fanfare that SIDC might be off or have calibration problems, the Belgian Government might say “well, if you can’t even count correctly, perhaps it is time to stop doing it…”
“On the images you selected both observatories have the same speck count [16] and very nearly the same total spot count [38 and 36 – not weighted]. So, no difference on that carefully cherry picked image pairs.”
1176 8 4-6, 1183 6 5, 1181 1 0, area 2 6 3
No, specks 16 for both, real spots 22 for Catania and 20 for Locarno.
The date was not cherry picked. As stated it is difficult to find images with the same good seeing this time of year.
There are hundreds of drawings each year…
The 37mm has a max seeing of greater than 3 arc seconds, good conditions allow around 1 arc second. This is important in determining the Wolf threshold size.
No, because Wolf observed for a dozen years before ever using the 37mm and that time is when his scale was set. He multiplied by 1.5 to bring the 37mm up to his real threshold. But, nobody knows pr can know what his ‘threshold’ was. It is un-knowable which is why nobody in his right mind uses [or tries to resurrect] Wolf’s original method.
I have shown that Catania count more pores under the same conditions as Locarno and will continue to do so.
I showed you that Locarno on the example you selected was the best. Shall I show you again: http://www.leif.org/research/Catania-Locarno-March-30.png
http://www.leif.org/research/Catania-Locarno-2-March-30.png
Did you even bother to study them carefully or at least look at them?
This is your interpretation that I take offense to.
It is clear from reading your website and from your comments [and Robert’s] that you think that there was some monkey business going on. There very word ‘inflated’ [that you use all the time] bears the same connotation. You can take offense all you will, your statements on this still stand. Or are you retracting them?
“It was not Wolf’s data. He wasn’t even born. The data Wolf used was auroral counts scaled to sunspot numbers. Making any discussion of threshold meaningless.”
You are confusing the issue. The Wolfer 0.6 k factor to align with Wolf was never tested in grand minimum conditions.
If you plot the k-factor as a function of the sunspot number, you’ll find that there is no clear correlation. Nobody has observed the past 300 years during Grand Minimum conditions. The Dalton was not a Grand Minimum.
You admit the L&P effect is now just a greater proportion of specks which places the Wolfer k factor in doubt during these times. The threshold maybe meaningless to you, but to others that are trying to reproduce history the threshold is important.
The L&P effect [which you say is bad science, but now invoke when it suits] is indeed a dark horse in this. but that has nothing to do with the threshold if all the spots are gone. You cannot reconstruct history before 1849 by referring to Wolf’s ‘threshold’ as Wolf did not observe then. For the Dalton period, he used counts of Swedish aurorae as a proxy for the sunspot number.
“NOAA come to a similar count because they deliberately try to align themselves with SIDC”
Could you explain how this is done and why it did not happen pre 2001?
I have asked them for their exact procedure, perhaps they’ll answer soon. They ‘try’, but do not quite succeed. Perhaps they think their result is good enough for Government work as we say here in the U.S. of A. At any rate, it is not only NOAA that is the issue. EVERY other observer have a pre-2001 problem, or rather SIDC has. As far as I can ascertain there has been no change at Locarno. It is possible that Locarno was used used as reference instead of SIDC and that SIDC somehow changed their complicated calculation [ http://www.leif.org/EOS/Clette_JASR8745.pdf ]. This is one of the things we eant to clear up at the workshop [if not sooner].
The LSC is calibrated by the threshold that Wolf used in his 37mm telescope. The replica LSC 37mm cannot determine penumbra under 333 pixels.
The problem is that having a penumbra is NOT the criterion for counting it as a spot or calling it a pore. The ‘real’ threshold was [as far a we know] twofold: the spot should look black in the 80mm [not grey] and it should not be so small that it could only be seen under very good seeing.
The Wolf’s Dalton numbers are backed up by the Group Sunspot Number which has 117 extra observers.
No, there were a total of 32 observers during 1800-1820 if which several were already used by Wolf. An additional problem was that many of these observers only made very few observations. Finally, the Group Sunspot Number has a very uncertain calibration at that time and is likely too small by some 40%. We have very little knowledge of what the actual number was. The geomagnetic record suggests something analogous to the 1900-1920 period. And in any event, the ‘threshold’ is completely irrelevant for this as Wolf was not observing.
“Can we now put this to rest?”
Not likely.
My next door neighbor believes that the Earth is only 6000 years old, and nothing and nobody can shake that belief. He also say ‘not likely’.

Geoff Sharp says:
April 14, 2011 at 5:36 pm
That’s a doosy and quite ironic really. You have shown no movement from your position even when hard data (Cat/Loc specks)is presented.
You could substantiate that by marking on my figure of the two groups which spot is a ‘speck’. Then we could count. You also ignore the observational evidence that I have shown you that the k-factor [once you are above ~60mm] does not depend on the aperture of the telescope: http://www.leif.org/research/Wolf-80mm-Telescope.png . If anything it gets slightly worse with increasing aperture, the reason for the stop down at Locarno: Sergio found with varying sizes of the cardboard stop that 80mm gave better resolution [most specks] than if the original 150mm aperture was kept [which is what they have at Catania]. So, no wonder that Locarno has a sharper image than Catania [were it not for the seeing]. You can see the cardboard piece on the telescope here: http://www.specola.ch/img/cupola.JPG
“April 8, 2011 at 6:52 am
The long-term average ratio is 0.64.”
No your statement read:
Leif Svalgaard says:
April 11, 2011 at 7:58 pm
I had carefully given you the time [6:52] of my statement, yet you just pick another statement [7:58]:
“The time of undercounting by SIDC is marked with red, open triangles. The mean of those points is 0.601. Partly due to my needling them, SIDC is beginning to improve on that as the green triangles show. Their long term ratio is 0.659.”
Note that I said ‘Their long term ratio is 0.659′, referring to the green triangles [and they do just have that ratio]. Sad that you can get such things wrong. Most be deliberate.
“No devil. This is standard regression. I can describe the process thusly: To scale one observer to another you plot the values of the observers against each other.”
I think you are making this way too hard, and in the process misrepresenting the data. If I do a simple check on your graph it is obvious the monthly values for NOAA and SIDC are incorrect.
Science is hard if done correctly. I think you didn’t even look at what I did. What you see on the graph are curves brought onto the same scale using the scale factors found by the regression. The scaled numbers would be slightly different from the raw data. This is the whole point of the exercise: if the number are not different from the raw data, they have not been scaled or brought onto the same scale.
If I plot the actual SIDC monthly figures and the NOAA monthly figures x 0.6 (std Wolfer factor) and just let the numbers fall I get a very different result.
The NOAA figures should not be multiplied by 0.6 as NOAA does not use the standard Wolfer factor, but by 0.64 [empirically found] for all the data as I said. You have not understood the ‘details’ in spite of my effort. Perhaps you didn’t even try. If you think it is wrong, point out where.
This working on the assumption that the amateur records match NOAA.
They do in the sense that they do not show any jump in 2001, but numerically they will not as they are not on the NOAA scale. See below on new information on the NOAA counts.
“There are hundreds of drawings each year…”
See how many days you can find this year where both drawings have the same seeing at 1 or 2 and with 30 mins of each other.
Who said ‘this year’. There are decades of data.
Any use of the Waldmeier method infers inflation along with the use of the Wolfer 0.6 factor as it is not allowing for the higher speck ratio.
If the speck ratio is higher, the Waldmeier method would give a lower number [thus deflation]. Let’s take an example: on a day there is a big spot [counted as 3], two smaller spots [counted as 2 each] and 3 specks [counted as 1 each], this gives a total of 3+2+2+1+1+1 = 10 [speck ratio 3/6=0.5]. Now increase the speck ratio by demoting one of the medium spots to a speck, then the sum becomes 3+2+1+1+1+1 = 9, thus smaller than 10 for the larger speck ratio of 4/6 = 0.67. Unless you stipulate that in a Grand Minimum there are actually MORE spots and specks which does not seem reasonable.
Telescope placement and number of observers plus 24 hour observing is also an inflation that is part of the modern system, this is very different from “deliberate mal-counting”
The Zurich observers had a simple rule: you only look at the Sun ONCE a day. This is still followed by Locarno, so the Waldmeier jump is not caused by round-the-clock observing. And more observers does not translate into more spots, as spots come and GO. If observer A sees 5 spots in the morrow, it is just as likely that observer B would see 4 spots in the afternoon as he would see 6. More observers does not change the average.
“I have asked them for their exact procedure, perhaps they’ll answer soon. They ‘try’, but do not quite succeed.”
So you really have no clue how they deliberately try to align.
They answered me today. They took over the old American Sunspot Number [which did careful alignment as I explained], but since the Navy no longer uses the old nomograms, there is no longer a reason to align with the Zurich scale, and the last many years they have not even tried to align, they just use k=1. [This was, in fact, news to me as they just stopped without telling anybody about it]. So now we don’t need to worry about how they align, because they don’t. They simply take the averages of the various observers. This seems to work well enough [and really: they don’t care, because the NOAA numbers are not made for long-term studies, but for immediate consumption in practical applications].
If the L&P effect is just extra speck ratio, then all is good. But unfortunately the title of the paper suggests otherwise.
The L&P effect is that, what were specks disappear and what were medium spots become specks. Example: 2 big spots, 4 medium spots, 10 specks. Due to L&P we now have, say, 0 big spots, 2 medium spots, and 4 specks. I don’t think you even read [or understood] their paper [otherwise you would not claim this]. What the L&P is, is a shift of the distribution towards smaller field strength [and thus less visibility]. Here is the evolution of the number of spots since 2001: http://www.leif.org/research/Livingston-Penn-Distribution.png The blue curves show the distributions for each year of 2001-2004, the green for 2005-2008, and the red for the latest years 2009-2011 [all of cycle 24]. You can clearly see the shift from blue, through green, to red.
“The ‘real’ threshold was [as far a we know] twofold: the spot should look black in the 80mm [not grey] and it should not be so small that it could only be seen by the 80mm under very good seeing.”
This further strengthens the LSC threshold.
Nonsense, you said that the 37mm was the key. And the why throw data away? That Wolf did it was already in the 1870s realized to be a mistake and serious observer does that any more. Are you a serious observer?
Looking at the daily records there are some years in the early 1800′s that have missing days but the data would seem more reliable than using a proxy.
Not ‘some years’, most years. and you do not understand the nature of the beast. Even with perfect coverage, it would do us no good as we would not know how they related to Wolf’s series. What Wolf tried to use was comparison with something [aurorae] that could be hoped to have a constant relation with sunspots and then use that scaling to calibrate the sunspot series. Again, this has nothing to do with any threshold.
Wolf would have dovetailed his reconstruction into his own count which is based on his threshold and proxy matching.
He almost cut the sunspot count in half based on the auroral counts. His threshold had nothing to do with it. He was trying to compensate for variable seeing, by not using spots/specks that were only visible [with his acuity] under exceptional seeing. The 37mm automatically did that for him, but he had to up his count by 50% on account of missing too much. You see, this was his admission that he knew it was wrong to omit the smallest spots [he had no other choice because he traveled a lot]

Geoff Sharp says:
April 14, 2011 at 5:36 pm
It is far more likely that you have it the wrong way around and the SIDC look to be over counting during two periods prior to 2001.
On commenting on http://www.leif.org/research/SIDC-Undercounting.png Frederic Clette in email today agrees:
“We find something similar using a bunch of core stations of the SIDC network (excluding Locarno).”
Time for you to give up your illusion and accept the facts.

Geoff Sharp says:
April 15, 2011 at 12:03 am
You need to be clearer in your statements. It can be read two ways.
Only if you want to misundersdtand. As it stands it is very clear: the green triangles have a factor of 0.66.
The NOAA figures should be factored by the standard Wolfer 0.6, this is how its been done since the late 1800′s, to do otherwise is plain wrong.
The 0.6 applies to the Zurich observers [including Locarno]. Every other observer including the ones that make up the NOAA network [USAF – SEON] will have a different k-factor. On http://www.vds-sonne.de/index.php?page=gem/res/results.html you can find a lot of information about this (including all the ‘amateur’ records). Including the k-factors for every observer in the SONNE network: e.g. for 2009 4Q:
Brettel,G. Refr. 90/1000 0.806
Bullon,J.M. Refl. 200/2000 0.976
Bullon,J.M. Refr. 70/ 350 1.018
Bullon,J.M. Refr. 120/1000 0.545
Gieseke,R. Fegl. 50/ 300 1.135
Hofmann,W. Refr. 80/ 400 2.945
Joppich,H. Refr. 60/ 900 0.773
Karlsen,N. Refr. 100/1000 0.869
Morales,G. Refl. 90/2000 0.822
Schott,G.-L. Refr. 80/ 910 1.371
Smit,F. Refr. 80/1200 0.903
Willi,X. Refl. 200/1320 1.166
As you can see applying the 0.6 for everyone is juts ‘plain wrong’.
Of course you will see a difference from 2001 if the overall long term difference is applied. I have also compared NOAA with the 0.64 factor and it still doesn’t look like your graph.
Then do it right. Here is a spreadsheet with the data: http://www.leif.org/research/SIDC-SWPC%20comparisons.xls it also produces the graph.
You have also elected to line up the records on the upslope of SC23 which produces a spurious outcome.
For all observers except SIDC the records line up throughout. SIDC agrees with all the others up to 2001, so it makes sense to align SIDC [only] on the upslope of SC23 [and it has really nothing to do with upslopes or solar cycles. Correct would have been to say 1996-2000, as this is not solar related but has to do with a defect in the SIDC processing]
it is obvious that SIDC is counting correctly since 2001 but perhaps their count could be in question before that.
SIDC agrees with everybody before 2001, so counted correctly then. SIDC disagrees with everybody [NOAA and the amateurs] after 2001, so SIDC is wrong after 2001.
We can compare the F10.7 flux values against both records
If you so [as I have told you many times] F10.7 agrees well with the Zurich and SIDC sunspot number up to about 1990. From there on the sunspot numbers are too progressively too low. This is independent confirmation of the L&P effect, which will hit every observer equally.
The question still remains how does NOAA come up with a very similar value without using the Waldmeier system?
They don’t. NOAA is higher. Since 1996 NOAA’s average was 79.90 and SIDC’s was 51.95 or 53.8% higher [k=0.65, if you calculate the average of k for each month you get 0.64], so they do not come up with a very similar value and it is not a ‘bias’ [it is just the value established from the data]. So there is no question anymore after we have learned that they do not even try to align themselves.

Leif Svalgaard says:
April 18, 2011 at 10:20 pm
By multiplying NOAA by the long-term k-factor you distribute the discrepancy over the whole plot, making it difficult to see. Multiply NOAA by 0.672 for 1996-2001 and by 0.611 thereafter and show us. Then you can see the difference.
I said that clumsily. What I meant was that if SIDC were not undercounting, then the k-factor should be the same after 2001 as before. since before it was 0.672, you should multiply ALL the NOAA values 1996-today by the assumed constant 0.672 and compare to SIDC. Or conversely, if you think SIDC after 2001 is correct then ALL the NOAA values before 2001 should be multiplied by 0.611. This is, of course, a separate plot. either you align before 2001 or you align after 2001. What SIDC is not so sure about is whether 2001 is the best cut-over time, they would like to think that their problem started a bit earlier. I can live with a progressive change from 1998 to 2001. The main point is that they know they have a problem. It does not take a long time to recognize that. It may take a long time to figure out why.

Geoff Sharp says:
April 19, 2011 at 3:40 am
All that does is align the 2 records.
By doing that, you see the difference, otherwise you are just hiding the decline.
There is no doubt there is a shift at 2001
Good, then we can get that out of the way/
but what I think you are not seeing, is that the SIDC are possibly over counting before 2001. The F10.7 records suggest the same
Simply plotting SSN as a function of F10.7 [as I have shown you so many times] shows that SIDC is undercounting after 2001. Here is such a plot:
http://www.leif.org/research/Yearly-SSN-vs-F107.png
To help you understand the plot: pick an F10.7 value on the X-axis. Follow a vertical line upwards. You will then see that the line first encounter the red symbols [SSN after 2001] and then later blue symbols [before 2001]. This means that for a given value of F10.7, the SSN since 2001 is lower [i.e. undercounted] than before. Perhaps you now would claim that the Canadians had a calibration problem around 2001 and that they were measuring too high values before that. Comparison with the Japanese flux values shows that this is not the case.
Now, most of the difference between the blue and red data points is not really due to sunspot counting problems, but the L&P [but since you think L&P is bad science, nonsense, and non-existent, you would have to accept that the SSN compared to F10.7 is indeed too low before 2001].
To show that the SIDC undercounting is a genuine defect in their series [and not just the L&P effect], we can directly compare with NOAA: http://www.leif.org/research/Monthly-SIDC-vs-NOAA.png
And find the same thing: the red data points are below the blue. SIDC is undercounting. Comparing with ‘the rest of the world’ shows the same thing.
There is a very large divergence between Ri and NOAA in 1998
since 1998 is just near the borderline when undercounting began, it is a poor example.
(1)The Waldmeier method is further skewing the records because of a higher incidence of larger spots within groups.
(2) Spots are getting a higher weighting than observed post 2001.
The Waldmeier method has been in effect for decades and did not change near 2001
There are examples of single spots with a score of 4 & 6 that might interest you, along with other groups that might not score as high today…see what you think?
Since Cortesi is doing the counting as he has for 54 years, I see no difference [unless you are claiming he is going blind]
This may only be evident if comparing with some of their other stations.
comparing with the ‘rest of the world’ [as I have shown you repeatedly] shows the same as comparing with NOAA and F10.7: SIDC is undercounting, and all stations [incl. SIDC] are further undercounting due to L&P.

Geoff Sharp says:
April 20, 2011 at 4:49 pm
and then the x-y scatter plot routine.
The x-y scatter plot is the standard way scientists use to calibrate one instrument against another. Read here how Waldmeier suggested using x-y plot of Rz and F10.7 for precisely that purpose: http://www.leif.org/EOS/W-CCCIV.pdf Study it carefully. To check that you have even looked at it, tell us what the last word on page 6 is.
Then in the next comment, we’ll tackle how to do this for cycle 23 [which you liked so much].

Leif Svalgaard says:
April 20, 2011 at 9:45 pm
Read here how Waldmeier suggested using x-y plot of Rz and F10.7 for precisely that purpose: http://www.leif.org/EOS/W-CCCIV.pdf Study it carefully. To check that you have even looked at it, tell us what the last word on page 6 is.
You failed the test…

Geoff Sharp says:
April 20, 2011 at 11:37 pm
http://www.landscheidt.info/images/ri_f10_xy.png
Now that you have seen the light that the x-y plot is a powerful technique [used by everybody in the whole wide world to compare instruments and check calibrations], we can take the next step. Find from the x-y plot between Canadian F10.7 flux 1947-1991 what the scaling formula between F10.7 and SSN is, i.e. how to calculate SSN from F10.7. We use 1991 as the cutoff, because the Canadians moved the observatory from Ottawa to Penticton in 1991. Then plot the calculated SSN [blue] and the observed SSN [pink] since 1947: http://www.leif.org/research/SSN-Obs-Scaled-Canadian-F107.png
You can then see, by eye, how well F10.7 predicts the SSN, until some time between 1999 and 2001, after which the SSN is observed to be lower than what we would expect from the F10.7 flux. The deficit is due to 1) SIDC undercounting by 12% and more importantly 2) the L&P effect removing the small specks, shifting the distribution towards lower field strengths and thus lower spot visibility http://www.leif.org/research/Livingston-Penn-Distribution.png
Such is the story. Comparisons with NOAA and ‘the rest of the World’ shows that the 12% is a SIDC undercount, as we have seen now several times. There really is anything mysterious here. SIDC has to figure out how their undercount comes about; it is not due to issues about Locarno as they see the same undercounting excluding Locarno from the calculation of Ri. It is not due to Waldmeier weighting as that has been done since at least 1945.

Geoff Sharp says:
April 21, 2011 at 8:32 pm
My overlay clearly shows an anomaly between F10.7 and sunspot numbers during 2001-2002, which both NOAA and the SIDC show. This is direct evidence of a decoupling of flux and sunspot number that needs to be investigated but also should be removed when doing comparisons.
All the overlays [mine included] show this and the decoupling is direct evidence of the L&P effect. Remember that the SIDC undercounting is only a small part of the discrepancy [see slides 23-26 of http://www.leif.org/research/Eddy-Symp-Poster-1.pdf ]. The greater part is the L&P effect which, of course, all observers, NOAA and SIDC too, will show. So, such a decoupling [still going on] is the most exciting news in solar physics for many decades, namely that the sunspot number no longer is a faithful representation of solar activity.
The SIDC undercounting is established by slide 23 of the above link, that does not involve F10.7, so has nothing to do with the decoupling.

Geoff Sharp says:
April 26, 2011 at 11:15 pm
So it is worse than we thought. The 2 largest months so far of SC24 and only 6 days covered. The Layman’s darkness record continues to be at odds with L&P.
Livingston observes on average one week per month, giving an unbiased record of L&P. The Layman’s measures are uncalibrated junk, designed to support an agenda.
Geoff Sharp says:
April 26, 2011 at 11:27 pm
This has gone on for some time and still myself and the SIDC are not convinced by your proposal. I believe your methods and results would not stand up to proper peer review.
The SIDC does not like the idea of discarding the 0.6 factor, but does not dispute that they undercount and are actively engaged in trying to figure out why. All parties involved [SIDC, NOAA, us, other researchers] have agreed to a workshop in September to sort out the mess. This is peer-review of the highest carat which might actually produce progress rather than just controversy.
Your methods of comparing the SIDC count is erroneous. Sunspot counts should not be grouped together at your discretion that favours an outcome.
Those are standard methods for calibrating instruments and measurements. The outcome is determined by the data.
You have not provided any proof of SIDC undercounting via the Locarno drawings. (there is ample evidence of overcounting at Locarno as a bi product of the Waldmeier system where the SIDC is higher than NOAA)
The Waldmeier system has been unchanged since the 1940s. SIDC recognizes that they undercount even when Locarno is excluded, so Locarno is but a convenient, red herring.
The F10.7 values do not support your case. When dissected the F10.7 values support overcounting during certain periods.
Even if you could find isolated instances [which is hard, http://www.leif.org/research/F107%20and%20SSN%202.png ], the overall result is plain. F10.7 is not the only measure that shows the undercount since ~2000. The sunspot areas support the undercounting: http://www.leif.org/research/SSN-vs-Sunspot-Area.png and the Calcium K-line measurements at NSO also show the same thing: http://www.leif.org/research/SSN-vs-CaK3.png not to speak about comparing with NOAA and the rest of the World: http://www.leif.org/research/SIDC-Undercounting.png . Now, the SIDC undercount [12%] is but a small part of the discrepancy caused by the much larger L&P effect, so that has to be kept in mind. Even if SIDC revises their count [as is expected], the L&P result still stands. One can hope that L&P will disappear [which it might as it eventually must], but so far there is no sign of that: http://www.leif.org/research/Livingston-Penn-Distribution.png