Guest essay by Eric Worrall
Why pre-judging the value of potential stepping stones to a solution, stifles scientific innovation.
Kenneth Stanley, one of the world’s top artificial intelligence researchers, has produced a fascinating presentation, on why focusing on an objective can sabotage innovation.
Consider how you would teach a robot to solve a maze. The obvious thing to do, is to teach the robot to do its best to reach the end of the maze – to reward movements which bring the robot closer to the end of the maze, and punish movements which cause the robot to move away from its objective.
However, researchers quickly discovered that if you give these logical seeming instructions to a robot, something unfortunate happens.
The obsession with finding a solution, and on focusing resources on actions which seem to bring the robot closer to the solution, blinds the robot to the possible paths which actually lead to a solution.
How do you solve this dilemma? Ken’s solution to preventing this kind of hangup is incredibly simple – the way almost every ingenious breakthrough is simple. Instead of punishing the robot if it makes a move which takes it away from its goal, you reward the robot for innovating – for always doing something new. The only caveat is, the robot should try to avoid repeating itself.
Focussing only on trying something new avoids prejudging potential stepping stones to the solution. Until you know what the solution is, there is no reliable way of knowing how valuable a potential stepping stone might be. The only way to solve the maze in my example, is to take a path which leads away from the solution – a path which an entity obsessed with reaching the solution would find very difficult to take.
Or to put it another way, if 3 decades of attempts to build a model which works, has failed to deliver results, maybe its time to stop repeatedly butting your head into the same wall.
This approach to problem solving, focusing on novelty, does not prevent you from monitoring the process, to see whether a solution has been found – solving the problem is the ultimate goal. All novelty search means is that you keep your opinions to yourself – you let the robot do its thing, regardless of how silly any individual move might seem at the time, until it finds the solution on its own terms.
The following is Ken’s presentation “Novelty Search and the Myth of the Objective”. Ken’s presentation covers a lot more than my quick summary – including valuable insights into why scientific advance has stalled in a number of disciplines.
Update – Ken Stanley has published a book, available for pre-order on Amazon, which discusses his ideas in more depth.
Click here to hear an interview with Ken, about his new book.
Mr Worrall
Thanks for the video. It recommends what I have already practiced for some time.
You may have seen some of my dozen or more correlations, among many others you may not think much of the results, but all of them are product of the process described in the video.
I plot set of data and if I find it interesting I do spectral analysis, if that looks interesting take next step etc, etc. Here is an example:
Dr. S goes on about his SSN revision, I read his paper and find that he uses geomagnetic ‘Y – east component’ as one of the metrics. Now, I go to the geomagnetic data and plot annual values of Y component, expecting to see sunspot cycles. Not a thing, but than I realised it looks like the AMO, and here it is:
http://www.vukcevic.talktalk.net/GMEC-AMO.gif
Dr. S pronounced it as a ‘spurious’ correlation, adding ‘vuk is not stupid but …misleading people’.
I will watch the video’s second half a bit later. Very much recommended !
Perhaps it is true that the “y” component correlates well with the AMO and it is still not the driver of the AMO, but since the “y” component can be correlated to solar activity in general, then perhaps the “y” component is simply a good proxy for the solar energy reaching into the depths of the ocean!
It will soon become apparent who is doing the ‘misleading’. I have great respect for Dr. S and his work, but there are major issues with his and the IPCC’s 0.1% solar variation constraint that deserves re-evaluation.
I try to look at anyone who doesn’t understand that the Sun caused the ~ 0.8 degree warming (big deal!) during the Modern Maximum as being slow to recognize the accumulative effect of solar heating upon the ocean (and the reverse). The hard of learning is evident in those already set in their ways.
In that vein, the idea of ‘misleading’ can then be attributed to ‘not understanding’. We can all relate to that on some level, I think. Given that, it has been my personal focus to make that understanding easy and very evident to all, a work-in-progress at this stage.
Keep on truckin’ Vuk!
Hi Bob
Y component is the weakest one of all (in the N. Atlantic it is about 50% of X and less than 10% of Z. It is simply an indication (or partial proxy if you will) of some other ( I would say two other) far more energetic process/es.
Bob
There are two data bases for geomagnetic data : ETHZ and NOAA.
Here
http://www.vukcevic.talktalk.net/AMO-GMY.gif
I have plotted both sets of data, for a single location in the far end of N. Atlantic (sub-polar gyre area, recently discussed on WUWT here .
Since the AMO oscillation follows the geomagnetic Y component, I maintained the AMO time scale and moved GM Y graph to the right along the time scale.
Dr. S’s sunspot correction metric (geomagnetic east Y component) has more to it than Dr. S might like to credit it with. Read all about it here:
http://www.leif.org/research/Sunspot-Number-Corrections.pdf
That’s why I’ve been patient, but it’s a long wait until August… Thanks to both of you for barely putting up with each other, again (and the new links & data) 😉
Unfortunately, Vuk is spouting nonsense as usual. I try, as usual, to teach him how things work, but, as usual, he is hard of learning, but while he is a lost cause one may hope that the lurkers out there are not so unfortunate to share that same affliction.
vukcevic April 4, 2015 at 2:16 pm
Two papers (quoted above) contradict each other
No they do not. It is you who has not understood what they said [as usual].
vukcevic April 4, 2015 at 2:27 pm
Btw. Section of the Pedersen current transversing polar region (see the graph above) I was referring to, is not vertical but horizontal and induces telluric currents. Two vertical sections definitely do not do so. I am a bit sceptical about Hall currents claim.
As usual, you have no idea what you are talking about. Fukushima’s theorem does not require the Pedersen currents to be vertical. And what they induce is irrelevant for the Hall current in the ionosphere that produces rY. As for being skeptical: in order to be that you should know what is going on. Since you do not [and are unwilling to learn], you cannot be skeptical, only incredulous.
Again, you are economical with the truth. The variableto consider is not the Y-component, but the amplitude of the diurnal change of the Y-component. The amplitude of that oscillation is a VERY good representation of the solar cycle. In fact, the best one we have, see e.g. slides 31-38 of http://www.leif.org/research/Keynote-SCOSTEP-2014.pdf
I have a hard time believing that you are ignorant enough [although you might be; you tell me] so that you can’t see the difference between the Y component itself [which arises from deep within the core and has nothing to do with the Sun] and the [much smaller] regular variation of the Y-component that is caused by currents in the ionosphere created by solar EUV radiation.
To start, just to make it absolutely clear as it is stated before, Y component and its variability (diurnal or annual) is far too weak to cause any significant effect; it is simply an indication, or a partial proxy, of some other far more energetic process.
I am not certain of the mechanism linking two (day-light and annual variability), thus I have not suggested one. Equally, I am not prepared to accept your total rejection that daytime and annual (accumulation of daily) changes are totally unrelated.
On page 31 of the paper you quoted (I have looked at once or twice before) there is rather complex flow chart. Because of my limited knowledge of whole complexity of the events I am happy to accept it as a conventional representation of reality.
There are two paths between the yellow coloured boxes (EUV to SFMfC) with number of external inputs. Some of the events involved are just ‘day-light’ transitory, while others leave residual effect from one day to the next, thus propagating the cumulative effect further along the time line into the annual variability.
Just to mention briefly, I have looked into number of papers and presentations (Svalgaard et al) and I am of the view that there is serious omission which would cast doubt on accuracy of an exact SSN – rY relationship. It appears that authors wrongly assume that the geomagnetic storms disturbances never happen around 8-9 am, but more about that on some other occasion.
Equally, I am not prepared to accept your total rejection that daytime and annual (accumulation of daily) changes are totally unrelated.
Then learn that the annual changes are NOT the accumulation of the daily ones. The latter are due to currents in the E-layer that disappear at night, thus no accumulation. And are cyclic: In the morning, the excursion is to East [or West in the opposite hemisphere] as the Sun rises and in the afternoon the excursion is the same amount to the West [or East] as the Sun sets, before returning to the same level as before the morning excursion.
This daily oscillation is due to external sources [the ionosphere maintained by EUV]. The slow change of the level from day to day and thus accumulated from year to year is due to internal sources deep in the core of the Earth and has nothing to do with the Sun.
lsvalgaard April 3, 2015 at 7:11 pm
Again, you are economical with the truth. The variableto consider is not the Y-component, but the amplitude of the diurnal change of the Y-component.>/i>
No sir !
Economical with the truth would be to ‘conceal’ the multi-decadal annual variability of the Y component, as it has been the case in most of the papers devoted to the subject.
– Both graphs quoted from your paper do show title as you have it in the papers.
– Annual variability of the Y component is made up of the final aggregate of the daily measurements, it doesn’t just pop-out on the 1st January (or any other individual day) of each relevant year and then goes away until next year.
– If you care to look at any of your annual normalised graphs you will notice that the peak amplitude is 50-60nT which is line with peaks of the multi-decadal annual variability on my graph.
Just a coincidence ? Ehh, to many coincidences and spurious correlations to your liking ?
Have a happy Easter.
Just a coincidence ? Ehh, to many coincidences and spurious correlations to your liking ?
Here are some examples of ‘Vuk-science’ http://www.tylervigen.com/
Failed again to account for the science we are supposed to take for granted.
No sir, some of us do not take anything for granted
You are still avoiding to address residual changes that occur in just one branch of your paper’s flow chart (page 31) EUV …- Ion formation – Recombination – Ionospheric conductivity – Dynamo process. -…
There are further problems with your ‘sunspot like graph’, there is a magic cut off below ~34nT, there is now link to actual daily values for at least one station, even for one individual year, for purpose of verification for the data ‘vacuum’ below the ~34nT.
In another paper you have an illustration graph of a day-light change (reproduced in my comment April 2, 2015 at 12:09 pm, from Hobart, Tasmania, more than 100 years old. Why not something more recent?
This is what a more recent magnetograph from 1- 3 March 2015 looks like.
Blue line is horizontal component H, red line is declination angle D, from which East component Y can be calculated as Y = H Sin(D).
Above linked graph for any day for two middle quarters (6am – 6pm) in no way can reproduce the ‘Taz’ graph from your articles.
Now, you might claim that these are geomagnetic disturbances produced by geomagnetic storms.
Yes sir, that is the most likely case, but I could not find a single reference to geomagnetic storms in the crucial paper http://arxiv.org/pdf/1407.3231v1.pdf .
A simple search would show following:
the solar activity effect on the changes in the Y component intensity are:
Solar minimum ~ 22 nT,
Solar maximum ~ 36 nT,
Moderate Geomagnetic storm ~ 70 nT,
Severe Geomagnetic storm ~200 nT usually spread over couple of days
Here is graph showing one of the aspects of geomagnetic activity
http://www.vukcevic.talktalk.net/Ap40.gif
The graph shows that for the last 80 years the geomagnetic storms show sort of sunspot cycles pattern but out of phase (delayed), but more importantly that the ‘cycle’ maxima are not proportional to the amplitude of the corresponding sunspot cycle. Smoothed version of the above graph shows peaks more even cycle ‘train’, a bit reminiscent of your rY annual graph..
Using above umbers it can be calculated for a year of a Solar maximum, for an average cycle
365 days x 36nt = annual daily average = 36
20 days of moderate storms x 70nT = annual daily average = 3.84
3 severe storms x 200 = annual daily average = 3.29
sum of the geomagnetic storms gives annual daily average of 7.13 or 20% of the 35 for a year of Solar maximum
This is a significant contribution and since the stronger sunspot cycles (eg SC19) are geomagnetically proportionately weaker than lesser cycles it follows that:
Using daily rY component changes to estimate (or even calculate) sunspot cycle intensity is an unreliable metric.
Let me continue your education:
1) The ‘cutoff’ at 34 nT is because 34 nT is the lowest rY can go. It reaches that value at every solar minimum.
2) The data for Hobarton is data obtained at that station in 1848 when there was an observatory operating there. The observatory operated during 1841-1854 and not since. My rY graph is based on numerous [dozens] observatories during 1840 and 2014.
3) The rY variation is caused by solar EUV, not by the solar wind.
4) Geomagnetic storms cause intensification of the Ring Current [Van Allen Belts] and the Auroral Electrojects [seen e.g. at Tromsoe]. Those currents flow East-West, hence their magnetic effect is North-South. The rY [which is East-West] is thus insensitive to those currents and their effects average out to nothing. Which is why rY was chosen in the first place.
None of this is controversial. All of this is well-understood in quantitative detail. That you don’t understand anything is a defect of you, not of rY.
Using daily rY component changes to estimate (or even calculate) sunspot cycle intensity is an unreliable metric
Quite the contrary, as this simple plot shows:
http://www.leif.org/research/rY-F107-SSN.png
The rY is an excellent measure of EUV, F10.7, and SSN for the reasons I have explained.
To continue your education:
As I explained, geomagnetic storms have little effect on rY. In the plot below, I show the variation of the three components [H,D,a dn Z] for Kakioka [Japan] since 2015, March 1st. A strong magnetic storm occurred on 17th March. The effect on H of the intensification of the Ring Current is a severe depression of the Horizontal Component. On Z, the effect is a strong increase of the Vertical Component. It takes about a week for those effects to go relax away. But there is only a minor, short-lived disturbance on D [which is essentially the Y-component: Y = H sin(D)] which simply adds a bit of random noise to the values. That noise averages out over the many days that make up a year, so has no effect on the yearly average rY.
http://www.leif.org/research/Diurnal-Var-Kakioka-During-Storm.png
Using daily rY component changes to estimate (or even calculate) sunspot cycle intensity is an unreliable metric
It is generally accepted and well-understood that
“[20]1. Sq(Y) is strongly controlled by solar activity and reflects the characteristics of each solar cycle.
[21] 2. The solar activity dependence is caused almost completely by the solar activity control of the ionospheric conductivity…” from http://www.leif.org/EOS/jgra50386-Sq.pdf
So, Sq(Y) [or as I call it, rY] is an excellent indicator of solar activity.
Small variations of underground conductivity and of the slow [and varying from station to station] secular variation of the main field are minimized by computed the average rY from many stations across the globe.
“The advantage of the Y component is that Sq is the clearest in the component in middle latitudes, where most observatories that provide long-term data are located, most clear in the component, and the Y component is less sensitive to disturbances in the geomagnetic field” [thus no geomagnetic storm contamination.
Here are some examples of ‘Vuk-science’ http://www.tylervigen.com/
Sir, you swerved again away from discussing shortcomings of the Sunspot revision proposal.
And what shortcomings do you have in mind?
2.The data for Hobarton is data obtained at that station in 1848 when there was an observatory operating there. The observatory operated during 1841-1854 and not since. My rY graph is based on numerous [dozens] observatories during 1840 and 2014.
Why use the extinct Tasmanian ‘Taz’ from 160 years ago, why not a more recent data: London, Paris, Potsdam, Moscow etc
Just give us one weeks hourly, and one year daily data files links. I’m sure they must exist else you wouldn’t use the graphs without the data.
4. ) Geomagnetic storms cause intensification of the Ring Current [Van Allen Belts] and the Auroral Electrojects [seen e.g. at Tromsoe]. Those currents flow East-West, hence their magnetic effect is North-South.
Pedersen currents flow North-South and vice versa at different latitudes (partial neutralising effect is only in a very narrow band) .
http://www.vukcevic.talktalk.net/Image1.gif
Decades ago it was shown by Chapman that a cloud of charged solar particles approaching polar regions induces electric currents in the Earth’s surface layers causing globally sudden change in the Horizontal component of the geomagnetic field , a direct result of these currents, as it can be seen on any magneto-graph.
East component Y = H * Sin (D).
Since H changes during any storm and D 0 degrees, in severe storms H changes up to 5% peak to peak and D moves up to 2 degrees, it is plainly obvious that the
geomagnetic storms do cause daily, hourly, even down to seconds changes in the East Geomagnetic Component Y
By ignoring effect on geomagnetic storms you are failing to address shortcomings of the rY metric for revising SSN data.
The AMO – Y correlation I found (and you would prefer to not know about) it is most likely to the great extent due to the above.
You are not paying attention. Try reading this:
http://www.leif.org/research/Reconstruction-Solar-EUV-Flux.pdf
If I want to know what rY was in 1848 it makes sense to use data from 1848, wouldn’t you think.
On Slide 7 you can see that to measure the effect of EUV one has to stay away from high [and low] latitudes. Only stations within the two red boxes are to be used for this. The ring current and the polar regions show other effects not related to solar EUV. Do yourself a favor and try to learn something instead of posting irrelevant factoids [that you don’t even understand].
The Pedersen and field-aligned currents [orange] have no effect on the magnetic field at ground level. What we see is the effect of the [yellow] Hall currents.
Again, try not to pontificate on things you do not understand.
At mid-latitudes where H = 20000 nT, a severe storm [only a handful occurs in the most active solar cycles – none so far in cycle 24] has dH = 400 nT [the limit for Kp=9]; if D changes by a random 2 degrees the change in Y is a random 13 nT, which when averaged over a year is negligible. It is painfully obvious that you have not thought this through [or at all]. Again: the diurnal variation of Y is caused by solar EUV [not solar wind storms]. The secular [slow] changes from year to year are caused by convective movements deep in the core of the Earth, and have nothing to do with the Sun, or the climate. Gauss showed us how to tell from a magnetic variation if the source was external [rY] or internal [secular changes to Y].
D 0
D is more or less than zero
singes disappeared !
not much noise in this one
http://flux.phys.uit.no/cgi-bin/plotgeodata.cgi?Comps=dhz&tint=3hrs&block=0&day=2&mnt=03&year=2015&site=bou1a&update=+Update.gif
Look at the whole month:
http://www.leif.org/research/Boulder-HDZ-March.2015.png
and note that D is a bit noisy when we have a storm, but goes down and up such as to have no lasting effect.
Just as I showed for Kakioka
http://www.leif.org/research/Diurnal-Var-Kakioka-During-Storm.png
It is about time you learn something. Try to make an effort.
You constantly try wriggle out of the essence of the problem, when you find yourself in a deep ditch. First you say there is no east-west shift, then it is noise, then it cancels itself out, while your paper clearly states it is min-max for the day, or is it at 9am & 3pm, which one is it?
Noise or no noise, now you are migrating towards equator (my graph which you originally decide to attack is for 60N, top of the subpolar gyre), any excuse but no actual data!
Now you say “The Pedersen and field-aligned currents [orange] have no effect on the magnetic field at ground level”.
Absolutely wrong!
The interplanetary electric field affects the Arctic’s ionosphere, generating global circuit of Pedersen currents in the ionosphere, the effect propagates southwards. Resulting time-varying magnetic fields induce electric currents in the ground !
Then you say “severe storm [only a handful occurs in the most active solar cycles – none so far in cycle 24”
Wrong again !
There was one only 2 weeks ago
http://thewatchers.adorraeli.com/data/uploads/planetary-k-index%20march16-18-2015%20kp-8-g4-severe.gif
I shall leave it here, until you produce actual data.
Wiggle matching is not good enough to eliminate Grand Solar Maximum of 20th century. No data no contest.
Data for all observatories are here: http://www.wdc.bgs.ac.uk/catalog/master.html
The min and max must be taken at 8 am and 2 pm. rY is the average difference between the 2 pm and the 8 am Y-values. Those are the times when the effect from the Sq vortex are largest.
A truly severe storm [with an amplitude of 400 nT or more] has Kp = 9. None so far in this cycle.
The average value of Y anywhere on Earth has nothing to do with solar activity, climate, magnetic storms.
http://en.wikipedia.org/wiki/Naoshi_Fukushima
“He is best known for proving that, under certain conditions, the magnetic field from a field-aligned current and the magnetic field from the associated Pederson current in the ionosphere would exactly cancel at the surface of the Earth.”
Although the conditions are not exactly met, they are close enough that it is generally accepted that the effect we see at ground-level is due to Hall currents [East-West electrojets].
Figure 2 of http://www.swsc-journal.org/articles/swsc/pdf/2013/01/swsc130002.pdf shows the Hall current East-West electrojets [by scientist at Tromsoe…]
“The dynamics, sizes and shapes of the auroral ovals are determined by the interaction between the geomagnetic field and the solar wind. This interaction allows for currents flowing in and out of the upper atmosphere, and these currents are closed by horizontal currents in the ionosphere. Owing to channels of enhanced conductivity produced by particle precipitation, large Hall currents flow along the auroral ovals. These are termed the auroral electrojets. In the dusk sector in the northern hemisphere, the electrojet flows eastward, and in the dawn sector it flows westward.” Hence their magnetic effects are North-South.
Again, try to learn from this.
“severe storm [only a handful occurs in the most active solar cycles – none so far in cycle 24”
Another excuse, now has to be a ‘truly severe’, 400nT, funny that.
BGS and NOAA mark a sever storm at around 200nT and above.
I will look at BGS data and see what can be extracted out of it, but for now here is a BGS graph for change in D directly relating to the East geomagnetic component Y = H Sin (D)
http://www.geomag.bgs.ac.uk/images/dDtimeseries.jpg
more than a bit like AMO (or my top graph) but with shifted time scale.
Not Funny. You claimed ” in severe storms H changes up to 5% peak to peak”. 5% of 20,000 nT is 1000 nT. The limit for Kp = 9 is a mere 400 nT http://isgi.latmos.ipsl.fr/des_kp_ind.html None that strong [not of your 1000 nT] this cycle. The currents in the ionosphere that cause rY are Hall current as your link correctly says.
“Related ground-based magnetic (primarily north-south)” and “PSD time series of horizontal and vertical magnetic fields recorded at 7 monitoring stations in northern Chile exhibit a pronounced temporal variation of the east-west component with an amplitude of about 5 dB for periods between 100 and several thousand seconds. This pronounced variation of By is observed between solstice seasons and daytime and nighttime. Seasonal variations are also observed for the north-south and vertical magnetic components but with 3 orders of magnitude smaller power.” This is BTW for short period variations not the several tens of thousands of seconds of the daily variation. Bottom line: What we observe are East-West Hall currents as I have said. Those have no effect on rY. Try to avoid to pontificate about things you do not understand.
Naoshi Fukushima was wrong about Pedersen currents
See Space Physics 2012 paper
http://onlinelibrary.wiley.com/doi/10.1029/2012JA018008/full
Abstract: ……….”The ground-based measurements of magnetic and electric fields show statistically significant coherences with the interplanetary electric field derived from solar wind and interplanetary magnetic field data of the Advanced Composition Explorer satellite. The interplanetary electric field (IEF) penetrates the polar ionosphere from where it propagates toward equatorial latitudes by waveguide transmission, with ionosphere and solid Earth acting as conducting boundaries.
… We conclude that the IEF drives primarily a global circuit of Pedersen currents in the ionosphere. Resulting time-varying magnetic fields induce electric currents in the ground. Related ground-based magnetic (primarily north-south) and electric (primarily east-west) signals vary coherently at all local times and seasons. Conversely, magnetic signals caused by the IEF-driven Hall currents depend much on local time and season. We show for the first time that these ionospheric Hall currents cause no induction in the ground, but they generate magnetic signatures that are confined to the waveguide between ionosphere and Earth’s surface.”
So it is Hall currents (yellow on the graph) that do not induce telluric currents. You are confusing Pedersen and Hall currents. What might be excuse this time?
The situation is as follows: the S(Y) current vortex is a Hall current in the ionosphere flowing North or South in the Morning and Evening with resulting magnetic effect in the East-West direction [rY]. This is what is related to the EUV and the sunspot number. As shown in my diagram about causes, there is also currents induced in the ground [at depth] and in the oceans. The effect of those currents decrease the observed rY by some 25% depending on the location, hence the need for normalizing the record to a common station [Niemegk]. Special conditions apply to the polar caps and the equatorial regions which complicate matters, but are irrelevant for and avoided by the use of rY derived from midlatitude stations only. Hence the superb correspondence between rY and solar activity.
And BTW, Fukushima’s Theorem holds provided that the field-aligned currents are perpendicular to the ground which they are approximately in the polar regions [corresponding to the Figure you showed]. The situation near the equator is different as the field lines are nearly horizontal there. So Fukushima was not ‘wrong’. What is wrong is to believe [as you assert] that the theorem should hold where it is not applicable.
Uh, thanx for pointing out my being economical with words above. It’s the daily change in the “y” component. Haste makes waste – and I’ve seen you do it too.
Your new SSNs will enable me to tune my solar model, and thank you for being so meticulous and thorough with your reconstruction, so that we can therefore better understand how solar activity affected the Earth during the past three hundred years.
Certainly, without a correct measure of solar activity it is hopeless [and indeed stupid] to try to assess the influence of solar activity, if any. Unfortunately this truism is widely ignored.
Two papers (quoted above) contradict each other, and since they were published shortly one after the other two sets of authors may have not be aware of contradictions.
To be honest, I had more than enough for one day, but maybe tomorrow or in a day or two, I will show graph that combines solar toroidal magnetic oscillations with what as you said above “caused by convective movements deep in the core of the Earth” produces multi-decadal ‘oscillation’ coincident in time a form with the annual changes in the geomagnetic east component Y as shown in my second graph, and coincidently when delayed with the AMO.
You say that this ‘has nothing to do with climate’ .
Maybe or maybe not.
Btw. Section of the Pedersen current transversing polar region (see the graph above) I was referring to, is not vertical but horizontal and induces telluric currents. Two vertical sections definitely do not do so. I am a bit sceptical about Hall currents claim.
Reblogged this on Rebekah Slusher and commented:
This applies to so many areas and disciplines! Several social programs come to mind, where we keep doing the same things over-and-over, expecting different results.
So many scientific discoveries are made by accident. Can robots be trained to detect these?
It depends what you mean by a discovery. Training AI systems to detect shifts in financial markets is big business these days. AI systems are used extensively in structural chemistry, to tease out details of difficult structures from available clues. Google language translation is not quite the joke it used to be.
As computers become more powerful, and insights into how intelligence works becomes more advanced, it seems likely to me that AI will take an increasing role in fields such as pure research.
Novelty: Milky Way is corrugated and much larger than we thought
http://news.rpi.edu/content/2015/03/09/rippling-milky-way-may-be-much-larger-previously-estimated
Does this affect the amount of dark matter??
One step forward towards eliminating the ‘need’ for the three darks: dark matter, dark energy and dark (gravity) force
Something I see all the time is that people — even those with a firm understanding of the scientific method — have a model of some phenomenon burned in their mind, and this precludes them from entertaining input from other models. If they hear one sentence fragment that contradicts their pet model, they rise to quash any further discussion.
Personally I try to think in terms of a ranked list of models for a given phenomenon. Even if the second best model is totally lame relative to the (current) best model, I keep my eye on it. I try to understand it in toto. I give it a fair chance.
The ranked positions can change at any time with new input, but this process can get short-circuited by a closed mindset.
I suspect people get bullied out of this approach because if one entertains more, shall we say, colorful options, one quickly gets labelled as a pseudo-scientist or crank or whatever. There is unseen peer pressure, even if it comes from ourselves.
Frankly, I find there is little downside to keeping a mental ranked list of possible theories for some phenomenon. It’s not like mental storage space is limited. The upside is that it promotes an open mind (but not so open that one’s brain falls out).
Does anyone else think in these terms?
You have a correct perception Mr. Photon. It happens all the time everywhere.
Max, I agree with this assessment. This is sort of how I do it. There is just one nuance that I will state. I first noticed it in graduate math courses. Often in approaching a problem I get a sense of gravity or direction or a way to go or a feeling. The impetus is initiating and does not deter proper techniques once I am under way. But there is definitely a feeling or a sense that I am going in a certain pathway. I am often wrong or I can’t snag the root of the problem sometimes, but it has helped out numerous times. I don’t know what to call it. Novelty, wonder…it is real and it precedes disciplined scientific activity.
Call it a “gut feeling”. Intuition.
Friendly reminder:
Don’t forget to check out the lunar eclipse!
Oops … that’s not the song I wanted.
Yeah if the moon had been that big this is what it would have looked like (~3 billion years ago):
http://i.dailymail.co.uk/i/pix/2015/03/23/13/26EC666100000578-3007624-image-a-16_1427115769595.jpg
Here we go … (much better) …
This theme of novelty was and is popular for logical science type puzzles. It is great fun to explore, like Eric’s example of the maze.
Geoff
http://www.geoffstuff.com/martin.pdf
It IS great fun. So much for my productive evening though.
(I think the trick is to use circles 😉
This is an example of the reader getting set-up by the examples. The 2 examples are complex as a whole, so equally complex patterns of arrangement of the parts are needed. I tried a few patterns for arranging internal parts of the square, failing to find a solution.Then I realized as was looking for far too complex of a solution.The box is a simple shape, so I tried a simple pattern: Running 4 equally spaced parallel lines through the box, either vertically or horizontally, divides the box into five equal sized rectangles.
SR
I probably should have specified that the width of each rectangle should be 1/5 of the whole square so no one can find a way to misunderstand my solution.
SR
SR,
Congrats, you solved the problem by discarding the ?misleading info in the examples. It took me a couple of hours, many years ago, before I hit my forehead and went DUH!! Some really respected mathematicians laboured over it in Sci Am until their agony was relieved in the issue a month later.
But now, how would you work the logic if you wanted to write a computer program to address the puzzle as put?
Cheers Geoff.
Geoff,
Under 2 minutes, but then I’ve always been considered simple-minded ;o)
Wait, wait, wait.
The instructions say there are at least TWO solutions.
I took that to mean that the trivial solution is dividing the square by five rectangles (which I got in about five seconds), and set about to find a non-trivial solution.
Am I the only one reading the instructions this way? Did anyone find a non-trivial solution?
Okay, I’ll admit to being tricked by complexity.
Rotating the trivial solution 90 degrees is another solution. That’s “at least two.”
I thought of that in 10 seconds, but thought there was something more interesting to the problem … which I suppose is interesting.
” you reward the robot for innovating”
Robot reaches pulls self on top of walls, walks along walls until reaching its goal. Innovation.
I seriously thought of making the wall walking solution my second scenario – but it probably violates the “physics” of the simulation world… 🙂
Innovation. Think – outside of the box LoL
Science may have entered ‘post normal’ without any help from whacked out philosophers who nowadays don’t ‘philo’ the ‘sophus’. The frustration of physicists of the past (say) 75 years since the close of the golden age is palpable. All these desperate brains with nothing to do. It is not even a search for the novel. As long as they take as chiseled in stone what is ‘known’ about general relativity/gravity they, like the robot above, are destined to come up with dark matter, strings and other ridiculous things. Having discovered something wrong with gravity of big things like galaxies and perhaps small things like, electrons and protons, and quarks, etc for which they had to invent totally new sets of forces: strong forces and weak forces and intermediate forces all to deal with the same matter – this is what they do instead of looking for (an)other factor(s) or variable(s) that makes Newton’s and Einstein’s gravity, Van der Walls, and intra atomic forces into special cases of a new and more general theory.
What’s the next step after 75 years of scientific silence in the search for novelty? Why it’s to invent stuff and then beat, bend and wrought the data until it fits. Not only is it ‘novel’ its profitable. And when this is finally abandoned? The end game for science would seem to be to to evolve like music. The great composers seem to have used up all the notes and there is nothing left but to become a performer of the great works of others. Lab coats and horn-rimmed glasses become the tuxes and tails of the performer.
This is good insight for problem-solving, keep trying something new.
It’s how evolution works and it’s also how the immune system operates (g.o.d. = generator of diversity).
I assume most of readers have gone elsewhere by now, but
as promised in my comment on April 4, 2015 at 2:16 pm
here it is:
http://www.vukcevic.talktalk.net/SUN-and-AMO.htm
There are things worthy of debate. That is not one of them.
Regardless of your ignorance about the basic physics of conductivity, currents, induction, plasma, etc there is no doubt that the diurnal variation of the geomagnetic East component is an exquisite measure of solar activity, expressed both by the F10.7 cm microwave flux from the Sun and the International Sunspot Number, with correlation coefficients of 0.99. In addition, we know the physical processes that are involved so there is no question about correlation vs. causation. The data is public and readily available, so anybody can check the calculations. There is nothing new nor controversial in this, the effects being known for centuries:
http://www.leif.org/research/rY-F107-SSN.png
The present comment brings a fitting closure to the matter.
The present comment brings a fitting closure to the matter.
I strongly doubt it .
You still have to get it accepted, and no sign of that at the moment.
I looked at some of the BGS data (thanks for the link), for the H component there are solid numbers, however D changes from morning to the afternoon are tiny (order of 0.1 degree at Lerwick), calculating SinD differences at these values is prone to a relatively large uncertainty with ratios of 36 to 15,000, or approximately 1:400.
As you can see from my post, there is an ‘uncomfortable’ link of the main source of natural variability AMO to the matters solar. Eeven the NASA’s new climate supremo, btw: we both went to the same university, perhaps he is just being kind, has found something of interest, else it would have been dispatched to the BoreHole
http://www.realclimate.org/index.php/archives/2015/04/unforced-variations-april-2015/comment-page-1/#comment-628160
calculating SinD differences at these values is prone to a relatively large uncertainty
Nonsense, the Sine Function can be accurately calculated with any precision that you want, hundreds of decimal places if so desired. No uncertainties at all.
R^2 is a neasure of how much of the variation is the same for two series. The statistical significance of the regression for F10.7 on rY is of the order of 10^(-40) with the 68 degrees of freedom.
About ‘uncomfortable’ links: they only become so when significant, so as I said: no need for further discussion of your spurious correlation.
R^2 of 0.97 and 0.98.
Autocorrelation ?
Natural (unforced – as Gavin calls it) variability is the next big thing. Mann and co are now talking about subpolar gyre slowing down (cooling in another words), Gavin kept on his blog my ‘spurious’ correlation. Soon they will need a reason for downturn, no major volcanoes, no aerosols to blame, Chinese pumping CO2 and temps stalled. Only thing they can blame is sun, which conveniently is going to take a rest. You have no chance of selling your revision, it came 10-15 years too late.
Irony of it all is that you may well be right, and that there is not much to it above 0.1C, but that will not matter.
Re. sun-earth oscillation phase differences; recently I came across rather strange phenomenon: the Earth has two kinds of year: astronomic and geomagnetic, and they are of different length !
I hope you had a good holiday break.
See you elsewhere.
The Revision of the Solar Activity is ongoing, and need not be ‘sold’ as it is firm. It will likely be adopted by IAU in August. It is obvious that there is a 0.1C solar cycle effect. All the rest is spurious and in some cases plain garbage. Such does not deter some people in their search for said garbage.
You must remember that the rY connection is not my idea. It has been known a long time and is widely accepted and its physics is known. All I have done is to use all available data [some Gigabytes] to construct a complete time series. You could benefit from reading http://www.leif.org/research/Radio-Ionosphere-Magnetism-and-Sunspots.pdf
posted comment in a wrong place.
see further below: April 6, 2015 at 3:40 pm
I once programmed a game sometimes called “Othello” or “Reversi”. Since I programmed it, I knew its algorithm and could easily beat it, no matter how intelligent I tried to make it. Then I added some uncertainty, random weighting of various moves. Sometimes its move is stupid, as are some of mine, and sometimes its moves seem brilliant although of course there’s no such thing; it is a mixture of algorithm and random. I let it look for the “long shot” several moves in advance but whether it decides to try for the long shot is somewhat random.
Wow , I never expected to encounter an example I was going to mention .
I find this article interesting , but I’m not overly impressed . Perhaps my greatest professor , and one of my strongest supporters was Don Campbell whose Blind variation and selective retention , 1960 , and Evolutionary Epistemology make very similar arguments .
The actual images and their algorithms remind me a lot of Brad Nelson’s Forth Haiku . See his very impressive presentations at the last couple of Silicon Valley Forth Interest Group Hangouts , particularly February’s meeting .
Getting back to Othello , Martin Gardner had a column about it in his Mathematical Games column in Scientific American back around 1963 when I first met a computer , an IBM 1620 . Essentially its a game where the object is to find a path from one side of a board to another . As Michael2 says , programming the winning strategy was pretty easy . However , having the computer figure out whether it had a connected path from one side to the other was not . Essentially it had to run a maze and know when it got all the way to the other side .
I worked quite a while on it in the available FORTRAN and kept running into the need to keep tables of state I had not anticipated . Only somewhat later , when I learned Algol , which has recursion built in , did I succeed . ( In an APL , I later implemented it in about 5 very short definitions . )
In a psychology course on the theory of learning , I submitted a paper titled Maze Running Behavior in the CDC 3400 emphasizing the necessity of recursion to provide path memory . The significance of this insight was , I feel , under-appreciated and I only got a B .
At any node , it simply tried “left” , then “forward” , then “right” , or backed up and tried the next direction at the last not exhausted node . It had no metric sense of distance from it’s goal , just success if it got thru . So , in the present terms , the only thing the “program” required was the requirement of novelty – not repeat a previous path .
Yea, thanks for the link. I looked at it not long ago, will do again.
I find it rather odd that the flow chart (here on page 32) I mention before doesn’t leave any longer term residuals, and that annual change is approximately same as the daylight max-min difference.
How they calculate annual, just look at two dates a year apart?
However, nothing in it about mesosphere, the one I forgot it existed, and you concluded (!wrongly!) ‘ vuk is not stupid ..etc’.
Talking about ‘stupid’, when I got the latest correlation, I couldn’t believe how good it was, but I am not that stupid to think that it was or will be there for ever, possible but highly unlikely to be coincidence, even 150 years is far more than could be expected.
16 years is the second strongest component after 21 in the Jackson data, but I have no clue where it comes from.
Is there a point of asking: have you ever come across 16 yr period ?
I searched for it and found it only in the Himalayan monsoons, after vague reference by a NZ scientist. I found data, did spectrum and it is there.
A long shot idea: Monsoons take millions of tones of water (in a short time, takes years for it to get back) from equatorial region to the higher latitudes, and in doing so changing rate of rotation (angular moment and so on).
Oh well, never mind ….
I have no particular conviction about any of this stuff , but it it is lot of fun and keeps me out of pubs; if it wasn’t for my daughter’s homework you wouldn’t ever heard of me.
good night
Here it is
http://www.vukcevic.talktalk.net/HM.gif
(data 1870-2000) with elNino, 16 years and Miyahara (not only at Maunder min GCR) 28 years.
Many things that look odd are quite natural, e.g. that the Moon has the same angular size as the Sun.
The annual change can be calculated in many ways: e.g. by computing the average for each year, or by a fancy low-pass filter. The diurnal variation over a year is always positive [greater than 34 nT], while the annual change is negative for some locations and positive for some locations. Averaged over longitudes at a given latitude the secular change is close to zero. The mesosphere is not mentioned because the electric currents that cause the diurnal change does not flow in the mesosphere. Occasionally a large solar flare will produce enough X-rays to ionize the D-layer giving rise to a Sudden Ionospheric Disturbance [SID http://solar-center.stanford.edu/SID/activities/activity.html ] for something like an hour. These are not cumulative and have negligible effect on rY. About 16 years [and other cycles]: if you look for cycles you will find cycles, most of them will mean nothing, as this one:
http://jonova.s3.amazonaws.com/graphs/temp-co2/us_post_causes_global_warming_lrg.jpg
Yours fall in the same category.
The Aa index is very similar to the Ap index, but not exactly the same since it is calculated from only the two stations’ data Canberra (Australia) and Hartland (UK).
Because of only two stations being used for Aa, Aa is inferior to Ap which is based on about a dozen stations. So Aa is a poorer index than Ap and less representative of true geomagnetic activity, hence more likely to be show spurious effects.
even 150 years is far more than could be expected.
No, if you have 60-100 years cycles then in 150 years you only have about two degrees of freedom and can easily get a couple of cycles to match, just like getting two heads in a row when tossing a coin.
Correlation I had in mind is not directly related to Y, it is this
http://www.vukcevic.talktalk.net/Sun-Earth.gif
Significant 16 years (above noise !) only appears in monsoon, Jackson geomagnetic, Aa but not Ap index and only Arctic temps data.
Thanks anuway.
Since Aa and Ap are just two expressions of the same thing, this is a sure sign of a spurious, non-significant signal.
The Aa index is very similar to the Ap index, but not exactly the same since it is calculated from only the two stations’ data Canberra (Australia) and Hartland (UK)
vukcevic April 7, 2015 at 1:49 am
The Aa index is very similar to the Ap index…
Because of only two stations being used for Aa, Aa is inferior to Ap which is based on about a dozen stations. So Aa is a poorer index than Ap and less representative of true geomagnetic activity, hence more likely to be show spurious effects.
I will say that VUK’S correlations are never straight forward and there is to much of this adjustment then that adjustment to make it come to some sort of fit. To much adjusting and accounting for my taste.
I am simple when it comes to this stuff maybe because my knowledge is not as great but I like correlations that are straight forward and either they work and hold up or they do not.
Vuk’s conclusions on the AMO versus solar activity , have been shown by others to be in direct opposition to what he shows. Some have made the point that during weak solar periods the AMO is positive and vice versa.