Southern Weather Discovery

Handwriting recognition casts new light on climate change from NIWA on Vimeo.

Be a volunteer! Check it out here.

HT/Steven Mosher

0 0 votes
Article Rating
Newest Most Voted
Inline Feedbacks
View all comments
March 16, 2020 10:58 pm

NIWA – an example of a useless org that if the Govt sacked them all and shut them down on a Friday – nobody would notice on Monday.

Patrick MJD
Reply to  wazz
March 16, 2020 11:23 pm

Having worked for NIWA as a contractor 20 years ago I would say that is a fair assessment of the organisation. Some very very self-important and precious people were there then, I am sure not much has changed.

Reply to  Patrick MJD
March 17, 2020 4:40 am

They were also give the legal right by an appeals court judge to make up their own data and have no obligation to provide source data, explanation of the methods used or anything which could be required for reproducibility and scientific validation.

NIWA is a non-scientific body now.

As proof of their bias and lack of objectivity, look about 02:20 in the video. The “day it snowed everywhere” was apparently either ” the last vestige of different climate we left behind” ( ie proof of global warming ) or “the first vestige of climate change impacting our region ( ie “extreme weather” due to climate change ).

Heads it’s global warming , tails it’s climate change.
Whatever the data shows, it’s exactly what we predicted, we have all angles covered.

The likely possibility that is just one point on the range of natural variability does not seem to enter their activists heads.

Reply to  wazz
March 16, 2020 11:33 pm

“NIWA – an example of a useless org that if the Govt sacked them all and shut them down on a Friday – nobody would notice on Monday.”
Really, what do you base that on?

Reply to  Simon
March 17, 2020 1:08 am

Old weather observations aren’t so “valuable” …..they keep getting homogenised and changed and ignored by climate scientists

Steven Mosher
Reply to  Gerry
March 17, 2020 5:20 am

actually , we use them to prove adjustments work.

Reply to  Steven Mosher
March 17, 2020 8:22 am

…and if they don’t they need adjusting.

Reply to  Steven Mosher
March 17, 2020 9:41 am

You use adjusted data to prove that your adjustments work?

paul courtney
Reply to  Steven Mosher
March 17, 2020 10:52 am

MarkW: It would be interesting to see Mr. Mosher’s response, we might learn exactly what “old weather observations” are used to “prove” his work. Does he use 1930’s “hottest evah” unadjusted data (Mr. Mosher, Tony Heller can help you find it), or something else? You may have caught him hoisting his own petard here.

Gerald Machnee
Reply to  Steven Mosher
March 17, 2020 1:54 pm

“work” – changed to suit your beliefs.

Mike McMillan
Reply to  Steven Mosher
March 18, 2020 10:52 am

In proving adjustments work, I assume Steven means that they produce the desired outcome.

Reply to  Gerry
March 17, 2020 4:01 pm

‘Old’ weather observations must give geologists a wry smile. Apparently even they didn’t exist before the introduction of MS Windows in 1985 and by 1989 in the hands of experts we were all doomed if we didn’t get rid of plant food in the atmosphere by 2000-

What’s the latest very very very last tipping point and dooming with Adjustocene Mosher? Will they all announce it in Glasgow or COP out with Covid19? Only Greta could show up and run it all on Skype with the taxeating doomsters self quarantining working from home in their coastal mansions and condos. Stay tuned folks but get back to work with the manufacturing as the Green overlords need their home deliveries on the front porch. There’s an economy to run.

Reply to  Simon
March 17, 2020 1:34 am

What would be missed Simon?

Australia shutdown it’s Climate Commision and they formed there own organization the Climate Council which is funded by donations. So at least we got the leeches and clowns off taxpayer dollars and we lost nothing.

Patrick MJD
Reply to  Simon
March 17, 2020 2:30 am

Typical comment from someone who has no idea, never worked for NIWA, never exposed to their “culture” and “practices”. Well done Simon!

March 16, 2020 11:19 pm

Vimeo freezes accounts after malware hunts for logins, coronavirus map app infected with evil code, and more

Reply to  cinaed
March 17, 2020 4:45 am

Yes, I clicked on something like that yesterday. Maybe in Guardian article. Browser got very sluggish, way too much disk activity. I cut the connection and killed FFx straight away.

I don’t store important log-ins anyway but it looked suspicious.

March 16, 2020 11:40 pm

Great idea………until they adjust the hell out of the pristine remote site data (of which quite a few will be) to match other non pristine data.

March 17, 2020 12:32 am

Kiwi taxpayers fleeced by the academic class and MS marketing machine!
Hey, , , 0.something of a degree really matters yes; NO.
And the main numbers are just fine apart from the occasional outlier which can be discounted.
A ‘quality’ sample of this largish archive is as good as the whole data given the inconsequential variations in over time.

Patrick MJD
Reply to  Warren
March 17, 2020 2:33 am

The IRD in NZ is rapacious…people commit suicide over tax bills, and the IRD STILL demand payment after your death! Assholes! I was an unpaid GST collector for the IRD for a while.

March 17, 2020 1:50 am

I think it sounds great! I don’t know that my tech skills are up to it but if we end up self-isolating I might give it a go. All information is useful (until it gets adjusted).

Patrick MJD
March 17, 2020 2:35 am

IIRC, there are only 3 thermometers used to calculate (Make up) the national average temperature for NZ. One on mainland (South Island) and two on the north island, Wellington and Dorkland.

March 17, 2020 2:44 am

NIWA is one of the best at making its weather station data easily and freely accessible to the public … but one of the worst at reconstructing the past climate of NZ. Their 7-station series only goes back to around 1910, a relatively cold time in NZ.

Berkeley-Earth probably would have gotten a good result for NZ back to around 1850, but they only used the data available in GHCN, which is very sparse, and the BE answer for NZ before around 1940 appears to be dominated by the almost purely maritime climate of several nearby tiny islands, and fails to reproduce the fluctuating temperatures of NZ stations such as Auckland.

I currently have the honor of having the best Tmax reconstruction for NZ back to the 19th century:

Reply to  climanrecon
March 17, 2020 4:37 am

Great work there!

Tom Abbott
Reply to  climanrecon
March 17, 2020 2:07 pm

“Their 7-station series only goes back to around 1910, a relatively cold time in NZ.”

1910 was a cold time in the U.S., too.

March 17, 2020 5:26 am

Working in various departments at a government Department known as the United States Postal Service, one learns some daft things from an engineering department that was ranked in the top global experts for Optical Character Recognition (OCR).

Still, every few years, some lower level manager would dream up a wonderful “What If” solution to their archaic methods of records (data) collection, retention, recall, etc.
Those imaginary scenarios always involve Information Technology devising some miraculous OCR solution for non-IT departments, that eliminates people. Often without involving those tiresome engineers who insist on silly things like standards and definitions.

None do.

Let’s start with the basics:
1) Paper with ink, pencil marks, stains form the basis for written communication and records collection.
Ink alone comprises an astounding array or products meant to leave often colorful marks.

Writing with ink starts with fresh ink and ranges through declining marking ability as the ink runs out to a final often discontinuous vague markings.

Pencil leads are a wide range of compounds all intended to allow a substance, e.g. paper, to accept and retain some eroded lead/charcoal.
Again, depending upon the compound, that marking ranges from bold thick lead layers through slight shades of harder and often older leads.
There are substantial reasons people are told to use pencils with 2B lead for filling out important papers like income tax forms, college exams and election vote submissions.

All of this comes before the near infinite myriad of forms people devise to record information upon or the common practices of the day in writing/print styles, error corrections, symbols designation.

A quick example are all typewritten communications prior to the 1990s where the number 1, the small l and the capital I were the same. All to save complexity in the typewriter or print devices of the day.

Ignoring the issues surrounding imaging written or printed information upon documents (Contrast, Reflectance, Resolution, etc.) one must deal with the issues surrounding basic communication detail itself.
Is that a Capital I?
or the number 1?
Or perhaps a lower case L?

Frankly, it takes a human brain to note all of the surrounding context to guess the information intent. Even then, it is almost impossible to decode documents where the capital I, number 1 and lowercase l are the same.

2) Writing conventions are very different even within small time frames.
Is that a french 7?
Is that a french Z?
Both of which have Nothing to do with the language or culture of France; instead it is a horizontal line through the supporting vertical slanted line. Supposedly to definitively separate 7s from 1s or Zs from 2s…

Is that a 9 or a written g?
Is that a 6 or a sloppy C or G?

3) Data recording forms are wildly different in size, paper composition, design, format, columns and line entries.

Returning to the USPS; where addressing standards are well known and practised for decades; where paper composition and size are defined by regulations; where letter fonts and size are defined by regulations.
Handwritten mail are still conundrums. Frequently requiring humans to decipher.

The Post Office Operates Remote Encoding Centers (REC) where images of mail that that OCR failed to interpret are sent to queues that are fed to people via computers and screens.

As a typical government operation, the Data Conversion Operators, i.e. employees, are tasked with strict and often optimistic production standards. Meaning that employees are pressured to complete and input addresses.

All errors in OCRing addresses end up in customer mailboxes. Often accruing cross outs and additional writing as customers and USPS employees try to interpret who really is supposed to receive a letter.

USPS engineers and IT employees were actually quite clever back in the 1990s. OCR reading, even of well defined bar code markings are utterly dependent upon reflectance and contrast. What one machine reads under certain conditions another machine may fail to read.
USPS engineers and IT workers devised a method that when a mail piece is successively OCRd, they print a bar code on the back side of the envelope using ultraviolet activate ink.

Printed black on white addresses and bar codes, under the best conditions are successively usually read successfully at less than 95%. UV activated bar codes are always successfully read at better than 98%.

Anyone who has tried to read and decode old handwritten documents can attest to the difficulty.
e.g. Census forms:
Every Census used different forms. Frequently there are regional differences.
Every Census form was handwritten, frequently very sloppily.

In trying to decode information about my ancestors, I have flipped through many pages of Census forms filled in by the same Census taker trying to definitely decode their writing.
Fancy cursive writing styles sloppily written can be nearly opaque to understanding.

What every OCR attempt often results in is that all results must be carefully assessed for errors and corrected; repeatedly.
A change in columns that rearranged where certain data are entered may never be identified.

Good luck Niwa…

Abolition Man
March 17, 2020 6:34 am

You can hardly expect the sociopaths and psychopaths embedded in gov’t bodies and gorging on their lifeblood to willingly give up their cushy lives! They already have the ill-educated and low intellect parts of the public bamboozled with the CAGW hoax. Who cares if the last three interglacials were warmer than the current one; it’s still the hottest EVAH! We are in a period (the Quaternary) with the lowest temps and CO2 levels of the last 250,000,000 years and yet a marginal increase is claimed to be catastrophic! Too bad the plants don’t seem to agree!
It seems that the cure for global warming is exactly the same as the one for global cooling and sounds suspiciously like the goals of the International Communist Party from 60 or 70 years ago! Destroy the liberal Western democracies because their workers are too prosperous to accept the yoke of slavery that is hidden in the works of Marx and Mao and Lenin.
Of course, many leaders in Gang Green would just as soon see humanity perish so as to “protect” the Earth from the rapacity of Mankind. Ironic when you consider that without human influence the atmospheric CO2 level in the next interglacial or two would decrease below the 150ppm threshold of death for most plants! To paraphrase Westmoreland (I think): We must destroy life on the planet to save the planet!

March 17, 2020 11:19 am

There should be some value in digitizing these older records — but only if the records are then left untouched as raw data.

The problem is that there are already plans afoot to adjust the data once it is digitized, as if the past is not proper evidence of the past….

“Steven Mosher — March 17, 2020 at 5:20 am

actually , we use them to prove adjustments work.”

What can that actually mean? “Adjustments work”? Work for what? Change the data? Of course, they change the data….that’s the problem.

Do they somehow improve our understanding of the past? Not if we change the data…..

Bryan A
Reply to  Kip Hansen
March 18, 2020 12:24 pm

Adjustments work
Especially if those adjustments alter the date in a desired skewed direction

Patrick MJD
Reply to  Kip Hansen
March 18, 2020 11:59 pm

Not only is there a drive to digitise data, and then “adjust” it, there is a drive to digitise *ALL* books too. That’s what Google wants to do. So you will have access to only “approved” digitised book content, ie, “adjusted”. No more going to a library to read an actual book, which is where I did most of my science studies.

March 18, 2020 4:10 am

great article about weather . thanks for valuable post , if anyone want to know about pollution in hindi than click here Cause of Water Pollution

Verified by MonsterInsights