Mar 25 2008
Good Metadata Necessary for Corrections?
In my last post, I stated that I don’t understand the time of observation correction. This was because when I was looking at the metadata, I could find no evidence that the time of observation had changed, even though it was obvious that there was a correction being applied. I decided to double check the metadata against other metadata published online by NCDC. I went to the USHCN FTP site and downloaded a file called “station.history.Z”. From previous experience, I knew that this file was out of date, but I was surprised to learn that it was last updated in November 1995. I didn’t have much choice though, as I don’t know of any other metadata available.
I have selected the metadata from the same surface station as the last post; namely State University, MS - COOP ID 228374. I excluded some of the unnecessary bits to make it a little easier to decode.
Start End Lat Lon DPL TofO 05 99 1882 05 99 1883 33 28 088 49 999 999 9999 05 01 1884 12 31 1889 33 28 088 49 999 999 03HR 01 01 1890 09 30 1890 33 27 088 47 015 ESE 03HR 10 01 1890 01 31 1891 33 27 088 47 999 999 03HR 02 01 1891 06 30 1910 33 27 088 47 999 999 9209 07 01 1910 01 31 1915 33 27 088 47 000 000 20SS 02 01 1915 05 31 1939 33 27 088 47 003 N 9SS9 06 01 1939 09 03 1948 33 27 088 48 010 SW 9917 09 04 1948 10 25 1955 33 27 088 48 000 000 9179 10 26 1955 03 01 1956 33 28 088 48 009 N 1717 03 02 1956 08 31 1960 33 28 088 48 800 000 1717 09 01 1960 05 31 1962 33 28 088 48 000 000 1717 06 01 1962 08 31 1964 33 28 088 48 000 000 1708 09 01 1964 10 01 1978 33 28 088 48 000 000 0808 10 01 1978 05 03 1983 33 28 088 48 000 000 0808 05 04 1983 99 99 9999 33 28 088 47 010 NE 0808
The first three columns represent the beginning date of the record. The next three columns represent the ending date of the record. Both are in the MM DD YYYY format. The next two columns are the latitude, followed by two columns for the longitude. DPL stands for distance from previous location, and is the next two columns. There are many more columns in the actual data file, but the one I want to look at today is the time of observation. This is the last column in the data above.
This particular column has four ASCII elements. The first two represent the observations time of the precipitation measurement, rounded to the nearest hour. The last two elements represent the observation time of the temperature measurement, again rounded to the nearest hour. The last record in the station history file is for the year 1983 and at this time both the precipitation and the temperature were recorded at 8:00AM.
Going up that column, notice that the last change in the temperature observation time occured on August 31, 1962. Before that, the temperature observations were recorded at 5:00PM.
The next interesting notation in the time of observation is “9179″ for the records starting in 1948 and ending in 1955. This means that there was one time recorded for the time of observation, but it was not noted if this was for the temperature or precipitation. It would make sense to me that this notation might mean that both observations were recorded at the same time. In this case, that would mean that the time of observation was 1700 or 5PM; the same as the previous and subsequent records.
The next change in observation time probably occured in 1939. The record here is “9SS9″, meaning that sunset was the recording time, but it was not noted if this is for precipitation or temperature. Again, it is likely because it was used for both observations.
In essence, there are four dates when the time of observation changed. These are 06-01-1962, 06-01-1939, 07-01-1910, and 02-01-1891. When I looked at the TOBS-RAW data, now these dates correspond perfectly with the changes in observation time. The major exception is 1983. Even when looking at this metadata, I see no records of a change in observation time around this period.
|
[Correcton 27/Mar/2008: This plot actually shows the RAW data subtracted from the FILNET data. For the actual TOBS-RAW see this plot.]
These metadata are obviously what is being used for the TOBS adjustment. However, it shows that the metadata in the MMS is wrong; at least for the time of observation. Graphic showing that the MMS metadata shows no evidence of a change in observation time is shown below.
|
This does raise two important points. Why is the metadata in the MMS different than that in the station.history file? And why hasn’t the station.history file been update since 1995?
Related Posts:
8 Responses to “Good Metadata Necessary for Corrections?”
To reduce spam, comments are automatically closed 30 days after the last comment. If you would like to comment on any closed thread, please use the contact form at the top of this page.


Nathan,
I looked at some more of the original handwritten observer reports for the following months:
Jan 1990 8:00 AM
Jan 1995 7:00 AM
Jan 2000 no time recorded
Feb 2000 no time recorded
Jan 2001 times vary each day
Send me an email and I’ll re you the originals.
So if Mike C’s hand-written notes are correct, then the TOBS didn’t change at 1982 - and was constant through to 1995
- and a minor (1hr) correction should be applied from 1995-2000
- meaning that the apparent temp rise since 1982 is just an artifact of the wrong correction factor being applied
- interesting!
I wonder how many more sites have this same sort of problem?!
Phil, Just to clarify, they are the origonal handwritten observer’s reports at the station.
I like your style. AGW leftist need to pursue truth even when it makes “their side” uncomfortable. Also skeptics need to be skeptical of skepticism and especially of some of the sophistry from McI and the blatant stupidity of his hoi polloi (I differentiate the two).
anhow, Feymnanlike kudos man.
Mike C - yes, I understand
i.e. the notes *are* correct, unless USHCN has a valid reason for using a different correction factor post-1982
Has anyone contacted USHCN to ask them what their explanation is?
i had a sense the history file would help, good catch
This still leaves the question of what are they actually using? Is there a another file or database that is the real source for the corrections?
[Reply: I think they're using an updated version of the station.history file, but haven't updated it on their FTP site yet.]
Oddly, the MMS “Phenomena” tab shows a 1/2006 change in temperature Obs time from 0800 to 0700, while the “Data Products” tab shows a consistent 0800. 0800 to 0700 isn’t a big deal (given that we don’t even know whether this is standard time or statutory time, ie daylight/standard according to local statute), but still it’s remarkable that MMS is internally inconsistent.
BTW, the CA thread on TOBS, which I sort of started, is at http://www.climateaudit.org/?p=2106. My take on it is at comment #110. As I remarked elsewhere, “I used to be a TOBS denialist, but now I’m a True Believer!”.