Mar 25 2008

Good Metadata Necessary for Corrections?

Published under Climate Change, Land Use

real_metadata_inset.jpgIn my last post, I stated that I don’t understand the time of observation correction. This was because when I was looking at the metadata, I could find no evidence that the time of observation had changed, even though it was obvious that there was a correction being applied. I decided to double check the metadata against other metadata published online by NCDC. I went to the USHCN FTP site and downloaded a file called “station.history.Z”. From previous experience, I knew that this file was out of date, but I was surprised to learn that it was last updated in November 1995. I didn’t have much choice though, as I don’t know of any other metadata available.

I have selected the metadata from the same surface station as the last post; namely State University, MS - COOP ID 228374. I excluded some of the unnecessary bits to make it a little easier to decode.

  Start       End      Lat    Lon    DPL    TofO
05 99 1882 05 99 1883 33 28  088 49 999 999 9999
05 01 1884 12 31 1889 33 28  088 49 999 999 03HR
01 01 1890 09 30 1890 33 27  088 47 015 ESE 03HR
10 01 1890 01 31 1891 33 27  088 47 999 999 03HR
02 01 1891 06 30 1910 33 27  088 47 999 999 9209
07 01 1910 01 31 1915 33 27  088 47 000 000 20SS
02 01 1915 05 31 1939 33 27  088 47 003 N   9SS9
06 01 1939 09 03 1948 33 27  088 48 010 SW  9917
09 04 1948 10 25 1955 33 27  088 48 000 000 9179
10 26 1955 03 01 1956 33 28  088 48 009 N   1717
03 02 1956 08 31 1960 33 28  088 48 800 000 1717
09 01 1960 05 31 1962 33 28  088 48 000 000 1717
06 01 1962 08 31 1964 33 28  088 48 000 000 1708
09 01 1964 10 01 1978 33 28  088 48 000 000 0808
10 01 1978 05 03 1983 33 28  088 48 000 000 0808
05 04 1983 99 99 9999 33 28  088 47 010 NE  0808

The first three columns represent the beginning date of the record. The next three columns represent the ending date of the record. Both are in the MM DD YYYY format. The next two columns are the latitude, followed by two columns for the longitude. DPL stands for distance from previous location, and is the next two columns. There are many more columns in the actual data file, but the one I want to look at today is the time of observation. This is the last column in the data above.

This particular column has four ASCII elements. The first two represent the observations time of the precipitation measurement, rounded to the nearest hour. The last two elements represent the observation time of the temperature measurement, again rounded to the nearest hour. The last record in the station history file is for the year 1983 and at this time both the precipitation and the temperature were recorded at 8:00AM.

Going up that column, notice that the last change in the temperature observation time occured on August 31, 1962. Before that, the temperature observations were recorded at 5:00PM.

The next interesting notation in the time of observation is “9179″ for the records starting in 1948 and ending in 1955. This means that there was one time recorded for the time of observation, but it was not noted if this was for the temperature or precipitation. It would make sense to me that this notation might mean that both observations were recorded at the same time. In this case, that would mean that the time of observation was 1700 or 5PM; the same as the previous and subsequent records.

The next change in observation time probably occured in 1939. The record here is “9SS9″, meaning that sunset was the recording time, but it was not noted if this is for precipitation or temperature. Again, it is likely because it was used for both observations.

In essence, there are four dates when the time of observation changed. These are 06-01-1962, 06-01-1939, 07-01-1910, and 02-01-1891. When I looked at the TOBS-RAW data, now these dates correspond perfectly with the changes in observation time. The major exception is 1983. Even when looking at this metadata, I see no records of a change in observation time around this period.

state_univ_ms_diff2.png

[Correcton 27/Mar/2008: This plot actually shows the RAW data subtracted from the FILNET data. For the actual TOBS-RAW see this plot.]

These metadata are obviously what is being used for the TOBS adjustment. However, it shows that the metadata in the MMS is wrong; at least for the time of observation. Graphic showing that the MMS metadata shows no evidence of a change in observation time is shown below.

state_univ_tobs.jpg

This does raise two important points. Why is the metadata in the MMS different than that in the station.history file? And why hasn’t the station.history file been update since 1995?

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading ... Loading ...

Related Posts:

  • I Guess I Don’t Understand the Time of Observation Correction
  • Lawrenceville, New York: The Perfect Surface Station?
  • Zeroth Order Corrections to Temperature Record
  • United States Climate Reference Network
  • Correcting for Bias in the Surface Temperature Record
  • 8 Responses to “Good Metadata Necessary for Corrections?”

    1. Mike Con 25 Mar 2008 at 4:53 pm

      Nathan,

      I looked at some more of the original handwritten observer reports for the following months:

      Jan 1990 8:00 AM
      Jan 1995 7:00 AM
      Jan 2000 no time recorded
      Feb 2000 no time recorded
      Jan 2001 times vary each day

      Send me an email and I’ll re you the originals.

    2. Philon 25 Mar 2008 at 5:22 pm

      So if Mike C’s hand-written notes are correct, then the TOBS didn’t change at 1982 - and was constant through to 1995
      - and a minor (1hr) correction should be applied from 1995-2000
      - meaning that the apparent temp rise since 1982 is just an artifact of the wrong correction factor being applied

      - interesting!

      I wonder how many more sites have this same sort of problem?!

    3. Mike Con 25 Mar 2008 at 5:49 pm

      Phil, Just to clarify, they are the origonal handwritten observer’s reports at the station.

    4. TCOon 25 Mar 2008 at 5:53 pm

      I like your style. AGW leftist need to pursue truth even when it makes “their side” uncomfortable. Also skeptics need to be skeptical of skepticism and especially of some of the sophistry from McI and the blatant stupidity of his hoi polloi (I differentiate the two).

      anhow, Feymnanlike kudos man.

    5. Philon 26 Mar 2008 at 3:54 am

      Mike C - yes, I understand
      i.e. the notes *are* correct, unless USHCN has a valid reason for using a different correction factor post-1982

      Has anyone contacted USHCN to ask them what their explanation is?

    6. steven mosheron 26 Mar 2008 at 6:20 am

      i had a sense the history file would help, good catch

    7. BarryWon 26 Mar 2008 at 6:44 am

      This still leaves the question of what are they actually using? Is there a another file or database that is the real source for the corrections?

      [Reply: I think they're using an updated version of the station.history file, but haven't updated it on their FTP site yet.]

    8. Hu McCullochon 27 Mar 2008 at 5:32 pm

      Oddly, the MMS “Phenomena” tab shows a 1/2006 change in temperature Obs time from 0800 to 0700, while the “Data Products” tab shows a consistent 0800. 0800 to 0700 isn’t a big deal (given that we don’t even know whether this is standard time or statutory time, ie daylight/standard according to local statute), but still it’s remarkable that MMS is internally inconsistent.

      BTW, the CA thread on TOBS, which I sort of started, is at http://www.climateaudit.org/?p=2106. My take on it is at comment #110. As I remarked elsewhere, “I used to be a TOBS denialist, but now I’m a True Believer!”.

    Trackback URI |

    To reduce spam, comments are automatically closed 30 days after the last comment. If you would like to comment on any closed thread, please use the contact form at the top of this page.