NEON EDUCATION bio photo

NEON EDUCATION

Devoted to open data and open source in science and education.

View All Tutorials

This tutorial is a part of a series!

Click below to view all lessons in the series!

Tags

LiDAR (9)
R programming (70)
Remote Sensing (12)
Data Visualization (4)
Hyperspectral Remote Sensing (7)
Hierarchical Data Formats (HDF5) (24)
Spatial Data & GIS (18)
Time Series (15)
Phenology (7)
Raster Data (8)
Vector Data (6)
Metadata (1)
Git & GitHub (6)
(1) (1) (1)

Tutorial by R Package

dplyr (8)
ggplot2 (17)
h5py (1)
lubridate (time series) (6)
maps (1)
maptools (3)
plyr (2)
raster (32)
rasterVis (raster time series) (3)
rgdal (GIS) (23)
rgeos (5)
rhdf5 (21)
sp (7)
scales (4)
gridExtra (4)
ggtheme (0)
grid (2)
reshape2 (3)
plotly (6)

View ALL Tutorial Series




Twitter Youtube Github


Blog.Roll

R Bloggers

Overview

R Skill Level: Intermediate

Goals / Objectives

After completing this activity, you will:
  1. Understand how to extract and plot spectra from an HDF5 file.
  2. Know how to work with groups and datasets within an HDF5 file.

What you'll Need

  • R or R studio to write your code.
  • The latest version of RHDF5 packag for R.

R Libraries to Install

  • rhdf5: source("http://bioconductor.org/biocLite.R") ; biocLite("rhdf5")
  • raster: install.packages('raster')
  • rgdal: install.packages('rgdal')
  • plyr: install.packages('plyr')

Data to Download

Download NEON Teaching Data Subset: Imaging Spectrometer Data - HDF5
These hyperspectral remote sensing data provide information on the National Ecological Observatory Network's San Joaquin Experimental Range field site. The data were collected over the San Joaquin field site located in California (Domain 17) and processed at NEON headquarters. The entire dataset can be accessed from the NEON website.

In this tutorial, we will extract a single-pixel’s worth of reflectance values to plot a spectral profile for that pixel.

#first call required libraries
library(rhdf5)
library(plyr)
library(ggplot2)

First, we need to access the H5 file.

#Define the file name to be opened
f <- 'NEON-DS-Imaging-Spectrometer-Data.h5'
#look at the HDF5 file structure 
h5ls(f,all=T) 

##   group        name         ltype corder_valid corder cset       otype
## 0     / Reflectance H5L_TYPE_HARD        FALSE      0    0 H5I_DATASET
## 1     /        fwhm H5L_TYPE_HARD        FALSE      0    0 H5I_DATASET
## 2     /    map info H5L_TYPE_HARD        FALSE      0    0 H5I_DATASET
## 3     / spatialInfo H5L_TYPE_HARD        FALSE      0    0   H5I_GROUP
## 4     /  wavelength H5L_TYPE_HARD        FALSE      0    0 H5I_DATASET
##   num_attrs  dclass          dtype  stype rank             dim
## 0         6 INTEGER  H5T_STD_I16LE SIMPLE    3 477 x 502 x 426
## 1         2   FLOAT H5T_IEEE_F32LE SIMPLE    2         426 x 1
## 2         1  STRING     HST_STRING SIMPLE    1               1
## 3        11                                  0                
## 4         2   FLOAT H5T_IEEE_F32LE SIMPLE    2         426 x 1
##            maxdim
## 0 477 x 502 x 426
## 1         426 x 1
## 2               1
## 3                
## 4         426 x 1

Next, we can read the spatial attributes of the file.

#r get spatialInfo using the h5readAttributes function 
spInfo <- h5readAttributes(f,"spatialInfo")

#r get attributes for the Reflectance dataset
reflInfo <- h5readAttributes(f,"Reflectance")

Read Wavelength Values

Next, let’s read in the wavelength center associated with each band in the HDF5 file.

#read in the wavelength information from the HDF5 file
wavelengths<- h5read(f,"wavelength")
#convert wavelength to nanometers (nm)
#NOTE: this is optional!
wavelengths <- wavelengths*1000 ## Extract Z-dimension data slice

Next, we will extract all reflectance values for one pixel. This makes up the spectral signature or profile of the pixel. To do that, we’ll use the h5read function.

#extract Some Spectra from a single pixel
aPixel<- h5read(f,"Reflectance",index=list(54,36,NULL))

#reshape the data and turn into dataframe
b <- adply(aPixel,c(3))

#create clean data frame
aPixeldf <- b[2]


#add wavelength data to matrix
aPixeldf$Wavelength <- wavelengths

head(aPixeldf)

##    V1 Wavelength
## 1 237     382.27
## 2 240     387.28
## 3 298     392.29
## 4 278     397.30
## 5 207     402.31
## 6 235     407.32

Scale Factor

Then, we can pull the spatial attributes that we’ll need to adjust the reflectance values. Often, large raster data contain floating point (values with decimals) information. However, floating point data consume more space (yield a larger file size) compared to integer values. Thus, to keep the file sizes smaller, the data will be scaled by a factor of 10, 100, 10000, etc. This scale factor will be noted in the data attributes.

#grab scale factor
scaleFact <- reflInfo$`Scale Factor`

#add scaled data column to DF
aPixeldf$scaled <- (aPixeldf$V1/scaleFact)

#make nice column names
names(aPixeldf) <- c('Reflectance','Wavelength','ScaledReflectance')
head(aPixeldf)

##   Reflectance Wavelength ScaledReflectance
## 1         237     382.27            0.0237
## 2         240     387.28            0.0240
## 3         298     392.29            0.0298
## 4         278     397.30            0.0278
## 5         207     402.31            0.0207
## 6         235     407.32            0.0235

Plot Spectral Profile

Now we’re ready to plot our spectral profile!

qplot(x=aPixeldf$Wavelength, 
      y=aPixeldf$ScaledReflectance,
      xlab="Wavelength (nm)",
      ylab="Reflectance")


Get Lesson Code

(some browsers may require you to right click.)