NEON EDUCATION bio photo

NEON EDUCATION

Devoted to open data and open source in science and education.

View All Tutorials

Tags

R programming (52)
Hierarchical Data Formats (HDF5) (15)
Spatial Data & GIS (22)
LiDAR (10)
Raster Data (14)
Remote Sensing (25)
Data Visualization (4)
Hyperspectral Remote Sensing (18)
Time Series (15)
Phenology (7)
Vector Data (6)
Metadata (1)
Git & GitHub (7)
(1) (1) (14)

Tutorial by R Package

dplyr (7)
ggplot2 (16)
h5py (2)
lubridate (time series) (6)
maps (1)
maptools (1)
plyr (2)
raster (26)
rasterVis (raster time series) (3)
rgdal (GIS) (24)
rgeos (2)
rhdf5 (11)
sp (5)
scales (4)
gridExtra (4)
ggtheme (0)
grid (2)
reshape2 (3)
plotly (5)

View ALL Tutorial Series




Twitter Youtube Github


Blog.Roll

R Bloggers

Lesson Objectives

At the end of this tutorial you will be able to

  • open an HDF5 file with Python.

Data to Download

Download NEON Teaching Data Subset: Sample Tower Temperature - HDF5

These temperature data were collected by the National Ecological Observatory Network’s Flux Towers . The entire dataset can be accessed from the NEON website.

Download NEON Teaching Data Subset: Imaging Spectrometer Data - HDF5

These hyperspectral remote sensing data provide information on the National Ecological Observatory Network’s San Joaquin Experimental Range field site. The data were collected over the San Joaquin field site located in California (Domain 17) and processed at NEON headquarters. The entire dataset can be accessed from the NEON website.

Python Code to Open HDF5 files

The code below is starter code to create an H5 file in Python.

if __name__ == '__main__':
	# import required libraries
	import h5py as h5
	import numpy as np
	import matplotlib.pyplot as plt

	# Read H5 file
	f = h5.File("NEON-DS-Imaging-Spectrometer-Data.h5", "r")
	# Get and print list of datasets within the H5 file
	datasetNames = [n for n in f.keys()]
	for n in datasetNames:
		print(n)
	
	# extract reflectance data from the H5 file
	reflectance = f['Reflectance']
	# extract one pixel from the data
	reflectanceData = reflectance[:,49,392]
	reflectanceData = reflectanceData.astype(float)

	# divide the data by the scale factor
	# note: this information would be accessed from the metadata
	scaleFactor = 10000.0
	reflectanceData /= scaleFactor
	wavelength = f['wavelength']
	wavelengthData = wavelength[:]
	#transpose the data so wavelength values are in one column
	wavelengthData = np.reshape(wavelengthData, 426)

	# Print the attributes (metadata):
	print("Data Description : ", reflectance.attrs['Description'])
	print("Data dimensions : ", reflectance.shape, reflectance.attrs['DIMENSION_LABELS'])
	# print a list of attributes in the H5 file
	for n in reflectance.attrs:
	print(n)
	# close the h5 file
	f.close()

	# Plot
	plt.plot(wavelengthData, reflectanceData)
	plt.title("Vegetation Spectra")
	plt.ylabel('Reflectance')
	plt.ylim((0,1))
	plt.xlabel('Wavelength [$\mu m$]')
	plt.show()
    
	# Write a new HDF file containing this spectrum
	f = h5.File("VegetationSpectra.h5", "w")
	rdata = f.create_dataset("VegetationSpectra", data=reflectanceData)
	attrs = rdata.attrs
	attrs.create("Wavelengths", data=wavelengthData)
	f.close()