TERRA-REF - software

Data-Driven Artificial Intelligence for Calibration of Hyperspectral Big Data

dlebauer — Wed, 18 Aug 2021 17:57:52 +0000

July 15, 2021

This paper describes the pipeline used to produce calibrated hyperspectral images from the two hyperspectral cameras deployed under field conditions. The pipeline implements radiometric calibration, reflectance calculation, bidirectional reflectance distribution function (BRDF) correction, soil and shadow masking, and image quality assessment. This was a true tour de force!

Calibrating these cameras was one of the project's most challenging efforts - the light environment is changing, and impacted not only by the sun but also by the big white box that both reflects and shades, the complex canopy, the slow moving camera and timing of calibration, not to mention broken cameras and the variety of radiometers and calibration targets that were put to use. The issues on GitHub include an interesting record of the discussions, challenges, and iterations of different approaches that we took over the years.

Sagan, Vasit, et al. "Data-Driven Artificial Intelligence for Calibration of Hyperspectral Big Data." IEEE Transactions on Geoscience and Remote Sensing (2021).

Vasit Sagan, Maitiniyazi Maimaitijiang, Sidike Paheding, Sourav Bhadra, Nichole Gosselin, Max Burnette, Jeffrey Demieville, Sean Hrtling, David LeBauer, Maria Newcomb, Duke Paul, Kyle Peterson, Nadia Shakoor, Abby Stylianou, Charles Zender, Todd Mockler

sagan2021dda_small.pdf

New Video Tutorials for Accessing TERRA REF Data

dlebauer — Thu, 24 Oct 2019 21:47:04 +0000

trait data

software

Oct. 11, 2019

Kristina Riemer and David LeBauer

Kristina has begun publishing new videos on the TERRA REF YouTube channel that demonstrate how to access and use some key TERRA REF datasets.

Because of the diversity and size of the data collected by TERRA REF as well as the number of different databases and ways to access it, it can be challenging to find a specific data set. We want to enable users to access, get, and use the data of interest, so we have been developing a set of tutorials that show users how to access and analyze TERRA REF data. Funding from the Midwest Big Data Hub supported presentation of online webinars and videos to accompany these tutorials.

Kristina presented four one hour walkthroughs of these tutorials that cover accessing and using trait data, weather data, and image data, and an introduction to getting TERRA REF data via the Breeding API.Look for more videos to be posted in the coming weeks. Please remember to like, subscribe, and comment on our videos and as always, you can find us on Slack and Github. If you do something cool with our data, please consider improving an exisitng tutorial or contributing your own!

TERRA-REF Data Processing Infrastructure

dlebauer — Thu, 31 Jan 2019 16:35:16 +0000

trait data

standards

field scanner

computing pipeline

analysis workbench

reference data

software

sensor data

algorithms

July 22, 2018

The Transportation Energy Resources from Renewable Agriculture Phenotyping Reference Platform (TERRA-REF) provides a data and computation pipeline responsible for collecting, transferring, processing and distributing large volumes of crop sensing and genomic data from genetically informative germplasm sets. The primary source of these data is a field scanner system built over an experimental field at the University of Arizona Maricopa Agricultural Center. The scanner uses several different sensors to observe the field at a dense collection frequency with high resolution. These sensors include RGB stereo, thermal, pulse-amplitude modulated chlorophyll fluorescence, imaging spectrometer cameras, a 3D laser scanner, and environmental monitors. In addition, data from sensors mounted on tractors, UAVs, an indoor controlled-environment facility, and manually collected measurements are integrated into the pipeline. Upt to two TB of data per day are collected and transferred to NCSA at the University of Illinois where they are processed.

In this paper we describe the technical architecture for the TERRA-REF data and computing pipeline. This modular and scalable pipeline provides a suite of components to convert raw imagery to standard formats, geospatially subset data, and identify biophysical and physiological plant features related to crop productivity, resource use, and stress tolerance. Derived data products are uploaded to the Clowder content management system and the BETYdb traits and yields database for querying, supporting research at an experimental plot level. All software is open source under a BSD 3-clause or similar license and the data products are open access (currently for evaluation with a full release in fall 2019). In addition, we provide computing environments in which users can explore data and develop new tools. The goal of this system is to enable scientists to evaluate and use data, create new algorithms, and advance the science of digital agriculture and crop improvement.

Burnette, Willis, Kooper, Maloney, Ward, Shakoor, Newcomb, Rohde, Fahlgren, Sagan, Sidike, Terstriep, LeBauer

Burnette et al 2018 Article