Atmospheric Radiation Measurement (ARM) Data Archive

Lead Investigator: 
Participating Staff: 
Collaborators: 
Argonne National Laboratory, Brookhaven National Laboratory, Los Alamos National Laboratory, Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, National Renewable Energy Laboratory, Pacific Northwest National Laboratory, Sandia National Laboratories
Sponsors: 
US DOE, Office of Science, Office of Biological and Environmental Research (BER)
Start Date: 
1992
End Date: 
Ongoing

The ARM Program was created to develop several highly instrumented ground stations to study cloud formation processes and their influence on radiative transfer. This scientific infrastructure includes two mobile facilities, an aerial facility, and data archive available for use by scientists worldwide through the ARM Climate Research Facility (ACRF). The primary objective of the ARM Project is improved scientific understanding of the fundamental physics related to interactions between clouds and radiative feedback processes in the atmosphere. The ARM Archive provides storage and access to all of the data generated or assembled by the ARM Program and the ACRF. The Archive provides access to data files by way of internet access to any researcher that submits data requests. The Archive supports hardware for data-intensive processing associated with reprocessing and processing of very large data sources, and also provides assistance with reprocessing and large data processing tasks (retrieving input, installing and supervising processing tools, storing results). 

The Archive includes several integrated components that provide basic storage and retrieval services. The functional components include: a data reception and cataloging system, a database for metadata about the millions of stored files, a user interface and web server for data retrieval specification, and a mass storage system (automated tape libraries) for file storage. The storage processing component of the Archive records information about the data files (name, size, date received, etc.) and supervises the distribution of the data files to the permanent storage (mass storage system and backup copy). The database not only tracks the inventory of the Archive, it also supports searches for requested data based on sites, facilities, date range, instruments, and measurements. The database monitors incorrectly named data files and files with insufficient documentation, and also controls the retrieval of requested files and generates reports on data usage and user attributes. The web hosted user interfaces include query, catalog, and graphics based logic. The mass storage system includes a hierarchy of disk and tape storage media. It is capable of storing millions of data files and 100’s of terabytes of data. It can also store or retrieve 100’s of gigabytes of data per day. The Archive also includes an online storage structure and navigation interface for non-continuous and specialized measurements generated by the ARM Intensive Operational Periods.

Significance

The ARM program conducts scientific field experiments studying atmospheric radiation balance, cloud feedback processes, and other atmospheric and environmental issues. Information gleaned from these field experiments is meant to improve scientific understanding of atmospheric radiative energy transfer, cloud formation, and the parameterization of these functions in general circulation models (GCM) for global climate change research. Measurements collected by the ACRF include numerous parameters about radiation, meteorology, water vapor, aerosols, and clouds. These measurements generate enormous amounts of data, which are made available to the scientific community through the World Wide Web. The Archive is the chief repository for these data and provides a gateway for access to them. The ARM Data Archive is intended to facilitate climate change research by providing climate related data about the Earth’s atmosphere in standardized formats free of charge. New efforts are now included to add high value data sets from the ARM Archive and Carbon Dioxide Information and Analysis Center (CDIAC) data center into the Earth System Grid “data environment.” This will further facilitate climate modeling researchers working with climate observations during the evaluation of model results.