The Breakthrough Listen Search for Intelligent Life: Public Data, Formats, Reduction and Archiving. (arXiv:1906.07391v2 [astro-ph.IM] UPDATED)

Breakthrough Listen is the most comprehensive and sensitive search for extraterrestrial intelligence (SETI) to date, employing a collection of international observational facilities including both radio and optical telescopes. During the first three years of the Listen program, thousands of targets have been observed with the Green Bank Telescope (GBT), Parkes Telescope and Automated Planet Finder. At GBT and Parkes, observations have been performed ranging from 700 MHz to 26 GHz, with raw data volumes averaging over 1PB / day. A pseudo-real time software spectroscopy suite is used to produce multi-resolution spectrograms amounting to approximately 400 GB hr^-1 GHz^-1 beam^-1. For certain targets, raw baseband voltage data is also preserved. Observations with the Automated Planet Finder produce both 2-dimensional and 1-dimensional high resolution (R~10^5) echelle spectral data.

Although the primary purpose of Listen data acquisition is for SETI, a range of secondary science has also been performed with these data, including studies of fast radio bursts. Other current and potential research topics include spectral line studies, searches for certain kinds of dark matter, probes of interstellar scattering, pulsar searches, radio transient searches and investigations of stellar activity. Listen data are also being used in the development of algorithms, including machine learning approaches to modulation scheme classification and outlier detection, that have wide applicability not just for astronomical research but for a broad range of science and engineering.

In this paper, we describe the hardware and software pipeline used for collection, reduction, archival, and public dissemination of Listen data. We describe the data formats and tools, and present Breakthrough Listen Data Release 1.0 (BLDR 1.0), a defined set of publicly-available raw and reduced data totalling 1 PB.