Embracing Statistical Challenges in the Information Technology Age

Information Technology is creating an exciting time for statistics. In this article, we review the diverse sources of IT data in three clusters: IT core, IT systems, and IT fringe. The new data forms, huge data volumes, and high data speeds of IT are contrasted against the constraints on storage, tr...

Full description

Bibliographic Details
Main Author: Yu, Bin
Other Authors: CALIFORNIA UNIV BERKELEY DEPT OF STATISTICS
Format: Text
Language:English
Published: 2006
Subjects:
Online Access:http://www.dtic.mil/docs/citations/ADA446888
http://oai.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA446888
Description
Summary:Information Technology is creating an exciting time for statistics. In this article, we review the diverse sources of IT data in three clusters: IT core, IT systems, and IT fringe. The new data forms, huge data volumes, and high data speeds of IT are contrasted against the constraints on storage, transmission and computation to point to the challenges and opportunities. In particular, we describe the impacts of IT on a typical statistical investigation of data collection, data visualization, and model fitting, with an emphasis on computation and feature selection. Moreover, two research projects on network tomography and arctic cloud detection are used throughout the paper to bring the discussions to a concrete level. The original document contains color images. Sponsored in part by the National Science Foundation.