Listed below are some of the most generally-useful publications and sites for statistical data.
Here is a list of important statistics-generating agencies.
Many of the sources in this subject arrangement are important data sources.
The Office of Management and Budget publishes an annual overview (with links) of Federal statistical resources.
There are two offical Federal Government sites which are intended to act as central, one-stop sources of data from lots of agencies and sources:
Well-established site, with data from 100 agencies.
Relatively new site, primarily useful for datasets.
Annually-published compilation of statistical data -- excellent place to start. No longer published, but 1878-2012 Volumes are available online.
Time-series data. A more current update was published commercially and is accessible here .