The Geographic Names Information System (GNIS), developed by the USGS in cooperation with the U.S. Board on Geographic Names (BGN), contains information about almost 2 million physical and cultural geographic features in the United States and its territories.
Comma delimited files are provided via a free download mechanism (no FTP access available)
http://geonames.usgs.gov/index.html