Small cities in the US started out as centers for the agriculture industry. Since agriculture has become more productive, they don't provide enough jobs to prevent small cites from collapsing. Some small cities, like Rochester, NY, survived and prospered for some time because new businesses were formed that prospered for awhile after the decline of its agricultural base. It has struggled since the decline of Eastman Kodak and Xerox which were central to its economy. Large cities typically have more businesses so they are not as dependent upon the survival of a particular business activity. They are typically located near educational institutions which seed new business formations or they have other geographical advantages such as nice weather or access to transportation centers.
Krugman concludes that the demise of small cities had little to do with globalization. They were primarily affected by changes in the agriculture industry. He does not discuss the decline of large cities like Detroit which were dependent upon the auto industry which suffered from foreign competition. More generally, he does not offer explanations for the shifting of our economy from a manufacturing base to a services economy.