“Sadly, [Detroit] is not the only municipality in trouble,” Glenn Hubbard, economist, Columbia University Graduate School of Business dean and Mitt Romney campaign adviser, told CNBC’s “Squawk Box” on Monday. “A lot of state and local governments have too much debt, too generous public pensions. We need a national conversation on how to fix this.”

(Read More: Pension over-promises need renegotiating: Hubbard)

One of the thorniest questions that conversation will need to address: who will pay to clean up these financial messes? Will it be the millions of retirees owed trillions of dollars in benefits, the bondholders who lent states and cities trillions more, or local taxpayers who may have to pay more to cover the shortfalls or see deeper cuts in public services?

Regardless, the painful process will likely play out for years.

“Moving pension plans is like steering a blimp: You turn the wheel and you go 6 miles before it starts to turn,” said John Tuohy, Arlington County, Va., deputy treasurer, who chairs the pension committee of the Government Finance Officers Association. “In the political process, that kind of patience is very difficult.”

Many state and local governments have set aside enough money to comfortably make good on promised retirement benefits. Seventeen states have funded more than 80 percent of their projected pension liability, a level that’s generally seen as financially sound. Most of the rest have been scrambling to make up investment losses inflicted by the 2008 market collapse and the shortfalls in sales, property and incomes taxes produced by the Great Recession.

But even as the economy and housing markets have recovered, most states are still falling behind in closing their pension funding gaps. In the last year, 34 states have seen their pension funds stretched further as they’ve failed to make the full contributions needed to meet the projected cost of retirement promises.

Much like a family that fails to save regularly to build a retirement nest egg, shortchanging those contributions increases the risk that the fund eventually will go broke.

(Read more: If Detroit cuts pensions, will your city be next?)

Nine states—Hawaii, Alaska, Kansas, Rhode Island, New Hampshire, Louisiana, Connecticut, Kentucky and Illinois—have now set aside less than 60 percent of what they need. Illinois has saved just 43 cents to cover every dollar of what it needs to pay 350,000 retirees and 500,000 current plan participants who are counting on a pension check.

In Detroit, city officials argue that pension payments to retirees simply have to be cut because the money just isn’t there to pay them. But union officials there and in other cash-strapped cities say that’s the city’s problem.

“Our members were promised certain things,” said Tom Ryan, president of the firefighters’ union in Chicago, where years of underfunding have prompted proposals to cut workers’ retirement benefits. “They enter dangerous situations every day, and the only thing they want to look forward to when they can no longer perform their duties is to be able to retire with some sort of security. People expect us to be there, and we are always there. We expect that the city holds up their end of the bargain when we signed on to be firefighters and paramedics for the city of Chicago.”

Without a pension check, public sector workers face a bleak retirement. Many are ineligible for Social Security.

“If we were talking about doing this to people with Social Security there would be rioting in the street,” said Ryan. “But because it’s public servants on pensions it seems to be OK to do this.”

Most cities and states with funding gaps still have time to fix the problem. Of the roughly 20,000 municipalities in the country, only a handful have completely run out of cash and been forced to seek shelter in bankruptcy court.

(Read more: Detroit joins list of failed US towns)

What’s less clear is whether those states and cities have the political will to make the necessary, unpopular decisions, according to Jean-Pierre Aubry, assistant director of state and local research at the Center for Retirement Research at Boston College.

“Even the ones that are really up against it have somewhere around 10 to 15 years of a window to turn things around,” he said. “That’s not a whole lot of time. And it’s not clear if these governments that couldn’t make contributions when times were good and the economy was better will be able to make them now or going forward.”

That political challenge is playing out in Illinois, where a $95 billion pension fund gap has deadlocked the state legislature over reform proposals for two years. In March, the nation’s fifth most populous state settled with the Securities and Exchange Commission after the agency charged Illinois with fraud for misleading bond investors about its pension funding problems between 2005 and 2009.

(Read more: Detroit bankruptcy case could bring unwanted change for munimarket)

After lawmakers adjourned in May without fixing the problem, the state’s credit rating was slashed, raising future borrowing costs. Earlier this summer, Gov. Pat Quinn used a budget line-item veto to freeze lawmakers’ salaries effective August. Last week, the Illinois House speaker and Senate president hauled Quinn into court to get their pay restored….more here