• Taliban fighters began entering the Afghan capital of Kabul on Sunday, the last city to have been thus far spared takeover by the militants amid their breakneck blitz across the country.
• Since President Joe Biden's April decision to withdraw U.S. troops from Afghanistan, the Taliban have made stunning battlefield advances with now nearly the entirety of the nation under their control.

WASHINGTON — Taliban fighters began entering the Afghan capital of Kabul on Sunday, the last city to have been thus far spared takeover by the militants amid their rapid sweep of the country in the wake of U.S. forces departing.

A Taliban spokesperson said the fighters intended to negotiate a "peaceful surrender" of the city.

"Until a peace agreement is agreed, the security of the city and its residents is the responsibility of the government and they should guarantee it," Taliban spokesman Zabihullah Mujahid said in a statement.

Two U.S. defense officials confirmed to NBC News that the Taliban also seized Bagram Air Base, a development that comes less than two months after the U.S. military handed over the once-stalwart airbase to the Afghan National Security and Defense Force.

The group began emptying out Parwan prison there which has an estimated 5,000 to 7,000 prisoners, including hardened Taliban and al Qaeda fighters, according to the officials, who spoke on the condition of anonymity.

In 2012, at its peak, Bagram saw more than 100,000 U.S. troops pass through. It was the largest U.S. military installation in Afghanistan.

Since President Joe Biden's April decision to withdraw U.S. troops from Afghanistan, the Taliban have made stunning battlefield advances with now nearly the entirety of the nation under their control.

The group previously captured the strategic city of Ghazni, which had brought their front line within 95 miles of Kabul, a staggering development that spurred the deployment of 5,000 American troops back into the country to help with evacuations.

Britain and Canada also rushed troops into Kabul to evacuate their embassies.

The State Department has issued repeated calls for U.S. citizens to leave Afghanistan immediately, warning that its ability to assist citizens is "extremely limited" due to deteriorating security conditions and reduced staffing.

Despite being vastly outnumbered by the Afghan military, which has long been assisted by U.S. and coalition forces, the Taliban seized Kandahar and Herat, Afghanistan's second- and third-largest cities last week. The group also took the strategic town of Pul-e-Alam, a city that has one of the four main roads to Kabul.

Read more: Afghanistan’s war will spread beyond its borders as Taliban advances, senior negotiator warns

The Pentagon has previously said that the continued Taliban offensive across the country runs against a commitment made last year by the group to engage in peace talks with the Afghan government.

The peace talks, which are hosted in Qatar, have since stalled.

"What we're seeing on the ground is that the Taliban continues to advance and to assume control of district and provincial centers that clearly indicates that they believe it is possible to gain governance through force, through brutality, through violence, through oppression, which is at great odds with their previously stated goal of actually wanting to participate in a negotiated political solution," Pentagon spokesman John Kirby told reporters recently.

He added that while the Pentagon is concerned to see such advances by the Taliban, the Afghan military must now leverage the nearly two decades of training from U.S. and NATO coalition forces.

"They have the advantage in numbers, in operational structure, in air forces and in modern weaponry and it's really about having the will and the leadership to use those advantages to their own benefit," Kirby said.

"The recipe can't be just a constant U.S. presence in Afghanistan that never ends," he added.

Last week, Biden told reporters at the White House that he does not regret his decision to withdraw American troops from Afghanistan, despite shocking gains by the Taliban.

"Look, we spent over a trillion dollars over 20 years, we trained and equipped with modern equipment over 300,000 Afghan forces," Biden said.

"Afghan leaders have to come together," the president added. "They've got to fight for themselves, fight for their nation."

CNBC's Natasha Turak contributed to this report from Dubai, United Arab Emirates.