Panetta said at a joint news conference with Karzai that the meeting is to be held during the week of Jan. 7.
The two leaders will use the meeting to discuss Afghanistan's future. Karzai said he and Obama will discuss the number of U.S. troops that will remain in Afghanistan after the combat mission ends there in December 2014.
Panetta also said that a suicide car bomber who staged an attack at the Kandahar Air Field killed one American serviceman and wounded three others. Two Afghan civilians were also killed in the attack Thursday. He told reporters that the bomber was aiming at a U.S. armored vehicle. It wasn't clear if the U.S. casualties were inside that vehicle.
Taliban spokesman Qari Jusuf Ahmedi claimed responsibility for the attack in an email, saying a suicide car bomber had targeted foreign military vehicles that were stopped near the gate of Kandahar Airfield.
It was unclear if the attack had anything to do with Panetta's unannounced visit to Kandahar Airfield. Kandahar is a huge and sprawling facility that houses more than 20,000 service members from 20 countries and has more than 11,000 civilian contract workers.