The 46-year-old American started the fight brightly, moving around the ring to neutralize Valuev's long reach advantage. The 7-foot (2.14-meter) Russian, the tallest and heaviest champion ever, struggled to close down Holyfield but began asserting his jab as the fight wore on.
One judged scored the bout a draw, while the others had Valuev winning 116-112 and 115-114.
"Of course I am disappointed," Holyfield said. "I thought I had done enough to get the win. Now I have to go home and think about my future."
Holyfield (42-10-2) was attempting to become the oldest heavyweight champion. He had not fought since losing a one-sided decision to then-WBO champ Sultan Ibragimov more than a year ago.
Holyfield was criticized for this latest comeback attempt, but the "Real Deal" looked in great shape and was never seriously hurt by Valuev (51-1), who has avoided the top heavyweights and did little to improve his standing in the division. He was vulnerable to Holyfield's right hooks, even if the Russian also was never stunned.
"He made me work very hard for the win," said Valuev, the overwhelming favorite and underwhelming winner. "Holyfield was unbelievable with his speed. The fight was fought at a great tempo for the whole 12 rounds."
Holyfield was the busier fighter from the start, and his footwork did much to reduce Valuev's few punches to little effect at Hallenstadion.
The Russian carried a weight advantage of almost 100 pounds (45 kilograms), and planted himself in the center of the ring, failing often to cut off Holyfield or unleash punches of power.
Holyfield appeared to accept the decision. Still, he offered only grudging praise of his conqueror.
"His hands are not as slow as everyone thinks they are," Holyfield said.
For Valuev, his first defense of the WBA title was the biggest win on a resume that includes no fights against the Klitschko brothers, who hold the other major heavyweight belts.
Holyfield's manager, Ken Sanders, asked Valuev to offer a rematch. The Russian's management said it was possible sometime next year.
Holyfield, 0-4-1 in his last five title fights, said he hoped for another chance at a title.
"I'm not really interested in fighting just to fight," he said. "I'm interested in being the undisputed champion again."
He was last the undisputed champion — owner of the WBC, WBA and IBF titles — in 1992.
Struggling with financial problems, Holyfield may try to use this better-than-expected result to score a big payday against another champion or Riddick Bowe, who has made a comeback.
Holyfield won one of three epic battles against Bowe in the 1990s, and beat James "Buster" Douglas, George Foreman and Mike Tyson twice to become a dominating figure in boxing.
The fight was in front of 12,500 largely pro-Holyfield fans, who gave his entrance a standing ovation. He wore a red robe fit for a king. Valuev was booed, and there were even scattered whistles while the Russian national anthem played.
The fans similarly showed their displeasure when the judges' scorecards were read out.