A 12-time Olympic medal winning swimmer wanted for questioning by Brazilian police returned to the United States before officers could seize his passport to ensure he was available for questioning, officials said Wednesday.

A Brazilian judge has issued a search and seizure warrant for US Olympic swimmer Ryan Lochte and teammate James Feigen as questions emerge about their account of being robbed at gunpoint early Sunday in Rio de Janeiro.

Two other swimmers, Jack Conger and Gunnar Bentz, were removed by Brazilian authorities on Wednesday night from their flight before it departed Rio de Janeiro to the United States, US Olympic Committee spokesman Patrick Sandusky told CNN Sports contributor Christine Brennan.

Federal police say Lochte left Brazil on Monday on a commercial flight, two days before the judicial order. Feigen is believed to still be in Brazil.

Lochte said he wouldn't make up a story like this, NBC's Matt Lauer reported Wednesday night. He said Lochte told him he and his three teammates were victims. NBC is broadcasting the Games in Rio.

Lochte compared the way police questioned him to a time in the United States when he talked with police after being robbed, Lauer said. Lochte said the Brazilian police asked few questions during what he thought was a casual interview. Lauer relayed that Lochte said he didn't think police expressed any skepticism over his account.

Police arrived Wednesday morning at the Olympic Village to take the swimmers' passports and gather further testimony from them about the reported robbery, Sandusky said earlier.

But the athletes were not there.

"The swim team moved out of the village after their competition ended, so we were not able to make the athletes available," Sandusky said in a statement, adding that the committee would "continue to cooperate with Brazilian authorities."

Lochte, Feigen, Bentz and Conger reported being robbed at gunpoint early Sunday in a taxi bound for the Olympic Village by individuals posing as armed police officers, the US Olympic Committee said Sunday.

The robbery -- during which one of the bandits was said to have put a gun to Lochte's forehead -- reportedly occurred after the athletes left a party at the French hospitality house. Lochte told Lauer on Wednesday, however, that the gunman pointed the gun at him, but it wasn't against his head.

Civil police say they are still looking for the taxi driver the athletes say they were with when they were robbed. Police say they have asked the driver to come forward.

(CNN)