An undocumented immigrant from Ecuador, held for deportation when he delivered pizza to an Army base in Brooklyn, left an immigration detention center Tuesday after a federal judge ordered his release.

"Thank you, thank you for everything! Thank you to the press!" Pablo Villavicencio said to members of the media after he was released. "I love you!" he said, kissing one of his daughters.

Villavicencio walked out of an immigration detention facility in Kearny, New Jersey shortly before 9 p.m. and was greeted with hugs from his jubilant wife, two young daughters, ages 2 and 4, and supporters.

Adriene Holder, the attorney-in-charge of the civil practice at the Legal Aid Society, said the rule of "law, humanity and morality'' prevailed and the Villavicencio family has "finally received a crucial measure of relief from their 53-day nightmare.''

"This decision should serve as a rebuke against the Trump Administration and its merciless crusade to tear families apart,'' Holder said in a statement.

Six hours after hearing legal arguments, U.S. District Judge Paul Crotty ruled that Villavicencio can remain in the United States while he exhausts his right to try to gain legal status.

"Although he stayed in the United States unlawfully and is currently subject to a final order of removal, he has otherwise been a model citizen,'' Crotty wrote.

"He has no criminal history,'' the judge added. "He has paid his taxes. And he has worked diligently to provide for his family.''

Villavicencio was arrested on June 1 while delivering pizza to Fort Hamilton base. Guards requested identification, and then ran a background check, which showed he had been ordered to leave the U.S. in 2010 but stayed.

At the time of his arrest, Villavicencio had an application pending to stay in the United States. His wife, a U.S. citizen, sponsored him and they began the process in February.

At Tuesday's hearing, Crotty put a government lawyer on the spot over the effort by immigration authorities to enforce the 2010 deportation order. He questioned the need to detain and quickly deport Villavicencio, who is 35 years old.

"Well, the powerful are doing what they want, and the poor are suffering what they must,'' the judge said after hearing Assistant U.S. Attorney Joseph Cordaro defend the government's actions.

"I mean, is there any concept of justice here or are we just doing this because we want to?'' the judge asked. "Why do we want to enforce the order? It makes no difference in terms of the larger issues facing the country.''

Cordaro argued for the case to be transferred to New Jersey because Villavicencio was being detained there by U.S. Immigration and Customs Enforcement. He said legal precedent dictated that New Jersey was the proper venue, an argument the judge rejected.

Cordaro said Villavicencio would still be able to pursue his application to become a legal U.S. resident after he is deported.

The case has attracted widespread attention amid a crackdown by the Trump administration on illegal immigration. Trump, a Republican, has said his policies are designed to keep the country safe.

Gov. Andrew Cuomo, a Democrat, said the federal government has "cruelly'' kept Villavicencio from his wife, Sandra Chica, and two daughters "for no legitimate reason.''

Villavicencio's two girls, who are U.S. citizens, played with toys on Tuesday as courtroom spectators around them observed the legal arguments. Villavicencio was not in court.

Another judge had already temporarily blocked his deportation last month.

Elected officials in New York City praised the judge's decision to release the father of two:

Attorneys for the advocacy group Make the Road, meanwhile, have recommended that undocumented immigrants who rely on the IDNYC cards stay away from federal facilities, including military bases, in the wake of Villavicencio and a Brooklyn couple being detained at military bases after showing their IDNYC cards.

That couple was released on bond Monday.