Chief Justice John Roberts has declined Democrats' request to discuss recent incidents surrounding conservative Justice Samuel Alito related to flags associated with far-right movements that flew outside of his homes.

The move comes one day after Justice Alito rebuffed calls to recuse himself from cases related to former President Donald Trump and the Jan. 6, 2021, insurrection at the U.S. Capitol.

In letters to members of Congress on Wednesday, Alito said his wife was responsible for flying an upside down flag over his home in 2021 -- used as a symbol of the "Stop the Steal" movement -- and an “Appeal to Heaven” flag -- which dates back to the Revolutionary War but in recent years has become associated with Christian nationalism and support of Trump -- at his New Jersey beach house last year.

Both flags were were carried by Capitol rioters on Jan. 6 and have become symbols of support for Trump. News of the flags was first reported by The New York Times. 

Last week, Illinois Sen. Dick Durbin, the chairman of the Senate Judiciary Committee, and Rhode Island Sen. Sheldon Whitehouse, a member of the panel who chairs its subcommittee on federal courts, sent a letter to Chief Justice Roberts asking for a meeting and urging him to take steps to ensure Alito's recusal. In a letter of his own, Roberts wrote that he "must respectfully decline" the request for a meeting, citing "separation of powers concerns."

"Separation of powers concerns and the importance of preserving judicial independence counsel against such appearances," Roberts wrote. "Moreover, the format proposed — a meeting with leaders of only one party who have expressed an interest in matters currently pending before the Court — simply underscores that participating in such a meeting would be inadvisable."

Justices decide for themselves when to step aside from cases, Roberts noted. Alito said he concluded nothing about the flags, both of which he said were flown by his wife outside their homes in Virginia and New Jersey, required his recusal.

"In considering whether [flying the upside down flag] requires recusal, an unbiased and reasonable person would take into account the following facts," Alito wrote in the letter to Sens. Durbin and Whitehouse on Wednesday. "As I have stated publicly, I had nothing whatsoever to do with the flying of that flag. I was not even aware of the upside-down flag until it was called to my attention. As soon as I saw it, I asked my wife to take it down, but for several days, she refused."

"My wife and I own our Virginia home jointly. She therefore has the legal right to use the property as she sees fit, and there were no additional steps that I could have taken to have the flag taken down more promptly," he continued, later adding that his wife "is a private citizen who possesses the same First Amendment, rights as every other American. She makes her own decisions, and I have always respected her right to do so."

Of the "Appeal to Heaven" flag, Alito wrote: "I recall that my wife did fly that flag for some period of time, but I do not remember how long it flew. And what is most relevant here, I had no involvement in the decision to fly that flag."

He went on to say that his wife "is fond of flying flags," while he is not, and said that he was "not familiar" with the flag, nor its connection to the "Stop the Steal" movement -- and his wife similarly unaware of any deeper meaning.

He went on to charge that those calling for his recusal are "motivated by political or ideological considerations or a desire to affect the outcome of Supreme Court cases."

Democrats in the House and Senate have continued to call for Alito's recusal from cases involving Trump and Jan. 6 defendants. Just hours after Alito released his statement on Wednesday, a group of 19 House Democrats signed on to a letter urging Chief Justice Roberts to put pressure on Alito to recuse himself.

Spectrum News' Ryan Chatelain and The Associated Press contribted to this report.