Bill Cosby is continuing to fight back against allegations of sexual assault during his long career, as his lawyers said Tuesday that they will ask a federal judge to toss the defamation lawsuit filed by three women who have accused him.

The motion should be submitted by Friday, the lawyers said according to a Billboard.com report.

The lawsuit was brought by Tamara Green, a California lawyer who has accused Cosby of drugging and sexually assaulting her back in the 1970s. Two women from Florida joined in the lawsuit, which was filed in December 2014. Therese Serignese claims that Cosby raped her in 1976, and Linda Traitz said he tried to drug and grope her in 1970.

Cosby tried to brand them as liars, prompting the women to file the defamation suit, they said. The suit seeks an unknown amount of money.

Cosby has repeatedly denied any wrongdoing despite scores of women coming forward alleging that he frequently attempted to drug and rape women. Even if there were compelling evidence of the crimes, the statute of limitations has passed for almost all cases, meaning he isn’t facing any charges.

The lawsuit was filed in Springfield, Mass., about 45 miles from Cosby’s home. The suit will allow them to air their allegations, according to the women’s lawyers.

Cosby is best known for playing Dr. Cliff Huxtable on The Cosby Show in the 1980s and early 1990s. The new allegations have forced him to postpone or cancel a number of his shows on his comedy tour that is currently going on.

Lawyers for the women will have until late March to file a reply to the motion to dismiss, and Cosby’s lawyers would be able to respond to that.