The trustee handling the liquidation of Bernard Madoff’s assets sued his sons, brother and niece for almost $199 million, claiming that they bilked investors out of almost $199 million when they treated the business like the “family piggy bank.”

Irving Picard, the trustee, sued Madoff’s sons, Mark and Andrew, his brother Peter, and his niece, Shana on Friday, for transferring customer money for their personal use and for breaching their fiduciary duties as principals of Bernard L. Madoff Investment Securities (BLMIS).

“Simply put, if the Family Members had been doing their jobs—honestly and faithfully—the Madoff Ponzi scheme might never have succeeded, or continued for so long,” the complaint said.

BLMIS was run as a family concern with Peter acting as chief operating officer and Shana, Peter’s daughter, who is a lawyer, acting as chief compliance officer. Bernard’s sons Mark and Andrew were given the titles co-director of trading.

After Madoff turned himself into the FBI in December and confessed that he had been running a $65 billion Ponzi scheme, the trustee discovered that trades investors thought were being made in their accounts were all a sham. His sons have said that they called the FBI after their father told them about the fraud.

Madoff is currently serving his 150-year prison term after he pleaded guilty to his crimes.

Now, the trustee is alleging that Madoff’s family members who were part of the business knew or should have known that their supposed profits from investment accounts were a total fiction. Not only that, but they took handsome salaries and commission to fund extravagant lifestyles, the trustee claimed.

Peter, the trustee alleged, got more than $20 million from BLMIS for his personal use, including about $4.5 million toward the purchase of a Park Avenue apartment and about $275,000 for an Aston Martin automobile.

BLMIS funds paid all aspect of Mark’s high-end lifestyle that included a $5.6 million apartment in Manhattan, a $6.7 million home in Nantucket, Mass., and a $2.2 million house located in Greenwich, Conn., according to the lawsuit. Andrew got millions for New York luxury apartments, almost $70,000 for a Connecticut marina, while Shana bought a $2.9 million house in East Hampton, N.Y. and $30,000 for an interior decorator with BLMIS money, the trustee alleged.

Investors have, all along, voiced suspicions that family members knew more than they were saying about Madoff’s scheme.

“I don’t understand why conspiracy wasn’t part of the plea,” an investor who identified himself as Mr. Newberger said in court when Madoff pleaded guilty back in March.

His family has maintained that he kept them in the dark about the investment business.

Attorneys for the family members did not immediately return calls from HedgeFund.net seeking comment.