The Justice Department is now involved. Civil lawsuits are on the rise against broker commissions; this presents a challenge for the real- estate industry.

The residential real-estate industry is bracing for a challenge to the commissions charged by its sales agents, one that could put downward pressure on the fees paid by home buyers and sellers.

The Justice Department is investigating home sale commissions, and in a wide-ranging executive order President Biden asked the Federal Trade Commission to adopt rules to address unfair or exclusionary practices in the real-estate industry.

Several civil lawsuits challenging industry rules and practices around commissions have survived initial procedural challenges and drawn support from the Justice Department, putting added pressure on traditional broker fees.

Buy Financial Times and The Economist Combo 3 Years Save 77% Off

The politically powerful real-estate industry has survived past challenges to its commission structure, but consumer advocates say rising home prices have exacerbated concerns about excessive fees.

“The litigation and the government attention that the industry is getting now is unprecedented,” said Stephen Brobeck, a senior fellow at the Consumer Federation of America and a longtime critic of the industry.

At issue are the commissions real-estate agents earn for the sale of a home, typically around 5% to 6% of the sale price. For a home sale at the recent national median price of about $375,000, a 5% commission would be $18,750—or for a $1 million home, it would be $50,000.

Very high-end properties tend to have somewhat lower commission rates.

Home buyers end up contributing to these commissions as part of the purchase price, but often have little room to negotiate since it is the home sellers who generally set the commissions for agents on both sides of the deal.

Consumer advocates say this contributes to excessive commissions and point to the National Association of Realtors’ rules as the biggest roadblock to change. Those rules require sellers to offer commissions to would-be buyers’ brokers, which consumer groups say encourages sellers to offer high rates for buyer agents as a way to attract more potential buyers.

Industry critics also say that the fees are opaque to most buyers, and say the advent of online home search engines has diminished the traditional role buyers’ agents play in connecting buyers and sellers.

The National Association of Realtors says a tight sales market and rising prices have made real-estate agents more important than ever, and it says that commissions are fully negotiable and declining. The average national commission rate is currently in the range of 4.9% to 4.94%, according to industry news and research site RealTrends, down from 5.40% in 2012.

Subscribe to Financial Times Digital Subscription FT Epaper 3 Years Buy Now and Save 70% Off

The NAR says that the current commission structure also encourages more competition by giving all participants in local multiple-listing services equal access to information on available properties. It also helps make homes more affordable to first-time and lower-income buyers, the trade group says, because they don’t have to pay their own agents up front.

“NAR remains committed to advancing and defending independent, local broker organizations that provide for greater economic opportunity and equity for small businesses and consumers, including first-time, low- and middle-income home buyers,” said NAR general counsel Katie Johnson.

News Corp, owner of The Wall Street Journal, operates Realtor.com under license from the National Association of Realtors.

Under the Trump administration, the Justice Department and the National Association of Realtors reached a deal to provide more disclosure on broker fees and make them more competitive. But this summer, the Justice Department under President Biden said it was withdrawing from the settlement so that it could pursue a broader investigation into broker commissions.

The Justice Department said that Americans paid more than $85 billion in home sale commissions last year, and that industry practices “may harm home sellers and home buyers.” Last week, the NAR challenged the department’s withdrawal, saying it would fight a Justice Department civil subpoena.

The civil subpoena issued by the Justice Department in July largely focuses on NAR and industry policies that could be shaping the commission structure, according to court records. For example, the subpoena requires NAR to “submit all documents relating to any policy, guideline, rule, or practice: a. requiring listing brokers to make an offer of compensation to buyer brokers to list a home” on a multiple-listing service.

The subpoena also probes practices that could restrict marketing of non-MLS listings—another subject of recent private lawsuits.

In a further sign of the mounting risks to the industry, the Justice Department has taken sides against NAR and other industry defendants, filing statements in several civil lawsuits. The Justice Department statements generally dispute the NAR’s contention that a 2008 settlement with the department gave the government’s blessing to the current commission structure.

Mr. Biden also has pushed the Federal Trade Commission to launch its own review. The president said in a recent executive order targeting competition problems that the FTC should consider rules to curb “unfair tying practices or exclusionary practices in the brokerage or listing of real estate.”

The FTC didn’t respond to a request for comment.

The Justice Department investigation could lead to an antitrust suit and eventual sanctions on the industry. The prospect of FTC action is more distant, given the procedural hurdles that the agency faces in adopting new rules.

The current Justice Department investigation began earlier this year. After the change in administrations, the department was eager to pursue a broader look at the industry and unilaterally withdrew from the settlement pact with NAR.

The department is concerned that an array of practices creates a closed marketplace for the buying and selling of homes, locking in higher fees and preventing maverick firms and agents from successfully offering lower-cost models.

These concerns are especially true at a time of potentially disruptive change in the industry, accelerated by the Covid-19 pandemic, which for example saw a boom in virtual models for touring homes and navigating the real-estate market.

Several civil suits against the industry seeking class-action status have been filed by private law firms, including one by Cohen Milstein and Susman Godfrey filed in U.S. District Court in Chicago that survived initial procedural challenges but hasn’t yet been granted class-action status.

The lawsuit contends NAR rules stifle competition, which Ms. Johnson disputes. Realtors “already are competing on commission, from offering varied commission models to flat fees,” she said.

U.S. District Judge Andrea Wood in Illinois sided with the plaintiff home sellers in denying the defendants’ motion to dismiss the case.

“Plaintiffs’ allegations plausibly show that the Buyer-Broker Commission Rules prevent effective negotiation over commission rates and cause an artificial inflation of buyer-broker commission rates,” the judge wrote last year. “But-for Defendants’ conspiracy, each Plaintiff would have paid ‘substantially lower commissions.’”