“The family torment is an open sore. There is such a profound breach there,” royal biographer Catherine Mayer tells PEOPLE

Prince Harry and Prince William’s lack of interaction during the Duke of Sussex’s recent trip to the U.K. to see King Charles amid the news of his cancer diagnosis highlights the hurt between the brothers.

Prince Harry, 39, has made no secret of his desire for reconciliation and is thought to have reached out to Prince William, 41, but there were “no plans” to meet, a palace source said at the time of the visit. A former courtier suggests that the Prince of Wales’ “principal concern is his wife” Kate Middleton amid her recovery from abdominal surgery, yet Harry spent just 24 hours in his home country to briefly see his father without a visit with his brother’s family.

“It would be the normal run of things to go and see your sister-in-law who’s had a serious operation and also see your niece and nephews. But it isn’t normal. It’s terribly sad,” a source close to the royal household tells PEOPLE exclusively in this week’s cover story.

“The family torment is an open sore. There is such a profound breach there,” adds Catherine Mayer, author of the biography Charles: The Heart of a King.

ODD ANDERSEN/AFP via Getty, Andy Stenning – WPA Pool/Getty Prince William at the coronation of King Charles and Queen Camilla in London on May 6, 2023; Prince Harry at the coronation of King Charles and Queen Camilla in London on May 6, 2023.

Trust continues to be a concern following the Duke of Sussex’s memoir, Spare, and the revealing Netflix docuseries with his wife, Meghan Markle. During Harry’s visit with his father, which lasted about 30 minutes, Queen Camilla made a deliberate effort to be in the room.

DOMINIC LIPINSKI/POOL/AFP via Getty

Prince Harry and Prince William at an unveiling of a statue of their mother, Princess Diana at The Sunken Garden in Kensington Palace, London on July 1, 2021, which would have been her 60th birthday.

“This is all to do with William’s wish to protect the institution of the monarchy, which he feels Harry can’t be trusted with. I don’t think William will ever agree to readmission of Harry into the family unless he clearly apologizes and moves forward,” royal author Robert Lacey tells PEOPLE.

Prince Harry may have another opportunity to reconnect with his family in May when he’s expected to visit the U.K. for the 10th anniversary of the Invictus Games. However, a reconciliation with his brother is unlikely, and it’s even more doubtful that King Charles will mediate any healing between his sons.

“Of course he would like Harry to be the prodigal son and come back and would be much happier if William and Harry were friends again. But he realizes that is just not about to happen,” says royal biographer Ingrid Seward, author of My Mother and I.

Adds a palace insider: “Families are complicated. They may be a high-profile family, but they still have a lot of the same issues as everyone else.”