Yolanda Foster's Love of Fame Contributed to Split From David, Says Source

David and Yolanda Foster‘s marriage broke down in part over her pursuit of fame on The Real Housewives of Beverly Hills, according to a source close to the family.

In the days since the couple announced the end of their four-year-marriage, there has been much speculation over exactly what lead to their split.

And a source tells PEOPLE how David, 66, struggled with his wife’s bid to star on the Bravo show – and indeed, only appeared on the show to make her happy.

“Yolanda’s obsession with fame for her and her children is the cause of the split,” a source tells PEOPLE. “David didn’t want to be part of the show, but he knew how important it was to her. He wanted to make her happy. ”

A source close to Yolanda counters: “David enjoyed being on the show. He gained a whole new set of fans through Housewives.”

David and Yolanda announced their split last week after nine years together and four years of marriage.

“What’s so hurtful now is that there are these stories perpetuating that he’s a horrible guy, who’s left her out to dry,” says a source.

“David would have probably worked on the marriage,” adds the source. ” He’s 66 and he doesn’t want to be alone and on his fourth divorce.”

Yolanda has three children from a previous marriage to Mohamed Hadid: Bella, 19, and son Anwar, 16, and supermodel daughter Gigi, 20.

RELATED VIDEO: Yolanda Foster’s Heartbreaking Confessions

Calling into Watch What Happens Live Tuesday, Yolanda indicated that her battle with Lyme disease put a strain on the marriage.

“We had a beautiful love story that I will treasure forever. Unfortunately it was not strong enough to withstand the circumstances that came our way,” Yolanda said. “The truth is that we both worked through the difficult time through the best of our ability, so rather than judging what went wrong, I pick to honor the great times that we shed the last nine years. There were a wealth of good times.”