Whitney Port Says She 'Definitely' Wants Another Baby, Is Seeing Fertility Specialist Soon

Whitney Port continues to have hopes of expanding her family after she suffered a miscarriage last month.

The fashion designer, 36, tells PEOPLE that she and her husband Tim Rosenman "definitely want another baby" and will be talking to a fertility specialist to "start that process" at the end of this month.

Port revealed she and husband Rosenman suffered a pregnancy loss on Nov. 17, two weeks after announcing she was seven weeks pregnant with "likely another unhealthy pregnancy." Port has previously suffered two miscarriages and a chemical pregnancy.

"It is a daunting thought having to go through this process again and all the unknowns, but it is something that we're looking into figuring out because I know that while I'm scared to get pregnant again and scared to miscarry again and scared for the newborn phase and scared for the breastfeeding, I'm scared for all of it, but I know that if I think about my life in 20 years and I look back that, I will regret not going through those things," she explains.

Due to her history of miscarriages, the With Whit podcast host — who also shares 4-year-old Sonny Sanford with Rosenman — says her doctor told her it's "safest" to speak to a fertility specialist, though she has "no idea what that looks like."

"It's totally new territory for me, but it's just a little bit too scary to leave it to fate again at this point with my history," she adds.

Subscribe to our new 12-episode weekly podcast, Me Becoming Mom, to hear celebrity moms open up exclusively to PEOPLE about their extraordinary roads to motherhood.

While the couple would love to have another child, Port says she's "trying to go with the flow" and continuing to "tell [herself] that even if we can't have another baby, we are still so blessed to obviously have what we have."

"Whatever happens next is really just extra and I really do believe that," she continues.

Shortly after she shared the news of her pregnancy loss with the world, Port unveiled an emotional, raw audio diary on an episode of her Dear Media podcast With Whit during which she discussed how she was feeling after finding out she had lost the baby.

"I just feel I've realized that my place on social media is founded in vulnerability," she tells PEOPLE. "It's normalized now to talk about these things, but I think that every person has a different following, a different perspective, a different voice, a different experience. And so the more that I can share my perspective, the more people like me out there will feel less alone."

The reality star says she and Rosenman eventually explained to Sonny what had happened with their baby without getting into too much detail.

She says they initially told their son that "there was a possibility that there could be a baby growing inside my belly, but we still have to wait and see."

"I'm not sure how much he understood or grasped," she continues. "And then when it did happen, we ended up telling him, 'It stopped growing inside Mommy's belly, but we're going to keep working on it.' And he just said in response, 'Well maybe you just need some more time.' "

"He has since made little comments here and there about wanting someone to play with and wanting a brother or a sister. And it pings my heart a little bit, but there's really nothing I can do but take a step by step at this point," Port adds.

In her YouTube update shared on Nov. 17, Port and Rosenman said that at their latest visit the "doctor didn't hear a heartbeat."

"I went today for the eight [week] and four day ultrasound, and last week the baby had doubled and he heard the heartbeat, and then this week there was no heartbeat," Port tearfully explained, adding that the doctor "said it's done — that when I had that ultrasound that … looked like there was no embryo inside and the yolk sac was thin. That was maybe a sign of it not being healthy from the beginning."

Port shared in July 2019 that she had suffered a miscarriage, and announced that she had another pregnancy loss in January of this year.