Ryan Lochte Says He's Worried He'll 'Become a Failure' if He Doesn't Reach Tokyo Olympics

Ryan Lochte is feeling the pressure to qualify for the Tokyo Summer Olympics.

The 36-year-old athlete recently spoke to Sports Illustrated about his hopes for reaching this year's Games, where he would become the oldest-ever men's Olympic swimmer for the United States.

"Everyone around me is putting a lot of pressure on me — more than I've ever had in my entire life," said Lochte, who won 12 medals, including six gold medals, at past Games to currently rank as the second-most decorated swimmer in Olympic history.

"I feel it from everyone," the athlete added of the pressure to SI. "Like my family. The people that live in this house. My agent. It's just everyone."

"I feel like, if I don't [make it], I'll become a failure," Lochte admitted.

Lochte recently spoke to PEOPLE about his preparations for the Tokyo Games, sharing that he's made it to every single swim practice at a quarantine-friendly facility near his family's Gainesville, Florida, home.

"There's been times when I don't want to wake up and go to the swimming pool, but she kicks my a-- out of the bed and is like, 'You're going to practice. You've got a job to do,' " Lochte said of wife Kayla, with whom he shares daughter Liv Rae, 22 months, and son Caiden Zane, 4 next month.

"She's been my rock," added Lochte. "I've been through a lot of tough times and she's been right there pushing me, keeping my head up and motivating me to just become a better person. We're a good team."

The one-year delay of the Tokyo Games due to the COVID-19 pandemic, while devastating for most athletes, gave Lochte even more time to practice, he told PEOPLE.

"I thought I was ready for 2020, and when they postponed it I started looking at the positive side of having another year under my belt," he said. "I'm going to get stronger, mentally prepare myself even more, I'm just going to be a better athlete. I'm so excited because I have so much determination and I'm just ready to go!"