So how do athletes get to be mentally tough? Many of them have survived tough times either at home or during their training.
"Adversity is a necessary pre-condition," Middleon said in an e-mail to Discovery News. "Without adversity in your life, mental toughness does not exist. Without adversity in your life growing up, how does one develop the self-disciplines and the know-how to overcome future adversities."
TV viewers have probably seen examples of this phenomenon, such as the athletes who come from troubled homes, for example, or return from devastating injuries to greater success.
RELATED: Olympics: How U.S. Athletes Fool Their Bodies
Gould says that overcoming adversity doesn't mean you have to abuse or hurt young athletes to make them tough.
Obviously we don't want something terrible to happen," Gould said. "But sometimes we put athletes in situation that are over their head once in a while."
That may mean training swimmers to practice with water-filled goggles, or putting young tennis phenoms in matches they probably won't win so they can learn, instead of cleaning up against weaker opponents.
RELATED: Olympic Doping Has a 50-Year History
"Good coaches do simulations," Gould said. "You are not making kids fail, but putting them in situations where they fail sometimes."
Mental toughness also comes into play during competitions when things don't always go as planned and the athlete has to manage distractions that are out of his or her control.
"Part of being mentally tough is do you have a plan A where everything goes perfectly, plan B something has gone wrong, plan C is all heck breaks loose," Gould said. "You have some strategies to go to in these events."