The mother of an alleged sexual abuse victim of Jerry Sandusky says that the boy felt he didn't have the power to say no to the former Penn State football coach.

The boy, referred to in court papers as "Victim 1," lived in fear of implicating the popular coach, his mother told ABC News.

"I had said, 'You know, maybe we should have come to this conclusion earlier — you should have told me,'" the mother, whose name is being withheld, said she told her son. "He was like, 'Well, I didn't know what to do … you just can't tell Jerry no.'"

Video   Sandusky investigation expands into Texas Nov. 11, 2011: Top Talkers: The investigation into former Penn State assistant football coach, Jerry Sandusky, has expanded beyond Pennsylvania and into Texas. In the grand jury indictment of Sandusky, a victim alleges Sandusky assaulted him during a trip to the '99 Alamo Bowl.

Sandusky, now 67, would often have the boy stay at his home after they met when he was 11 in 2005 through the Second Mile program, which the coach founded for at-risk youth. According to the grand jury investigation, Sandusky "indecently fondled Victim 1 on a number of occasions, performed oral sex on Victim 1 on a number of occasions and had Victim 1 perform oral sex on him on at least one occasion."

The boy's mother said her boy would act out violently to intentionally become grounded and avoid seeing Sandusky. Once, he asked her how to look up information on sex offenders.

"[I] proceeded to ask him if there was something he needed to tell me, if there was something going on … it wasn't 'til a month later when he indicated he was uncomfortable with leaving the school with him, and [Sandusky] pulling him out of classes at school," she said.

She expressed her concerns to the school and was quickly called to the school where she learned more about the situation.

"I'm infuriated," she said. "Even if they had the slightest inclination that anything inappropriate was going on it should have been reported, or at least brought to my attention," she said. "I didn't even know he was leaving the school with my child, taking him out of classes. They never told me that."

Sandusky was barred from the school district as soon as the mother had expressed concern about the coach's relationship with her son.

The boy testified to the grand jury that when staying in Sandusky's basement, the coach would come down and get into bed, crawling underneath him and running his arms up and down the boy's back to "crack" it. He testified that this led to further inappropriate touching during the summer of 2005 through 2006, when he was in seventh grade. This soon led to inappropriate sexual contact.

"I was horrified. I was absolutely horrified," Victim 1's mother said. "I knew some details but I didn't know that it was that, I didn't know it was that bad. It's caused a lot of nightmares, for him and I both."