CLEVELAND - Angered and betrayed by LeBron James' decision to leave for Miami, Cavaliers owner Dan Gilbert accused the NBA's MVP of quitting during the playoffs.

Gilbert, who posted a letter to Cavs fans on the team's website shortly after James announced his plans to sign with the Heat, told The Associated Press late Thursday night that it's "accountability time" for James.

"He has gotten a free pass," Gilbert said in a phone interview with The AP. "People have covered up for (James) for way too long. Tonight we saw who he really is."

Gilbert feels James quit on the Cavs during their second-round series against the Boston Celtics, who rallied from a 2-1 deficit to eliminate Cleveland.

"He quit," Gilbert said. "Not just in Game 5, but in Games 2, 4 and 6. Watch the tape. The Boston series was unlike anything in the history of sports for a superstar."

The Cavaliers were beaten by 32 points in Game 5. During the game, James appeared distracted and uninterested, often glaring at Cleveland's coaches as the Cavs tried to foul to get back into the game in the second half. James also made some puzzling postgame comments, saying he had "spoiled" people with his play over seven seasons.

Slideshow   Fits for a King Reactions in Cleveland, Chicago, New York and Miami after LeBron James' decision to sign with the Miami Heat.

Gilbert also said he believes James quit on the Cavs in Game 6 of their series in 2009 against Orlando.

"Go back and look at the tape," he said. "How many shots did he take?"

Gilbert, who has owned the Cavs for five years, said he was most disappointed by James' behavior in the months leading up to the superstar's announcement that he is going to Miami to play with Dwyane Wade and Chris Bosh. Gilbert said James never returned a single phone message or text since the end of the season and that the Cavs were not informed of James' decision until he went on the air.

Gilbert said Rich Paul, one of James' business partners, called the Cavs moments before the announcement.

"LeBron James needs to go to another team with two superstars already so he can win a championship," Gilbert said. "We will win a championship before (the Heat) do."

Gilbert said he now wishes he had done some things differently with James, who spent seven seasons with the Cavs.

LeBron signs with Miami Wade says Lakers, not Heat, are team to beat Cavs fans back owner, offer to help pay fine Cleveland begins erasing LeBron's big presence Bosh was first domino to fall for Heat supertrio PBT: LeBron, Wade, Bosh leave $15M on table PBT: LeBron becomes the ultimate facilitator Ventre: There has to be no ‘I’ in super team Celizic: Time for LeBron to put up or shut up   Vote: How many titles will Miami win?

"It's not about him leaving," Gilbert said. "It's the disrespect. It's time for people to hold these athletes accountable for their actions. Is this the way you raise your children? I've been holding this all in for a long time."

In Miami, Wade said he was stunned by Gilbert's comments.

"I think I'm happy that I have the owner that I have here in Miami," Wade told The Associated Press late Thursday night. "I'm happy Micky Arison is my owner. I couldn't believe it. I'm speechless. It's very unfortunate and I think it makes LeBron that much better about his decision.

"We knew 'Bron would take some backlash," Wade added in his interview with AP. "I told him he's a strong man for it."

Also on NBCSports.com Beyond the Arc: No. 11 Mich. St. takes big step forward Chandler: Jeremy Lin was special even in 8th grade What NFC teams need this offseason  |  And AFC DeMarco: Dodgers could become a contender Follow us on Twitter: @NBCSports  |  And Facebook