After compiling a 40–28 win-loss record (including playoffs) in four seasons with the Raiders, Gruden replaced the fired Tony Dungy as head coach of the Tampa Bay Buccaneers in 2002, via a high-stakes trade that included Tampa Bay's 2002 and 2003 first-round draft picks, 2002 and 2004 second-round draft picks
, and $8 million in cash. The trade took place for a number of reasons, including Davis' desire for a more vertical passing attack rather than Gruden's horizontal pass attack, the fact that Gruden's contract would expire a year after the trade, and Davis' uncertainty over whether Gruden was worth as much money as his next contract was sure to pay him. Gruden signed a five-year contract with the Buccaneers worth $17.5 million.
The Bucs' search had taken more than two months, and had proven to be a major embarrassment to the Buccaneer organization. Tampa Bay had expressed an interest in Gruden, but Davis had originally refused to release him from his contract. The team subsequently interviewed several other coaches and believed a deal was in place with Bill Parcells, before Parcells backed out, reportedly because his choice for General Manager, Mike Tannenbaum, told him not to accept the job because of the salary cap difficulties that Tampa Bay was about to endure
. With the franchise's search floundering, the coach they wanted having only one year remaining on his deal, and the immediate hire of Dungy by the Indianapolis Colts, many fans and sports commentators began to openly question if the Bucs had made the right move by dismissing Dungy. Only a big splash hire could quiet the storm, and this may have been the primary motivation for the Bucs to give up as much as they did to acquire Gruden.
Immediately after arriving in Tampa, Gruden significantly retooled the offense with the addition of numerous free agents.
His determination to fix the under-performing offense so often maligned during Dungy's tenure inspired Tampa's defense to another #1 ranking, which helped the team to a 12–4 season. Both the offense and defense hit their stride in the playoffs; the Buccaneers posted a playoff per-game point differential of 23 points per game in victory, tied with the 1992 Dallas Cowboys for the highest average playoff margin of victory by a Super Bowl winner in the free agency era.
Fans were especially satisfied with a victory in the NFC Championship game against the Philadelphia Eagles, the team that had defeated Tampa Bay in the Wild Card round two years running by the combined score of 52-12, and Gruden was especially satisfied with a dominant win over his old team, the Raiders, in Super Bowl XXXVII. Despite the Super Bowl win, there were many, including players on the Buccaneers like Warren Sapp, who attributed Gruden's win primarily to the defense that coach Tony Dungy and defensive coordinator Monte Kiffin had created during Dungy's tenure with the Bucs. Gruden, for his part, publicly and graciously thanked Dungy for his contributions upon accepting the Lombardi Trophy at the Super Bowl XXXVII postgame ceremony.
His mantra for the 2002 season was "Pound the Rock", a reference to never giving up. Gruden even went as far as to display a large chunk of granite in the locker room, a tactic mimicked by the Jacksonville Jaguars. (Their slogan, "Keep choppin' wood", was tainted when punter Chris Hanson injured his leg on an axe brought in to accompany a large log.) Upon returning to Tampa after winning Super Bowl XXXVII, he led a capacity crowd at Raymond James Stadium in chanting the phrase. However, it seemingly disappeared from the lexicon the following year, and was not aggressively marketed or displayed on stadium video boards.
In the two years following Gruden's Super Bowl win, the Bucs went 7–9 and 5–11 respectively, implying to many Dungy supporters that Gruden had simply taken over a strong team and then driven it into the ground. However, the high draft picks sacrificed by the team to acquire Gruden, along with salary-cap issues and failed draft choices forced upon him by the now-departed Rich McKay (with whom Gruden had a bitter relationship) limited Gruden's ability to field the teams he wanted after that successful Super Bowl-winning season.
With no emerging talent in the fold and no money to afford replacements, the team was decimated by injuries to many of the Super Bowl stars, including Joe Jurevicius, Greg Spires, Shelton Quarles,and (cornerback), as well as acrimony with highly-paid veterans such as Sapp, Keyshawn Johnson and Keenan McCardell.