Tampa Bay had the tying run at third base in the eighth, but B.J. Ryan got pinch-hitter Toby Hall to ground out, turning the Rays away empty-handed.
Ryan wrapped things up in the ninth to earn his 10th save.
Coming off a three-game sweep at the hands of the Colorado Rockies, the Blue Jays were in trouble early against the Rays. Julio Lugo led off the game with a double and advanced on a bunt single by Carl Crawford.
Jonny Gomes followed with a line drive that seemed destined for the gap in right-center, but Hinske made a tremendous diving catch, holding the Rays to just a sacrifice fly.
"Hinske's play was a game-saver," Gibbons said. "If that ball gets through, you're looking at a whole different game. That may be a blowout-type inning."
An infielder until last year, Hinske is experiencing the outfield for the first time, but he played Gomes' drive like a seasoned professional.
"I took a good angle to the ball," he said. "I realized about halfway there that it was hanging up there a little bit and I was going to have a chance to make a play on it.
"I just laid out for it and caught it. It was cool -- it was exciting."
There was more nifty defense in the second, when Frank Catalanotto started an inning-ending double play, sliding feet-first to catch a sinking liner from Joey Gathright, then throwing to first base to double up Josh Paul.
"Those were two great plays that were key for the ballgame, and we were able to hold on at the end," said Glaus, who wasted no time in wiping out Tampa Bay's early 1-0 lead by smacking a two-run homer in the first, his 13th. The long ball struck a sponsor's display in the outfield seats and won a car for a lucky Jays fan.
Tampa Bay's Russell Branyan tied the game in the second with a leadoff homer to center, his fifth, but Hinske put the Jays back in front with a solo shot in the bottom of the second.
Hinske's drive came with a dose of controversy, striking a metal railing on the bullpen bench over the left-field fence and bouncing back into play. Crawford threw the ball back in and Hinske was tagged as he rounded the bases, but second-base umpire Greg Gibson ruled that the ball had gone out, giving Hinske his second homer of the season.
"When I got to second, I saw the umpire put his arm up and make the home run sign, so I kept going," Hinske said. "Then the ball came back, and I was like, 'What's going on?' "
The Jays added two more in the third. Lyle Overbay doubled home Catalanotto, then scored on Gregg Zaun's sacrifice fly.
A second sac fly by Zaun in the eighth inning gave Toronto some valuable insurance after the Rays had closed the gap to one.
Janssen turned in a solid if unspectacular outing, retiring eight of the last nine batters he faced. He allowed four runs on six hits over 6 2/3 innings, walking none and striking out three.
"He did what he had to do," said Gibbons. "He kept us in the game. It wasn't pretty -- it was exciting but the bullpen came through."
Janssen struggled with his command, but he concentrated on keeping the ball down.
"I knew I wasn't going to blow anybody away today -- not that I do normally, but I just knew I had to mix it up and not make too many mistakes," he said. "I got into a little bit of a groove in the fourth, but from the first pitch, I knew I didn't have great stuff."