The episode's title is a reference to the 1990 mobster film "Goodfellas".
When Bender says "Ask not for whom the bone bones, it bones for thee," he is referencing John Donne: "never send to know for whom the bell tolls; it tolls for thee,"; the famous passage from Donne's Meditation XVII begins with another quotable phrase, "No man is an island". Ernest Hemingway referenced J.D.'s first phrase in the title of, For Whom the Bell Tolls.
An asteroid collides with Bender, and immediately afterward, the Shrimpkin civilization appears on his metal casing. This references the theory, known as Panspermia, that life can spread across vast distances of space by "hitching a ride" on space debris. 
Fry and Leela visit the "Monastery of Teshuvah". Teshuvah is the Hebrew word for repentance.
The use of a technologically advanced telescope to search for Bender is similar to the use of Professor Farnsworth's Smellescope in A Big Piece of Garbage. The Smellescope is also used, but dismissed as too low powered to locate Bender - the Professor mentions that Bender has a very mild odor.
An observatory located in a monastery is also a reference to The Nine Billion Names of God by Arthur C. Clarke.
Bender's godlike influence upon the diminutive Shrimpkins is expanded to include more serious implications in the "Microcosmic God" by Theodore Sturgeon and the The Twilight Zone episode The Little People. The tiny creatures in Alan Dean Foster's short story, Gift Of A Useless Man depend on the main character in a way similar to the Shrimpkins building their civilization 'on' Bender. The God Entity appears almost identical to the God Entity portrayed in Stanislaw Lem's "Voyages of Professor Tarantoga" screenplay.
Bender: But why would God think in binary? Unless ... you're not God, but the remains of a computerized space probe that collided with God. "God": That seems probable. This references the first Star Trek movie, in which a massive and powerful, albeit emotionally immature alien intelligence, was found to be human technology that had been found, modified with extremely advanced technology, and sent back. Star Trek V: The Final Frontier posits the concept of a being godlike in power that is nonetheless not God as humans conceive of it, i.e. not the creator of the universe, nor in fact good.
Billy West states on the audio commentary that the voice of "God" was based on the opening announcer from The Outer Limits.
This episode is one of only a few that deals with the religious issues of the Futurama universe. After Bender's unsuccessful attempt at godhood he encounters a god-like entity in space. During the conversation between these two the episode touches on the ideas of predestination, prayer, and the nature of salvation in what Mark Pinsky, who wrote a book on Futurama and Religion, referred to as theological turn to the episode which may cause the viewer to need "to be reminded that this is a cartoon and not a divinity school class". By the end of the conversation, Bender's questions still have not been fully answered and like many of the conversations between humans and God in the Bible, Bender is left wanting more from the voice than it has given him.
The astronomically long distance backwards pan as the God Entity becomes aware of Fry's plea to have Bender back is similar to one in the movie "Contact" and a short short 1968 film "Cosmic Zoom" by Eva Szasz.
Ken Keeler, writer and executive producer, said, "I took great pains in the script never to say that the Galactic Entity (as we called it) was in fact God, and fought some battles over that point during the rewrite. 
The piece Bender plays on the piano is Chopin's Polonaise in C Minor. Ken Keeler performed the piece. 
This episode won the first Writers Guild of America Award for animation in 2003
Series creator Matt Groening cites it as one of the best episodes of the series and was quoted as saying he planned to explore the idea of the "God" figure in a DVD movie, 19 months before Bender's Big Score was released.