According to Yardbirds drummer Jim McCarty, the basic outline for "Over Under Sideways Down" was inspired by Bill Haley and His Comets' "Rock Around the Clock". The group had heard Haley's song on the radio after a gig and considered adapting it in their own style. At a later recording session, guitarist Jeff Beck started by playing the song's bassline and the group were soon working out an arrangement. Beck then switched to lead guitar and came up with the intro. McCarty recalled:

"We needed an intro, and Jeff peeled one out, an instantly recognisable peel that completely took us by surprise ... Over and over we listened back to that line, going back and forth over whether it belonged in the song. And then like a flash of lightning, we realised that it did. More than that, it made the song."