Coventry winger McSheffrey spent a period on loan at Elland Road last season during the club's promotion campaign, but after a series of unspectacular performances was not offered a permanent deal in the summer.
The 28-year-old played 11 times for Leeds, scoring one fortunate attempt, but has been back on form for the Sky Blues this term - netting four times in 10 Championship appearances.
"Gary's obviously moved on. We were excited when we signed him last season and it was an opportunity for him to get his career back on track," he told Leeds' official website.
"He played his part in us getting promotion and when summer came around we felt there were other avenues for us to go down.
"He went back to Coventry from Birmingham and he's looking like he's found his home from home, and he's done well there. I have to make decisions which I think are right."