Assuming EVERYTHING is constant except for air temperature, you will fall faster in hot air and slower in cold air because the drag force will decrease with a decrease in air density, and air density will decrease with a decrease in air temperature.
Force of drag = 0.5 X density of fluid (in this case air) X velocity^2 X Cd (drag coefficient) X A (reference area)
Sources:
http://en.wikipedia.org/wiki/Drag_(physics)
http://www.engineeringtoolbox.com/air-properties-d_156.html