I generally try to take them down in the afternoon the day they're emptied. The yard waste and recycling bins are later in the day so I don't bother taking the general trash bin in until they're all empty.
But *sometimes* I don't get to it until the following morning.