In Colorado, it's my understanding if you want a full refund, you will lose your points. If you want to retain the points used to draw the tag, you have to forgo the refund.
On the other hand, you can return the tag, and forfeit the tag fee and retain your points. You do not get a point for the current year that you drew however, which will set you back one year from the pack. I did that in 2010 when I drew one of the top tags, but didn't have time to go on the hunt.