Let me begin by saying I will be 72 this October. I work out in a gym 3 or 4 days a week doing mostly cardio stuff on the treadmill and stairmaster. I also do leg work on the weight machines. My workouts take about 1 1/2 hours. During the spring/summer/fall, I walk 3 to 4 miles over hilly terrain near my place 3 to 4 days a week in the evenings.
The hardest part for most hunters (IMHO) is getting used to the altitude, especially the ones who live at less than around 3000 feet. I live at 6800 feet so I have a distinct advantage in that respect.
At my age and even with a total knee replacement that is 10 years old, I can usually keep at it all day. Each summer I do several high altitude hikes in a local wilderness area to fish. Usually do the 10 or so mile roundtrip in a day and hike to 11,000 to fish.
Just keep at it, and at my age, no looking back as they might be catching up!!!