Sounds like you are up for some fun...
Things I have loved doing with my son:
Hikes through parks, arboretums, forest. Take a special fun snack, a picnic lunch, some warm herbal tea with honey, and a small blanket. Bring bubbles. Sing songs.
Drawing in nature. A trip to the park can be extended by collecting a bunch of leaves and tracing/drawing them, or any other thing you find enjoyable. Kids get to see the shapes of the plants when we trace around them.
Or, make a little 'adventure bag' for them, which fits onto their bodies comfortably. They can put a little trail mix, some toy binoculars (or you can make some from toilet paper tubes), a few tissues and bandaids... and then put their little treasures (pinecones, rocks, acorns) into the bag as they find them.
Go to a beach, or the sandy shore of a river. Play, throw rocks, throw leaves in the water and watch them float away. Collect rocks of a color.
Make a fun 'list' of things to do or find in your neighborhood. I like to do this when kids are reluctant to go for a walk. Draw picture of each item they're likely to find during their walk (squirrels, a jay or crow, a cat, a bike.... or your neighborhood 'kid' landmarks like a mailbox or funny yard decorations), then let the kids bring a crayon and cross the items off as they are found. Or bring sidewalk chalk and mark a 'trail' of X's (or the first letter of their names, my preschoolers LOVED that one) to your destination and look for them on the way back.
Buy postcards together at a store or cafe, write them to grandparents or other persons beloved by your son, and then take him to the post office to buy stamps to send it. (or use your own stamps and go to a big mailbox and let him put them in.)
Tour the fountains or public art in your city. Most cities will have a website which lists good walking tours; google your town and 'fountains tour' and you can usually come across some good information. I used it to see when the fountains were on during the season as well. It's pretty disappointing to get there and see nothing b/c the water is turned off. (Learn from my mistake!)
Play together with a big box. We've turned boxes into log cabins, spaceships, a cave, and my son has had a big box in his room for the past month. It has a skylight door and a side door. He likes to do his homework in it with a flashlight. Do not underestimate the potential of a box!
Make a fort. Go some place in nature where it's okay to do a little moving of branches and just build together. It's fun. Or build one with blankets and pillows under a table, between chairs, behind the couch...
Now, something I want to say: remember that these memories, from this age, are going to be mostly for you. So, don't sweat the small stuff, just have fun, and take pictures if you like what's going on. Kids don't need things to be all-out exciting or perfect-- they have the best fun when WE are having fun with them. :)