The libraries often have family story time that will accomodate both ages. Go online to www.wccls.org.
Try the recreation center nearest you for classes or camps. It is tough to find a class for the little ones age anywhere close to the times of the older one. I'm struggling with that, but while the older one is in class, maybe you could take the tot to the playground. Many rec. centers offer indoor play park. Check the Tualatin Hills Recreation Center site for these offerings at your local rec. center.
Go online to MeetUp and see if they have any play groups available for your needs.
You might set up a scavenger hunt for your kids the next time you go to the park or hike one of the many trails. There is something called letterboxing, a treasure hunt of sorts with boxes hidden in and around Portland and many other locales (if on vacation). Search for letterboxing with kids for more info.
Blow bubbles outside on your porch or watch/help them ride their bikes.
Good luck on finding some happy times.
D.