C.F.
The best approach I can suggest is to let her do it, but put a limit on it that she enforces on herself, or make it a learning activity.
If she's collecting words, I wouldn't go through and 'weed them out', because when she figures it out (and she will), you will lose her trust in you and that things at home are safe and waiting for her to get home.
So, two ideas:
1) Give a her a big shoebox or even a copy-paper box and some safety scissors. Tell her she can fill it up with words, but no more. When it gets full, she will need to go through it to weed out what she no longer wants. Emphasize that she can put anything in the box, but the box must be able to close, and that that is the only box she can use for this. Cover the box with plain paper (or wrapping paper) and let her decorate it and put it in a special place in her room/closet/etc. where she can get to it whenever she wants.
2) Take it another direction - buy her some composition notebooks (so the pages don't fall out) and some glue sticks and encourage her to cut out and then paste the words in there and do some artwork on each page (word=bird, draw a picture of a bird, etc.) As she gets older/more into reading, encourage her to string words together into sentences. She could use a couple of words she's found and then write in the connecting words to make a complete sentence. You'll end up with a bunch of books, but they will make for great memories, and are much easier to archive.
Hey, as habits go (and collecting), this one is pretty harmless. Mine picked up little rocks from the playground at preschool until she tried to bring home a bunch of them in her knapsack (like 15lbs worth!). That time, we had a conversation with the school director who let her know nicely that the rocks needed to stay on the playground for everyone...you get the idea. = )
Good luck!