First, you should consider yourself lucky. When my kids are scared, they come in yelling and wake everyone up. Or, they yell until I go in there. If he's coming in and making himself at home at the foot or side of your bed, he's being very courteous, and you should appreciate that. Not many kids will.
As long as he's not seeing anything inappropriate, and I assume you do that with a locked door, then I don't see anything wrong with that. He's scared and gets comfort by being near you. If it's not bothering you or waking anyone, I see no harm in letting him do it. He will outgrow it, and as the floor isn't too comfortable, it probably won't last too long.
You might try talking to him about what bothers him and see if it's an easy fix. My kids all need night lights. Two need closed doors and one needs hers open. All kids are different.
I wouldn't worry too much about it.