Why do I keep getting stories from people I'm not following on my inbox page?

At first it used to happen occasionally but now it happens all the time and it's annoying. Is there a way to disable this feature?