If you're not using the dryer, I suspect it isn't dryer lint at all.
What kind of insulation do you have in your attic (or perhaps even the walls)? If it's the fluffy blown-in type, chances are, what you're finding is insulation that is somehow being sucked into your living space from the attic (or walls).
Look closely at the linty debris you're finding and compare it to insulation in your attic or walls. If it looks similar -- you'll know the origin of your problem.
Q
Logical
Q