They tried, but it didn't work out too well. What they did manage to figure out is that the little bit of residue they leave behind is something similar to gunpowder.

[Kay comes down from the crows nest. Picking off more people would be easy, but he figures that'll do for now. Most will burn up in the morning, and as for the spiders-]

The spiders will calm down in the daylight. If they're not inside, no reason to go out of your way.