Do you have entries in your log where the IP resolves to something like *.caida.org? This may be skitter. Check out the webpage at http://old.caida.org/Tools/Skitter/. If it really was skitter and you want that traffic to stop bothering you, send email to skitter-configs@caida.org and tell them to stop pinging your IPs.
