Due to the complexity of DNSBL 5, there is more going on here than just a regular DELETE FROM, in our databases. We do have honeypots and reporters that has started to send referring data to each ip that is blacklisted.
Our local honeypots are currently also not only blacklisting hosts - they are also saving each spamMessage as a referring entry since some of the ISP's are asking "What happened? Why are we blacklisted?". To make it easier for those requests we are also from version 5.0 saving the reason of it (as long as there is any).
Each time the API is requesting for a removal from the blacklist, this is happening:
- What referrers exists for this IP? (How many times has the IP actually been reported?)
- Is this the only IP that is linked to each referrer found?
- If the answer is yes, mark the IP as deleted and delete the referrer
- The DNSBL are stopping reporting this host as deleted.
- If no more reports are made from this ip, it will be purged
- If there is more reports being made from this ip, it will be reactivated as listed again.
- Removals will take longer time for each time it has been listed
(Referrers are quite unique due for example the Date-field in a spam message)