AbuseIPDB Integration module

Quote:

---

Originally Posted by marcopolo

Great solution this modification in the code introduces an additional check to prevent unnecessary IP abuse checks.

If the current page is a 'page_not_found', the IP abuse check will be skipped.
If the visitor is a known web spider or bot, the IP abuse check will also be skipped.

This reduces unnecessary API calls to AbuseIPDB when traffic comes from known web spiders or bots, which are usually harmless.

PHP Code:

---

// Do not execute the check for the 'page_not_found' page or for known spiders
if ($current_page_base == 'page_not_found' || (isset($spider_flag) && $spider_flag === true)) {
    return;
} 


---

---

I've changed the modification to this to exclude spiders & 404. Just have to wait until tomorrow to check if block caching is now fully working, but on test mode all seems well. This mod seems to be evolving quickly, but it will do wonders to keep the worst offenders away & reduce my manual blocking workload massively.

Thank you all.

Also, as an aside in case anyone else wants to include an error message on their block page, I'm still serving a 403 but now echoing a browser message as follows.

Code:

---

  header('HTTP/1.0 403 Forbidden');
                echo 'You are forbidden! Your IP Address is marked as malicious in the abuseipdb.com database';
                zen_exit();

---

Any chance we can get a github link to avoid having to add, delete, rinse, repeat?

Quote:

---

Originally Posted by dbltoe

Any chance we can get a github link to avoid having to add, delete, rinse, repeat?

---

Here is the GitHub link: https://github.com/CcMarc/AbuseIPDB.git

I Updated to the latest commit from github, which was made three hours ago, as the API results caching wasn't working on mine. My API calls reset at midnight and by 7am GMT I had hit 3.5k API calls. Checking the calls log, there were multiples for the same IP. The latest commit, saving sessions to database appears to have fixed this. Good work :)

Quote:

---

Originally Posted by johnjlarge

I Updated to the latest commit from github, which was made three hours ago, as the API results caching wasn't working on mine. My API calls reset at midnight and by 7am GMT I had hit 3.5k API calls. Checking the calls log, there were multiples for the same IP. The latest commit, saving sessions to database appears to have fixed this. Good work :)

---

Are you on v2.0? Switched from session caching to database caching for improved performance and reliability.

Quote:

---

Originally Posted by marcopolo

Are you on v2.0? Switched from session caching to database caching for improved performance and reliability.

---

Yes, I'm on version 2.0 and fully up to date as of just a moment ago with the changes on github.

A couple of ideas, in the abuseipdb_api_call_2023_05.log the IPs of spiders which appear in spiders.txt are still showing as blocked, for example

2023-05-26 16:51:52 IP address 54.236.1.11 API call. Score: 63
(this is pinterest bot, which is still allowed to browse the site, but perhaps we could avoid adding and spider sessions to the api log)

Also, maybe beyond the scope of this plugin, but perhaps we could stop blocked IPs from showing in whos online?

Blocking is working really well, just thinking how the plugin could evolve. Perhaps even excluding anything in spiders.txt for even doing an api call, so it never shows in logs & doesn't use up any api hits?

I run a relatively busy & established site with over 8000 products which has been running on zen cart since 2005, so I have a lot of traffic to test this out. So far today, most normal users have scored a 0 which is as expected, but the block log has blocked a fair few really malicious IPs today, so this plugin could prove invaluable for protecting sites from the worst offenders.

Quote:

---

Originally Posted by johnjlarge

Yes, I'm on version 2.0 and fully up to date as of just a moment ago with the changes on github.

A couple of ideas, in the abuseipdb_api_call_2023_05.log the IPs of spiders which appear in spiders.txt are still showing as blocked, for example

2023-05-26 16:51:52 IP address 54.236.1.11 API call. Score: 63
(this is pinterest bot, which is still allowed to browse the site, but perhaps we could avoid adding and spider sessions to the api log)

Also, maybe beyond the scope of this plugin, but perhaps we could stop blocked IPs from showing in whos online?

Blocking is working really well, just thinking how the plugin could evolve. Perhaps even excluding anything in spiders.txt for even doing an api call, so it never shows in logs & doesn't use up any api hits?

I run a relatively busy & established site with over 8000 products which has been running on zen cart since 2005, so I have a lot of traffic to test this out. So far today, most normal users have scored a 0 which is as expected, but the block log has blocked a fair few really malicious IPs today, so this plugin could prove invaluable for protecting sites from the worst offenders.

---

The latest v2.0.4 release of the AbuseIPDB module is now live on GitHub. This update introduces a new feature that allows you to enable or disable known spiders from bypassing IP checks. Additionally, in the previous v2.0.3 release, I added an IP Cleanup feature that automatically deletes expired IP records. You can enable or disable this functionality and configure the IP record expiration period in the admin settings.

I'm facing an issue with the spider detection code. The purpose of this code is to detect spiders/crawlers, if enabled will bypass the api call and creates a log log if enabled as well. However, I've noticed that the log file is not being created even though I can see spiders accessing my site.

I suspect there might be an issue with the spider detection logic or the file logging process. I would greatly appreciate it if someone with more knowledge and experience could take a look at the code snippet and help me identify the problem.

PHP Code:

---

            // Skip API call for known spiders if enabled
                if (isset($spider_flag) && $spider_flag === true && $spider_allow == 'true') {

                    // Check if logging is enabled for allowed spiders
                        $log_file_name_spiders = 'abuseipdb_spiders_' . date('Y_m') . '.log';
                        $log_file_path_spiders = $log_file_path . $log_file_name_spiders;
                        $log_message = date('Y-m-d H:i:s') . ' IP address ' . $ip . ' Spider - Score: ' . $abuseScore . PHP_EOL;

                    if ($spider_log_enabled == 'true') {            
                        file_put_contents($log_file_path_spiders, $log_message, FILE_APPEND);
                    }

                return 0; // Return 0 score for spiders or whatever default value you want
                } 


---

I have already verified that the variables ($spider_flag, $spider_allow, $spider_log_enabled, etc.) are correctly set and the paths for the log files are valid.

Could someone please review the code and provide insights into why the log file is not being created? Any suggestions, improvements, or alternative approaches to spider detection in Zen Cart are also welcome.

Thank you in advance for your assistance!

Being lazy and not downloading the plugin, if $log_file_path is set to DIR_FS_LOGS, then the $log_file_name_spiders should have a leading '/' since DIR_FS_LOGS doesn't end in that character.