Is that 2.2 million CIDRs, or actual addresses? I use IPFW with tables for about 20k CIDRs. I don't see any significant server load. It seems to me nginx has a big enough task that it makes sense to offload the blocking to something that is more tightly integrated to the OS. At a bare minimum, block OVH and Hetzner. People bash the Russians and old Soviet block countries for hacking, but OVH aby gariac - Nginx Mailing List - English
I don't know how to state this without being insulting, but Kodi is designed to be used by dumb people. That is how I use it. It seems pointless to me to try to hack Kodi into doing something it wasn't meant to do. That is why I called that example an edge case. There is a YouTube plugin for Kodi. http://kodi.wiki/view/Add-on:YouTube Load up Kodi and go hack yourself. That is the best way tby gariac - Nginx Mailing List - English
Apparently there is a scheme to feed urls to kodi. https://m.reddit.com/r/kodi/comments/3lz84g/how_do_you_open_a_youtube_video_from_the_shell/ Block/ban as you see fit. ;-) These people are edge users of Kodi. But you may want to search the interwebs to see if someone is attempting to write a kodi plugin for your service. The vast majority of the Kodi plug-ins are third party.by gariac - Nginx Mailing List - English
Kodi is the renamed xbmc. I use it myself, but I never "aimed" it at a website. I just view my own videos or use the kodi plug-ins. You can install it yourself on a PC and see it is intended to be just a media player. It really isn't any different that seeing VLC as the agent. Perhaps someone wrote a plugin for your website. Make that a poorly written plugin ;-) Do you offer your mp4by gariac - Nginx Mailing List - English
If you get hammered, even serving the 403-page is actually noticeable traffic. --------- Nginx rate limiting works very well. _______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginxby gariac - Nginx Mailing List - English
Original Message From: Cox, Eric S Sent: Tuesday, November 1, 2016 3:35 PM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: RE: Blocking tens of thousands of IP's Currently we track all access logs realtime via an in house built log aggregation solution. Various algorithms are setup to detect said IPS whether it be by hit rate, country, known types of attacks etc. These IPS are tyby gariac - Nginx Mailing List - English
Original Message From: Cox, Eric S Sent: Tuesday, November 1, 2016 8:16 AM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: Blocking tens of thousands of IP's Is anyone aware of a difference performance wise between using return 403; vs deny all; When mapping against a list of tens of thousands of ip? Thanks ------------- I started a thread on blocking via nginby gariac - Nginx Mailing List - English
On Sat, 22 Oct 2016 17:40:56 -0400 "itpp2012" <nginx-forum@forum.nginx.org> wrote: > The idea is nice but pointless, if you maintain this list over 6 > months you most likely will end up blocking just about everyone. > > Stick to common sense with your config, lock down nginx and the > backends, define proper flood and overflow settings for nginx to deal > with,by gariac - Nginx Mailing List - English
http://pastebin.com/7W0uDrLa If you need an extensive list of hacker requests (over 200), I put this log entry on pastebin. As mentioned at the top of the pastebin, the hacker used my IP address directly rather than my doman name. I have a "map" that detects typical hacker activity. Perhaps in my "map" of triggers, I should look for bypassing the domain name, that is requestby gariac - Nginx Mailing List - English
http://pastebin.com/tZZg3RbA/?e=1 This is the access.log file data relevant to that fake googlebot. It starts with a fake googlebot entry, then goes downhill from there. I rate limit at 10/s. I only allow the verbs HEAD and GET, so the POST went to 444 directly. I replaced the domain with a fake one. _______________________________________________ nginx mailing list nginx@nginx.org http://mailmby gariac - Nginx Mailing List - English
_______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginxby gariac - Nginx Mailing List - English
_______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginxby gariac - Nginx Mailing List - English
If you dig through some old posts, it was established that the deny feature of nginx isn't very effective at limiting network activity. I deny at the firewall. What remains is if you should deny dynamically or statically. Original Message From: c0nw0nk Sent: Tuesday, September 27, 2016 11:42 AM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: Re: 444 return code and rate limby gariac - Nginx Mailing List - English
Your reply does not agree with the documentation. https://httpstatuses.com/444 Original Message From: B.R. Sent: Tuesday, September 27, 2016 10:09 AM To: nginx ML Reply To: nginx@nginx.org Subject: Re: 444 return code and rate limiting Responding 444 is... a response. It is not anything else than a (non-standard, thus 'unknown', just like 499 nginx chose to illustrate client-siby gariac - Nginx Mailing List - English
I pulled this off the rate limiting thread since I think the 444 return is a good topic all on its own. "But under a DoS attack I always feel those values would be better being "444" since the server won't respond and cut's the connection rather than waste bandwidth on a client who is opening and closing connections fast as a bullet." Looking at the times in my nginx accesby gariac - Nginx Mailing List - English
You might want to check out tinfoilsecurity.com to evaluate Naxsi. Microsoft uses them for azure. I pass all their tests. As I stated a few times, I only serve static pages. I can get away with homebrew hacking detection. But I think you are kidding yourself if you think a stack of WAF rules isn't a CPU burden. There is no free lunch. Someone supporting 500 vhosts probably should segreby gariac - Nginx Mailing List - English
_______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginxby gariac - Nginx Mailing List - English
For one thing, I have trouble making fail2ban work. ;-) I run sshguard, so the major port 22 hacking is covered. And that is continous. I don't know if fail2ban can read nginx logs. I thought you need to run swatch, which requires actual perl skill to set up. In any event, my 444 is harmless other than someone not getting a reply. I find hackers try to log into WordPress. I find Google trys toby gariac - Nginx Mailing List - English
I doubt I could patch source. (I know my limits.) But reverse DNS seems very useful. Someone should fix the module. Original Message From: A. Schulze Sent: Monday, September 26, 2016 12:33 AM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: Re: fake googlebots / nginx-http-rdns lists: > Nginx has a reverse DNS module: > https://github.com/flant/nginx-http-rdns for an older vby gariac - Nginx Mailing List - English
_______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginxby gariac - Nginx Mailing List - English
I got a spoofed googlebot hit. It was easy to detect since there were probably a hundred requests that triggered my hacker detection map scheme. Only two requests received a 200 return and both were harmless. 200 118.193.176.53 - - [25/Sep/2016:17:45:23 +0000] "GET / HTTP/1.1" 847 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-&by gariac - Nginx Mailing List - English
I had too many false positives with Naxsi and debugging is difficult. In any event, using Naxsi doesn't eliminate the need to block bad referrals, so you still need the map module. I have passed tinfoilsecurity.com flogging, as well as one of the transversal testers. So this is more than just security theater. I flag all the hackers with a 444, then use scripts to display the 444 log entrieby gariac - Nginx Mailing List - English
Possibly map uses a hashing scheme to do the matches, so it could be more efficient than a series of ifs. That is something the programmers would know. Every situation is different. I don't find the maps I use to be detrimental, especially if you are preventing further operations by the nginx. I can tell you a trimmed about a third of my network traffic by aggressively blocking scrapers and othby gariac - Nginx Mailing List - English
I suspect the map module can do that more efficiently. There is an example of how to use the map module in this post: http://ask.xmodulo.com/block-specific-user-agents-nginx-web-server.html The code is certainly cleaner using map. I use three maps, specifically for bad user agent, bad request, and bad referrer. Original Message From: Anoop Alias Sent: Saturday, September 24, 2016by gariac - Nginx Mailing List - English
I serve no ads. I even pulled my piwik so that my sites can be surfed no script. Can you clickjack an encrypted page? How would the browser handle two certs? Original Message From: c0nw0nk Sent: Thursday, September 22, 2016 1:57 PM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: Re: (Semi-OT) Clickjacking countermeasure If you read the OWASP page it will also mention about headeby gariac - Nginx Mailing List - English
I saw that, but I took the path of least resistance. The method I mentioned was sufficient to pass the tinfoilsecurity.com test. To tinfoils's credit, they provided three references on Clickjacking, one of which is the website you suggested. Original Message From: c0nw0nk Sent: Thursday, September 22, 2016 1:34 PM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: Re: (Semi-OT) Clickby gariac - Nginx Mailing List - English
I ran one of these website inspection services on my website and it was deemed to be subject to Clickjacking. This might be a false positive since I don't use frames, but the info on this link was enough to make the error go away. I chose "DENY" since I don't use frames. https://geekflare.com/add-x-frame-options-nginx/ The inspection was from tinfoilsecurity.com. If you are blockiby gariac - Nginx Mailing List - English
I'm assuming at this point if cookies are too much, then logins or captcha aren't going to happen. How about just blocking the offending websites at the firewall? I'm assuming you see the proxy and not the eyeballs at the ISP. I have my hacker detection schemes in nginx. I flag the clowns, yank the IPs every day or so, and block the IP space of any VPS, colo, etc. I have blocked so mucby gariac - Nginx Mailing List - English
What about Roboo? It requires a cookie on the website before the download takes place. (My usual warning this is my understanding of how it works, but I have no first hand knowledge.) I presume the hot linkers won't have the cookie. https://github.com/yuri-gushin/Roboo Original Message From: c0nw0nk Sent: Tuesday, September 13, 2016 1:09 AM To: nginx@nginx.org Reply To: nginx@nginx.orgby gariac - Nginx Mailing List - English
Re-reading the original post, it was concluded that multiple connection don't effect the rate limiting. I interpreted this incorrectly the first time: "Nginx's limit_rate function limits the data transfer rate of a single connection." But I'm certain a few posts, perhaps not on the nginx forum, state incorrectly that the limiting is per individual connections rather than allby gariac - Nginx Mailing List - English