Yes I see after looking at the various plugins on GitHub it seems they replace the & ampersand string with & when they pull contents from the HTML. They also fake / spoof referrers and can change user-agents etc but they do it properly not like the person who has ended up in my logs. As you said they did it is badly. I feel this could be a loosing battle if they are spoofing the useby c0nw0nk - Nginx Mailing List - English
gariac Wrote: ------------------------------------------------------- > Apparently there is a scheme to feed urls to kodi. > > https://m.reddit.com/r/kodi/comments/3lz84g/how_do_you_open_a_youtube > _video_from_the_shell/ > > Block/ban as you see fit. ;-) These people are edge users of Kodi. > > But you may want to search the interwebs to see if someonby c0nw0nk - Nginx Mailing List - English
gariac Wrote: ------------------------------------------------------- > Kodi is the renamed xbmc. I use it myself, but I never "aimed" it at a > website. I just view my own videos or use the kodi plug-ins. You can > install it yourself on a PC and see it is intended to be just a media > player. It really isn't any different that seeing VLC as the agent. > > Peby c0nw0nk - Nginx Mailing List - English
So with Nginx my access.logs show allot of Kodi user agents from what I look up online Kodi is a app that runs on Phones, TV sticks, Mac, PC etc and it is used for watching live TV I reckon its a pretty abusive app or service since there is allot going around about IPTV and how illegal it is. The issue I have is I am receiving allot of spammy errors from them like this. [02/Nov/2016:06:46:by c0nw0nk - Nginx Mailing List - English
You should check your application sounds like that is compressing its pages. A simple test is this create a empty html file and serve that from a location and check the headers. location = /test.html { root "path/to/html/file"; } if the headers on that have no gzip compression as set in your nginx config then you know its your web application gzipping.by c0nw0nk - Nginx Mailing List - English
Thanks :) I thought the more servers I have within my upstream location would mean I should also increase my keepalive to suit for best performance etc.by c0nw0nk - Nginx Mailing List - English
FastCGI : upstream fastcgi_backend { server 127.0.0.1:9000; keepalive 8; } server { ... location /fastcgi/ { fastcgi_pass fastcgi_backend; fastcgi_keep_conn on; ... } } Proxy : upstream http_backend { server 127.0.0.1:80; keepalive 16; } server { ... location /http/ { proxy_pass http://http_bacby c0nw0nk - Nginx Mailing List - English
So this is one of those issues it is most likely a bad configuration but my robots.txt file is returning a 404 because of another location because I am disallowing people to access any text files but I do want to allow only the robots.txt to be accessed. location /robots.txt { root 'location/to/robots/txt/file'; } #This is to stop people digging into any directories looking for files that onby c0nw0nk - Nginx Mailing List - English
What I would say to do is write IP's from your toolkit or what ever you are using for reading your access.log and those that trigger and spam the 503 error within milliseconds or what ever range it is you can do an API call and add those IP's to be blocked at a router level. With CloudFlare you can have CloudFlare block those IP's before they reach your server like so https://api.cloudflare.comby c0nw0nk - Nginx Mailing List - English
It is a response by the time the 444 is served it is to late a true DDoS is not about what the server outputs its about what it can receive you can't expect incoming traffic that amounts to 600Gbps to be prevented by a 1Gbps port it does not work like that Nginx is an Application preventing any for of DoS at an application level is a bad idea it needs to be stopped at a router level before it hitsby c0nw0nk - Nginx Mailing List - English
Francis Daly Wrote: ------------------------------------------------------- > On Mon, Sep 26, 2016 at 07:41:12PM -0400, c0nw0nk wrote: > > Hi there, > > > Whats a good setting that won't effect legitimate decent (I think I > just > > committed a crime calling some of these companies decent?) crawlers > like > > Google, Bing, Baidu, Yandex etc. >by c0nw0nk - Nginx Mailing List - English
So to prevent flooding / spam by bots especially since some bots are just brutal when they crawl by within milliseconds jumping to every single page they can get. I am going to apply limit's to my PHP block limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s; limit_conn_zone $binary_remote_addr zone=addr:10m; location ~ \.php$ { limit_req zone=one burst=5; limit_conn addr 10; }by c0nw0nk - Nginx Mailing List - English
Anoop Alias Wrote: ------------------------------------------------------- > Ok .. reiterating my original question. > > Is the usage of if / map in nginx config more efficient than say > naxsi ( > or libmodsecurity ) for something like blocking SQL injection ? > > For example, > https://github.com/nbs-system/naxsi/blob/master/naxsi_config/naxsi_cor > e.ruby c0nw0nk - Nginx Mailing List - English
So I want to find the best optimal settings for serving large static files with Nginx. >=2GB I read that "output_buffers" is the key. Would also like to know if it should be defined per location {} that the static file is served from or across the entire server via http {} and any other settings that should be in place or left at defaults. Also curious if any of this would evby c0nw0nk - Nginx Mailing List - English
If you read the OWASP page it will also mention about header stripping etc and proxies that will remove the X-Frames headers there is no real way to stop proxies framing your site but the X-Frame-Options combined with that JavaScript is a good way to start it will stop the majority. Also break their proxies is what I like to do. For example I combine it with not allowing people to browse witby c0nw0nk - Nginx Mailing List - English
https://www.owasp.org/index.php/Clickjacking_Defense_Cheat_Sheet Inside your <head> </head> tags. <style id="antiClickjack">body{display:none !important;}</style> <script type="text/javascript"> if (self === top) { var antiClickjack = document.getElementById("antiClickjack"); antiClickjack.parentNode.removeChildby c0nw0nk - Nginx Mailing List - English
Thanks for the information so based of what that resource says and from what I understand surely that field should only say "anonymous" or "username" if on those files / folders in my Nginx config I use "auth_basic" ? http://nginx.org/en/docs/http/ngx_http_auth_basic_module.html The fact they are inputting that header unlike everyone else just alerts me. Becaby c0nw0nk - Nginx Mailing List - English
So in my access logs all my other logs the $remote_user is empty. But for only this one single IP that keeps making requests the $remote_user has a value. CF-Real-IP: 176.57.129.88 - CF-Server: 10.108.22.151 - anonymous [21/Sep/2016:18:54:52 +0100] "GET /media/files/29/96/2b/701f56b345ce531192645ddb532a8fd7.mp4 HTTP/1.1" Status:503 206 "http://www.networkflare.com/" &quoby c0nw0nk - Nginx Mailing List - English
Yeah the reason it does not work behind CloudFlare is because the limit_conn and limit_req is blocking the CloudFlare server IP for making to many requests. So that is why i am reciving the DOS output "503 service unavailable" And I don't fancy building a whitelist of IP's since it would require manually updating allot. The cloudflare server IP's would need excluding from the $binary_by c0nw0nk - Nginx Mailing List - English
Il test further with it but it definitely did not work with the following using nginx_basic.exe (it was blocking the cloudflare server IP's from connecting) http { #Inside http real_ip_header CF-Connecting-IP; limit_req_zone $binary_remote_addr zone=one:10m rate=30r/m; limit_conn_zone $binary_remote_addr zone=addr:10m; server { # server domain etc here location ~ \.mp4$ { limit_by c0nw0nk - Nginx Mailing List - English
itpp2012 Wrote: ------------------------------------------------------- > c0nw0nk Wrote: > > Yes I can't test it at the moment unfortunately with the realip > module > > due to the fact i use "itpp2012" Nginx builds > > http://nginx-win.ecsds.eu/ They do not come compiled with the > realip > > module (for now ?) > > Of course this moduleby c0nw0nk - Nginx Mailing List - English
itpp2012 Wrote: ------------------------------------------------------- > c0nw0nk Wrote: > > Yes I can't test it at the moment unfortunately with the realip > module > > due to the fact i use "itpp2012" Nginx builds > > http://nginx-win.ecsds.eu/ They do not come compiled with the > realip > > module (for now ?) > > Of course this moduleby c0nw0nk - Nginx Mailing List - English
B.R. Wrote: ------------------------------------------------------- > You were just told the best way to get a meaningful > $binary_remote_addr > variable using CloudFlare, with the added bonus of a list of network > ranges > to use with set_real_ip_from to only filter out CloudFlare's IP > addresses > as sources to be repalced and avoid false positives. > > Uby c0nw0nk - Nginx Mailing List - English
Reinis Rozitis Wrote: ------------------------------------------------------- > > But that book says it is to reduce the memory footprint ? > > Correct, but that is for that specific varible. > > You can't take $http_cf_connecting_ip which is a HTTP header comming > from > Cloudflare and prepend $binary_ just to "lower memory footprint". > There iby c0nw0nk - Nginx Mailing List - English
Reinis Rozitis Wrote: ------------------------------------------------------- > > I just found the following : > > > https://books.google.co.uk/books?id=ZO09CgAAQBAJ&pg=PA96&lpg=PA96&dq=$ > binary_ > > > limit_req_zone $binary_http_cf_connecting_ip zone=one:10m > rate=30r/m; > > limit_conn_zone $binary_http_cf_connecting_ip zone=addr:10m;by c0nw0nk - Nginx Mailing List - English
Reinis Rozitis Wrote: ------------------------------------------------------- > > I just found the following : > > > https://books.google.co.uk/books?id=ZO09CgAAQBAJ&pg=PA96&lpg=PA96&dq=$ > binary_ > > > limit_req_zone $binary_http_cf_connecting_ip zone=one:10m > rate=30r/m; > > limit_conn_zone $binary_http_cf_connecting_ip zone=addr:10m;by c0nw0nk - Nginx Mailing List - English
I just found the following : https://books.google.co.uk/books?id=ZO09CgAAQBAJ&pg=PA96&lpg=PA96&dq=$binary_ To conserve the space occupied by the key we use $binary_remote_addr It evaluates into a binary value of the remote IP address So it seems I should be doing this instead to keep the key in memory for that IP small to reduce the memory footprint. limit_req_zone $binary_httby c0nw0nk - Nginx Mailing List - English
gariac Wrote: ------------------------------------------------------- > I'm assuming at this point if cookies are too much, then logins or > captcha aren't going to happen. > > How about just blocking the offending websites at the firewall? I'm > assuming you see the proxy and not the eyeballs at the ISP. > > I have my hacker detection schemes in nginx. I flagby c0nw0nk - Nginx Mailing List - English
> gariac Wrote: > ------------------------------------------------------- > > What about Roboo? It requires a cookie on the website before the > > download takes place. (My usual warning this is my understanding of > > how it works, but I have no first hand knowledge.) I presume the > hot > > linkers won't have the cookie. > > > > https://giby c0nw0nk - Nginx Mailing List - English
I was going to do a cookie method but its bad because on browsers with no cookies that make legitimate requests (first time visitors maybe that don't have a cookie set) or browsers on legitimate users who disable cookies or use extensions / add-ons to only whitelist cookies from sites they specifically allow like facebook, youtube etc. So that's why I decide to peruse the connection and requestby c0nw0nk - Nginx Mailing List - English