Welcome! Log In Create A New Profile

Advanced

NGINX with VHosts from DB (MySQL)

Posted by neilstuartcraig 
NGINX with VHosts from DB (MySQL)
September 19, 2011 05:03PM
Hi all

Firstly, thanks to everyone who has contributed to NGINX - it's looking really good from what i have seen, a very, very worthy replacement for the ageing Apache.

I've just started using NGINX after years and years of using Apache and more recently lighttpd. I work for a company which develops, maintains and hosts websites for some large companies and at present we use a Windows .NET platform - needless to say, i don't like that and want to change it!

I am currently working on the beginnings of a project to look at alternatives to our current platform and I believe that NGINX is the best web server for our purposes...so i want to make sure it fits our need at the same time as hoping to contribute to the project wherever i/we can - i think that's only right.

A major issue for us (and probably many other ISP-style hosting companies) which I would like to solve though is that i'd like to serve Vhosts from a DB back-end rather than a flat/text config file (currently i am favouring the Percona fork of MySQL) so i wanted to ask if anyone has started a module to achieve that? If not, i am happy to do it, i will need to dust off my C skills though as i haven't written C for about 10 years! Lighttpd has a great mysql-vhost module but lighttpd is not as well maintained as NGINX.

Many thanks guys
Neil
Re: NGINX with VHosts from DB (MySQL)
October 04, 2011 07:03PM
I've been looking for something like this for months now, and I don't think there is a module which does this - meaning I was looking into creating something like this too, alas I've not found much time yet.

I've noticed some solutions/work arounds in apache and lighttpd rebuild the vhost files on a restart, or require a restart when a new host is added to the database. It'd be interesting to perhaps incorporate the drizzle module (http://wiki.nginx.org/HttpDrizzleModule) to query on a new request for servers with huge amounts of vhosts, but I'm not sure if there would be a noticeable drop in efficiency, but nginx's speed may improve the viability of it?

So you're not alone.
Re: NGINX with VHosts from DB (MySQL)
October 05, 2011 04:44PM
Hi Nick
Yes indeed, I agree with you on those workarounds...we need a proper solution!
I will take a look at drizzle, thanks for the suggestion. I'll have to dust off my C skills!
I'll post back here when I have some progress, might take a while but I'll do it.
Cheers
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 07:29AM
Reading the data (from the DB) on startup is easy, refreshing it every X minutes is a bit harder but quite doable.
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 07:52AM
That's quite interesting having it refresh. Would that put much of a strain on the running of things? Particular for ISPs like the one Neil is representing with large numbers of vhosts? Or would having a module which queries the database when a HTTP request is made be more efficient?

Maybe it would depend on the type of setup you've got going. I mean, for servers holding many hosts with not much demand on them could be better having the query method, but hefty sites which don't have frequent changes made to files/records would be better with having a refreshed version. I'm not sure, just some pondering I thought I'd impose, perhaps a test would better answer these questions.
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 07:54AM
> That's quite interesting having it refresh. Would
> that put much of a strain on the running of
> things? Particular for ISPs like the one Neil is
> representing with large numbers of vhosts? Or

What's a large number?
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 08:03AM
Well, a few hundred? thousands? :-\ I'm just trying to stir up debate here to see what would be the best direction for a module like this. Perhaps Neil has more of an idea of how large his database would be for him.

I'm just wondering if a select statement per HTTP request is better than a refresh every 10 mins or so, particularly for large systems?



Edited 1 time(s). Last edit at 11/09/2011 08:04AM by nick.
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 08:05AM
That's not a large number. :p

You're joking when you say 1 database request per HTTP request, right?
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 08:15AM
Okay, a large number. ;-)

I wasn't really meaning to put any numerics to it initially, but in your opinion how many database request would you estimate would be done?
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 09:02AM
About 1 per minute



Edited 1 time(s). Last edit at 11/09/2011 09:03AM by Olaf van der Spek.
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 09:37AM
Are you referring to 1 reload per minute?
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 10:36AM
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 11:41AM
So would you think the CPU consumption and resources of 1 reload per minute for millions of vhosts in the database would be less than just using a pool of database connections to do a lookup on top of a HTTP request?

Because what if you only have a couple of hundred frequently accessed hosts, they'd have to wait for the others to be reloaded, but if you had some sort of caching and had it just look up in the database when a host not in the cache is requested, would that not be more efficient?
Re: NGINX with VHosts from DB (MySQL)
November 09, 2011 12:23PM
The refresh should be non-blocking, so there won't be any waiting.
Millions is quite a large number. :p
If you do have a very large number of infrequently used vhosts then a more complex design *might* be better.
But I'd start with the simple design.
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 89
Record Number of Users: 7 on March 06, 2014
Record Number of Guests: 229 on August 01, 2014
Powered by nginx    Powered by FreeBSD    PHP Powered    Powered by Percona     ipv6 ready