Thank you Marcus. I never think about those kind of FS issues. files
should not exceed hundred thousand in the worse case scenario so I
think file system should be the better solution for me.

On Mar 25, 2009, at 11:07 AM, Marcus Clyne wrote:

> Hi Brice,
>
> Firstly, I'm not talking about a full-blown database that's serving
> files, but a lightweight front to the files (as PBMS is). PBMS is
> just an HTTP front to a storage engine for MySQL, and doesn't deal
> with SQL or anything like that. It's really like serving the
> content directly out of a large file, in a pretty similar way to
> many caches do.
>
> Ordinarily, the filesystem will be much faster than even the
> lightest of fronts to a db, but if you have milllions of files, then
> each file will have metadata associated with it (which takes up
> space - usually at least 4KB) and the filesystem has to cope with
> all the files, and many filesystems struggle when you start getting
> to large numbers of files, and it slows things down.
>
> If you have billions of files, then you couldn't even serve them off
> a normal 32-bit fs, because you'd run out of inodes (I believe).
>
> For thousands of files, or tens of thousands, you'd be fine, though,
> and the filesystem will definitely be quicker than PBMS.
>
> When I did my tests with PBMS, I created 2M objects, and put them in
> the database with MySQL, then served them using PBMS. My benchmarks
> showed that the number of req/s I could serve was similar to Apache
> 2 at best, and about 20% slower at worst (it depended on the index
> of the file). That might not seem great, but that was serving from
> 1 file vs 2M. I tried creating files in a hierarchical structure,
> and after I got to around 400k (? I can't quite remember), my system
> almost completely stalled, so I stopped trying to add files.
>
> In most scenarios, the filesystem will be quicker, but not always.
>
> Cheers,
>
> Marcus.
>
>
> Brice Leroy wrote:
>>
>> On Mar 24, 2009, at 1:55 PM, Marcus Clyne wrote:
>>
>>> Michael Shadle wrote:
>>>> not to mention that this is only useful if the OP is storing
>>>> files in
>>>> the database to begin with, I believe.
>>>>
>>>> if it's filesystem, then X-Accel-Redirect is the way to go.
>>>>
>>> Yes, if the filesystem is the storage mechanism, then I'd agree.
>>> If you have a very large number of files, though, storing them in
>>> the DB can be more efficient than using the filesystem (depending
>>> on platform, file number, directory hierarchy etc).
>>
>> How can the filesystem can be slower than DB to serve huge video
>> files ? That's completely on the opposite to my culture ! Can you
>> explain me how this is possible ?
>> My situation will be serving thousand(maybe more later) of
>> different big files(between 200MB and 4GB) to different users.
>>
>> Brice
>>
>>
>
>