Hi, about performances

[Alexandre Bique]

Hi,

I am using MogileFS and testing it's performance.
My goal is to be able to push (and maybe overwrite) about 2000 files
of ~10KB per seconds with no replication.

I did some tests and I got ~333 files per secondes.

What performances can I expect from MogileFS, and what can I do to
improve performances ?

Thanks a lot.

--
Alexandre Bique

[dormando]

You're going to have to give us a *lot* more information about your test
setup and goals. Exact hardware, OS, software configuration, database,
etc. Have you analyzed the bottleneck you're hitting at all?

[Kiall Mac Innes]

Hi Alexandre,

I guess the first to to consider is your storage hardware - how many storage servers, with how many devices each? are the devices raw disks or RAID? etc etc.

Followed by your DB server - bearing in mind that MogileFS only performs writes to the 1 master DB server. 2k new files a second translates to roughly 4k new records per second between the file and file_on tables alone, add in all the reads and thats a fair few QPS! You'll need to ensure your DB setup is up to the task...

Anyway ... The more info you supply ... the more likely you'll get useful advice ;)

Thanks,
Kiall

[David Birdsong]

On Thu, Mar 18, 2010 at 10:30 AM, Kiall Mac Innes <ki...@managedit.ie> wrote:
> Hi Alexandre,
>
> I guess the first to to consider is your storage hardware - how many storage
> servers, with how many devices each? are the devices raw disks or RAID? etc
> etc.
>
> Followed by your DB server - bearing in mind that MogileFS only performs
> writes to the 1 master DB server. 2k new files a second translates to
> roughly 4k new records per second between the file and file_on tables alone,
> add in all the reads and thats a fair few QPS! You'll need to ensure your DB
> setup is up to the task...
>

on that note, i recently moved my master to an ssd disk and saw db
read/write throughput jump considerably.

[David Mc Anulty]

Sounds like your trying to use a hammer as a stapler here. If you
arn't using replication with mogile, your kinda missing the whole
point.

2000 * 10 kilobytes = 20 megabytes....

Why not just store these small non-replicated files direcly in memory?
You could use an existing shared memory device like /dev/shm or create
your own loopback mounted ramdrive.

On Mar 18, 10:07 am, Alexandre Bique <bique.alexan...@gmail.com>
wrote:

[Akzhan Abdulin]

To achieve required performance we use nginx as WebDAV implementation for MogileFS storage nodes.

mogstored runned in None mode.

2010/3/18 Alexandre Bique <bique.a...@gmail.com>