Re: [Lustre-discuss] Benchmarking Working Group
faibis...@emc.com
-----Original Message-----
From: Jeffrey...@Dell.com [mailto:Jeffrey...@Dell.com]
Sent: Friday, March 23, 2012 4:55 PM
To: faibish, sorin; Richard.V...@netapp.com; openbe...@lists.opensfs.org; dis...@lists.opensfs.org; lustre-...@lists.lustre.org
Subject: RE: Benchmarking Working Group
Sorin,
Is the intent to add some benchmarks that utilize the ideas and concepts of sfs2013 to the list? Or is the intent to possible add sfs2013 itself to the list of benchmarks?
[SF] My intention was to include Lustre as one of the workloads of sfs2013 as a special profile to the benchmark. Regardless, I was approved by sfs committee to add a Lustre specific profile to sfs2013. I hope with the help of the Lustre community to create a realistic and representative profile to be included in sfs2013 additional to other benchmarks that this group will decide to do. The advantage of such a benchmark will be that vendors can publish results officially to the SPEC.org reviewed by a peer audit group. Hope this makes sense
Sorin Faibish
Distinguished Engineer
EMC - Office of CTO
Thanks!
Dr. Jeff Layton
HPC Enterprise Technologist
Dell | Research Computing
mobile + 1 678 427-5819
jeffrey...@dell.com
-----Original Message-----
From: openbenchm...@lists.opensfs.org [mailto:openbenchm...@lists.opensfs.org] On Behalf Of faibis...@emc.com
Sent: Friday, March 23, 2012 1:12 PM
To: Richard.V...@netapp.com; openbe...@lists.opensfs.org; dis...@lists.opensfs.org; lustre-...@lists.lustre.org
Subject: Re: [Openbenchmark] Benchmarking Working Group
Per the discussion during the call, attached please find the documentation of sfs2013 benchmark (currently called netmist).
Please take a look and let me know if it is of interest to continue on this path of action for the Lustre benchmark. I will be attending the LUG and plan to introduce sfs2013 benchmark. Thank you for your understanding and patience
Sorin Faibish
Distinguished Engineer
EMC - Office of CTO
_____________________________________________
From: Vanderbilt, Richard [mailto:Richard.V...@netapp.com]
Sent: Monday, March 19, 2012 12:25 PM
To: openbe...@lists.opensfs.org; dis...@lists.opensfs.org; lustre-...@lists.lustre.org; faibish, sorin
Subject: [Openbenchmark] Benchmarking Working Group
When: Friday, March 23, 2012 11:30 AM-12:30 PM (GMT-05:00) Eastern Time (US & Canada).
Where: 1(866)844-9418 id 4088226016
The OpenSFS Benchmarking Workgroup aims to provide an I/O benchmark suite to satisfy the benchmarking requirements of the scalable parallel file system users and facilities. Towards this end, as a first step, the workgroup aims to characterize the I/O workloads from small- to very large-scale scalable parallel file systems deployed at various high-performance computing and parallel computing facilities and institutions. Using these gathered characteristics, the workgroup will then collect and build the required I/O benchmarks to emulate these workloads and provide required documentation about the benchmark suite.
The workgroup meeting will reoccur every Friday at 8:30am Pacific time. 1 hour each. The dial-in is as follows: 1(866)844-9418 id# 4088226016.
Also note: we will be having our first face to face meeting on Sunday April 22nd from 2:00-5:00pm at the LUG. Details to come.
On Tuesday April 24 4:10-5:00pm there will be a Benchmark panel discussion moderated by Sarp Oral. Panelist will be Jeremy Filizetti (DOD), Andrew Uselton (NERSC), Eric Barton (Whamcloud), Nathan Rutman (Xyratex), Stephane Thiell (CEA), and Stephen Simms (IU).
If you are interested in contributing to this work group, please email Sarp Oral at ora...@ornl.gov<mailto:ora...@ornl.gov> or myself at richard.v...@netapp.com<mailto:richard.v...@netapp.com>.
Sincerely,
Sarp Oral, PhD
National Center for Computational Sciences Oak Ridge National Laboratory ora...@ornl.gov<mailto:ora...@ornl.gov>
865-574-2173
Richard Vanderbilt Co-chair
Netapp Inc.
(408)822-6016
<< File: ATT00001.txt >>
_______________________________________________
Lustre-discuss mailing list
Lustre-...@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
faibis...@emc.com
The benchmark, any benchmark is as good as the profile of the workload it wants to represent as close as possible as many applications behavior. It is obvious that there are applications running on Lustre that are not representative to typical usage of Lustre FS. The art here is to find the best workload generator that will represent the majority of the apps using Lustre. Yet when a users selects a Lustre FS server he needs to give them a clear picture if Lustre is the right FS as well as which Lustre vendor has a better implementation. It is almost impossible for a user to use a benchmark for a single type of application. What majority of users do is build their own micro-benchmark specific to the application they us, of course if they have a real workload characterization of the application. In my experience many apps do not have a definition of the workload and prefer to select servers using a more general benchmark such as sfs, SPC, and others. This is the value of published benchmark data that are general enough to allow to compare different implementations. As a disclaimer a benchmark cannot ever represent all the behavior of all the apps using a FS but can represent accurately the average goodness of a FS implementation assuming a typical average usage of the FS. That being said I need the help of the LUG community to build the best most representative average workload definition for sfs2013.
I hope this explains the reason behind using general benchmarks and sfs2013 in particular
Sorin Faibish
Distinguished Engineer
EMC - Office of CTO
-----Original Message-----
From: Henry Newman [mailto:h...@instrumental.com]
Sent: Saturday, March 24, 2012 10:39 AM
To: faibish, sorin; Jeffrey...@Dell.com; Richard.V...@netapp.com; openbe...@lists.opensfs.org; dis...@lists.opensfs.org; lustre-...@lists.lustre.org
Subject: RE: Benchmarking Working Group
Have some SPEC data is interesting in my opinion only if the workload maps to user application. Much of Lustre usage from what I have seen is defensive I/O and therefore is write intensive. How does that map is the question at hand.
Thanks
hsn
________________________________________
Henry Newman | CEO/CTO
Instrumental, Inc | High Performance Innovation
1450 Energy Park Drive Suite 375
St. Paul, MN 55108-5274
Direct: 651-280-4801
Fax: 651-280-4839
Toll-free: 1.800.886.6188
E-Mail: h...@instrumental.com
Web: http://www.instrumental.com
STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and
proprietary material for the sole use of the intended recipient. Any review
or distribution by others is strictly prohibited. If you are not the
intended recipient please contact the sender and delete all copies.
Sorin,
Thanks!
Sincerely,
<< File: ATT00001.txt >>
_______________________________________________
Openbenchmark mailing list
Openbe...@lists.opensfs.org
http://lists.opensfs.org/listinfo.cgi/openbenchmark-opensfs.org
STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and proprietary material for the sole use of the intended recipient. Any review or distribution by others is strictly prohibited. If you are not the intended recipient please contact the sender and delete all copies.
Jeffrey...@dell.com
Is the intent to add some benchmarks that utilize the ideas and concepts of sfs2013 to the list? Or is the intent to possible add sfs2013 itself to the list of benchmarks?
Thanks!
Sincerely,
<< File: ATT00001.txt >>
_______________________________________________
Jeffrey...@dell.com
Thanks for the clarification - much appreciated. Interesting idea as well.
Thanks!
Jeff
Henry Newman
Thanks
hsn
________________________________________
Henry Newman | CEO/CTO
Instrumental, Inc | High Performance Innovation
1450 Energy Park Drive Suite 375
St. Paul, MN 55108-5274
Direct: 651-280-4801
Fax: 651-280-4839
Toll-free: 1.800.886.6188
E-Mail: h...@instrumental.com
Web: http://www.instrumental.com
STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and
proprietary material for the sole use of the intended recipient. Any review
or distribution by others is strictly prohibited. If you are not the
intended recipient please contact the sender and delete all copies.
-----Original Message-----
From: openbenchm...@lists.opensfs.org [mailto:openbenchm...@lists.opensfs.org] On Behalf Of faibis...@emc.com
Sent: Friday, March 23, 2012 4:35 PM
Sorin,
Thanks!
Sincerely,
<< File: ATT00001.txt >>
_______________________________________________
Openbenchmark mailing list
Openbe...@lists.opensfs.org
http://lists.opensfs.org/listinfo.cgi/openbenchmark-opensfs.org
STRICTLY PERSONAL AND CONFIDENTIAL. This email may contain confidential and proprietary material for the sole use of the intended recipient. Any review or distribution by others is strictly prohibited. If you are not the intended recipient please contact the sender and delete all copies.
_______________________________________________