create_lmtconfig config file syntax
Jason Hill
I've been looking over getting LMT 2.6.4 going here at ORNL with Lustre 1.8.4.
A while back you answered a question for me about building a config file for
create_lmtconfig to parse - and I must admit I didn't specify everything fully.
So here's what we have:
widow2 (48 OSS nodes, 7 OST's per OSS)
Hostnames: widow-oss[5-8][a,b,c][1-4]
So the first bracketed quantity refers to a Scalable Cluster (3 DDN 9900
couplets, 12 OSS nodes), the second is the Scalable Unit inside the Scalable
Cluster, and finally there's 4 OSS nodes per Scalable Unit.
They lay out like:
widow-oss5a1 - ost0
widow-oss5a2 - ost1
widow-oss5a3 - ost2
widow-oss5a4 - ost3
widow-oss5b1 - ost4
...
widow-oss8c4 - ost47
widow-oss5a1 - ost48
...
I've been poking around in create_lmtconfig and it seems to not handle more
than one expansion inside of brackets nicely. Here's a snippit of the sql that
it generates from a stanza like this:
ost {
name widow-oss[5-8][a,b,c][1-4]
uuid widow2-OST{HEXINDEX}
skip 48
numdevs 7
}
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5[a:b:c][1-4]'), 'widow2-OST0000', 'widow-oss5[a:b:c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss6[a:b:c][1-4]'), 'widow2-OST0001', 'widow-oss6[a:b:c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss7[a:b:c][1-4]'), 'widow2-OST0002', 'widow-oss7[a:b:c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss8[a:b:c][1-4]'), 'widow2-OST0003', 'widow-oss8[a:b:c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5[a:b:c][1-4]'), 'widow2-OST0004', 'widow-oss5[a:b:c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss6[a:b:c][1-4]'), 'widow2-OST0005', 'widow-oss6[a:b:c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss7[a:b:c][1-4]'), 'widow2-OST0006', 'widow-oss7[a:b:c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss8[a:b:c][1-4]'), 'widow2-OST0007', 'widow-oss8[a:b:c][1-4]', '', '0');
I also tried putting the oss name to widow-oss[5-8][a-c][1-4] and got this:
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5[a-c][1-4]'), 'widow2-OST0000', 'widow-oss5[a-c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss6[a-c][1-4]'), 'widow2-OST0001', 'widow-oss6[a-c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss7[a-c][1-4]'), 'widow2-OST0002', 'widow-oss7[a-c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss8[a-c][1-4]'), 'widow2-OST0003', 'widow-oss8[a-c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5[a-c][1-4]'), 'widow2-OST0004', 'widow-oss5[a-c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss6[a-c][1-4]'), 'widow2-OST0005', 'widow-oss6[a-c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss7[a-c][1-4]'), 'widow2-OST0006', 'widow-oss7[a-c][1-4]', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss8[a-c][1-4]'), 'widow2-OST0007', 'widow-oss8[a-c][1-4]', '', '0');
I can put in a large quantity of stanzas with the correct startidx, or I can
just write a script to specify each OST device individually - but if there's
something I can use inside the hostlist_expand or quadrics_expand_range from
create_lmtconfig I'll do that.
Thanks!
--
-Jason
-------------------------------------------------
// Jason J. Hill //
// HPC Systems Administrator //
// National Center for Computational Sciences //
// Oak Ridge National Laboratory //
// e-mail: hil...@ornl.gov //
// Phone: (865) 576-5867 //
-------------------------------------------------
Jim Garlick
I'm pretty sure we can't handle multiple ranges per hostlist like you
need. Sorry :-(
A custom shell script probably is going to be the easiest thing here.
Jim
> --
> You received this message because you are subscribed to the Google Groups "lmt-discuss" group.
> To post to this group, send email to lmt-d...@googlegroups.com.
> To unsubscribe from this group, send email to lmt-discuss...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/lmt-discuss?hl=en.
>
>
Jason Hill
Thanks for the reply. You can't solve everyone's problems :)
--
Jason
Jason Hill
I think I have what *should* work, but I'm not getting that result as of now.
Here's a snip of the config file I'm feeding to create_lmtconfig.
ost {
name widow-oss5a1
uuid widow2-OST{HEXINDEX}
skip 48
numdevs 7
startidx 0
}
ost {
name widow-oss5a2
uuid widow2-OST{HEXINDEX}
skip 48
numdevs 7
startidx 1
}
ost {
name widow-oss5a3
uuid widow2-OST{HEXINDEX}
skip 48
numdevs 7
startidx 2
}
ost {
name widow-oss5a4
uuid widow2-OST{HEXINDEX}
skip 48
numdevs 7
startidx 3
}
I think I understand the man page correctly here and am specifying things
correctly.
Now here's the SQL that create_lmtconfig builds:
insert into OSS_INFO (FILESYSTEM_ID, HOSTNAME) values ('1', 'widow-oss5a1');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a1'), 'widow2-OST0000', 'widow-oss5a1', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a1'), 'widow2-OST0001', 'widow-oss5a1', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a1'), 'widow2-OST0002', 'widow-oss5a1', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a1'), 'widow2-OST0003', 'widow-oss5a1', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a1'), 'widow2-OST0004', 'widow-oss5a1', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a1'), 'widow2-OST0005', 'widow-oss5a1', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a1'), 'widow2-OST0006', 'widow-oss5a1', '', '0');
insert into OSS_INFO (FILESYSTEM_ID, HOSTNAME) values ('1', 'widow-oss5a2');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a2'), 'widow2-OST0001', 'widow-oss5a2', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a2'), 'widow2-OST0002', 'widow-oss5a2', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a2'), 'widow2-OST0003', 'widow-oss5a2', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a2'), 'widow2-OST0004', 'widow-oss5a2', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a2'), 'widow2-OST0005', 'widow-oss5a2', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a2'), 'widow2-OST0006', 'widow-oss5a2', '', '0');
insert into OST_INFO (OSS_ID, OST_NAME, HOSTNAME, DEVICE_NAME, OFFLINE) values ((select OSS_ID from OSS_INFO where HOSTNAME='widow-oss5a2'), 'widow2-OST0007', 'widow-oss5a2', '', '0');
So it seems to be ignoring the skip paramater. I would expect that oss5a1 would
get OST0001, OST0030 (48), OST0060 (96)... and so on.
I'm happy to admit I'm mis-reading the manpage.
Thanks again,
-Jason
On Thu, Sep 16, 2010 at 12:05:19AM -0400, Jim Garlick wrote:
Jim Garlick
as hostranges, so will this do what you want?
ost {
name test-oss5a[1-4],test-oss5b[1-4],test-oss5c[1-4],test-oss6a[1-4],test-oss6b[1-4],test-oss6c[1-4],test-oss7a[1-4],test-oss7b[1-4],test-oss7c[1-4],test-oss8a[1-4],test-oss8b[1-4],test-oss8c[1-4]
uuid test-OST{HEXINDEX}
skip 48
numdevs 7
}
Jim
> --
> You received this message because you are subscribed to the Google Groups "lmt-discuss" group.
> To post to this group, send email to lmt-d...@googlegroups.com.
> To unsubscribe from this group, send email to lmt-discuss...@googlegroups.com.
> For more options, visit this group at http://BLOCKEDgroups.google.com/group/lmt-discuss?hl=en.
>
Jason Hill
Perfect! I knew there was an easy fix out there. I appreciate all the mental
cycles.
Thanks!
-Jason