Newbie questions about CQL and the CIM object model

[Dave Kirby]

I am part of a team writing a tool to map servers to the storage volumes they use, and have lots of questions about CQL and CIM and hope someone here can answer them.  Please bear in mind that I am a programmer rather than a storage administrator, and am still getting up to speed with the technology and terminology.

1) Are there any good tutorials on CQL?  The only one I have found is a one-page overview at http://www.wbemsolutions.com/tutorials/DMTF/wbem-cql.html but that does not go very deep.  I also have the CQL Specification but that is very dense and confusing and leaves me with more questions than it answers. 

2) My biggest question with CQL is how to traverse an association from one object to another.  WQL has the "ASSOCIATORS OF..." query type but this is not supported in CQL.

3) is there a standard way to get from the WWPN of an HBA card to the storage volumes that are configured for that server?  For IBM Storewiz I have found the class IBMTSSVC_HardwareIdStorageVolumeViewn which does exactly what I want - the description says "Represents a mapping between a StorageHardwareID (SVC host) to a Volume (SVC VDisk)".  However this is obviously IBM specific - is there a standard way of getting this information across storage vendors?   Or even vendor specific ways for non-IBM systems?

I have lots of other questions but that enough to be going on with.  Any help will be greatly appreciated.

Dave K

[Gris Ge]

On Wed, May 14, 2014 at 03:33:08AM -0700, Dave Kirby wrote:
>
> 1) Are there any good tutorials on CQL? The only one I have found is a
> one-page overview at http://www.wbemsolutions.com/tutorials/DMTF/wbem-cql.html
> but that does not go very deep. I also have the CQL Specification but that is
> very dense and confusing and leaves me with more questions than it answers.

This document might help:
http://www.dmtf.org/standards/search?keys=&title=&dsp=DSP0202&tid=All

> 2) My biggest question with CQL is how to traverse an association from one
> object to another. WQL has the "ASSOCIATORS OF..." query type but this is not
> supported in CQL.

Please use Associators() or AssociatorNames() call instead.
The "DMTF: CIM over HTTP" document might help:
http://www.dmtf.org/standards/search?keys=&title=&dsp=DSP0200&tid=All

If you are using Python pywbem:
http://sourceforge.net/apps/mediawiki/pywbem/index.php?title=Upcalls#Associators

>
> 3) is there a standard way to get from the WWPN of an HBA card to the storage
> volumes that are configured for that server? For IBM Storewiz I have found the
> class IBMTSSVC_HardwareIdStorageVolumeViewn which does exactly what I want -
> the description says "Represents a mapping between a StorageHardwareID (SVC
> host) to a Volume (SVC VDisk)". However this is obviously IBM specific - is
> there a standard way of getting this information across storage vendors? Or
> even vendor specific ways for non-IBM systems?

You can refer to SMI-S Block book:
"Masking and Mapping Subprofile"

There are two ways to get LUN masking information:

Method A, introduced by SMI-S 1.6:

CIM_StorageVolume # LUN
|
| CIM_ProtocolControllerForUnit # Associate class
v
CIM_SCSIProtocolController # You can treat this as a host
| # group or initiator group.
|
| CIM_AssociatedPrivilege # Associate class
v
CIM_StorageHardwareID # CIM_StorageHardwareID['Initiator'] is WWPN

Method B, defined in SMI-S 1.4, 1.5 and 1.6. (deprecated in 1.6):
# You'd better use the method B unless you checked the profile
# registration status of your SMI-S provider.

CIM_StorageVolume
|
| CIM_ProtocolControllerForUnit # Associate class
v
CIM_SCSIProtocolController #
|
| CIM_AuthorizedTarget
v
CIM_AuthorizedPrivilege
|
| CIM_AuthorizedSubject
v
CIM_StorageHardwareID # CIM_StorageHardwareID['Initiator'] is WWPN

There is also an experimental SMI-S profile: "Group Masking and Mapping
Profile". But I see no vendor implemented that profile.

>
> I have lots of other questions but that enough to be going on with.
> Any help will be greatly appreciated.

Yes. The whole DMTF/SNIA/CIM/WBEM/SMI-S things and SNIA SMI-S big PDFs
confused me too.

Feel free to ask any questions.
I have gained pretty a lot helps on SMI-S and willing to contribute back.
>
> Dave K
>

Hope that helps.
Best regards.

--
Gris Ge

[Dave Kirby]

Thanks Gris, that is very helpful. 

I get the impression that CQL is not used much and has minimal support from vendors - is that the case?  Some of the systems I have tried it on do not even seem to support "SELECT * FROM ...", you have to specify a list of attributes to select.  This seems a pretty basic omission to me. 

[Jim Davis]

Not exactly. CQL is used quite heavily (at least the basic features) for Indication Subscription, but the operation ExecQuery is not mandated by SMI, so that is not supported by many vendors. 

Jim Davis
CTO
910-528-5511

On May 16, 2014, at 11:24 AM, Dave Kirby <dave.x...@gmail.com> wrote:

Thanks Gris, that is very helpful. 

I get the impression that CQL is not used much and has minimal support from vendors - is that the case?  Some of the systems I have tried it on do not even seem to support "SELECT * FROM ...", you have to specify a list of attributes to select.  This seems a pretty basic omission to me. 

--
You received this message because you are subscribed to the Google Groups "SMI-S Developers Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smi-s-developers-...@googlegroups.com.
To post to this group, send email to smi-s-devel...@googlegroups.com.
Visit this group at http://groups.google.com/group/smi-s-developers-group.
For more options, visit https://groups.google.com/d/optout.

[Jim Davis]

Hi Gris,

I am not sure what you are asking, but CQL is not standalone support - it has to be used by something. CQL is mandatory for Indication Subscription, not for ExecQuery, since ExecQuery is not mandatory.

also, CQL has many features/levels of support (14 I think), so not all levels are required. 

Jim Davis
CTO
910-528-5511

On May 18, 2014, at 3:27 AM, Gris Ge <f...@redhat.com> wrote:

On Fri, May 16, 2014 at 11:28:05AM -0400, Jim Davis wrote:

Not exactly. CQL is used quite heavily (at least the basic features)
for Indication Subscription, but the operation ExecQuery is not
mandated by SMI, so that is not supported by many vendors.


Jim Davis
CTO
910-528-5511

Just add something up.
The Indication profile is mandatory if a SMI-S provider want to use
SMI-S block book.[1]

In Indication profile(SMI-S 1.4rev6 CommonProfile book, PDF page 489), quote:
   43.1.6.5 Support for Query Languages
   For versions of the standard prior to 1.3.0, CQL had not been approved as a
   standard and was treated as recommended and experimental. For those early
   versions, WQL (also referred to as the SMI-S query Language) was the
   non-experimental query language.  For versions of the standard starting at
   1.3.0, CQL is mandatory for newly defined indication filters; WQL
   alternatives shall not be defined in the standard.

So CQL is mandatory since SMI-S 1.3.0.

But I found no standard documents mentioned how much should a SMI-S profile
support. And I also failed to find a way to know the CQL support status
of connected SMI-S in the runtime.

In stead of the CQL/SQL, I use this method to filter CIMInstance by myself:
   1. Use EnumerateInstances() or Associators() to get all possible
      CIMInstance, limit the return data via PropertyList parameter.
   2. Filter the CIMInstance by ourself.

I did some performance comparison, this method having nearly the same or
better[2] performance than CQL/WQL filter.

Generally, I would suggest leaving CQL/WQL for indication profile only and use
associators() for other profiles.

We can discuss on the detail if you can provide these informations:
   1. Which profile you are using.
   2. Which CIM_XXX you are using.
   3. Which property of CIM_XXX you are using.

Hope this helps.
Best regards.

[1] In SMI-S 1.4rev6 CommonProfile book, PDF page 43.
[2] It turn out to be a EMC SMI-S bug, they have that fixed already.

--
Gris Ge

[Jim Davis]

 CQL is defined in DSP0202 (http://www.dmtf.org/sites/default/files/standards/documents/DSP0202_1.0.0.pdf). You can get the feature level supported by either the WBEM Server Profile (CIM_GenericOperationCapabilitiesStructure) or by the deprecated class CIM_QueryCapabilities, if either is implemented.

The levels are

 "Basic Query", "Simple Join", "Complex Join",

 "Subquery", "Result Set Operations",

 "Extended Select List", "Embedded Properties",

 "Aggregations", "Regular Expression Like", "Array Range",

 "Satisfies Array", "Foreign Namespace Support",

 "Arithmetic Expression", "Full Unicode Support",

 "Conversion Utilities", "Property Scoping",

 "Association Traversal"

Jim Davis
CTO
910-528-5511

On May 19, 2014, at 1:25 AM, Gris Ge <f...@redhat.com> wrote:

On Sun, May 18, 2014 at 03:47:18PM -0400, Jim Davis wrote:

Hi Gris,

I am not sure what you are asking, but CQL is not standalone support -
it has to be used by something. CQL is mandatory for Indication
Subscription, not for ExecQuery, since ExecQuery is not mandatory.
also, CQL has many features/levels of support (14 I think), so not all
levels are required.

Hi Jim,

Those questions were for Dave Kirby. Sorry for the confusion.

Talk about CQL. I am interesting on the 'CQL levels of support', do you
have any document elaborate that?

I am looking for a capabilities check before I execute any CQL query.

Thanks.
Best regards.

Jim Davis
CTO
910-528-5511

--
Gris Ge

[Gris Ge]

On Mon, May 19, 2014 at 02:24:48PM -0400, Jim Davis wrote:
>
> The levels are
>
> "Basic Query", "Simple Join", "Complex Join",
> "Subquery", "Result Set Operations",
> "Extended Select List", "Embedded Properties",
> "Aggregations", "Regular Expression Like", "Array Range",
> "Satisfies Array", "Foreign Namespace Support",
> "Arithmetic Expression", "Full Unicode Support",
> "Conversion Utilities", "Property Scoping",
> "Association Traversal"

Great thanks. Jim.

[Gris Ge]

On Fri, May 16, 2014 at 11:28:05AM -0400, Jim Davis wrote:

> Not exactly. CQL is used quite heavily (at least the basic features)
> for Indication Subscription, but the operation ExecQuery is not
> mandated by SMI, so that is not supported by many vendors.
>
>
> Jim Davis
> CTO
> 910-528-5511
>
>

[Gris Ge]

On Sun, May 18, 2014 at 03:47:18PM -0400, Jim Davis wrote:

> Hi Gris,
>
> I am not sure what you are asking, but CQL is not standalone support -
> it has to be used by something. CQL is mandatory for Indication
> Subscription, not for ExecQuery, since ExecQuery is not mandatory.
> also, CQL has many features/levels of support (14 I think), so not all
> levels are required.

Hi Jim,

Those questions were for Dave Kirby. Sorry for the confusion.

Talk about CQL. I am interesting on the 'CQL levels of support', do you
have any document elaborate that?

I am looking for a capabilities check before I execute any CQL query.

Thanks.
Best regards.
>

[goldnerjeff]

Yeah good luck with that one :(

I have not found a valid implementation for ExecQuery. There is a property in both SLP and in the Server Profile (CIM_ObjectManagerCommunicationMechanism.FunctionalProfilesSupported[]), but you can't rely on that because it refers to the CIMOM feature, not the provider capability. I think Jim had proposed a solution for the provider functionality.

So for now, save your energy.