Enabling MSI

72 views
Skip to first unread message

Mike Lane

unread,
May 28, 2018, 12:53:57 AM5/28/18
to darwin-...@lists.apple.com
What needs to be done to enable MSI interrupt? I've a simple driver.  The filter is not getting called if I pass the msi index to the filterInterruptEventSource. However, the device supports MSI(x). No issues in drivers for other OS (Win). Any clues what might be happening? 

kIOPCIMSIXCapability and kIOPCIMSICapability indicate MSI(X) is supported however the device node under PCI section in system report indicate MSI=No which is strange. Where does this come from?

Have been struggling to get past this issue. Any help would be appreciated.

Henry Kannapell

unread,
May 29, 2018, 10:51:27 AM5/29/18
to darwin-...@lists.apple.com
The procedure is described in “Thunderbolt Device Driver Programming Guide” on the Apple developer site.

— Henry

> What needs to be done to enable MSI interrupt? I've a simple driver. The filter is not getting called if I pass the msi index to the filterInterruptEventSource. However, the device supports MSI(x). No issues in drivers for other OS (Win). Any clues what might be happening?
>
> kIOPCIMSIXCapability and kIOPCIMSICapability indicate MSI(X) is supported however the device node under PCI section in system report indicate MSI=No which is strange. Where does this come from?
>
> Have been struggling to get past this issue. Any help would be appreciated.

_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-drivers mailing list (Darwin-...@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/darwin-drivers/darwin-drivers-garchive-96018%40googlegroups.com

This email sent to darwin-drivers...@googlegroups.com

Mike Lane

unread,
May 30, 2018, 8:40:27 PM5/30/18
to darwin-...@lists.apple.com

 

Hi Henry


Yes I did read that doc. Here is some code from the driver.  (I replied with more code but got bounced due to size limit)


bool MyClass::initEventSources(IOService *provider)

{

   IOReturn err = kIOReturnSuccess;

    int msiIndex = -1;

    int intrIndex = 0;

    int intrType = 0;

    bool result = false;

    

    while ((err = fPciDevice->getInterruptType(intrIndex, &intrType)) == kIOReturnSuccess) {

        if (intrType & kIOInterruptTypePCIMessaged){

            msiIndex = intrIndex;

            IOLog("msiIndex: %d\n", msiIndex);

            break;

        }

        intrIndex++;

    }


    if (msiIndex == -1) {

        IOLog("MSI index was not found or MSI interrupt could not be enabled.\n");

        err = kIOReturnNoResources;

        goto done;

    }

    

    IOLog("MSI interrupt index: %d\n", msiIndex);


    fInterruptSource = IOFilterInterruptEventSource::filterInterruptEventSource(

                                                                          this,

                                                                          OSMemberFunctionCast(IOInterruptEventSource::Action,

                                                                                               this, &MyClass::handleInterrupt),

                                                                          OSMemberFunctionCast(IOFilterInterruptEventSource::Filter,

                                                                                               this, & MyClass::filterInterrupt),

                                                                          provider, msiIndex);


    if (!fInterruptSource) {

        IOLog("Unable to get filterInterruptEventSource");

        err = kIOReturnNoResources;

        goto done;

    }

    

    _msiIndex = msiIndex;


    err = fWorkLoop->addEventSource(fInterruptSource);


    if ( err != kIOReturnSuccess )

    {

        IOLog("unable to add filter event source: 0x%x",err);

        err = kIOReturnNoResources;

        goto done;

    }


    // Enable the interrupt delivery.

    fInterruptSource->enable();

}



Andrew White

unread,
Jun 6, 2018, 8:01:11 PM6/6/18
to Mike Lane, darwin-...@lists.apple.com

Grabbed some old code of mine and looked at it quickly.  One difference I note is that you are passing the IOService as a provider to IOFilterInterruptEventSource::filterInterruptEventSource while I passed:

 

IOPCIDevice * pciDevice = (IOPCIDevice *) getProvider();

 

--

Andrew

 

 

Mike Lane

unread,
Jun 7, 2018, 11:00:27 PM6/7/18
to Andrew White, darwin-...@lists.apple.com

Thanks. That didn't help. Earlier I was passing fPciDevice then changed to provider. NowI changed back to that. Earlier, I chased initial values of some device registers but no clues there. Basically, stuck at this point. 





From: Andrew White <andrew...@audinate.com>
Sent: Wednesday, June 6, 2018 5:00 PM
To: Mike Lane; darwin-...@lists.apple.com
Subject: Re: Enabling MSI
 

Chris Sarcone

unread,
Jun 8, 2018, 12:40:52 AM6/8/18
to darwin-...@lists.apple.com
Hi Mike —

The TBT developer guide is for MSI only and has not been updated for MSI-X. I’ll ask for DevPubs to update the document the next time they rev it. For MSI-X, there’s a bit more work you need to do.

Something like (some error checking omitted):

#define kMSIXVectorControlOffset 12
#define kMSIXCapabilityControlOffset 2
#define kMSIXCapabilityMSIXTableOffset 4


fPCIProvider->extendedFindPCICapability ( kIOPCIMSICapability, &msiCapability );
if ( msiCapability == 0 )
{
fPCIProvider->extendedFindPCICapability ( kIOPCIMSIXCapability, &msixCapability );
if ( msixCapability != 0 )
{
IOMemoryMap * map = NULL;
uint32_t      table = 0;
uint8_t       bar = 0;
uint16_t      control = 0;
control    = fPCIProvider->configRead16 ( msixCapability + kMSIXCapabilityControlOffset );
table  = fPCIProvider->configRead32 ( msixCapability + kMSIXCapabilityMSIXTableOffset );
bar    = ( kIOPCIConfigBaseAddress0 + ( ( table & 7 ) << 2 ) );
table &= ~7;
map = fPCIProvider->mapDeviceMemoryWithRegister ( bar );
if ( map )
{
fMSIXTable = ( volatile uint8_t * ) map->getAddress ( ) + table;

}
}

Clearing an interrupt:

OSWriteLittleInt32 ( fMSIXTable, ( ( vector * 16 ) + kMSIXVectorControlOffset ), 1 );

Enabling the interrupt:

OSWriteLittleInt32 ( fMSIXTable, ( ( vector * 16 ) + kMSIXVectorControlOffset ), 0 );

HTH,

— Chris


_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-drivers mailing list      (Darwin-...@lists.apple.com)
Help/Unsubscribe/Update your Subscription:


------------------

19191 Vallco Parkway

M/S 77-1SSG

Cupertino CA 95014

phone: (408) 974-4033

fax:   (408) 862-7577

email: sar...@apple.com


Mike Lane

unread,
Jun 8, 2018, 2:38:06 PM6/8/18
to Chris Sarcone, darwin-...@lists.apple.com

Hi Chris


The change didn't help yet.  Still debugging this. After seeing the code snippet from you,

I looked around in IOPCIFamily and noticed the following methods implement similar logic.


Any ideas why these weren't called automatically when I enabled msi and registered the 

the IOInterruptEventSource? Thanks!


IOPCIMessagedInterruptController::allocateDeviceInterrupts

IOPCIMessagedInterruptController::enableVector



From: Darwin-drivers <darwin-drivers-bounces+mikemlane=outlo...@lists.apple.com> on behalf of Chris Sarcone <sar...@apple.com>
Sent: Thursday, June 7, 2018 9:40:41 PM
To: darwin-...@lists.apple.com
Subject: Re: Enabling MSI
 

Mike Lane

unread,
Jun 9, 2018, 11:09:35 PM6/9/18
to Chris Sarcone, darwin-...@lists.apple.com

Hi Chris


I went after why the MSI vector wasn't enabled automatically. Perhaps this is not supported yet. 


Any idea kIOPCIResourcedKey was true for my device? 


As a result, IOPCIBridge::getnubResources returns early without further device initialization for the MSI. The call resolveMSIInterrupts never gets called which in turn calls allocateDeviceInterrupts that inits msiTable for the device. Since msiTable was null, the vector wasn't enabled in IOPCIMessagedInterruptController::enableVector method 


pci10de,e0f: family specific matching fails

PXSX: family specific matching fails

Matching service count = 1

IOPCIBridge::getNubResources - 0xffffff806df50300

IOPCIResourced <——————————————!!!!


init() -> entry

MyPCIDevice::probe(pciXXXX,XXXX)

MyPCIDevice::start(pciXXXX,XXXX) <1>

MyPCIDevice[0xffffff8071fbbe80]::start(0xffffff806df50300)

pciXXXX,XXXX::requestPowerDomainState

initPCIConfigSpace

VendorID: 0xXXXX

DeviceID: 0xXXXX

Subsystem Vendor: 0xXXXX

Subsystem Device: 0x1b

Range@0x10 (0x00000000d4000000) mapped to kernel virtual address 0xffffff85785f1000

fBaseAddr: 0xffffff85785f1000

MSI is supported

MSIX is supported

resetDevice

initEventSources

msiIndex: 1

MSI interrupt index: 1


IOPCIMessagedInterruptController::registerInterrupt - 0xffffff806df50300 1 0xffffff8071710e80 0xffffff800849a660

IOPCIMessagedInterruptController::enableDeviceMSI - 0xffffff806df50300

IOPCIMessagedInterruptController::enableInterrupt - 0xffffff806df50300 1

IOPCIMessagedInterruptController::enableVector - 5 0xffffff84e6026140


vector->interruptRegistered: 1

vector->interruptActive: 0

vector->interruptDisabledSoft: 0

vector->interruptDisabledHard: 0

vector->nub: 0xffffff806df50300

vector->source: 1

vector->target: 0xffffff8071710e80

vector->handler: 0xffffff800849a660

device->reserved->msiTable:0 <——————————————!!!!


MSI interrupt is successfully connected!


initDevice



Thanks!



From: Mike Lane
Sent: Friday, June 8, 2018 11:37:52 AM
To: Chris Sarcone; darwin-...@lists.apple.com
Subject: Re: Enabling MSI
 
Reply all
Reply to author
Forward
0 new messages