We are using the IsMobileDevice property to automatically redirect mobile devices to our customized mobile site, while keeping standard web browser traffic flowing into our conventional website. The problem we have is search engine spiders. The code inside the Machine.Config that is added via the MMIT is setting the IsMobileDevice property to TRUE for all devices except Mozilla. Spiders such as Google are not Mozilla, so when they attempt to index our site, they get redirected to the mobile site instead.
Has anyone encountered a similar problem, or have ideas on how best to resolve this?
Though I don't have a ready-made solution for you, I can point you in the right direction. Use the information below to compile a list of "user agent" strings that you wish to identify. Then you change your machine.config file so that it uses the list you have compiled.
BACKGROUND INFO When a web device (such as a browser, or a automated paged indexer) requests a page from a server, it includes a "user agent" string in the request. This string, called HTTP_USER_AGENT, identifies that the request is coming from Internet Explorer 5 or Netscape Communicator 6 or .... there are a huge number of possibilities.
The machine.config file contains a section named "browserCaps". This section contains code which attempts to parse or match the text in the user agent string to identify the browser. This section is designed to identify browsers used by end users (such as IE or Netscape). With MMIT installed, it also performs some minimal testing to identify mobile devices. To a large degree, the logic simply marks the user agent as being from a mobile device if it was not able to otherwise identify the user agent.
>From: "Brad Griffey" <bgrif...@dollar.com> >Subject: IsMobileDevice -- Mobile Devices and Search Engine Spiders >Date: Thu, 21 Nov 2002 17:40:57 -0600 >Lines: 14 >X-Priority: 3 >X-MSMail-Priority: Normal >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >Message-ID: <u7w3vdbkCHA.2472@tkmsftngp08> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >NNTP-Posting-Host: 208.2.126.4 >Path: cpmsftngxa08!tkmsftngp01!tkmsftngp08 >Xref: cpmsftngxa08 microsoft.public.dotnet.framework.aspnet.mobile:4985 >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>We are using the IsMobileDevice property to automatically redirect mobile >devices to our customized mobile site, while keeping standard web browser >traffic flowing into our conventional website. The problem we have is >search engine spiders. The code inside the Machine.Config that is added via >the MMIT is setting the IsMobileDevice property to TRUE for all devices >except Mozilla. Spiders such as Google are not Mozilla, so when they >attempt to index our site, they get redirected to the mobile site instead.
>Has anyone encountered a similar problem, or have ideas on how best to >resolve this?
I have a similar need in my application to detect if it is running on a PDA or via the Web. Here's what I use:
Private Function isPDA() As Boolean
Dim mc As MobileCapabilities = Request.Browser Dim op As String = ""
Dim isMyPalm As Boolean = mc.HasCapability ("isMyPalm", op) Dim isPocketIE As Boolean = mc.HasCapability ("isPocketIE", op) Dim isIE As Boolean = mc.HasCapability("isIE", op) Dim isBlazer As Boolean = mc.HasCapability ("isBlazer", op) Dim isOpera As Boolean = mc.HasCapability ("isOpera", op)
Return (isMyPalm Or isPocketIE Or isIE Or isBlazer Or isOpera)
End Function
Corresponding device filters needed in your Web.config:
>-----Original Message----- >We are using the IsMobileDevice property to automatically redirect mobile >devices to our customized mobile site, while keeping
standard web browser
>traffic flowing into our conventional website. The problem we have is >search engine spiders. The code inside the
Machine.Config that is added via
>the MMIT is setting the IsMobileDevice property to TRUE for all devices >except Mozilla. Spiders such as Google are not Mozilla, so when they >attempt to index our site, they get redirected to the
What I failed to mention in my previous post is that I used the isPDA function mainly to differenciate between PDA and cell phone users. The particular feature which needed this allowed it to be used on the web & pda, but not cell phones. So, the function probably should've been name isPDAOrWeb. In any event, you get the idea. You could just remove the IE and Opera lines. Incidentially, I added Opera support because I heard that Opera 7 supported Smartphones and PDAs with it's small-screen rendering. Later, I found out that by PDA they really meant devices which run on Symbian OS, which is mainly a cell phone OS. MyPalm and Blazer are WAP browsers which run on Palm Pilots. PocketIE runs on Pocket PCs.
>-----Original Message----- >I have a similar need in my application to detect if it is >running on a PDA or via the Web. Here's what I use:
>Private Function isPDA() As Boolean
> Dim mc As MobileCapabilities = Request.Browser > Dim op As String = ""
> Dim isMyPalm As Boolean = mc.HasCapability >("isMyPalm", op) > Dim isPocketIE As Boolean = mc.HasCapability >("isPocketIE", op) > Dim isIE As Boolean = mc.HasCapability("isIE", op) > Dim isBlazer As Boolean = mc.HasCapability >("isBlazer", op) > Dim isOpera As Boolean = mc.HasCapability >("isOpera", op)
> Return (isMyPalm Or isPocketIE Or isIE Or isBlazer >Or isOpera)
>End Function
>Corresponding device filters needed in your Web.config:
>>-----Original Message----- >>We are using the IsMobileDevice property to automatically >redirect mobile >>devices to our customized mobile site, while keeping >standard web browser >>traffic flowing into our conventional website. The >problem we have is >>search engine spiders. The code inside the >Machine.Config that is added via >>the MMIT is setting the IsMobileDevice property to TRUE >for all devices >>except Mozilla. Spiders such as Google are not Mozilla, >so when they >>attempt to index our site, they get redirected to the >mobile site instead.
>>Has anyone encountered a similar problem, or have ideas >on how best to >>resolve this?
I guess the concern I have with this fix is I'm tracking spiders based on their existing user-agent strings (or some other means of identifying them). I know that new spiders are created all the time, and existing search engines upgrade their existing spiders. I'm just worried that my user-agent list within my machine.config will need to be routinely maintained.
One of the methods I used prior to our site going .NET was to check the pixel height and width of the calling device (if it was less than 640x480 we assumed it was a mobile device). I'm assuming spiders would return 0x0 or null for this, which would still be a good way to detect them and flag them as non-mobile. Would this be a viable solution as well? At least it would prevent me from having to maintain a user-agent list in my machine.config.
> Though I don't have a ready-made solution for you, I can point you in the > right direction. Use the information below to compile a list of "user > agent" strings that you wish to identify. Then you change your > machine.config file so that it uses the list you have compiled.
> BACKGROUND INFO > When a web device (such as a browser, or a automated paged indexer) > requests a page from a server, it includes a "user agent" string in the > request. This string, called HTTP_USER_AGENT, identifies that the request > is coming from Internet Explorer 5 or Netscape Communicator 6 or .... there > are a huge number of possibilities.
> The machine.config file contains a section named "browserCaps". This > section contains code which attempts to parse or match the text in the user > agent string to identify the browser. This section is designed to identify > browsers used by end users (such as IE or Netscape). With MMIT installed, > it also performs some minimal testing to identify mobile devices. To a > large degree, the logic simply marks the user agent as being from a mobile > device if it was not able to otherwise identify the user agent.
> 2) Update the browserCaps section of machine.config > Look at the syntax of what is already in this section, then expand on it > with your new list.
> --- > Do you have any more questions on this issue?
> Thank you, Mike Moore > Microsoft, ASP.NET
> This posting is provided "AS IS", with no warranties, and confers no rights.
> -------------------- > >From: "Brad Griffey" <bgrif...@dollar.com> > >Subject: IsMobileDevice -- Mobile Devices and Search Engine Spiders > >Date: Thu, 21 Nov 2002 17:40:57 -0600 > >Lines: 14 > >X-Priority: 3 > >X-MSMail-Priority: Normal > >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 > >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 > >Message-ID: <u7w3vdbkCHA.2472@tkmsftngp08> > >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile > >NNTP-Posting-Host: 208.2.126.4 > >Path: cpmsftngxa08!tkmsftngp01!tkmsftngp08 > >Xref: cpmsftngxa08 microsoft.public.dotnet.framework.aspnet.mobile:4985 > >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
> >We are using the IsMobileDevice property to automatically redirect mobile > >devices to our customized mobile site, while keeping standard web browser > >traffic flowing into our conventional website. The problem we have is > >search engine spiders. The code inside the Machine.Config that is added > via > >the MMIT is setting the IsMobileDevice property to TRUE for all devices > >except Mozilla. Spiders such as Google are not Mozilla, so when they > >attempt to index our site, they get redirected to the mobile site instead.
> >Has anyone encountered a similar problem, or have ideas on how best to > >resolve this?
>From: "Brad Griffey" <bgrif...@dollar.com> >References: <u7w3vdbkCHA.2472@tkmsftngp08> <iUeBu3nkCHA.2528@cpmsftngxa08> >Subject: Re: IsMobileDevice -- Mobile Devices and Search Engine Spiders >Date: Mon, 9 Dec 2002 08:12:15 -0600 >Lines: 101 >X-Priority: 3 >X-MSMail-Priority: Normal >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >Message-ID: <ObH82z4nCHA.1448@TK2MSFTNGP11> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >NNTP-Posting-Host: mail.dollar.com 208.2.126.4 >Path: cpmsftngxa06!cpmsftngxa10!tkmsftngp01!TK2MSFTNGP11 >Xref: cpmsftngxa06 microsoft.public.dotnet.framework.aspnet.mobile:4993 >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>I guess the concern I have with this fix is I'm tracking spiders based on >their existing user-agent strings (or some other means of identifying them). >I know that new spiders are created all the time, and existing search >engines upgrade their existing spiders. I'm just worried that my user-agent >list within my machine.config will need to be routinely maintained.
>One of the methods I used prior to our site going .NET was to check the >pixel height and width of the calling device (if it was less than 640x480 we >assumed it was a mobile device). I'm assuming spiders would return 0x0 or >null for this, which would still be a good way to detect them and flag them >as non-mobile. Would this be a viable solution as well? At least it would >prevent me from having to maintain a user-agent list in my machine.config.
>> Though I don't have a ready-made solution for you, I can point you in the >> right direction. Use the information below to compile a list of "user >> agent" strings that you wish to identify. Then you change your >> machine.config file so that it uses the list you have compiled.
>> BACKGROUND INFO >> When a web device (such as a browser, or a automated paged indexer) >> requests a page from a server, it includes a "user agent" string in the >> request. This string, called HTTP_USER_AGENT, identifies that the request >> is coming from Internet Explorer 5 or Netscape Communicator 6 or .... >there >> are a huge number of possibilities.
>> The machine.config file contains a section named "browserCaps". This >> section contains code which attempts to parse or match the text in the >user >> agent string to identify the browser. This section is designed to identify >> browsers used by end users (such as IE or Netscape). With MMIT installed, >> it also performs some minimal testing to identify mobile devices. To a >> large degree, the logic simply marks the user agent as being from a mobile >> device if it was not able to otherwise identify the user agent.
>> 2) Update the browserCaps section of machine.config >> Look at the syntax of what is already in this section, then expand on it >> with your new list.
>> --- >> Do you have any more questions on this issue?
>> Thank you, Mike Moore >> Microsoft, ASP.NET
>> This posting is provided "AS IS", with no warranties, and confers no >rights.
>> -------------------- >> >From: "Brad Griffey" <bgrif...@dollar.com> >> >Subject: IsMobileDevice -- Mobile Devices and Search Engine Spiders >> >Date: Thu, 21 Nov 2002 17:40:57 -0600 >> >Lines: 14 >> >X-Priority: 3 >> >X-MSMail-Priority: Normal >> >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >> >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >> >Message-ID: <u7w3vdbkCHA.2472@tkmsftngp08> >> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >> >NNTP-Posting-Host: 208.2.126.4 >> >Path: cpmsftngxa08!tkmsftngp01!tkmsftngp08 >> >Xref: cpmsftngxa08 microsoft.public.dotnet.framework.aspnet.mobile:4985 >> >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>> >We are using the IsMobileDevice property to automatically redirect mobile >> >devices to our customized mobile site, while keeping standard web browser >> >traffic flowing into our conventional website. The problem we have is >> >search engine spiders. The code inside the Machine.Config that is added >> via >> >the MMIT is setting the IsMobileDevice property to TRUE for all devices >> >except Mozilla. Spiders such as Google are not Mozilla, so when they >> >attempt to index our site, they get redirected to the mobile site >instead.
>> >Has anyone encountered a similar problem, or have ideas on how best to >> >resolve this?
QUESTION IsMobileDevice is not accurate. How can I get a better determination of which type of device sent a given request?
The current method for testing if an incoming request is from a mobile device is IsMobileDevice. This tests for specifically known devices and then classifies all requests as mobile if "mozilla" is not included in the user agent string. However, there are many other user agent strings in use today which are not mobile devices. This includes "spiders" which search engines use to index web sites. If spiders get classified as mobile devices, then they get directed to the sites mobile section. The result is that the regular section of the site does not get indexed by the search engine.
ANSWER I agree with your idea of using the pixel height/width. I will research this further and let you know what I find.
Regarding the height/width. This is dependant upon the device sending its screen height/width as part of the request. Here are the related headers for a request from a Pocket PC (it also had a couple other headers which were not of interest for this topic): UA-OS: Windows CE (POCKET PC) - Version 3 0 UA-color: color16 UA-pixels: 240x320 UA-CPU: ARM SA1110 UA-Voice: FALSE UA-Language: JavaScript User-Agent: Mozilla/2.0 (compatible; MSIE 3.02; Windows CE; PPC; 240x320)
All of these are different from the headers sent by IE. Three are of particular interest: UA-OS, UA-pixels, and User-Agent. Parsing these header strings can reveal that this is a Pocket PC and that it's height/width is 240x320.
Since IE does not send height/width information, I expect that spiders will not send height/width information either.
I previously wrote about compiling a list of user agent strings and updating your machine.config file. Then the IsMobileDevice would use the new information and it would be more accurate. For those interested in seeing this reply, go to: http://www.google.com/groups?safe=images&ie= UTF-8&oe=UTF-8&as_umsgid=iUeBu3nkCHA.2528 @cpmsftngxa08&lr=&hl=en
Again, I will look into this further and post my findings. This will probably take several days to track down.
Thank you, Mike Moore Microsoft, ASP.NET
This posting is provided "AS IS", with no warranties, and confers no rights.
I'm still looking for more information on this. So far, your own idea of using window height/width, along with IsMobileDevice, is looking like it may be the best solution we can offer at this time. I am still looking into this and will keep you posted.
Thank you, Mike Moore Microsoft, ASP.NET
This posting is provided "AS IS", with no warranties, and confers no rights.
>From: "Brad Griffey" <bgrif...@dollar.com> >References: <u7w3vdbkCHA.2472@tkmsftngp08> <iUeBu3nkCHA.2528@cpmsftngxa08> >Subject: Re: IsMobileDevice -- Mobile Devices and Search Engine Spiders >Date: Mon, 9 Dec 2002 08:12:15 -0600 >Lines: 101 >X-Priority: 3 >X-MSMail-Priority: Normal >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >Message-ID: <ObH82z4nCHA.1448@TK2MSFTNGP11> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >NNTP-Posting-Host: mail.dollar.com 208.2.126.4 >Path: cpmsftngxa06!cpmsftngxa10!tkmsftngp01!TK2MSFTNGP11 >Xref: cpmsftngxa06 microsoft.public.dotnet.framework.aspnet.mobile:4993 >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>I guess the concern I have with this fix is I'm tracking spiders based on >their existing user-agent strings (or some other means of identifying them). >I know that new spiders are created all the time, and existing search >engines upgrade their existing spiders. I'm just worried that my user-agent >list within my machine.config will need to be routinely maintained.
>One of the methods I used prior to our site going .NET was to check the >pixel height and width of the calling device (if it was less than 640x480 we >assumed it was a mobile device). I'm assuming spiders would return 0x0 or >null for this, which would still be a good way to detect them and flag them >as non-mobile. Would this be a viable solution as well? At least it would >prevent me from having to maintain a user-agent list in my machine.config.
>> Though I don't have a ready-made solution for you, I can point you in the >> right direction. Use the information below to compile a list of "user >> agent" strings that you wish to identify. Then you change your >> machine.config file so that it uses the list you have compiled.
>> BACKGROUND INFO >> When a web device (such as a browser, or a automated paged indexer) >> requests a page from a server, it includes a "user agent" string in the >> request. This string, called HTTP_USER_AGENT, identifies that the request >> is coming from Internet Explorer 5 or Netscape Communicator 6 or .... >there >> are a huge number of possibilities.
>> The machine.config file contains a section named "browserCaps". This >> section contains code which attempts to parse or match the text in the >user >> agent string to identify the browser. This section is designed to identify >> browsers used by end users (such as IE or Netscape). With MMIT installed, >> it also performs some minimal testing to identify mobile devices. To a >> large degree, the logic simply marks the user agent as being from a mobile >> device if it was not able to otherwise identify the user agent.
>> 2) Update the browserCaps section of machine.config >> Look at the syntax of what is already in this section, then expand on it >> with your new list.
>> --- >> Do you have any more questions on this issue?
>> Thank you, Mike Moore >> Microsoft, ASP.NET
>> This posting is provided "AS IS", with no warranties, and confers no >rights.
>> -------------------- >> >From: "Brad Griffey" <bgrif...@dollar.com> >> >Subject: IsMobileDevice -- Mobile Devices and Search Engine Spiders >> >Date: Thu, 21 Nov 2002 17:40:57 -0600 >> >Lines: 14 >> >X-Priority: 3 >> >X-MSMail-Priority: Normal >> >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >> >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >> >Message-ID: <u7w3vdbkCHA.2472@tkmsftngp08> >> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >> >NNTP-Posting-Host: 208.2.126.4 >> >Path: cpmsftngxa08!tkmsftngp01!tkmsftngp08 >> >Xref: cpmsftngxa08 microsoft.public.dotnet.framework.aspnet.mobile:4985 >> >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>> >We are using the IsMobileDevice property to automatically redirect mobile >> >devices to our customized mobile site, while keeping standard web browser >> >traffic flowing into our conventional website. The problem we have is >> >search engine spiders. The code inside the Machine.Config that is added >> via >> >the MMIT is setting the IsMobileDevice property to TRUE for all devices >> >except Mozilla. Spiders such as Google are not Mozilla, so when they >> >attempt to index our site, they get redirected to the mobile site >instead.
>> >Has anyone encountered a similar problem, or have ideas on how best to >> >resolve this?
Thanks for your help with this. We had to implement a fix last week, so we went with the list of user-agent strings in the machine.config file. Although I would like to return to the pixel height-width solution at some point (which would require less maintenance), I wanted to let you know the user-agent solution is working for us, and spiders are now being properly identified as non-mobile devices.
> I'm still looking for more information on this. So far, your own idea of > using window height/width, along with IsMobileDevice, is looking like it > may be the best solution we can offer at this time. I am still looking into > this and will keep you posted.
> Thank you, Mike Moore > Microsoft, ASP.NET
> This posting is provided "AS IS", with no warranties, and confers no rights.
> >Subject: Re: IsMobileDevice -- Mobile Devices and Search Engine Spiders > >Date: Mon, 9 Dec 2002 08:12:15 -0600 > >Lines: 101 > >X-Priority: 3 > >X-MSMail-Priority: Normal > >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 > >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 > >Message-ID: <ObH82z4nCHA.1448@TK2MSFTNGP11> > >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile > >NNTP-Posting-Host: mail.dollar.com 208.2.126.4 > >Path: cpmsftngxa06!cpmsftngxa10!tkmsftngp01!TK2MSFTNGP11 > >Xref: cpmsftngxa06 microsoft.public.dotnet.framework.aspnet.mobile:4993 > >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
> >I guess the concern I have with this fix is I'm tracking spiders based on > >their existing user-agent strings (or some other means of identifying > them). > >I know that new spiders are created all the time, and existing search > >engines upgrade their existing spiders. I'm just worried that my > user-agent > >list within my machine.config will need to be routinely maintained.
> >One of the methods I used prior to our site going .NET was to check the > >pixel height and width of the calling device (if it was less than 640x480 > we > >assumed it was a mobile device). I'm assuming spiders would return 0x0 or > >null for this, which would still be a good way to detect them and flag them > >as non-mobile. Would this be a viable solution as well? At least it would > >prevent me from having to maintain a user-agent list in my machine.config.
> >> Though I don't have a ready-made solution for you, I can point you in the > >> right direction. Use the information below to compile a list of "user > >> agent" strings that you wish to identify. Then you change your > >> machine.config file so that it uses the list you have compiled.
> >> BACKGROUND INFO > >> When a web device (such as a browser, or a automated paged indexer) > >> requests a page from a server, it includes a "user agent" string in the > >> request. This string, called HTTP_USER_AGENT, identifies that the request > >> is coming from Internet Explorer 5 or Netscape Communicator 6 or .... > >there > >> are a huge number of possibilities.
> >> The machine.config file contains a section named "browserCaps". This > >> section contains code which attempts to parse or match the text in the > >user > >> agent string to identify the browser. This section is designed to > identify > >> browsers used by end users (such as IE or Netscape). With MMIT installed, > >> it also performs some minimal testing to identify mobile devices. To a > >> large degree, the logic simply marks the user agent as being from a > mobile > >> device if it was not able to otherwise identify the user agent.
> >> 2) Update the browserCaps section of machine.config > >> Look at the syntax of what is already in this section, then expand on it > >> with your new list.
> >> --- > >> Do you have any more questions on this issue?
> >> Thank you, Mike Moore > >> Microsoft, ASP.NET
> >> This posting is provided "AS IS", with no warranties, and confers no > >rights.
> >> -------------------- > >> >From: "Brad Griffey" <bgrif...@dollar.com> > >> >Subject: IsMobileDevice -- Mobile Devices and Search Engine Spiders > >> >Date: Thu, 21 Nov 2002 17:40:57 -0600 > >> >Lines: 14 > >> >X-Priority: 3 > >> >X-MSMail-Priority: Normal > >> >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 > >> >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 > >> >Message-ID: <u7w3vdbkCHA.2472@tkmsftngp08> > >> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile > >> >NNTP-Posting-Host: 208.2.126.4 > >> >Path: cpmsftngxa08!tkmsftngp01!tkmsftngp08 > >> >Xref: cpmsftngxa08
> >> >We are using the IsMobileDevice property to automatically redirect > mobile > >> >devices to our customized mobile site, while keeping standard web > browser > >> >traffic flowing into our conventional website. The problem we have is > >> >search engine spiders. The code inside the Machine.Config that is added > >> via > >> >the MMIT is setting the IsMobileDevice property to TRUE for all devices > >> >except Mozilla. Spiders such as Google are not Mozilla, so when they > >> >attempt to index our site, they get redirected to the mobile site > >instead.
> >> >Has anyone encountered a similar problem, or have ideas on how best to > >> >resolve this?
QUESTION The IsMobileDevice property is sometimes inadequate. It identifies some incoming requests as mobile which are more accurately described as "unknown". In particular, some search engines of interest to us are using User-Agent-Strings which are not currently recognized by the System.Web.Mobile.MobileCapabilities object. Currently, the MobileCapabilities object sets IsMobileDevice to True for unrecognized requests. The result is that requests from unrecognized search engines are directed to the mobile section of our web site. Then the main-stream section of our site does not get indexed by the search engine.
The requests sent by search engines and other automated queries are often called "spiders" or "crawlers". Many of the larger search engines use a User-Agent-String containing the word "mozilla" which identifies them as a desktop type browser. However, some search engines use other strings and even change their strings occasionally.
ANSWER I am glad to hear that you have been successful in setting up your own list of user-agent-strings.
I'm still looking into this. Third party user-agent-strings are not specifically supported. This means that the .NET Framework does not promise to recognize them. However, I am hoping I can find something for you that is easier to work with over time than for you to maintain your own list of user-agent-strings.
So far, I've found that the height/width is only useful if your list of user-agent-strings is fairly up to date with regard to mobile devices. I found that the framework does a good job of identifying known mobile devices and setting height/width information for them. Therefore, a device with IsMobileDevice set to True and no height/width is probably an UN-recognized request. So, if the browsercaps section of your machine.config file contains an up-to-date list of known mobile devices, then most requests that truly are mobile requests should have both IsMobileDevice set to True and have some height/width other than zero or null.
Again, I'm still looking for a better solution. I don't know if I will succeed. And, this may take a few more days.
Thank you, Mike Moore Microsoft, ASP.NET
This posting is provided "AS IS", with no warranties, and confers no rights.
QUESTION The IsMobileDevice property is sometimes inadequate. It identifies some incoming requests as mobile which are more accurately described as "unknown". In particular, some search engines of interest to us are using User-Agent-Strings which are not currently recognized by the System.Web.Mobile.MobileCapabilities object. Currently, the MobileCapabilities object sets IsMobileDevice to True for unrecognized requests. The result is that requests from unrecognized search engines are directed to the mobile section of our web site. Then the main-stream section of our site does not get indexed by the search engine.
The requests sent by search engines and other automated queries are often called "spiders" or "crawlers". Many of the larger search engines use a User-Agent-String containing the word "mozilla" which identifies them as a desktop type browser. However, some search engines use other strings and even change their strings occasionally.
ANSWER I am glad to hear that you have been successful in setting up your own list of user-agent-strings.
I looked into this and I had some co-workers help me. Our conclusion is that we don't have an ideal solution for you. Using height/width is not perfect. A superior option to height/width is to change the default value in machine.config for the IsMobileDevice property from True to False. With the value False, all devices not specifically recognized will be marked as NOT mobile.
Using height/width actually turns out to be very similar to doing just this. Except, that using height/width could mark devices as being NOT mobile when they did get recognized as a known mobile type, but one which does not have height/width specified.
That's the "easy" option. As you can guess, this leaves an opening for genuine mobile devices that are not recognized being directed to your HTML site rather than your mobile site. So, the easy solution forces you to choose whether unrecognized devices are always HTML or always mobile.
The more accurate solution is the same as given previously. Do some research to find the user-agent-strings that are important to you and add them to your machine.config file. Below is a repeat of that answer.
---- Here is a repeat of the previous post.
BACKGROUND INFO When a web device (such as a browser, or a automated paged indexer) requests a page from a server, it includes a "user agent" string in the request. This string, called HTTP_USER_AGENT, identifies that the request is coming from Internet Explorer 5 or Netscape Communicator 6 or .... there are a huge number of possibilities.
The machine.config file contains a section named "browserCaps". This section contains code which attempts to parse or match the text in the user agent string to identify the browser. This section is designed to identify browsers used by end users (such as IE or Netscape). With MMIT installed, it also performs some minimal testing to identify mobile devices. To a large degree, the logic simply marks the user agent as being from a mobile device if it was not able to otherwise identify the user agent.
I was just looking at my last response. I tried to make it legible to new comers to the thread. Now, I think I buried my answer to you.
The answer is that we don't have a lot more to offer. Earlier, you suggested using height/width in addition to IsMobileDevice. I found something that is superior to that, but it's still just an incremental improvement.
The height/width only gets set if the machine.config browsercaps section recognizes the device within the mobile section. So, height/width is a subset of recognized mobile devices. It you change the default value of IsMobileDevice from true to false, then you will have a better solution compared to using height/width with IsMobileDevice. Unfortunately, that's all I have to offer at this time (aside from the advice about finding and adding your own list of user-agent-strings).
What that leaves, is to either leave the default for IsMobileDevice set to true and risk redirecting non-mobile devices to your mobile site. Or, setting it to false and risk sending mobile devices to your HTTP site.
Clearly, it's better if you can add your own user-agent-strings.
Thank you, Mike Moore Microsoft, ASP.NET
This posting is provided "AS IS", with no warranties, and confers no rights.
>Subject: Re: IsMobileDevice -- Mobile Devices and Search Engine Spiders >Date: Mon, 16 Dec 2002 08:06:03 -0600 >Lines: 168 >X-Priority: 3 >X-MSMail-Priority: Normal >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >Message-ID: <eva4AxQpCHA.2432@TK2MSFTNGP12> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >NNTP-Posting-Host: mail.dollar.com 208.2.126.4 >Path: cpmsftngxa06!tkmsftngp01!TK2MSFTNGP08!TK2MSFTNGP12 >Xref: cpmsftngxa06 microsoft.public.dotnet.framework.aspnet.mobile:5028 >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>Mike,
>Thanks for your help with this. We had to implement a fix last week, so we >went with the list of user-agent strings in the machine.config file. >Although I would like to return to the pixel height-width solution at some >point (which would require less maintenance), I wanted to let you know the >user-agent solution is working for us, and spiders are now being properly >identified as non-mobile devices.
>> I'm still looking for more information on this. So far, your own idea of >> using window height/width, along with IsMobileDevice, is looking like it >> may be the best solution we can offer at this time. I am still looking >into >> this and will keep you posted.
>> Thank you, Mike Moore >> Microsoft, ASP.NET
>> This posting is provided "AS IS", with no warranties, and confers no >rights.
>> -------------------- >> >From: "Brad Griffey" <bgrif...@dollar.com> >> >References: <u7w3vdbkCHA.2472@tkmsftngp08> ><iUeBu3nkCHA.2528@cpmsftngxa08> >> >Subject: Re: IsMobileDevice -- Mobile Devices and Search Engine Spiders >> >Date: Mon, 9 Dec 2002 08:12:15 -0600 >> >Lines: 101 >> >X-Priority: 3 >> >X-MSMail-Priority: Normal >> >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >> >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >> >Message-ID: <ObH82z4nCHA.1448@TK2MSFTNGP11> >> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >> >NNTP-Posting-Host: mail.dollar.com 208.2.126.4 >> >Path: cpmsftngxa06!cpmsftngxa10!tkmsftngp01!TK2MSFTNGP11 >> >Xref: cpmsftngxa06 microsoft.public.dotnet.framework.aspnet.mobile:4993 >> >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>> >I guess the concern I have with this fix is I'm tracking spiders based on >> >their existing user-agent strings (or some other means of identifying >> them). >> >I know that new spiders are created all the time, and existing search >> >engines upgrade their existing spiders. I'm just worried that my >> user-agent >> >list within my machine.config will need to be routinely maintained.
>> >One of the methods I used prior to our site going .NET was to check the >> >pixel height and width of the calling device (if it was less than 640x480 >> we >> >assumed it was a mobile device). I'm assuming spiders would return 0x0 >or >> >null for this, which would still be a good way to detect them and flag >them >> >as non-mobile. Would this be a viable solution as well? At least it >would >> >prevent me from having to maintain a user-agent list in my >machine.config.
>> >> Though I don't have a ready-made solution for you, I can point you in >the >> >> right direction. Use the information below to compile a list of "user >> >> agent" strings that you wish to identify. Then you change your >> >> machine.config file so that it uses the list you have compiled.
>> >> BACKGROUND INFO >> >> When a web device (such as a browser, or a automated paged indexer) >> >> requests a page from a server, it includes a "user agent" string in the >> >> request. This string, called HTTP_USER_AGENT, identifies that the >request >> >> is coming from Internet Explorer 5 or Netscape Communicator 6 or .... >> >there >> >> are a huge number of possibilities.
>> >> The machine.config file contains a section named "browserCaps". This >> >> section contains code which attempts to parse or match the text in the >> >user >> >> agent string to identify the browser. This section is designed to >> identify >> >> browsers used by end users (such as IE or Netscape). With MMIT >installed, >> >> it also performs some minimal testing to identify mobile devices. To a >> >> large degree, the logic simply marks the user agent as being from a >> mobile >> >> device if it was not able to otherwise identify the user agent.
>> >> 2) Update the browserCaps section of machine.config >> >> Look at the syntax of what is already in this section, then expand on >it >> >> with your new list.
>> >> --- >> >> Do you have any more questions on this issue?
>> >> Thank you, Mike Moore >> >> Microsoft, ASP.NET
>> >> This posting is provided "AS IS", with no warranties, and confers no >> >rights.
>> >> -------------------- >> >> >From: "Brad Griffey" <bgrif...@dollar.com> >> >> >Subject: IsMobileDevice -- Mobile Devices and Search Engine Spiders >> >> >Date: Thu, 21 Nov 2002 17:40:57 -0600 >> >> >Lines: 14 >> >> >X-Priority: 3 >> >> >X-MSMail-Priority: Normal >> >> >X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 >> >> >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 >> >> >Message-ID: <u7w3vdbkCHA.2472@tkmsftngp08> >> >> >Newsgroups: microsoft.public.dotnet.framework.aspnet.mobile >> >> >NNTP-Posting-Host: 208.2.126.4 >> >> >Path: cpmsftngxa08!tkmsftngp01!tkmsftngp08 >> >> >Xref: cpmsftngxa08 >microsoft.public.dotnet.framework.aspnet.mobile:4985 >> >> >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.mobile
>> >> >We are using the IsMobileDevice property to automatically redirect >> mobile >> >> >devices to our customized mobile site, while keeping standard web >> browser >> >> >traffic flowing into our conventional website. The problem we have is >> >> >search engine spiders. The code inside the Machine.Config that is >added >> >> via >> >> >the MMIT is setting the IsMobileDevice property to TRUE for all >devices >> >> >except Mozilla. Spiders such as Google are not Mozilla, so when they >> >> >attempt to index our site, they get redirected to the mobile site >> >instead.
>> >> >Has anyone encountered a similar problem, or have ideas on how best to >> >> >resolve this?
|
