Firebase hosting: Googlebot blocked by robots.txt

[winacod...@gmail.com]

Hello, 

I want to display ads using ad sense in my angular/firebase application but I got this issue Googlebot blocked by robots.txt

I tried to do my owen robots.txt and deploy it in firebase hosting but it does not work.

Google ad sense continues to refuse my account and I cannot display ads

Thanks

[Alex Memering]

Firebase Hosting does not automatically provide a `robots.txt` page.  You should make sure that you do not have a redirect or rewrite set up that is covering that path and that the `robots.txt` page that you are trying to deploy is in your public directory.

Hope that helps,

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/26d38b01-b5a6-4d18-8b52-b2f512749a93%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Alex Memering |

Software Engineer |

meme...@google.com

[winacod...@gmail.com]

Hi Alex;

Thanks for your replay.

robots.txt is in my public directory. with 'Firebase deploy' all my public folder will be deployed but the file is not reachable that's why I have the previous error.

This is my firebase.json file

{

"database": {

"rules": "database.rules.json"

},

"functions": {

"predeploy": [

"npm --prefix ./functions run lint",

"npm --prefix ./functions run build"

],

"source": "functions"

},

"hosting": {

"public": "wwwroot",

"ignore": [

"firebase.json",

"**/.*",

"**/node_modules/**"

],

"rewrites": [

{

"source": "**",

"destination": "/index.html"

}

],

"headers": [

{

"source": "**/*.html)",

"headers": [

{

"key": "Cache-Control",

"value": "max-age=0, no-cache"

}

]

},

{

"source": "**/*.@(jpg|jpeg|gif|png)",

"headers": [

{

"key": "Cache-Control",

"value": "max-age=3300"

}

]

},

{

"source": "**/*.@(css|js)",

"headers": [

{

"key": "Cache-Control",

"value": "max-age=3500"

}

]

}

]

},

"storage": {

"rules": "storage.rules"

}

}

 

In the robots file I put Allow for all user-agent.

I don't get it !

[Alex Memering]

That seems correct to me.  When you go to `/robots.txt` on your domain does it show up?

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/7e28cfc8-2f9c-477e-92b8-f97dd8d983db%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[winacod...@gmail.com]

Yes, it shows up (attached file..). I tired to deploy in gcloud and I got the same issue. I thins the robots file is overwritten, this is the content of my file

User-agent: *
Allow: /

and this is what robots file look like in https://googleads.g.doubleclick.net/robots.txt

User-Agent: *
Allow: /ads/preferences/
Allow: /gpt/
Allow: /pagead/show_ads.js
Allow: /pagead/js/adsbygoogle.js
Allow: /pagead/js/*/show_ads_impl.js
Allow: /static/glade.js
Allow: /static/glade/
Disallow: /
Noindex: /

[Alex Memering]

Are you sure that you deployed the correct file?  The `/robots.txt` that you have in your public folder locally should be the one that gets deployed with the Firebase CLI.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/1c370bbc-61cd-42ca-aae8-3a943797e8e9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[winacod...@gmail.com]

Yes, I am sure since Firebase take my folder and deploy it

But I don't have any ways to check if the folder is in Firebase hosting

[Michael Bleigh]

When you visit your hosted website at /robots.txt you will see what we serve. We do not serve different traffic to bots so if you see the content you expect that is what the crawler will see as well.

On Tue, Feb 26, 2019, 6:53 AM <winacod...@gmail.com> wrote:

Yes, I am sure since Firebase take my folder and deploy it

But I don't have any ways to check if the folder is in Firebase hosting

--

You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/b2d9f14d-944a-41dd-87d2-e7d9237fc88a%40googlegroups.com.