404 after project wrongly suspended for social engineering and then reinstated

[Tycho Bellers]

I am using app engine to host a website for my Discord bot.

My website was wrongly suspended for "social engineering" and shortly after was reinstated after an appeal (this has happened twice already).

The emails I received mentioned a specific URL of my website:

https://discord-dictionary-bot.uc.r.appspot.com/docs/settings

It has now been a few days after the project was reinstated, but I still cannot access the home page (https://discord-dictionary-bot.uc.r.appspot.com) and the docs/settings page (https://discord-dictionary-bot.uc.r.appspot.com/docs/settings). They just return a 404 with the following message:

"Error: Page not found

The requested URL was not found on this server"

All other pages of the website work fine, but since the homepage doesn't work you need to know the exact URL to visit them. (For example https://discord-dictionary-bot.uc.r.appspot.com/docs/commands)

Interestingly, you can still visit the home page and docs/settings page if you specify default (and only) service in the URL: https://default-dot-discord-dictionary-bot.uc.r.appspot.com/

How can this be fixed?

[Darrell (Cloud Platform Support)]

Hi,

This sounds like a routing issue. Can you try redeploying your app as a new version and migrate all traffic[1] to that version?

If that does not work, can you provide some information about how you are routing your requests[2]?

[1] https://cloud.google.com/appengine/docs/standard/nodejs/migrating-traffic#migrating_traffic_to_a_new_version [2] https://cloud.google.com/appengine/docs/standard/python3/how-requests-are-routed

[Tycho Bellers]

I have already tried redeploying a new version, but that does not have any effect.

The only routing configuration I have is in app.yaml:

runtime: nodejs14

handlers:
# Serve all static files with urls ending with a file extension
- url: /(.*\..+)$
static_files: dist/\1
upload: dist/(.*\..+)$
# catch all handler to index.html
- url: /.*
static_files: dist/index.html
upload: dist/index.html

[goya]

I have used the GitHub repository linked in your website to reproduce this issue, and it has worked properly without any changes, so maybe you could try to redeploy again. Another thing you could try is to deploy into a new service and see if you get the same behaviour there too. 

If this issue of the main page not showing properly continues to occur, I suggest you open a support ticket as a deeper investigation might be needed.

[Tycho Bellers]

Deploying to a new service works, but then you have to specify that service in the URL which is not ideal. I still haven't managed to get it working again with just the default URL (https://discord-dictionary-bot.uc.r.appspot.com/).

I don't think I'm able to open a ticket because I'm just using the free tier :(

[Felipe Bergallo Corral]

Darrell mentioned traffic allocation, but in your response you stated that your only routing configuration is in your app.yaml file. What Darrell was referring to was the versions page, which should contain every version you've uploaded to App Engine so far (unless you've deleted one or more versions), with a percentage of traffic allocation.

Since you haven't mentioned mentioned traffic migration and the issue is specifically with the default URL, you can change where that URL points to. Which means you can make it so that https://discord-dictionary-bot.uc.r.appspot.com is an alias of https://default-dot-discord-dictionary-bot.uc.r.appspot.com instead of whatever version it is an alias of now. 

Another option would be to remove the faulty version from App Engine entirely, like so:

Simply select the version you want to delete, then click the delete button (in my case I only have one version, and it's both stopped and has traffic allocation, so I can't delete it at the moment).

[Tycho Bellers]

I have already tried to deploy a new version, and have deleted all other versions. I'm not sure how to create an alias for the URL.

One thing I've noticed is that the main page also does not show up in Traces or Logs, but everything else does:

This is my config for the only version I have:

runtime: nodejs14

env: standard

instance_class: F1

handlers:

  - url: /(.*\..+)$

    static_files: dist/\1

    require_matching_file: false

    upload: dist/(.*\..+)$

  - url: /.*

    static_files: dist/index.html

    require_matching_file: false

    upload: dist/index.html

  - url: .*

    script: auto

automatic_scaling:

  min_idle_instances: automatic

  max_idle_instances: automatic

  min_pending_latency: automatic

  max_pending_latency: automatic

network: {}

service_account: discord-dic...@appspot.gserviceaccount.com

[Felipe Bergallo Corral]

Ok, I've noticed something that I didn't before:
Whether or not the path (url) you use has a trailing slash or not affects the outcome. For example:
https://discord-dictionary-bot.uc.r.appspot.com/docs/settings -> 404 (not the catch-all you've set up)
https://discord-dictionary-bot.uc.r.appspot.com/docs/settings/ -> Settings page
https://discord-dictionary-bot.uc.r.appspot.com/docs/commands -> Commands page
https://discord-dictionary-bot.uc.r.appspot.com/docs/commands/ -> Black page (header loads, body doesn't; catch-all not triggered)
https://discord-dictionary-bot.uc.r.appspot.com/docs/languages -> Languages page
https://discord-dictionary-bot.uc.r.appspot.com/docs/languages/ -> Languages page (I expected this to not work but it does)
https://discord-dictionary-bot.uc.r.appspot.com/ -> Redirects to https://discord-dictionary-bot.uc.r.appspot.com -> 404 (not the catch-all you've set up)

I think it's that VueRouter is having a bit of trouble, so I think you should change the routing in index.js like so:
const routes = [
        {
                path: '/home/',
                name: 'Home',
                component: Home,

                pathToRegexpOptions: { strict: true } 

        },
        {
                path: '/statistics/',
                name: 'Statistics',
                component: Statistics,

                pathToRegexpOptions: { strict: true } 

        },
        {
                path: '/docs/',
                name: 'Documentation',
                component: Documentation,
                redirect: '/docs/commands',

                pathToRegexpOptions: { strict: true } 

children: [
                        {
                                path: '/commands/',
                                name: 'Commands',
                                component: Documentation,

                                pathToRegexpOptions: { strict: true } 

                        },
                        {
                                path: '/settings/',
                                name: 'Settings',
                                component: Documentation,

                                pathToRegexpOptions: { strict: true } 
                        },
                        {
                                path: '/languages/',
                                name: 'Languages',
                                component: Documentation,

                                pathToRegexpOptions: { strict: true } 

                        }
                ]

And change the links you've used to have trailing slashes, that way all paths are uniform and the pathing used is Vue's strict pathing (which prior to 2017 was the default), which should keep Vue aware of all the paths that exist.
I'm not sure why it's only in your project that Vue is struggling though... though I can confirm that it is somewhat consistent, in default-dot-discord-dictionary-bot it also struggles with trailing slashes, though only in one specific instance: https://default-dot-discord-dictionary-bot.uc.r.appspot.com/docs/commands/

It seems to me that Vue may have some internal ruling that it follows and that it's breaking due to the ambiguity, hence my suggestion of being rather specific with the pathing in the routes constant and setting it to strict - that way it at least acts consistently and predictably.

[Tycho Bellers]

Hmm, that is interesting, although I'm not able to reproduce that behavior when hosting the website locally. (Other than the blank page when accessing /commands, which was caused by this line).

All these scenarios work fine when hosted locally:

http://localhost:8080/docs/settings   -> OK
http://localhost:8080/docs/settings/  -> OK
http://localhost:8080/docs/commands   -> OK
http://localhost:8080/docs/commands/  -> Blank page (header only)
http://localhost:8080/docs/languages  -> OK
http://localhost:8080/docs/languages/ -> OK
http://localhost:8080  -> OK
http://localhost:8080/  -> OK

Given that the only pages that are not working properly then are the settings page and the home page, and that the emails I received when my website was suspended specifically mentioned those two URLs, I believe that those URLs are somehow still being blacklisted by App Engine.

That would also explain why there are no logs when I visit /docs/settings but there are logs when I visit /docs/settings/

It seems like App Engine is blocking these URLs before my app even has a chance to route them.