Bots - JavaScript Challenge vs Anubis

[lib-s...@unbc.ca]

Hello,

Like many others, we are being hit with recent bot scraping "attacks". Our organization is looking at Cloudflare, but until that is in place I need a solution to keep our AtoM from falling over.

I first tried Anubis (https://anubis.techaro.lol/), and I have a question about that. But first, two questions about Javascript Challenge (https://www.accesstomemory.org/en/docs/2.9/admin-manual/security/js-challenge/), which I discovered via another post about bots in this group. Questions:

1. How would we know it's working?
2. Can someone at Atefactual please expand the "Finishing Up Installing JS Challenge" section to detail exactly how to perform those actions? Is there some  cache(s) to clear other than 'php symfony cache:clear'? How does one 'update composer dependencies'? I didn't use Composer in our install, I used the tarball install method. And is 'Rebuild BS5 theme' just re-running 'sudo npm run build', or is there more to it?

Now, my question about Anubis. I got it working just fine, except the site doesn't work for someone who's logged in.

Brief summary of Anubis/AtoM config:

1. Nginx listens on port 80, redirects to 443
2. Nginx listens on port 443, uses proxy_pass to send to Anubis
3. Nginx listens on port 8080, this is where all the usual atom config is. If traffic successfully passes through Anubis, it goes here.

As I said, the site works great with Anubis. However, if a staff member tries to sign in, then after entering credentials they are redirected to https://search.nbca.unbc.ca:8080/user, which produces an "ERR_CONNECTION_TIMED_OUT" in the browser. If the ":8080" is manually removed from the URL, the site loads and it shows the account signed in, but the main body text is "Sorry, you do not have permission to access that page". Trying to go to any page via the menus generates the same error.

Is this something that could be fixed on the AtoM side, or is it more likely something I'll have to change in Anubis?

I realize the chances of getting a good answer are slim, but I'm trying anyway just in case.

Thanks,

David 

[matthewb...@gmail.com]

Hi David,

Not an answer to your question, but I have blocked IP ranges on UFW and found that works pretty well.

The bots are coming from countries which would have no interest in the type of material on my atom instances anyway, so its not a huge loss. 

But you can block specific ips on ufw also. Just go into your logs and see them. I have done that also, and the bot traffic trails off. 

That way, the legitimate bots like google, meta, etc. etc. get through as their servers are based in the US and europe mostly.

Its a bit of a hack, but keeps things quiet on the server.

Matthew

[Brandon Uhlman]

We are running Anubis successfully, but a shared instance that protects several applications. To do that, it's configured using Anubis' subrequest authentication (https://anubis.techaro.lol/docs/admin/configuration/subrequest-auth).

As for why your users are getting redirected to port 8080, it sounds like AtoM isn't figuring out that it's getting proxied, so it's generating URIs at the hostname at which it, itself, is getting called. Are you setting the request headers as suggested in Anubis' suggested configuration for Nginx (https://anubis.techaro.lol/docs/admin/environments/nginx), in particular the "proxy_set_header Host" line? Another possibility, which we implemented on some of our applications for this reason, is to have the "usual AtoM config" listening not on a non-standard port like 8080 -- but on the standard port 443 on a different IP address (either a non-routable internal IP address, or 127.0.0.1), that way the request itself knows it is still being served on port 443, so if the application uses that port to generate URLs, it will fake it into generating correct ones.

If you're still experiencing problems and you could share your Nginx configuration for 443 and 8080 (either privately, or in the group), I'm happy to look and offer some suggestions.

Brandon

[lib-s...@unbc.ca]

Thank you for the suggestion, Brandon! I switched from using the Reverse proxy method to the Subrequest authentication mode, and it seems to be working correctly. At least on our Dev server, I’ll wait for our Archivist to confirm before I apply the change to Prod.

The only thing I had to do extra was tweak the “return 307” line in the @redirectToAnubis location block, as I ran into the “Redirect domain not allowed” error. "Redirect domain not allowed" on any domain when using Subrequest Authentication in docker · Issue #390 · TecharoHQ/anubis

 David