Upgrade to a newer SKU of Virtual Machine image

[Daryl W]

Azure keeps telling me this, has doubled my price, and the machine has gotten unstable the last few months. 

What is the process of upgrading to a new version of the SKU?

I have spent hours getting everything working like I want and this forced upgrade is annoying and also concerning. Are things going to migrate smoothly?

Any help is appreciated.

[rafael....@diladele.com]

Good morning, DRCBW,

We  have never tried in place upgrades of the SKU on Azure, here in the test lab we always deploy a new machine then import settings from the old machine - and yes some minor settings might need to be adjusted after it.

I would not worry about Azure recommendations - if it worked before ok I do not see any reason it cannot continue working so in the future.

When you say unstable - what exactly you mean?

The machine should *not* get unstable with time.

Best regards,

Rafael

[Daryl W]

Good morning,

Thanks for getting back to me. I was hoping to be able to do just that, setup another machine and migrate settings...

As for the stability issues, it 'feels' like it is an Azure thing. What I mean by that is they are notifying me about the depreciated SKU and telling me that I need to upgrade for "stability improvements" at the same time a machine that has run without hitch for the last 3 years suddenly starts randomly stopping responding and requires a reboot to get it working again. Along with that they literally doubled my monthly price. It all feels like "part of the package" to force me to upgrade.

Honestly, I have never liked Azure and would love to go somewhere else. However, I had originally tried DO only to have too many websites that had the DO IP address on a blacklist. Is there any other providers besides Azure that work well? Vultr?

Thanks

--
You received this message because you are subscribed to a topic in the Google Groups "Diladele Web Safety" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web-safety/ytprgIc7HUA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web-safety+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/web-safety/7abbdfe0-a088-4e8c-ae40-6cdb05a36895n%40googlegroups.com.

[rafael....@diladele.com]

Unfortunately we only run our test scenarios in Azure/AWS - and also sometimes on hetzner.de - but the latter does present more captchas from google/youtube so it might be as inconvenient as DO.

[Daryl W]

Ok thanks. I set up another machine in AWS and imported my license key. However when I attempt to backup my original machine I get this error Permission denied: '/opt/websafety/etc/gui.key'")] 

I SSH'ed into my machine but get the same error when trying to change permissions on that file.

Any idea's?

To view this discussion visit https://groups.google.com/d/msgid/web-safety/f8f3af83-642c-4517-8af0-efdeeb55cc49n%40googlegroups.com.

[Rafael Akchurin]

Just run the chown command with sudo?

Best regards,

Rafael

Op 12 dec 2024 om 21:17 heeft Daryl W <drcbw...@gmail.com> het volgende geschreven:



You received this message because you are subscribed to the Google Groups "Diladele Web Safety" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web-safety+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/web-safety/CAHxzj4dURSJE%2BE9j8x912%3D%3Dtfi5XnLosP0dPXX0tzJEoNwsOFw%40mail.gmail.com.

[Daryl W]

Ah yes. I thought I was SUDO but guess I wasn't. Got it working. Thanks

To view this discussion visit https://groups.google.com/d/msgid/web-safety/789FA870-C8AF-4AA3-8A7D-9AFE7D7AF9E1%40diladele.com.

[Daryl W]

So I got my new machine up and running, restored a backup, logged into the console and noticed there were updates. I blindly ran apt list --update and do-release-upgrade accepting all defaults and of course didn't have a snapshot of anything. Now Apache will not start. These are the errors:
mod_wsgi (pid=667): Failed to exec Python script file '/opt/websafety-ui/var/console/console/wsgi.py'.

File "/opt/websafety-ui/var/console/console/wsgi.py", line 13, in <module>
from django.core.wsgi import get_wsgi_application

ModuleNotFoundError: No module named 'django'

Is there an easy fix or would I be better off destroying this machine and starting over?

Also, over the last few years I have highly customized the heuristics.conf and the weighted.conf files. I noticed these are in json format on the new server. Is there an easy process to convert my highly modified files?

Thanks

[Rafael Akchurin]

The do release upgrade was a little too much. We still stick to ubuntu 22.04 instead of 24.04 :(

Best regards,

Rafael

Op 13 dec 2024 om 16:57 heeft Daryl W <drcbw...@gmail.com> het volgende geschreven:

So I got my new machine up and running, restored a backup, logged into the console and noticed there were updates. I blindly ran apt list --update and do-release-upgrade accepting all defaults and of course didn't have a snapshot of anything. Now Apache will not start. These are the errors:

To view this discussion visit https://groups.google.com/d/msgid/web-safety/fb975fc0-b216-4905-95cc-56a29bd0e1a0n%40googlegroups.com.

[Rafael Akchurin]

As for conf to json - it is very simple in structure - i will try to find the script we used for conversion and get back to you..

Best regards,

Rafael

Op 13 dec 2024 om 16:57 heeft Daryl W <drcbw...@gmail.com> het volgende geschreven:

So I got my new machine up and running, restored a backup, logged into the console and noticed there were updates. I blindly ran apt list --update and do-release-upgrade accepting all defaults and of course didn't have a snapshot of anything. Now Apache will not start. These are the errors:

To view this discussion visit https://groups.google.com/d/msgid/web-safety/fb975fc0-b216-4905-95cc-56a29bd0e1a0n%40googlegroups.com.

[Daryl W]

Ok no problem. I will destroy it and start over. No big deal since I hadn't really done anything with it other than a restore.

I believe you missed my second question:

Also, over the last few years I have highly customized the heuristics.conf and the weighted.conf files. I noticed these are in json format on the new server. Is there an easy process to convert my highly modified files?

To view this discussion visit https://groups.google.com/d/msgid/web-safety/81CC8393-F3F8-4F67-8CAA-1D5648FBDDE5%40diladele.com.

[rafael....@diladele.com]

The conf to json conversion script we used is attached.

[Daryl W]

Thanks for that. I have successfully converted by files and appended them to the weighted and heuristics json files. However, it doesn't appear that it is working. I had quite a bit of customizations for specific searches on mixed content sites like amazon and ebay. All the searches that should be blocked are allowed through.

To be quite honest, I am trying to wrap my head around why the switch to something that used to be as simple as this <shows and movies><30> to something that now looks like this     {
        "enable": true,
        "words": [
            "shows and movies"
        ],
        "weight": 30
    }

Also, the inability to comment a json file is frustrating. My old DG format files had all my customizations labeled as to what they do. With this json format, I have no idea what section of code does what. Or am I implementing this incorrectly?

Also, what am I missing that is causing this new json stuff not to work? The above is one actual example of one of my customizations. 

Thanks for your help.

[rafael....@diladele.com]

The JSON is needed to let the admin (later) manage those words in the Admin UI - parsing of conf structure is a little harder in Python.

As for detection - do not forget to enable HTTPS decryption - without it all proxy sees if the flow of encrypted bytes.

To add  comments, just add "comment" field to the JSON - as rule loader now do not care about actual JSON schema and just looks for tags which are known - simply ignoring other tags.

I frankly forgot to add  "comment" tag - will do that in the next version, added https://github.com/diladele/websafety/issues/2281 so not to forget.

Best regards,

Rafael

[Daryl W]

Ok, so this is a little pain for future gain. Fair enough and I look forward to having this built into the GUI.

I do have HTTPS filtering set for All Domains (with Exclusions) and enabled for the policy.

Does this work the same as before? My weight limit is set to the default of 150. In the DG format of previous versions, the example above added 30 to the total weight and any other matches either added or subtracted and if the total was less than 150, it was allowed. I assume this is doing the same totaling of all matches, or is it working differently?

[Rafael Akchurin]

Yes it sums up all components, no changes from previous versions.

Best regards,

Rafael

Op 14 dec 2024 om 18:45 heeft Daryl W <drcbw...@gmail.com> het volgende geschreven:

Ok, so this is a little pain for future gain. Fair enough and I look forward to having this built into the GUI.

To view this discussion visit https://groups.google.com/d/msgid/web-safety/03d149a0-90a4-427c-8ee5-aa87d1ae1079n%40googlegroups.com.

[Daryl W]

Well, it doesn't work.

I removed everything from the heuristics.json except for this:

[
    {
        "enable": true,
        "words": [
            " test"
        ],
        "weight": 400
    } 
]

I changed max URL weight to 40 in my policy.

Saved and restarted.

Searching google.com for test results in a block. This is expected behavior. This is the URL. https://www.google.com/search?q=test

However searching amazon.com for test, is allowed. This should have a score of 400. This is the URL https://www.amazon.com/s?k=test

To view this discussion visit https://groups.google.com/d/msgid/web-safety/BA30A171-D4B4-4587-89F4-1E9AF6322244%40diladele.com.

[Rafael Akchurin]

I will look into this tomorrow morning. If google is blocked and aws is not - maybe some other factors play a role not just json storage.

Best regards,

Rafael

Op 14 dec 2024 om 19:44 heeft Daryl W <drcbw...@gmail.com> het volgende geschreven:



To view this discussion visit https://groups.google.com/d/msgid/web-safety/CAHxzj4fsio-keRWQ4-ritNqaRssTdKwWigrmsuGdCJPK08jh_w%40mail.gmail.com.

[Daryl W]

Ok. Thanks for your help in this.

To view this discussion visit https://groups.google.com/d/msgid/web-safety/BCB3F609-F611-4E63-AE1F-8A7B5D0941A3%40diladele.com.

[rafael....@diladele.com]

Ok here are the results of the research:

- amazon.com is categorized as "online shopping" site

- by default the known "online shopping" sites are part of trusted category  - it means adult modules is not applied on those sites

- so as heuristics is part of adult detection

- if you search for test on amazon - the heuristics module is not triggered

- if you search for test on google - the heuristics module is triggered and you see the block.

Recommendation - remove the "online shopping" category from the trusted category in UI / Web Filter / Settings / Trusted Categories.

Best regards,

Rafael

[Daryl W]

Interesting. Apparently those setting were not included in the backup and I didn't think to check them. It fixed the problem. Thanks!

To view this discussion visit https://groups.google.com/d/msgid/web-safety/6e633202-bfd5-4374-8a3c-6553c6cef123n%40googlegroups.com.