good 404 vs bad 404

[guettli]

In systems which are accessed only via intranet we used to monitor http 404 responses.

Every time a 404 happens, an issue in our monitoring gets created.

This worked well in the past.

Now we have cases where a 404 is valid or should be ignored. No issue in the monitoring should be visible.

You could apply "separation of concerns" and keep on reporting all 404 responses
and do the filtering in the monitoring area.

But on the other hand the code knows more. I have places where I know that
this 404 should be ignored.

I am biased where the problem should get solved:
 
 - inside django app
 - inside monitoring


What do you think?

Does anybody make a differenence between "good 404 vs bad 404"?

Regards,
  Thomas

[Andreas Kuhne]

I think you should always report a 404 as a 404 regardless of the situation. 

The application shouldn't have to know if it is a valid 404 or an "invalid" - because from the applications point of view, the page (or item) couldn't be found. 

Just my 2 cents.

Regards,

Andréas 

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/fb152b91-ee44-42aa-8304-fb43eba43be1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Antonis Christofides]

My gut feeling says you should treat this in monitoring, however here are some questions:

1) Why do you monitor 404s at all?

2) Could you give some examples of 404s that are valid or should be ignored?

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALXYUbkcs%2BEqkNPLNhctqtEzB7Zd3oPKVoJUMNi8yzcXgON7bQ%40mail.gmail.com.

[ludovic coues]

I would tag the 404 in django and let the monitoring app do most of the work.

The tag would simply be an extra header, x-reason for example. It would take values like "no mapping for url" or "object not found".

This way, your django app is still doing no more than its job, indicating the ressources is unavailable, and this monitoring app have enough data to decide if the 404 should be logged or not.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALXYUbkcs%2BEqkNPLNhctqtEzB7Zd3oPKVoJUMNi8yzcXgON7bQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/32a6faf0-a248-b983-2ad3-6dbc2d833927%40djangodeployment.com.

[guettli]

Am Mittwoch, 1. März 2017 10:42:44 UTC+1 schrieb Andréas Kühne:

I think you should always report a 404 as a 404 regardless of the situation. 

The application shouldn't have to know if it is a valid 404 or an "invalid" - because from the applications point of view, the page (or item) couldn't be found. 

If I always report the 404 the monitoring people would hate me because they get too many false issues. Or they would switch off thinking and
say ... "nice, I get paid for pushing the button 'ignore-this'. That's my job - relaxed and money monthly. What do I want more?"

This was a joke, I hope it does not offend anybody.

[guettli]

Am Mittwoch, 1. März 2017 10:54:38 UTC+1 schrieb Antonis Christofides:

My gut feeling says you should treat this in monitoring, however here are some questions:

1) Why do you monitor 404s at all?

2) Could you give some examples of 404s that are valid or should be ignored?

I monitor 404 since it this question is about an application which runs only in the intranet.

No robots or other strange request come in.

99% of all requests are inside the application.

If there is a 404, then it is very likely that our application creates broken links.

Since we use reverse() every this is very seldom.

Up to now monitoring 404 was nice to have.

But now there are some parts where too many 404 responses get returned.

Concrete example: we render HTML mails. They often contain broken links.
I don't want to report these.

Can you understand my use case now? If not, please ask.

[guettli]

nice, I like this. Yes, an additional header ... Thank you.

Regards,

Andréas 

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/fb152b91-ee44-42aa-8304-fb43eba43be1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALXYUbkcs%2BEqkNPLNhctqtEzB7Zd3oPKVoJUMNi8yzcXgON7bQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

[C. Kirby]

There are lots of other status codes - perhaps your "good" or expected not found should be a 3xx code (Redirection) or a 4xx that isn't 404

https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

[Melvyn Sopacua]

On Wednesday 01 March 2017 08:21:34 guettli wrote:

> Concrete example: we render HTML mails. They often contain broken

> links. I don't want to report these.

 

That depends if the rendering makes the link broken (for example you fold soft wraps in quoted-printable incorrectly).

A different approach would be to treat the notification mails the way you treat spam. Notifications that are "user errors" are "spam", notifications that are "application errors" are "ham".

Using baysian filtering, the number of "user errors" should deminish greatly over time.

--

Melvyn Sopacua

[Vinicius Assef]

See suggestion below.

On 1 March 2017 at 13:21, guettli <guet...@gmail.com> wrote:
>
> Concrete example: we render HTML mails. They often contain broken links.
> I don't want to report these.

I would add a query string in these links to track them. E.g.,
"?autogenerated=1".

So, links with this query string should be filtered by monitoring
service and not notified.

[Antonis Christofides]

If what you want is locate broken links and ignore 404s resulting from emails, then Django's broken link detection functionality is probably all that you need, unless you really need to do it in monitoring and you can't possibly do it with emails. Ignoring 404s with no referrer has been added to 1.9. If I wanted to do this and I was using 1.8 I'd probably try to monkey-patch Django with backported code from 1.9.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/74679d0f-aa42-4885-8702-c8f6caf6a6bc%40googlegroups.com.

[C Kirby]

I'd argue that all of the 404s are bad - if your tools are generating broken links isn't it better to fix that than to band-aid it with what is a "good" vs "bad" 404?

On Thu, Mar 2, 2017 at 11:10 AM, Antonis Christofides <ant...@djangodeployment.com> wrote:

If what you want is locate broken links and ignore 404s resulting from emails, then Django's broken link detection functionality is probably all that you need, unless you really need to do it in monitoring and you can't possibly do it with emails. Ignoring 404s with no referrer has been added to 1.9. If I wanted to do this and I was using 1.8 I'd probably try to monkey-patch Django with backported code from 1.9.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com

On 03/01/2017 06:21 PM, guettli wrote:

Am Mittwoch, 1. März 2017 10:54:38 UTC+1 schrieb Antonis Christofides:

My gut feeling says you should treat this in monitoring, however here are some questions:

1) Why do you monitor 404s at all?

2) Could you give some examples of 404s that are valid or should be ignored?

I monitor 404 since it this question is about an application which runs only in the intranet.

No robots or other strange request come in.

99% of all requests are inside the application.

If there is a 404, then it is very likely that our application creates broken links.

Since we use reverse() every this is very seldom.

Up to now monitoring 404 was nice to have.

But now there are some parts where too many 404 responses get returned.

Concrete example: we render HTML mails. They often contain broken links.
I don't want to report these.

Can you understand my use case now? If not, please ask.

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/74679d0f-aa42-4885-8702-c8f6caf6a6bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/Ew7hDj-ELaw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-users+unsubscribe@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/172dd6e3-a6a9-145c-4963-063100332cd1%40djangodeployment.com.

[Antonis Christofides]

The Django documentation explains it nicely:

[Django] doesn’t bother to email [the admins] for 404s that don’t have a referer – those are usually just people typing in broken URLs or broken Web bots.

Antonis Christofides
http://djangodeployment.com

To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFRq704FmGtr40Km-aX%2Bj09RzWQLmXneDYohm%3D_DbOfCR6whyg%40mail.gmail.com.