In treeherder when you click on a job an info pane pops up at the bottom of the screen, if there are failures, they are displayed along with potential matching bugs.
Now if there is a failure that is unknown in the last 3 weeks and matches a pattern of "TEST-UNEXPECTED-* | testname | error", there will be a "NEW" [1] button to the left of the failure line. This will indicate a few things:
1) this failure is on an existing bug that hasn't had failures in the last 3 weeks on autoland
2) this failure is not seen and you should look into it (ideally gather a few more data points)
a) if this doesn't fail on retriggers, it is a new intermittent, so a new bug needs to be filed
b) if this fails again on a retrigger, it is most likely a regression caused by the revision!
This change is in place primarily for the sheriffs- it helps them pinpoint new failures which has become more difficult with "single tracking bugs". As developers this can be used as a way to help focus on NEW failures and not existing ones (complements the mitten icon!).
There are edge cases, so this isn't an absolute signal. Please leave feedback in #treeherder in Matrix.