Denial of service attacks / Quota depletion

[Daniel Faust]

I'm in a ugly situation.

Due to a 3rd party gadget in a wave which has my robot as a
participant, the bot is depleting it's Incoming Bandwidth quota after
only 4 hours.

That gadget -- judging from the bot's logs it's
http://www.razorcam.com/seconds_spent_by_everyone_reading.xml --
causes an OnBlipSubmitted every second.

This means that every second the entire context is being pushed into
the robot.

I tried looking at the participant list in the wave through the normal
wave client, but the list is huge (1000+), and there is no way to
scroll through it, so that I can't find my bot to remove it from
there. (Ok, fixed that, got a computer with a bigger screen and used
the ctrl- to make the page smaller)

I've seen a removeSelf or something similar in the API which allows a
bot to remove itself from a wave, but at that time, about a month ago,
it was not implemented.

Maybe there should be a place where we can submit robots/gadgets that
should be blacklisted due to unfair behaviour, and maybe the docs
should mention the side effects of implementing a highly active
extension.

Or maybe an extra element in the capabilities.xml file which defines
which waves, or submissions from certain robots or gadgets should be
excluded from being processed.

I truely doubt that razorcam.com had bad intentions, so I'm not angry
at all.

[Stephen George]

Daniel, I agree that it's difficult to remove a robot from a wave of
over 200 participants.

I like to use bounc...@appspot.com in that situation to avoid
having to sift through the list. Glad that you were able to resolve
it and save your exhausted bot.

[Olreich]

It doesn't seem like that gadget should be causing an onblipsubmitted
every second...

On Dec 1, 9:41 pm, Stephen George <sfgeo...@gmail.com> wrote:
> Daniel, I agree that it's difficult to remove a robot from a wave of
> over 200 participants.
>

> I like to use bouncy-w...@appspot.com in that situation to avoid

[cmdskp]

I guess if that gadget is storing the total seconds in it's wave
State, this would cause a BLIP_SUBMITTED every state update - every
second.

Daniel, I've seen my logs get filled up with repeated
DOCUMENT_SUBMIT's (my main robot only listens to it and SELF_ADDED)
from certain popular gadgets.

I really would like to have my robot evict itself at that point using
the above API method. I tried adding code to call Bouncy in to assist
my robot in escape :D Sadly, it came to the wave at my robot's call,
but didn't respond to the robot's cry of distress!

The other (disturbing choice) would be to get the robot to remove the
gadget from the blip submitted - which would amount to unethical robot
blip vandelism.

Currently, I have code prepared to use in the future, so my robot
detects over-frequent waves and stop responding to them for a time.
However, this just lowers my out-going Quota, not the in-coming which
is the problem.

[Olreich]

I think I figured out the issue. For every person, the blip submitted
event is happening, and the Robot is receiving many more than 1
request per second. It's receiving however many people there are
watching the wave requests. The problem is that there is no way for
the gadget itself to update independently of the users, and thus, each
user must submit an update request for themselves, rather than the
gadget updating every second with a list of users who have the gadget
running.

There is also no ability for the gadget to tell if a user stops
looking at them, compounding the issues stated above. The only way for
a gadget such as this to operate is to inundate the users (and Robots)
with requests (2.6k requests with 2 people over the course of a few
minutes). This is just unacceptable with the quotas for Robots set as
they are. The massive amount of data being passed around used 2% of my
"quota" in just 20 minutes. Assuming much larger scales, this sort of
architecture would never hold up. Gadgets cannot be mini-applications
unless they have the ability to reduce the load to something more in
the order of 1 request per second maximum. This reduces the bot's
load, but only by half, and it needs to be mitigated more. This can be
done via creating a new event for gadgets, so that Robots can ignore
them if there's serious issues with bandwidth usage.

Oftentimes Robots will not care about gadgets anyways, so having an
opt-in solution would be quite manageable. This is especially useful
for Robots that are solely gadget-extenders, meant to increase the
productivity and awesomeness of a gadget, in that they can ignore all
users blip-submitting and just care about when the gadget updates the
wave's state.

There are probably better solutions too, but there is a preliminary
one for consideration that hopefully gets the idea across.

[cmdskp]

I agree, a gadget open in all viewers' browsers can submit a state
change on a short Javascript timer event leading to a horrendous
amount of potential events.

Even worse if you consider malicious possibilities, like having a
robot insert one of these small gadgets in every blip in a wave. Upon
one user opening that one wave to read it, all robots will receive an
extended spike in BLIP_SUBMITTED, even worse if said user(s) leave the
wave open in a separate tab for days on end.

Pamela has said on the API issues that she believes that the new API's
GADGET_STATE_CHANGED will still cause a BLIP_SUBMITTED - so sadly,
it's not separating it from triggering other non-gadget change events.

I can understand that if they stopped the BLIP_SUBMITTED event on a
GADGET_STATE_CHANGED it would break most current robots checking for a
gadget change through BLIP_SUBMITTED.

This issue that requests a new TEXT_STATE_CHANGED event might be worth
starring so that robots can get a separate event from gadgets for text
only changes. I noticed they've added one for ANNOTATED_TEXT_CHANGED
in the new API, so this would complete the set and allow any robot to
process all events separately.

http://code.google.com/p/google-wave-resources/issues/detail?id=552&can=4&colspec=Stars%20ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary%20Internal

For those curious about the new API, more info here:

http://code.google.com/apis/wave/extensions/robots/protocol.html

[pamela (Google Employee)]

I think we will also need to re-think the Gadgets API, in terms of
reducing the noise they cause. We've discussed things like not letting
gadgets submit state until an object is clicked (requiring action from
the user, basically), and having gadgets change multiple types of
state, or indicate the importance of a particular state change. So,
yes, robots events changes could be a part of this, but so are gadgets
state changes.

It's a useful discussion; thanks for bringing it up.

- pamela

> --
>
> You received this message because you are subscribed to the Google Groups "Google Wave API" group.
> To post to this group, send email to google-...@googlegroups.com.
> To unsubscribe from this group, send email to google-wave-a...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-wave-api?hl=en.
>
>
>

[Daniel Faust]

> I tried adding code to call Bouncy in to assist my robot in escape :D
> Sadly, it came to the wave at my robot's call, but didn't respond
> to the robot's cry of distress!

Sounds like a good idea. If i'm guessing correctly, bouncy needs to be
added to the wave and then receive configuration messages via messages
posted to the wave, right? In that case, posting that extra message to
configure bouncy to drop the bot out of the wave seems like an
expensive operation.

Imagine the situation if bouncy could be added to a wave, and then,
via a cheap and direct urlopen() call from the bot to bouncy, be
configured behind the scenes to remove the bot from the wave. Could
that be possible? The only issue I see here is that that urlopen call
won't hand over bouncy any context which it will probably need to
remove the bot from the wave. On the other hand, bouncy could queue up
the request removal and execute it during the next incoming message
coming from the problematic wave.

one issue is that if bouncy is also receiving all that incoming data,
then it might be the case that it also hits the quota limit, which
might be the reason why bouncy didn't respond to your call. So maybe
having google remove the incoming bandwith quota limit on bouncy would
be an good thing to do. Or, if that won't happen, built a network of
bouncies, which take care of each other so that no more than one is in
a given wave, and where, if one dies, others could jump in. or the
affected bot would first query the system to see which bouncy bot is
still up and running.

There would also be the need of taking protective measures against
direct urlopen() configuration requests from sources which might have
bad intentions.

Probably doing something with the capabilities.xml would be a better
option. Or add a dynamic blacklist.xml file.

----------

The issue with gadgets requiring user interaction before being allowed
to submit data seems to be a good thing to desire. Because, as it
looks now, it's only necessary for someone to take a look at a wave,
and, as soon as a gadget gets loaded and submits data, the user gets
inserted as a participant. At least it looks that way, since some
waves have a large amount of participants whose inclusion couldn't be
explained by other means.

[antimatter15]

I think I recall that Bouncy doesn't respond to posts made by robots
(It can be added by robots though).

I would like to think the solution would be to have blip content regex
matching in compatibilities.xml

[cmdskp]

Yeah, I think I found that too, antimatter15 - since Bouncy worked for
my human account at the time I tried it on the same wave I was testing
my robot on.

The blip content regex solution won't help robots that don't remove
the triggering text content: e.g. text management robots, notifying,
etc.

But I have thought of another idea, have a whitelist of gadgets that
the robot specifies it wants to listen to in the capabilities.xml.

Are there any situations when a robot would want to listen to unknown
gadgets? The white-list (rather than black-list) would then allow a
similar method to the regex content filtering on what gadgets get
their events sent to your robot.

This could either be a list of gadget URL's or just a single regex for
the gadget URL's to match against before sending the gadget events to
the robot.

[pamela (Google Employee)]

I think we should be able to let robots in the new API filter the
GADGET_STATE_CHANGED event based on the URL of the gadget.
That would be the "whitelist" that you're looking for.

- pamela