security issue (strange behavior)

[@Mlaynes]

my page : http://igguestbook.appspot.com/

( a guestbook with some of geolocation and OAuth2.0 Authrization & Authentication )

over one month I have "attacks" of robots or hackers who type sexual ads on my guestbook I'm trying to control ...

I have restricted the submits on my page from the front-side (javascript) .. and from  the server side (GAE-Java backend) .. but the ads are still trying to write on my page .. and now, save ads "empty" ..

. and now my page sometimes throws: Error: Server Error

sometimes by the attacks, my page has exceeded  "Datastore Read Operation" quota .. but I see that now hasn't exceeded .. But still throw : "Error : Server Error"

any idea ?

best regards

@Mlaynes

http://mlaynessanchez.blogspot.com

 

 

 

 

[@Mlaynes]

Hi friend.

I see the problem still persists .. although my website has failed all day, dashboard graphs has reported increasing "Front End instance hours" quota... really, I believe this is a serious problem on GAE security

last 24h Chart

last 30d Graph :

at least has been locked and have not continued the recording of  empty ads.. of course, I have already cleaned my computer from viruses and spybots

any solution ?

 @Mlaynes

 http://mlaynessanchez.blogspot.com

 

 

 

[@Mlaynes]

forgot to say .. now, in my graphs you can see few empty recordings, but that's because many of them I've deleted .. and during the attack was averaging 100-200 sex ads per day on average, which of course also I've erased...

:/(

help me please !

@Mlaynes

[Per]

The easiest way to battle unsophisticated spam is to add a invisible honeypot field to your form. Ignore any posts for which the robot filled the field. http://stackoverflow.com/questions/8873961/how-do-i-add-honey-pot-fields-to-my-forms

Other than that, you will have to live with bots crawling your site. I wouldn't say that 100 comments per day is a lot. :)

[@Mlaynes]

Hi Per..

Thanks so much for reply.. how you indicate, it's really important consider honeypot techniques for forms (in fact, I will do it), but i don't understand why now  my page throw Error Server.. perhaps GAE has blocked my site unannounced ? ..this is probably ..but in a business scene, this one would be very bad

@Mlaynes

[@Mlaynes]

[Jeff Schnitzer]

If your page is showing a 500 error, you should see the cause of it in
the application logs.

Jeff

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengi...@googlegroups.com.
> To post to this group, send email to google-a...@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[@Mlaynes]

Hi Jeff

Thanks for reply too.. 

I had not really checked the log because the site has always worked very well .. Now I created a new version (my source works OK), it has loaded to GAE platform .. and it's same, it throw : Error Server (yes, it indicate 500: Error Server in the tab

my logs are :

as I say all work fine with my source and local test, really is throwing error where there is none..

 : (

@Mlaynes 

[Jeff Schnitzer]

That line number is the line number of the java file generated from your JSP. You can either try to find the compiled JSP, or you can just take start debugging the old-fashioned way with print statements.

Either way you should have more than enough direction to figure this out on your own.

Jeff

[@Mlaynes]

Ok Jeff..

Although the code this sample is not complicated, really it was  difficult to correct JSP file .. I was able locate the temporary file, but not clearly showed me the error .. neither old-fashioned way helped me .. finally cutting portions of the code, show to me the error on site model  it was what caused the problem (..by my corrections)

:/)

http://igguestbook.appspot.com/

 

thanks so much

@Mlaynes

http://mlaynessanchez.blogspot.com

[@Mlaynes]

Hi friends!

I see that empty ads recording continues .. but, I had restricted their visualizations. This is good moment to put into practice honeyput techniques in the fields of my forms, or other ways..

best to all..

@Mlaynes

 

 

[@Mlaynes]

hi to all again..

How you can see, the "attacks" of robots continues.. the result is that they are consuming my free quota on GAE service each day ....and my page will be broken almost all day

; (

 for lack of time,  I have not implemented honeyput on fields of my form yet, but I hope will do it soon to measure its effectiveness. Currently, I 've placed a restrictions on the fields of form  but it's insufficient , but at least the "empty ads" aren't show (of course, empty ads are my restriccions, other way, almost all will be sexual ads)

what surprises me is the technic that uses robots to save data even with the restrictions of Javascript and server side that I have placed (seeking to limit empty recordings, this barrier they are jumping smoothly)

well.. will continue my tests

@Mlaynes

[Jeff Schnitzer]

You have a web form online and spammers are filling it with spam? That's what happens when you put unauthenticated forms online. Put FB or Persona auth on it.

Jeff

--

[timh]

I have been meaning to post a similiar response.

I wouldn't expect anything but excessive abuse of an open form.

T

[Vinny P]

On Monday, April 15, 2013 11:36:49 PM UTC-5, Jeff Schnitzer wrote:

You have a web form online and spammers are filling it with spam? That's what happens when you put unauthenticated forms online. Put FB or Persona auth on it.

+1. Put any sort of authentication on it. You can easily activate Google Accounts sign in with a few lines of code: https://developers.google.com/appengine/docs/java/users/overview 

Also, I notice that your "Datastore Reads" is very high, but your "Datastore Writes" is actually fairly low ( https://groups.google.com/d/msg/google-appengine/O-juv811wok/7WUNMsByfUEJ ). You'd be able to keep your site up more if you copied your datastore data to memcache, and then when a request comes in, try to pull data from memcache before querying the datastore.

-----------------

-Vinny P

Technology & Media Advisor

Chicago, IL

@GOV on AppDotNet: https://alpha.app.net/gov

[@Mlaynes]

Hi friends..

interesting options...

1) put FB or Google+ OAuth on it.. yes, a good option, but I will maintain open textarea field some time to do some personal testings about this issue (the fields hidden are now recording  empty ads..)

2) yes Vinny, I had noticed that the "Datastore Read operation" quota it was much larger than "write operations".. your solution is very interesting (using memcache), of course I go to implement it

[@Mlaynes]

Ok.. these are some of solutions implemented about the issue:

1. eliminate the ugly error page thrown by GAE .. and it was easy, setting correctly our Guestbook.JSP file to show our personal errorPage (of course, the design can be better)

<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<%@ page errorPage="/errorPage.jsp" isErrorPage="true" %>

2.- to try eliminate or diminish"Datastore Read Operations", we have implemented using "memcache"..but as you will see, not happens something better..

to implement "memcache" I use fist "key-only" thecnic, because according to the manual has some advantages and because my uestbook is a grid (created with table tag)

A keys-only query returns just the keys of the result entities instead of the entities themselves, at lower latency and cost than retrieving entire entities:

Query q = pm.newQuery("select id from " + Person.class.getName());
List<String> ids = (List<String>) q.execute();

of course, then with each "key", it was worked on memcache service

but, now I see that seems any query operation on datastore consumes "Datastore Read Operation", even key-only operations.. so then, is there some best technic to work with memcache? is it possible to do query on memcache with GAE-Java?

can anybody tell me something about this ?

Note : I've not corrected yet recording of "empty ads" to keep the spam attack and use it to improve my techniques GAE-Java development, and of course, we will continue our testings

best to all

@Mlaynes

[Jeff Schnitzer]

The answers to your questions are in here:

https://developers.google.com/appengine/docs/

As someone that has spent significant time outside my own country, I appreciate that this presents some linguistic challenges.  However, a few hours or days spent reading the documentation will be more effective than a lifetime of asking questions here.

Jeff

[@Mlaynes]

actually estimated jeff.. this was one of my first GAE project that I built, and now with using specialized "javascript" frameworks and using MVC architecture (ExtJS4, Backbone, others), I have not really watched problems like with this simple application unsecure, but I find it interesting the observed effects, to take them into account in my other projects ..and of course overcome these, because in the GAE world all consumptions and costs, must be controlled

some free samples

http://goo.gl/wVU6O a first integration with ExtJS & OAuth2.0 protocol

http://goo.gl/SLu4G  working with spreadsheets

http://goo.gl/N6W4p GAE in action project

Comments like this should also be considered :

http://www.carlosble.com/2010/11/goodbye-google-app-engine-gae/

and although I love the GAE philosophy .. I see betting a lot for startups that for professional business developments .. and forums like this, are the ideal place to develop, share and overcome these challenges..

so any help is welcome..

best regards

@Mlaynes

[@Mlaynes]

Ok.. after several tests, it appears that our sample is stabilized now, although it is an unsecured application :

of course we will wait one more day to see if this is really a solution, but the graphics are optimistic about the end of a day of quotas, and even with the ongoing spam attack how you can see..

what was the proposed solution?

by front-side:

a single layer of protection (enabling / disabling our textarea according to a checkbox)

by server side:

preventing recordings of empty ads or other non-consistent (similar when working with trusted endpoints )

and of course, using memcache technics

we will detail more our code & solution after observing some time  the results