[google-appengine] Google Apps and BigTable

[Madame Or]

Madam, Sir,


I am aware that this Group is meant for Programmers but ultimately it
is people like me that use these products. May be you could open a
group for users of the integration of Google App Engine in Google
Apps.

My request is the following:

Can people that use the Google Apps Premier Edition use BigTable for
their own use?

Would it be possible to allow the forms in Zoho Creator to access
BigTable.

Please be aware that this is central to the objectives of my
enterprise.

Given the high sensitivity of my data it is very important that no one
except of course the individuals I would allow and the software I
would enable to access the data stored in that BigTable. Hence I would
like to know also if it is possible to prevent anyone, including
Google, to have access to my data stored in BigTable?


I am, Sir, yours sincerely.


Keren Or Shalom

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Geoffrey Spear]

On May 21, 4:20 am, Madame Or <v07768198...@gmail.com> wrote:
> Given the high sensitivity of my data it is very important that no one
> except of course the individuals I would allow and the software I
> would enable to access the data stored in that BigTable. Hence I would
> like to know also if it is possible to prevent anyone, including
> Google, to have access to my data stored in BigTable?

There's no way to prevent Google from accessing anything stored on
their servers. It's unlikely they'd do so without a good reason, but
if you want to ensure security, use strong encryption.

[hawkett]

Hi Geoffrey,

I'm sure you didn't mean your response to come across this way - but
it reads - 'encrypt your data, or expect google to look at it for
whatever reason they see fit'. As this thread alone shows, data
security is of huge importance to many people using or considering the
use of app engine.

Is this approach to data security also your perception of app engine
for business? http://googlecode.blogspot.com/2010/05/announcing-google-app-engine-for.html

Sorry for the semi-rant, but it's hard work convincing folks to put
their data in the cloud, and every authoritative (you are listed as an
API guru) statement that erodes their perception of the cloud vendor's
security practices makes that job harder. I guess this highlights
again the need for unambiguous statements on this - probably in the
form of an SLA. I recommend that anyone who has similar concerns star
this issue - http://code.google.com/p/googleappengine/issues/detail?id=501
- raised nearly 2 years ago. Ironic that the issue id is '501' - HTTP
speak for 'Not Implemented'. Some kudos though for starting that
process as of a couple of days ago - http://code.google.com/appengine/business/sla.html,
despite the lack of data privacy clauses. As noted in the issue, the
number one concern is data privacy, not uptime.

Colin

[Keren Or Shalom]

I don't believe it would be responsible for any business to count on the goodwill of a provider no matter how good his reputation to preserve the security of strategic data. It does not matter what might be the promise of Google I think the only way is the way that Geoffrey propose.

--
V07768198309

[hawkett]

I don't necessarily disagree with the approach - depending on your
organisation, it will probably provide the most reliable data
security.

All solutions carry a risk of a data breach, however, and the
encryption solution is no different. One of the arguments for cloud
data security is that organisations generally over-estimate their own
ability to secure their data, and that cloud data is actually more
secure - precisely because the cloud vendor has greater capability
(knowledge, processes, people, infrastructure) to provide that
security. From this perspective it may be *less* responsible to do
data security in-house.

So my point is that when a someone asks 'What is the data security
situation with app engine' - it would be nice to do better than 'you
better encrypt it'. Encryption might be the right answer for your
organisation - but some information from google around auditing
processes, employee access policies, notification when your data was
accessed, the reasons why your data might be accessed etc. would
probably lead most businesses to opt for a solution that didn't
involve them taking on the security burden themselves. Most
businesses would say 'ok, that sounds good - I understand breaches can
happen, but it sounds like you guys are pretty organised and take this
stuff seriously'.

I'm sure google has a great many policies and procedures in place to
secure app engine data, but when the presented options are 'encrypt
it' or 'we'll look at it if we want', well, its a much harder sell.

On May 22, 8:44 pm, Keren Or Shalom <v07768198...@gmail.com> wrote:
> I don't believe it would be responsible for any business to count on the
> goodwill of a provider no matter how good his reputation to preserve the
> security of strategic data. It does not matter what might be the promise of
> Google I think the only way is the way that Geoffrey propose.
>

> On 22 May 2010 21:07, hawkett <hawk...@gmail.com> wrote:
>
>
>
>
>
> > Hi Geoffrey,
>
> > I'm sure you didn't mean your response to come across this way - but
> > it reads - 'encrypt your data, or expect google to look at it for
> > whatever reason they see fit'. As this thread alone shows, data
> > security is of huge importance to many people using or considering the
> > use of app engine.
>
> > Is this approach to data security also your perception of app engine
> > for business?

> >http://googlecode.blogspot.com/2010/05/announcing-google-app-engine-f...

>
> > Sorry for the semi-rant, but it's hard work convincing folks to put
> > their data in the cloud, and every authoritative (you are listed as an
> > API guru) statement that erodes their perception of the cloud vendor's
> > security practices makes that job harder. I guess this highlights
> > again the need for unambiguous statements on this - probably in the
> > form of an SLA. I recommend that anyone who has similar concerns star

> > this issue -http://code.google.com/p/googleappengine/issues/detail?id=501

> > - raised nearly 2 years ago. Ironic that the issue id is '501' - HTTP
> > speak for 'Not Implemented'.  Some kudos though for starting that
> > process as of a couple of days ago -
> >http://code.google.com/appengine/business/sla.html,
> > despite the lack of data privacy clauses. As noted in the issue, the
> > number one concern is data privacy, not uptime.
>
> > Colin
>
> > On May 21, 2:08 pm, Geoffrey Spear <geoffsp...@gmail.com> wrote:
> > > On May 21, 4:20 am, Madame Or <v07768198...@gmail.com> wrote:
>
> > > > Given the high sensitivity of my data it is very important that no one
> > > > except of course the individuals I would allow and the software I
> > > > would enable to access the data stored in that BigTable. Hence I would
> > > > like to know also if it is possible to prevent anyone, including
> > > > Google, to have access to my data stored in BigTable?
>
> > > There's no way to prevent Google from accessing anything stored on
> > > their servers.  It's unlikely they'd do so without a good reason, but
> > > if you want to ensure security, use strong encryption.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > > To post to this group, send email to google-a...@googlegroups.com.
> > > To unsubscribe from this group, send email to

> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .

> > > For more options, visit this group athttp://
> > groups.google.com/group/google-appengine?hl=en.
>

> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .

[Keren Or Shalom]

I believe that the cloud is the solution and it does not hurt to encrypt.

My requirement is very stringent because if I succeed my system would become systemic.

[hawkett]

Encryption might be the right solution for your application, but it
does carry some significant limitations on your ability to use app
engine and increases your app's complexity significantly.

If you intend to encrypt your data, you will need to consider the
following issues (and probably more besides)

1. Encrypted data can be very difficult to query. Consider the query
"SELECT * FROM Document WHERE author = 'Joe' AND tag = 'Science'".
Both the author and tag fields cannot be encrypted, unless you intend
to also encrypt the query values. If you intend to encrypt these
fields as well, then things like ordering your data correctly become
very difficult. Debugging your queries won't be much fun either. Some
queries (such as those that match the beginning of a string) will not
be possible.

2. You need to decide whether you intend to encrypt the data on app
engine, or in the client (probably the browser). If you intend to do
it on app engine, then the means of encryption and decryption need to
form part of your uploaded application, and you need to consider how
you will protect the *means* by which data is encrypted and
decrypted. In this situation, the data will spend some time on app
engine in an unencrypted state. If you intend to encrypt in the
browser, then you seriously limit the useful work you can do with the
data on app engine.

3. If you intend to use the task queue, you may need to encrypt the
data in your tasks as well.

4. You cannot encrypt your application code - I'm not sure if google
does this for you, but I suspect not.

5. Structured data is more difficult to maintain if it is encrypted.
Consider the following database model

class Person(db.Model):
name : StringProperty()
address : StringProperty()
email : StringProperty()
....

To truly encrypt all of your data on app engine you would need to
encrypt each of these fields separately, or combine them into a single
encrypted field, which will make querying on them almost impossible.

6. Application logs are probably not encrypted (I doubt they are, but
not sure), and you need to be careful what data is exposed in your
logs.

7. Using the admin console to look at your data will be next to
useless. Your next best option would probably be an implementation
based on either the custom admin pages or the remote API.

8. Encryption reduces your application performance.

So when you say it does not hurt to encrypt, that may be true for your
application, but encrypting data in the data store presents
significant barriers for most applications, and can seriously limit
the amount of value you can get out of app engine. For most
applications it makes a lot of sense to understand the security around
the datastore itself, and see encryption as last resort.

If all you need is a cloud based data storage solution where you dump
encrypted blobs of data, then you may get better mileage using Google
storage for developers - http://code.google.com/apis/storage/

I'm trying to highlight that data encryption on app engine is not
actually a very simple solution to data security for most classes of
application. Given this is the case, its a pity the only other option
is to assume it is not secure - knowing what security measures google
has in place would be a real help for the majority of classes of
application.

On May 22, 11:01 pm, Keren Or Shalom <v07768198...@gmail.com> wrote:
> I believe that the cloud is the solution and it does not hurt to encrypt.
>
> My requirement is very stringent because if I succeed my system would become
> systemic.
>

> > > > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

> > e...@googlegroups.com>
> > > > .
> > > > > For more options, visit this group athttp://
> > > > groups.google.com/group/google-appengine?hl=en.
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine" group.
> > > > To post to this group, send email to google-a...@googlegroups.com
> > .
> > > > To unsubscribe from this group, send email to

> > > > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

[Keren Or Shalom]

As I read your answer I may be found a solution and you might tell me what it is worth.

We could for example encrypt the account number of a customer and not the transaction or the value of the transaction.

This way I could send a query on the encrypted value of the customer account which wouldn't increase in a significant way the time of treatment? What do you say?

May be you need to know more of why I need security and what the system is meant to do.

[Robert Kluin]

Keren,

   I think it is important to consider the data you are dealing with and what it means to make it secure.  We do something like what you mention.  If you can secure the account number or name or whatever perhaps it is sufficient. 

  One of our apps is effectively a front-end to an accounting system.  The details of transactions on our app are completely worthless to someone without also having access to info in the accounting system.  The backend system runs on secure client-site servers and talks to the GAE app through an interface. This means if someone had all the GAE data it would be almost worthless to them.

Robert

[Keren Or Shalom]

Seems that we are closing on the way to implement the idea.

Although your model seems to fit my needs my system is more like a database of transaction on a bank account.

One of the requirement being to make secure transactions through means of payment and electronic means of communications

do you have an idea about implementing that. 

Anyone got an idea of the price of a large scale implementation?

[Robert Kluin]

Our system is something like a checkbook. Checks are effectively
written from it, then passed off to be paid. We do not actually make
electronic payments from it. We are typically only concerned with
hiding 'personally identifiable' information; if we do that, we are
OK.

Robert





On Sun, May 23, 2010 at 12:15 PM, Keren Or Shalom

[Siva P Thumma]

I think the cloud is meant for open social, blogs, etc or more
straight forward, it is for the ones who wish to show off.
People coined an opinion saying " No one who has to ensure security,
would probably never use appengine or any cloud.. may be simpleDB of
amazon".

People who are considered of security, may have to wait till the
AppEngine folks introduce their SQL support. probably, that would have
more security.
and then, Domain specific access scenario in google apps would
probably help too.

But, I hope to see this discussion continue till it resolves in a
fruitful end-result. ( Something that is reliable, authorized
information. )

[Jawaad Mahmood]

Dude, you clearly don't appreciate the cloud for what it is. It has
nothing to do with showing off, and everything to do with keeping a
company working on things it is good at. It's about efficiency, and
automating a huge cost center in a company.

I know a cosmetics company here in Tokyo. They have an IT team. They
have an in-house server farm, and most of it is just for
advertisements / campaigns, etc... It is a huge expense (Server
manager + programmers + line from NTT + etc...). Most of all, it is
entirely outside of the company's expertise. The president/ceo is a
wonderful hard-working person, but she can't tell an iPhone from a car
phone. She certainly couldn't tell you whether the Sun or HP server
she bought was any good; it was just what the sales people told her.

Cloud to the rescue. She doesn't have to waste time with computer
salespeople. IT people move to jobs where they can do interesting
stuff instead of making sure a hard disk problem isn't causing a web
outage for a heavily flash-based site. Moreover, instead of spending
corporate mindshare on IT, Amazon gets a check and handles almost all
of the headaches (apparently even backups). It's a lot better than
having a full time CTO and a team of server engineers / web
programmers.

The idea that some form of SQL is somehow more secure than any other
form of database interaction is frankly ridiculous. Feel free to look
up the sordid history of addslashes and mysql_real_escape_string on
PHP if you don't believe me.

Storing someone's unencrypted important personal information on a
server you don't own is insane, but that's what public key encryption
and intermediary servers are for. It's a lot easier to maintain such
minor infrastructure as opposed to the difficulty in having a full
network to withstand 1000-2000 simultaneous connections (or more).
(If the encrypted part of your website is getting those many
connections, you probably can afford to deal with the problem in a
more traditional way.)

The cloud kicks ass when you need to do something quickly. It can be
made to kick ass with minor investments if you want to do something
securely.

--
Sincerely yours,

Jawaad Mahmood
http://www.jawaadmahmood.com
080-4204-7198

[Keren Or Shalom]

Dear Group Members:


I thought it over again. I found out that in effect the cloud does not today meet my requirement for security.

I definitively think that this is the place to discuss that project.

I think I have to tell this forum exactly what is my project and that will give you a better grasp on what is required.

First I don't believe that my  System will have to operate before mid September: it is a short term for common men it is a very long-term for that kind of technology.

My project is a very simple one from a technological point of view but it is very involved in the field of macro economy, money emission and game theory.

I have no doubt that each and everyone of you have the necessary mathematical knowledge necessary to understand those economic notion but very few of you are in a position to fully adhere to that system: as John Maynard Keynes stated famously in his General Theory of Employment Credit and Money:  "The ideas which are here expressed so laboriously are extremely simple and should be obvious. The difficulty lies, not in the new ideas, but in escaping from the old ones, which ramify, for those brought up as most of us have been, into every corner of our minds."

I have proved that sooner or later this economy will go down the drain. The cause being the very existence of credit.

The condition of that economic collapse are almost met. 

There is no scientific way to date that collapse however it is my estimate, at the present moment, that this collapse would take place in mid September.

I have determined that no public entity, as we know them today, would be in a position to crate the condition of a functioning economy in the short term. 

I have determined that the solution would be to create a credit free currency.

I have determined that the only solution would be to create a money emission institute which is in effect a data base. 

The access to this data base must be a wide range of means of telecommunication, what is today called means of payment.

Access can be made through the internet, which would be the easiest but I think that it should accept more simple means like very simple cellular phone or rudimentary dedicated electronic devices. 

Although the management of the database can't be delegated the means of payment, access to the data base and the creation of markets where supply and demand for good and services would meet, should be the role of private entities that would sell that kind of services. I call these services anonymous socio economic networks. They have to be anonymous in order for people to be able to reclaim their privacy.

If one or several of you want to get more involved in the project and take some real responsibility which are beyond my technical knowledge I believe they will have to understand, that is easy, and accept, that is not possible for everyone, the scientific basis of my economic conclusion. I am ready to give some of the necessary knowledge that would be required to grasp that subject.

The subject he would have to understand would be:

1- Why the economy will necessarily collapse and under what conditions.

2 - Why people will adhere to that system and why they will trust that currency.

3 -  How we will implement the monetary policy with a credit free currency.

...

I won't insult you by describing the kind of security such a system would require. 

That system resembles much that of the management of checking account on which there would be no credit.

Although at the present moment I have very little economic resources, anyone who would get involved readily understands that economic resources won't be an issue within a very short period of time.

The person or persons that would care to get involved in that kind of project need to have a lot of creativity being able to delegate work and probably, but that is less important, some managerial and leadership capacity. He would have to be able to imagine how a system on the cloud can be meet secure which, for now, is a real challenge.

The reason I came to the conclusion that the cloud would be the solution are:

- Speed and ease of implementation.

- Low cost.

If it is my role to provide the necessary economic knowledge it is for you to propose the technological solutions which is beyond my capacity.


I am, members of this group, yours sincerely.

--
V07768198309

[hawkett]

A small issue if the economy goes down the drain on the scale you are
talking about is that an internet where everyone has the capability to
connect to a cloud system is going to be difficult to maintain. It's
not going to be easy for google to make money, for individuals to pay
their internet bills, for internet companies to continue to deliver
access - indeed you may find it difficult for yourself to make money
or pay for the developers you are proposing to hire. What value will
money have? How will google pay its employees to continue to deliver
app engine - ad revenues are going to plummet. I think if your system
is anticipating the total collapse of the economy, then you may find
it difficult to deliver your system to what remains.

When you say credit free currency, do you mean credit free society?
How do you plan to make it credit free - through legislation? What's
to stop me lending some currency I have in your system to someone - or
making a purchase on their behalf in exchange for a purchase by them
of something more valuable at a later date? That scenario is
essentially an interest bearing loan. The system and those that use
it exist in a symbiosis - you can't change one without addressing the
other - it's hard to see how you will create a credit free society
without addressing the psychology of the people within that economy -
people need to *want* to avoid credit, and when you haven't got much,
and others seem to have so much, credit is an insidiously attractive
proposition - it's easy to market and sell. You'll probably need to
address the gulf between rich and poor, and marketing of impossible,
unaffordable goals to remove credit from society - perhaps you feel
that economic collapse will do that. Does your plan deal effectively
with the reality of the flawed, selfish human being? I guess I'm
asking about your second point - is it based on a global societal
enlightenment occurring due to the economic collapse?

If you don't have much money to back this venture then a few points
stand out

1. The cloud is *much* cheaper than any custom hardware installation
you can come up with. Security isn't just data security - you need to
manage backups, uptime, support, maintenance, facilities, redundancy,
disaster recovery - long list of stuff that will cost you buckets
before you even get started.

2. You need to do some risk assessment. There are always risks, and
many of them - there is no perfectly secure system, and it is
important not to try and sell such a thing. It is just a matter of how
you rate each risk, and important that the stakeholders understand the
risk and agree to it - the last thing you want to have is a
conversation where someone says 'You told me it was secure!!!'. It
can't be secure unless it is truly inaccessible to everything and
everyone, and you can't build a system around that :). Risk assessment
is generally subjective - what is the chance of someone obtaining a
password they shouldn't? What is the chance of the cloud vendor
exposing data? What is the chance that your code contains bugs? What
is the chance of the cloud vendor losing your data entirely -
http://gigaom.com/2009/10/10/when-cloud-fails-t-mobile-microsoft-lose-sidekick-customer-data/?
The list goes on. If you build a system where there is perceived value
in subverting it, then it will probably be subverted. It is much
better to cope with it than try to prevent it (you can't prevent it).
What you need to prevent against is the possibility that any
particular problem or combination of problems results in total
failure.

Some key questions you need to ask -

'What is the worst that can possibly happen?'
'How likely is it?'
'What can I do to reduce the worst that can possibly happen, and the
chance that it will happen - i.e. how do I mitigate my risk?'
'Can I afford to implement mitigation strategy X, Y or Z?'
'Am I prepared to accept the consequences of the risk being realised?'
Usually it is easier to answer yes to this question when the rewards
(not necessarily financial) are higher.

You need to deliver a system that allows you to answer yes to the last
question, for every risk you identify. Sometimes you cannot reach the
point where the answer is 'yes' - the risk is not worth the reward.

<slight political rant (IMO)>
A good example is the recent financial crisis. The banking sector took
risks, understanding that the worst possible outcome (for them) was
not actually as bad as is being portrayed - they felt that they would
still not fail because they commanded enough political power to rely
on federal funds. So what seemed incredibly risky behaviour by the
banks, was, in fact, not all that risky - because they had shored up
their political power to mitigate their risk. Consequently, the worst
that could possibly happen with all the risks the banks took was that
they would be fine. And largely, they are fine. Good risk management
they would say - it wasn't all that risky. Not for them. Incredibly
risky for the taxpayer, but that risk wasn't their concern - at least
not from their perspective. You can be sure that it was a possibility
they were aware of. The nation(s), however, totally lost a handle on
their risk management - the banks should never have been allowed to
reach a position where they could mitigate their risk with federal
funds - essentially the risk was transferred to the tax payer. If they
were unable to do that, then they wouldn't have taken the risks in the
first place, because they could not have answered 'yes' to the last
question. It is difficult to see how the nation(s) could have avoided
this problem though - the stakeholders (the citizens) have very
limited means by which to limit the subversion of power from within
their political process - most of these powerful people within
government are not elected.
</slight political rant (IMO)>

You might also consider an approach to security like Google's - they
offer a platform so compelling that many customers will use the system
despite google making no promises on security or reliability - 'use at
your own risk'. Is your product so compelling that you can mitigate
some of your risk by transferring it to your customers? Google also
benefits here from significant goodwill and reputation, but that takes
time and evidence - we have seen in their reaction to the Chinese
hacking scandal that they take the compromise of customer data very
seriously.

My gut feel is that the security you want, you can't afford - so use
the cloud, encrypt the personal data you need to, and mitigate the
remaining risk by transferring it to your users. Just make them aware
of the risk they are taking - you might find they'll accept that risk
and use your system anyway. Where google basically says 'Trust us,
we're Google', you need to say 'Trust me, it's on Google'. Most of the
customers that would go for the first statement will go for the
second.

You say it would not be responsible to trust google - but this is what
you are asking of your customers - to trust you with the security of
their information. Trust is a huge factor in risk assessment and
security. Many banks trust third party organisations with customer
information, or to write software that secures customer information.
You're going to have to trust someone.

I think history and common sense tells us that our economy will
probably go down the chute at some point - our system is pretty
tightly strung, and doesn't cope very well with fairly minor
disturbances. It's utterly dependent upon confidence, and totally
global. Natural systems manage risk through redundancy and diversity -
failure doesn't bring the whole thing down. I doubt our brilliant idea
of having one global economy is going to have nature going - 'Hey, why
didn't I think of that?' - more like - 'Tried that, didn't work - you
should have asked. In the meantime, here's a small icelandic volcano
to deal with - get the idea?'. We are selfish idiots ruled by selfish
idiots, urging each other to become more so. Probably not the best
architecture we could come up with. Our society could learn something
from the way we build distributed software.

> On 24 May 2010 07:18, SivaTumma <sivatu...@gmail.com> wrote:
>
>
>
>
>
> > I think the cloud is meant for open social, blogs, etc or more
> > straight forward, it is for the ones who wish to show off.
> > People coined an opinion saying " No one who has to ensure security,
> > would probably never use appengine or any cloud.. may be simpleDB of
> > amazon".
>
> > People who are considered of security, may have to wait till the
> > AppEngine folks introduce their SQL support. probably, that would have
> > more security.
> > and then, Domain specific access scenario in google apps would
> > probably help too.
>
> > But, I hope to see this discussion continue till it resolves in a
> > fruitful end-result. ( Something that is reliable, authorized
> > information. )
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .

[Gopal Patel]

@hawkett last line was the perfect answer to @keren.

instead of making a central system on cloud , designing a software
that is very highly distributed using highest encryption standard is a
way to go.

on some level , people at http://www.joindiaspora.com/ are doing that
thing at social networking level. ( personally i dont think doing this
on top of other http or https protocol is good idea. even
though creating/defining protocol is hard work , it should be done
directly on top of tcp/ip level. )

that way, i own my own server at my own location with my own data
fully encrypted and accessible only to me. then i delegate
data through defined protocol to other person or system ( be it a
company , a bank , a friend , a community etc.. ) with different level
of data and security access. same way, other system,
users and bank etc will delegate their data to my server to similar
encryption channels.

distributing data this way will remove "scalability need" altogether.
and application complexity will reduce dramatically as they do not
have to take care of 1000's of connection or users. but it will arise
problem for myself to maintain my server , will force to learn a new
software etc and learn the system, may be your own private virtual
server is a new commodity of the future.

my two cents. :D

> is the chance of the cloud vendor losing your data entirely -http://gigaom.com/2009/10/10/when-cloud-fails-t-mobile-microsoft-lose...

> ...
>
> read more »

[Gopal Patel]

@keren,

i may not have economical expertise , but one thing is sure , as long
as this world is running from greed and passion of people ( imho, 2
holiest emotion ) , the economy is not going to down
for sure.

On May 24, 4:52 pm, gops <patelgo...@gmail.com> wrote:
> @hawkett  last line was the perfect answer to @keren.
>
> instead of making a central system on cloud , designing a software
> that is very highly distributed using highest encryption standard is a
> way to go.
>

> on some level , people athttp://www.joindiaspora.com/are doing that

[hawkett]

@gops this style of system is the holy grail of distributed systems,
however the model you have suggested perhaps suffers from lack of
redundancy. If only you have your data (or some of your data), and
only you have access to it, then failure at that point represents
total loss. The solution is a network of trust and redundant copies of
your data and multiple people with the capacity to access it. The
biggest problem we have with this solution is bandwidth and network
latency. Keeping copies of data in different locations is expensive
on both those fronts - especially if you want decent consistency, and
especially between your computer and my computer. Another problem is
that if publicly available data on your node is popular - then you
need massive resources to be able to meet that demand - as you noted,
a problem is the need for you to maintain your node. Perhaps the data
distribution you mention is intended to solve these problems.

For many classes of application, the cloud is actually the best we can
do at the moment with the state of technology. Companies like google,
amazon etc. *do* distribute data like this. However to solve the
bandwidth and latency issues that come with the need for good
consistency, they need to manage the clusters. I expect that one of
the major reasons google is pushing hard for better bandwidth and
lower latency for everyone is to expand the possibilities for this
type of architecture. It won't be too long before our web browser is
also a web server. I don't doubt that this forms a large part of the
vision for Chrome OS. Another huge help will be getting people
comfortable with eventual consistency.

One of the key future applications of the social web is the
application of networks of trust that allow us to distribute data to
trusted nodes - the best people to store redundant copies of your data
are the people you trust personally. It sounds like the guys on that
link you posted are really pushing the boundaries of this stuff.

Resistance is futile :)

On May 24, 12:52 pm, gops <patelgo...@gmail.com> wrote:
> @hawkett  last line was the perfect answer to @keren.
>
> instead of making a central system on cloud , designing a software
> that is very highly distributed using highest encryption standard is a
> way to go.
>

> on some level , people athttp://www.joindiaspora.com/are doing that

[Duong BaTien]

Hi:

This is an interesting topic. I am a retired entrepreneur, professional
software developer, professional economist, economic professor and
professional engineer. I am planning to use cloud technology to
democratize the AwakeningBudh or Innate Intelligence in living persons.

All emotions are conditioned driven by your view (Understanding) and
motive (Volitional Formation) which are driven by Binding Thoughts.
There is binding and detachment-born thought in a sense that once a
thought and its action is completed, there is No Hanging Effect known by
many Loving-Kindness Samaritan while the binding though will
continuously haunt one's conscience (which is a part of the Innate Budh)
until one's Innate Budh is awaken and do something about it. Binding
Thought is a slavery pattern (fears, phobias, innumerable unmanageable
desires of the grasper and the grasped, etc) while detachment-born
thought enables one see Thing As It Is, outside the conditioned box of
slavery bounded in the Name of Anything. Thought is conditioned. When I
call your name you can answer right away. When I ask if you know so and
so over 40 years ago, you may take some time to recall. If thought is
conditioned, what is the Source of Thought?

That is the Innate Budh (Intelligence) that we prove it is a Sine Qua
Non of Living, since without some form of Intelligence, there is No
Living. It is an Observable Truth or mathematical axiom. From that
Observable Truth like Newton Third Law of Action and Reaction, one can
discover a process according to natural laws (like gravity force)
leading the "Right Living" toward Happiness and Evolution where
"Dependent Nature" is a part of the natural laws governing all processes
from gross manifestations to minute level of a thought. That Dependent
Nature enables one see Thing As It Is where 1 + 1 > 2 (a win-win
Strategic Moment), symbolized in 2 Knights on the same horse in the
symbol of Knights Templar. Greed and Passion are transformed into
"Sharing Happiness - Mitigating Sufferings" called ComPassion (or Common
Vibration of Energy) of one's Inner Circle and circles of inner circles.

Via democratizing the AwakeningBudh, one can see the Inner Dignity of
Living and have the "Right Understanding and Right Thought" to be a part
of the Crowd Wisdom that no tyranny and extreme dark force of violence,
of "To me Through Me and Only Me" in the past can be repeated. Software
open source is a movement toward that direction.

Hope each one of us can be a soldier of Sharing Happiness - Mitigating
Sufferings that go beyond the symbol of Tao's Yin-Yang of Evolution then
Degeneration.

Duong BaTien
DBGROUPS and BudhNet

[Baz]

I'm selling spider legs and bat wings at half price...

[Keren Or Shalom]

Please make your remarks shorts and focussed:

I will answer to all I have received but break them down to something we can digest:

hawkett said

A small issue if the economy goes down the drain on the scale you are
talking about is that an internet where everyone has the capability to
connect to a cloud system is going to be difficult to maintain. It's
not going to be easy for google to make money, for individuals to pay
their internet bills, for internet companies to continue to deliver
access - indeed you may find it difficult for yourself to make money
or pay for the developers you are proposing to hire.  What value will
money have? How will google pay its employees to continue to deliver
app engine - ad revenues are going to plummet.  I think if your system
is anticipating the total collapse of the economy, then you may find
it difficult to deliver your system to what remains.

It all depends on how much my system can create demand: you are reacting just as if the previous system didn't existed and we stayed in a vacuum. My system is designed to feel the vacuum.

--
V07768198309

[Keren Or Shalom]

hawkett said:

When you say credit free currency, do you mean credit free society?
How do you plan to make it credit free - through legislation? What's
to stop me lending some currency I have in your system to someone - or
making a purchase on their behalf in exchange for a purchase by them
of something more valuable at a later date? That scenario is
essentially an interest bearing loan.  The system and those that use
it exist in a symbiosis - you can't change one without addressing the
other - it's hard to see how you will create a credit free society
without addressing the psychology of the people within that economy -
people need to *want* to avoid credit, and when you haven't got much,
and others seem to have so much, credit is an insidiously attractive
proposition - it's easy to market and sell. You'll probably need to
address the gulf between rich and poor, and marketing of impossible,
unaffordable goals to remove credit from society - perhaps you feel
that economic collapse will do that. Does your plan deal effectively
with the reality of the flawed, selfish human being? I guess I'm
asking about your second point - is it based on a global societal
enlightenment occurring due to the economic collapse?

I am an economist not a dreamer: a credit is a contract. The willingness to sign it is based on the belief that it will be enforced.

If there is no enforcement of such a contract I bet you $1 that no body will do enter that kind of contract. Moreover I don't have to explain how data mining can help spot a transaction that is not a regular economic transaction but a credit.

[Keren Or Shalom]

hawkett:

1. The cloud is *much* cheaper than any custom hardware installation
you can come up with. Security isn't just data security - you need to
manage backups, uptime, support, maintenance, facilities, redundancy,
disaster recovery - long list of stuff that will cost you buckets
before you even get started.

You are right but I am not more in a hurry than anyone else and after the crash the number of cheap idle resources available will be tremendous. I am a very patient man I have waited for that for 15 years I bet you $1 that the owners of these resources have more money than I but a lot less patience.

[Keren Or Shalom]

hawkett:

2. You need to do some risk assessment. There are always risks, and
many of them - there is no perfectly secure system, and it is
important not to try and sell such a thing. It is just a matter of how
you rate each risk, and important that the stakeholders understand the
risk and agree to it - the last thing you want to have is a
conversation where someone says 'You told me it was secure!!!'. It
can't be secure unless it is truly inaccessible to everything and
everyone, and you can't build a system around that :). Risk assessment
is generally subjective - what is the chance of someone obtaining a
password they shouldn't? What is the chance of the cloud vendor
exposing data? What is the chance that your code contains bugs? What
is the chance of the cloud vendor losing your data entirely -

http://gigaom.com/2009/10/10/when-cloud-fails-t-mobile-microsoft-lose-sidekick-customer-data/?

The list goes on. If you build a system where there is perceived value
in subverting it, then it will probably be subverted. It is much
better to cope with it than try to prevent it (you can't prevent it).
What you need to prevent against is the possibility that any
particular problem or combination of problems results in total
failure.

I am not worried if a few thousand of password or account were broken into we can insure that. 

The problem is a system wide breach like the provider exposing or destroying data.

This is the subject we are dealing with. I don't believe in fixing a bad solution I believe in designing a good system

then implementing it. If people use that system and it fails no one will be willing to listen to my excuses.

You are asking me questions you never asked your banks: why do you hold me to a higher standard than your banks.

[Keren Or Shalom]

I have started this work in 1994 a lot before the last crisis. 

Yes they solved that one for reason I won't explain here.

 If I worked for 15 years on that system it is because 

I knew that ultimately there would be no way to save the system.

There are mathematical limits to the extent to which such a crisis can be solved and

that no matter how much liquidity you put in. This is why, we economists, call it

a Liquidity Trap. The Fed is well aware of that just Google: " 'Liquidity Trap' 'Zero Lower Limit' Site:federalreserve.gov"

[Keren Or Shalom]

You might also consider an approach to security like Google's - they
offer a platform so compelling that many customers will use the system
despite google making no promises on security or reliability - 'use at
your own risk'. Is your product so compelling that you can mitigate
some of your risk by transferring it to your customers?  Google also
benefits here from significant goodwill and reputation, but that takes
time and evidence - we have seen in their reaction to the Chinese
hacking scandal that they take the compromise of customer data very
seriously.

My gut feel is that the security you want, you can't afford - so use
the cloud, encrypt the personal data you need to, and mitigate the
remaining risk by transferring it to your users. Just make them aware
of the risk they are taking - you might find they'll accept that risk
and use your system anyway. Where google basically says 'Trust us,
we're Google', you need to say 'Trust me, it's on Google'. Most of the
customers that would go for the first statement will go for the
second.

You say it would not be responsible to trust google - but this is what
you are asking of your customers - to trust you with the security of
their information.  Trust is a huge factor in risk assessment and
security. Many banks trust third party organisations with customer
information, or to write software that secures customer information.
You're going to have to trust someone.

What would you think if your bank did 'trust' someone with your banking account.

I can't dump the risk on what you call 'my customers' it is what they want to see taken care of.

If I can't design a system that will satisfy the kind of security I deem necessary I won't do it.

I criticize enough central bankers for taking a job they cant make, I am not going to do just that.

[Keren Or Shalom]

hawkett said:

I think history and common sense tells us that our economy will
probably go down the chute at some point - our system is pretty
tightly strung, and doesn't cope very well with fairly minor
disturbances.  It's utterly dependent upon confidence, and totally
global. Natural systems manage risk through redundancy and diversity -
failure doesn't bring the whole thing down. I doubt our brilliant idea
of having one global economy is going to have nature going - 'Hey, why
didn't I think of that?' - more like - 'Tried that, didn't work - you
should have asked. In the meantime, here's a small icelandic volcano
to deal with - get the idea?'. We are selfish idiots ruled by selfish
idiots, urging each other to become more so. Probably not the best
architecture we could come up with.  Our society could learn something
from the way we build distributed software.

So this is one in a millennium opportunity to have an impact on Society:

It is your responsibility to take it or not.

[Keren Or Shalom]

gop said

@hawkett  last line was the perfect answer to @keren.

instead of making a central system on cloud , designing a software
that is very highly distributed using highest encryption standard is a
way to go.

on some level , people at http://www.joindiaspora.com/ are doing that

thing at social networking level. ( personally i dont think doing this
on top of other http or https protocol is good idea. even
though creating/defining protocol is hard work , it should be done
directly on top of tcp/ip level. )

that way, i own my own server at my own location with my own data
fully encrypted and accessible only to me. then i delegate
data through defined protocol to other person or system ( be it a
company , a bank , a friend , a community etc.. ) with different level
of data and security access. same way, other system,
users and bank etc will delegate their data to my server to similar
encryption channels.

distributing data this way will remove "scalability need" altogether.
and application complexity will reduce dramatically as they do not
have to take care of 1000's of connection or users. but it will arise
problem for myself to maintain my server , will force to learn a new
software etc and learn the system, may be your own private virtual
server is a new commodity of the future.

my two cents. :D

Seems that we are already getting somewhere... I think we must continue along these lines.

[Keren Or Shalom]

gop said:

@keren,

i may not have economical expertise , but one thing is sure , as long
as this world is running from greed and passion of people ( imho, 2
holiest emotion ) , the economy is not going to down
for sure.

greed and passion have one mathematical limit: the 0% lower limit on short term interest.

then comes another emotion: the deep depression.

[Keren Or Shalom]

What is the full (market price) of spider legs and bat wings?

Water has no value in the city you live in. 

How much can you sell it in the desert to a man dying from thirst?

I am saying that if you sell him $100  a gallon he will think he made a deal

whereas a week before, when he was his in Manathan apartment
he would have told you that at $1 / gallon you were out of your mind.

My profession unlike yours is a question of timing. 

If you can anticipate if and when there will be a demand for a product 

you can wait years. The main difference between economy, science, politics or religion is that one:

in science you must prove, in politics or religion you must believe
in economics even a chimpanzee will understand it when there is a lack of supply of bananas.

--
V07768198309

[Ikai L (Google)]

Guys, I want to encourage you to stay on topic. 

The issue here is encryption for data store in App Engine. As several posters have pointed out, there are no easy solutions for this. A shift towards the cloud has all the implications of not being able to physically secure the data. Using a service such as App Engine, something we common describe as a platform-as-a-service, you have much less control over your data, though we will bear the responsibility of providing as much security as we can. There is probably a workable compromise somewhere involving where we store cryptographic keys and ACLs, but for the foreseeable future, you will have to accept that if you are running your software on Google's platform, then Google's platform will be able to access your data. If you build a rich client, you can encrypt the data on the client.

--
Ikai Lan 
Developer Relations, Google App Engine

Twitter: http://twitter.com/ikai

Delicious: http://delicious.com/ikailan

----------------

Google App Engine links:
Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

[Baz]

It seems the world economic revolution and the dissolution of central banks will have to wait for an updated sdk :(

[Keren Or Shalom]

Updating a sdk is easier and faster than updating the brain of a banker.

We won't chose when we will have to implement that is something which is mathematically impossible to forecast even though we know that the system is already unstable. It is impossible by definition to know when it will start to behave in a non linear way.

The consequence of not being ready would be tremendous: we had the occasion of tasting locally what would be the social consequences but if the 1929 episode is an indication the political and military consequences could be disastrous.

I have started to work on this project since 1994. Although it took me 6 years to start to envision a solution and I haven't been able yet to persuade people of the necessity to find a solution, even after the Great Recession, I am a patient optimist.

I bet you $1 we will be ready

--
V07768198309

[Siva P Thumma]

@all, Hmm, now comes the need to have a look at this thread often.
what this thread could yield, is kind of puzzle.
@Jawaad Mahmood, Man, I am a developer, always appreciate bifurcating
competency. "But yet, the cloud is yet not secure." I mean there is no
way we can believe "if that prat somewhere got 1000s of followers -
really !!" This all social seams a nonsense. I mean the technology is
awesome, I appreciate, but can I be sure that a comment made on the
cloud is of a specific person as it claims ?

The same way, can we believe that something done over the cloud is
really done by the one the cloud claims ??
( This is clearly separate from the google apps domain specific
access, though. "But yet.." !! )

[Keren Or Shalom]

@SivaTumma does that mean I am suspect of having an hidden agenda

or being manipulated?

--
V07768198309

[hawkett]

> I am an economist not a dreamer: a credit is a contract. The willingness to
> sign it is based on the belief that it will be enforced.
> If there is no enforcement of such a contract I bet you $1 that no body will
> do enter that kind of contract. Moreover I don't have to explain how data
> mining can help spot a transaction that is not a regular economic
> transaction but a credit.

Enforcement is most often something the lender does. If the loan
recipient doesn't pay up, then that is a bad debt, and the lender
analyses this risk when assessing the candidate. The people
repossessing houses are employed by the bank, not the government.
True, the law is there to support this - but that was the question -
do you plan to legislate against credit, or are you expecting its
demise to be a natural outcome of the economic collapse? In general if
you legislate against something people want, then it ends up on the
black market - e.g. enforced by thugs with baseball bats. The point is
you need stop people wanting to give and receive credit to get rid of
it. I'm not sure I understand the data mining point - perhaps its
possible for your system to stop credit within it, but credit
transactions will be undertaken outside your system while there is
supply and demand for it - an alternative economy.

> I am not worried if a few thousand of password or account were broken into
> we can insure that.
> The problem is a system wide breach like the provider exposing or destroying
> data.
> This is the subject we are dealing with. I don't believe in fixing a bad
> solution I believe in designing a good system
> then implementing it. If people use that system and it fails no one will be
> willing to listen to my excuses.

You're talking about the difference between a waterfall project
methodology and an iterative project methodology. Your 'design,
implement, finished' approach sounds attractive, but is usually
significantly less effective than an iterative approach when
developing software.

When you talk about preventing against a system wide breach, if you
take on the work yourself - then *you* need to be responsible for
preventing against total breach or total loss of data - the problem
doesn't go away if you do it yourself. This is a massive undertaking
- even if you had an army of low costs resources - a project can only
improve efficiency so much no matter how many resources you throw at
it. If you are talking about a global currency system, then it will be
a target for attack.

My point is that I doubt you can start from scratch and build a cloud
platform that has the scalability, security and reliability
characteristics you are after, and convince your customers that you
have done so. Building it yourself doesn't sound like a good risk
mitigation strategy to me. For the position you are in, it would seem
the cloud is the right choice, and the risks you can't mitigate you
have to be prepared to cope with - tell your customers what the risks
are - 'It's on Google, but your account number is encrypted'. No
venture is guaranteed to succeed.

> You are asking me questions you never asked your banks: why do you hold me
> to a higher standard than your banks.

I thought I only scratched the surface. The questions I ask are ones I
would expect a bank - actually not even a bank - to ask of
themselves. Banks do ask these questions - and more besides. Which
ones in particular do you feel were unnecessary for your project?

> On 25 May 2010 00:25, Keren Or Shalom <v07768198...@gmail.com> wrote:
>
>
>
> > Please make your remarks shorts and focussed:
>
> > I will answer to all I have received but break them down to something we
> > can digest:
>
> > hawkett said
>
> > A small issue if the economy goes down the drain on the scale you are
> > talking about is that an internet where everyone has the capability to
> > connect to a cloud system is going to be difficult to maintain. It's
> > not going to be easy for google to make money, for individuals to pay
> > their internet bills, for internet companies to continue to deliver
> > access - indeed you may find it difficult for yourself to make money
> > or pay for the developers you are proposing to hire.  What value will
> > money have? How will google pay its employees to continue to deliver
> > app engine - ad revenues are going to plummet.  I think if your system
> > is anticipating the total collapse of the economy, then you may find
> > it difficult to deliver your system to what remains.
>
> > It all depends on how much my system can create demand: you are reacting
> > just as if the previous system didn't existed and we stayed in a vacuum. My
> > system is designed to feel the vacuum.
>

> >> > on some level , people athttp://www.joindiaspora.com/aredoing that

[hawkett]

Hi Ikai,

I'd like to draw a distinction between (a) the platform being able to
see the data, (b) employees being able to see the data (c) the
policies by which employees look at data (sanctioned by google) (d)
ilegal data access (not sanctioned by google - either by an employee
or an external attack) (e) the protections against d at a high level

At the moment google seems to pull all these elements into the one
basket - and say 'we can see your data'. That statement doesn't
actually carry any useful information, except perhaps by implication -
'we're not saying much because you wouldn't like it if you knew.'

Take for example this FAQ statement for Google Apps -
http://www.google.com/support/a/bin/answer.py?hlrm=en&answer=106887 -
it is an exceptionally helpful statement. With App Engine, unless I
have missed it, there are no such statements - just 'we can see your
data'.

Additional security is one thing, clarity and transparency is
another. You don't need cryptographic keys and ACL's to achieve an
improvement in people's ability to *understand* the data security of
their app on app engine.

Cheers,

Colin

[Keren Or Shalom]

hawkett says:

Enforcement is most often something the lender does. If the loan
recipient doesn't pay up, then that is a bad debt, and the lender
analyses this risk when assessing the candidate. The people
repossessing houses are employed by the bank, not the government.

True, the law is there to support this - but that was the question -
do you plan to legislate against credit, or are you expecting its
demise to be a natural outcome of the economic collapse? In general if
you legislate against something people want, then it ends up on the
black market - e.g. enforced by thugs with baseball bats. The point is
you need stop people wanting to give and receive credit to get rid of
it. I'm not sure I understand the data mining point - perhaps its
possible for your system to stop credit within it, but credit
transactions will be undertaken outside your system while there is
supply and demand for it - an alternative economy.

The only reason they can do that is that they have a law that allows them to do so.

If no laws allows them to do so it is illegal and then they can face retribution.

When you use a system, let's say a Gmail account you agree to something

(that thing that no one read and which is called an EULA or whatever.)

When you agree to use that currency you agree that you won't use credit.

If you break the EULA then the service provider disconnect you from the system.

So there are two point:

If you make a loan and the 'bank' won't let you repossess the assets of your customers

you are in deep trouble.

If we find a transaction with no good and services to justify it we have to conclude it is a credit.

Even the guy with the baseball bat needs an economy: he will think twice before he uses it.

As far as credit outside whatever you want but we have scientific evidence which show
that whoever will chose another system will have to regret it at some point. 

But look it is a free market.

People can and do have email accounts that allow for SPAMing, 

they just can't do that with Gmail.

The difference being that once you chose to use the 'spamming account' you won't be allowed into 'GMail'.

Like everything in life or in economy it is a matter of free choice.

--
V07768198309

[Keren Or Shalom]

Colin

Be sure that people at Google are fully aware of the issue and anyway the Market will

take care of it: very few decider would give away the control of strategic data at any price

very few will do it but being so dumb it is probable that they will be out of business long before 

the first data leak.

On the other way any decider given that he has technological assurance that his data are

secure would gladly put his IT on the cloud.

Myself I decided that for efficiency and privacy reason I wouldn't collect private information

about my 'customers' I don't need to tell them I won't do evil they know I can't.

--
V07768198309

[Keren Or Shalom]

hawkett says:

You're talking about the difference between a waterfall project
methodology and an iterative project methodology.  Your 'design,
implement, finished' approach sounds attractive, but is usually
significantly less effective than an iterative approach when
developing software.

When you talk about preventing against a system wide breach, if you
take on the work yourself - then *you* need to be responsible for
preventing against total breach or total loss of data - the problem
doesn't go away if you do it yourself.  This is a massive undertaking
- even if you had an army of low costs resources - a project can only
improve efficiency so much no matter how many resources you throw at
it. If you are talking about a global currency system, then it will be
a target for attack.

My point is that I doubt you can start from scratch and build a cloud
platform that has the scalability, security and reliability
characteristics you are after, and convince your customers that you
have done so. Building it yourself doesn't sound like a good risk
mitigation strategy to me. For the position you are in, it would seem
the cloud is the right choice, and the risks you can't mitigate you
have to be prepared to cope with - tell your customers what the risks
are - 'It's on Google, but your account number is encrypted'. No
venture is guaranteed to succeed.

I agree 100% with you still it does not allow me not to do due diligence 

as they say in the financial market.

I am a risk taker but I never play roulette.