Google Cloud Storage: Policy did not reference this field (blank)

[Joseph Wegner]

I'm working on sending some files to GCS via a Node.js program, using the POST object method.  I'm getting a pretty odd error that seems to indicate that I haven't referenced a field in my policy, but then it doesn't tell me what field.  The exact error XML is:

<?xml version='1.0' encoding='UTF-8'?>

<Error>

    <Code>InvalidPolicyDocument</Code>

    <Message>The content of the form does not meet the conditions specified in the policy document.</Message>

    <Details>Policy did not reference these fields: </Details>

</Error>

I've checked and double-checked that I don't accidentally have some field in my POST request that has a blank name.  My POST data looks like this:

---------------------------043351533357053995

Content-Disposition: form-data; name="key"

Joseph Test

---------------------------043351533357053995

Content-Disposition: form-data; name="bucket"

publicjs

---------------------------043351533357053995

Content-Disposition: form-data; name="GoogleAccessId"

SECRET

---------------------------043351533357053995

Content-Disposition: form-data; name="acl"

public-read

---------------------------043351533357053995

Content-Disposition: form-data; name="policy"

eyJleHBpcmF0aW9uIjoiMjAxNS0wNi0xNlQxMToxMToxMVoiLCJjb25kaXRpb25zIjpbWyJzdGFydHMtd2l0aCIsIiRrZXkiLCIiXSxbImVxIiwiJGFjbCIsInB1YmxpYy1yZWFkIl0sWyJlcSIsIiRidWNrZXQiLCJwdWJsaWNqcyJdXX0=

---------------------------043351533357053995

Content-Disposition: form-data; name="signature"

SeCrEtSeCrEtSeCrEtSeCrEtSeCrEt=

---------------------------043351533357053995

Content-Disposition: form-data; name"file"; filename="jquery-1.4.2.min.js"

Content-Type: text/plain

After that I pipe through my file data.  You can view the exact code for how I pipe it in here: https://github.com/josephwegner/Multipost

Any ideas what might be causing this?

Thanks,

Joe

[Google Storage Team]

Hi Joseph,

I think the problem is that you're not supplying a policy document. Per this doc:

The security policy describes what can and cannot be uploaded in the form. If you do not provide a security policy, requests are considered to be anonymous and will only work with buckets that have granted WRITE or FULL_CONTROLpermission to anonymous users. The policy document must be Base64 encoded. See the policy document section for more information.

One option would be to make your bucket anonymously writable. if that's untenable, you'll have to formulate and sign a policy document per the referenced cited above. A more secure and perhaps simpler approach would be to use a PUT request, which allows you to upload or overwrite an object using a simple Authorization header instead of a policy document.

Marc

Google Cloud Storage Team

--
You received this message because you are subscribed to the Google Groups "Google Cloud Storage" group.
To view this discussion on the web visit https://groups.google.com/d/msg/gs-discussion/-/UPFn0GLPxtEJ.
To post to this group, send email to gs-dis...@googlegroups.com.
To unsubscribe from this group, send email to gs-discussio...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gs-discussion?hl=en.

[Google Storage Team]

Joseph,

My colleague pointed out that you are including a policy (I couldn't find it in your git repo but I see it now in your multi-part form trace):

Content-Disposition: form-data; name="policy"

eyJleHBpcmF0aW9uIjoiMjAxNS0wNi0xNlQxMToxMToxMVoiLCJjb25kaXRpb25zIjpbWyJzdGFydHMtd2l0aCIsIiRrZXkiLCIiXSxbImVxIiwiJGFjbCIsInB1YmxpYy1yZWFkIl0sWyJlcSIsIiRidWNrZXQiLCJwdWJsaWNqcyJdXX0=

De-base64-ing that string gives us your policy document:

$ echo "eyJleHBpcmF0aW9uIjoiMjAxNS0wNi0xNlQxMToxMToxMVoiLCJjb25kaXRpb25zIjpbWyJzdGFydHMtd2l0aCIsIiRrZXkiLCIiXSxbImVxIiwiJGFjbCIsInB1YmxpYy1yZWFkIl0sWyJlcSIsIiRidWNrZXQiLCJwdWJsaWNqcyJdXX0=" | base64 -d

{"expiration":"2015-06-16T11:11:11Z","conditions":[["starts-with","$key",""],["eq","$acl","public-read"],["eq","$bucket","publicjs"]]}

Notice the '$' prefix you included in "$key", "$acl" and "$bucket". Those should be, simply, "key", "acl" and "bucket", with no prefix.

Marc

[Joseph Wegner]

Marc,

I attempted that change, but I'm now getting the error:

"The content of the form does not meet the conditions specified in the
policy document.</Message><Details>Policy did not reference these

fields: key"

I don't see how that could be possible, as the key field's condition
is simply that it starts with blank, so it should always pass.. I
thought?

Also, the example provided in Google's documentation uses "$key",
instead of "key". https://developers.google.com/storage/docs/reference-methods#policydocument

Joe

On Apr 5, 8:33 pm, Google Storage Team <gs-t...@google.com> wrote:
> Joseph,
>
> My colleague pointed out that you are including a policy (I couldn't find
> it in your git repo but I see it now in your multi-part form trace):
>
> Content-Disposition: form-data; name="policy"
>
> eyJleHBpcmF0aW9uIjoiMjAxNS0wNi0xNlQxMToxMToxMVoiLCJjb25kaXRpb25zIjpbWyJzdGF ydHMtd2l0aCIsIiRrZXkiLCIiXSxbImVxIiwiJGFjbCIsInB1YmxpYy1yZWFkIl0sWyJlcSIsIi RidWNrZXQiLCJwdWJsaWNqcyJdXX0=
>
> De-base64-ing that string gives us your policy document:
>
> $ echo
> "eyJleHBpcmF0aW9uIjoiMjAxNS0wNi0xNlQxMToxMToxMVoiLCJjb25kaXRpb25zIjpbWyJzdG FydHMtd2l0aCIsIiRrZXkiLCIiXSxbImVxIiwiJGFjbCIsInB1YmxpYy1yZWFkIl0sWyJlcSIsI iRidWNrZXQiLCJwdWJsaWNqcyJdXX0="
> | base64 -d
> {"expiration":"2015-06-16T11:11:11Z","conditions":[["starts-with","$key","" ],["eq","$acl","public-read"],["eq","$bucket","publicjs"]]}
>
> Notice the '$' prefix you included in "$key", "$acl" and "$bucket". Those
> should be, simply, "key", "acl" and "bucket", with no prefix.
>
> Marc
>

> On Thu, Apr 5, 2012 at 2:59 PM, Google Storage Team <gs-t...@google.com>wrote:
>
>
>
>
>
>
>
> > Hi Joseph,
>
> > I think the problem is that you're not supplying a policy document. Per this

> > doc<https://developers.google.com/storage/docs/reference-methods#postobject>
> > :

>
> > The security policy describes what can and cannot be uploaded in the form.
> > If you do not provide a security policy, requests are considered to be
> > anonymous and will only work with buckets that have granted WRITE or
> > FULL_CONTROLpermission to anonymous users. The policy document must be

> > Base64 encoded. See the policy document<https://developers.google.com/storage/docs/reference-methods#policydo...> section

> > for more information.
>
> > One option would be to make your bucket anonymously writable. if that's
> > untenable, you'll have to formulate and sign a policy document per the
> > referenced cited above. A more secure and perhaps simpler approach would be

> > to use a PUT request<https://developers.google.com/storage/docs/reference-methods#putobject>,

[Google Storage Team]

Joseph,

In a policy document, the name "key" (or "bucket" or "acl", etc.) should be used without the dollar sign when you're specifying a field name, like this:

     {"key": "travel-maps"},

However, when using a field name as part of a pattern matching element (e.g., "eq" or "starts-with") you should include the dollar sign prefix, like this:

     ["starts-with", "$key", "travel"],

So your original use of $key was correct (sorry for confusing you). Btw, for others interested in this subject, there's a parallel conversation on this topic here on stackoverflow.com, which has additional details and some sample code.

Marc

[Joseph Wegner]

Hi Marc,

Thanks for the response. Ironically, that StackOverflow is mine from
a previous issue with the same project. I suppose it may shed some
more light on my experience, though.

So, it sounds like I'm back at square one. If I'm understanding you
correctly, my policy document is correct, even with the "$key"
format. So any suggestions on what to change there?

I realize that this certainly is an issue on my side and not yours,
because I can upload from a regular HTTP form, but I haven't got a
clue what I'm doing wrong uploading from Node.js.

Thank you,
Joe

On Apr 9, 3:58 pm, Google Storage Team <gs-t...@google.com> wrote:
> Joseph,
>

> In a policy document, the name "key" (or "bucket" or "acl", etc.) should be
> used without the dollar sign when you're specifying a field name, like this:
>
>      {"key": "travel-maps"},
>
> However, when using a field name as part of a pattern matching element
> (e.g., "eq" or "starts-with") you should include the dollar sign prefix,
> like this:
>
>      ["starts-with", "$key", "travel"],
>
> So your original use of $key was correct (sorry for confusing you). Btw,
> for others interested in this subject, there's a parallel conversation on
> this topic here on

> stackoverflow.com<http://stackoverflow.com/questions/10032567/generate-google-cloud-sto...>,

> which has additional details and some sample code.
>
> Marc
>
>
>
>
>
>
>
> On Mon, Apr 9, 2012 at 12:33 PM, Joseph Wegner <j...@wegnerdesign.com> wrote:
> > Marc,
>
> > I attempted that change, but I'm now getting the error:
>
> > "The content of the form does not meet the conditions specified in the
> > policy document.</Message><Details>Policy did not reference these
> > fields:  key"
>
> > I don't see how that could be possible, as the key field's condition
> > is simply that it starts with blank, so it should always pass..  I
> > thought?
>
> > Also, the example provided in Google's documentation uses "$key",
> > instead of "key".

> >https://developers.google.com/storage/docs/reference-methods#policydo...

[Marc Cohen]

De-base64-ing your policy doc reveals...

{"expiration":"2015-06-16T11:11:11Z","conditions":[["starts-with","$key",""],["eq","$acl","public-read"],["eq","$bucket","publicjs"]]}

I'm wondering if the empty starts-with pattern might be your problem (["starts-with","$key",""]). I would try removing that clause or populating a non-empty (and matching) value to compare against your key. Let me know if that doesn't help and I'll see if I can reproduce your problem in Python.

Marc

On Tue, Apr 10, 2012 at 8:28 AM, Joseph Wegner <j...@wegnerdesign.com> wrote:

Marc,

Thanks for the response.  It's ironic - that StackOverflow article you posted is one of mine from a previous issue on the same project.  I've been able to fix that problem, but I suppose it sheds some more light on my situation.

So, what is my next step here?  We've confirmed that my policy document is correct...  So what do you think might be causing the problem?

Thanks,

Joe

Joseph,

> >> gs-discussion+unsubscribe@googlegroups.com.

> >> For more options, visit this group at
> >>http://groups.google.com/group/gs-discussion?hl=en.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Storage" group.
To post to this group, send email to gs-dis...@googlegroups.com.

To unsubscribe from this group, send email to gs-discussion+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/gs-discussion?hl=en.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Storage" group.

To view this discussion on the web visit https://groups.google.com/d/msg/gs-discussion/-/MQ-krMLEu_sJ.

To post to this group, send email to gs-dis...@googlegroups.com.
To unsubscribe from this group, send email to gs-discussio...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gs-discussion?hl=en.

--

Marc Cohen |

Developer Programs Engineer |

marc...@google.com |

206-552-0240