Feature request: collectstatic shouldn't recopy files that already exist in destination - Django developers

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Feature request: collectstatic shouldn't recopy files that already exist in destination
Received-SPF: pass (google.com: domain of jdu...@gmail.com designates 209.85.213.51 as permitted sender) client-ip=209.85.213.51;
References: <CAOKWLrHqJW+Db18n2Qbs8z4n4qVcfEJ5aZRPGuT6HkN47G5...@mail.gmail.com> <7516c233-e5f2-4109-8ecf-6f4a0054b798@googlegroups.com> <CAOKWLrHydT4M8LX8j9n5SrJRghZ1gWGM4RFbPGbO_zS_AzE...@mail.gmail.com> <BFC83DCA-157E-4577-A32D-C52302300...@gmail.com> <CAOKWLrGhKnhR7tiBhCO1UPp2-v-Ri5J1bTPyn-m2oPJtU1K...@mail.gmail.com> <12C76767-1C38-4C63-B401-85C8A48C2...@gmail.com> <CAOKWLrE0eBq1ZqTwduS+0yf9emLoYu1ZbDqJDV_k9rwKrQL...@mail.gmail.com> <0747d560-57e8-4d02-a62d-6755d2ea04d6@googlegroups.com> <CAACKYF5=+vUPO=EPe3YfVgay=4iu-pH_hVUiJBv87i87=kQ...@mail.gmail.com>
In-Reply-To: <CAACKYF5=+vUPO=EPe3YfVgay=4iu-pH_hVUiJBv87i87=kQ...@mail.gmail.com>
Mime-Version: 1.0 (1.0)
Content-Transfer-Encoding: 7bit
Content-Type: multipart/alternative;
	boundary=Apple-Mail-F8790631-99F0-4A87-A6DD-80AB74D8152B
Message-Id: <72634255-4651-455D-9202-6C6BBBDD3...@gmail.com>
Cc: "django-developers@googlegroups.com" <django-developers@googlegroups.com>
From: Jeremy Dunck <jdu...@gmail.com>
Subject: Re: Feature request: collectstatic shouldn't recopy files that already exist in destination
Date: Mon, 8 Oct 2012 20:14:48 -0700
To: "django-developers@googlegroups.com" <django-developers@googlegroups.com>


--Apple-Mail-F8790631-99F0-4A87-A6DD-80AB74D8152B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Would it be reasonable to have a backend-specific hook to determine a finger=
print, where that could be mtime or md5 or whathaveyou as long as equality (=
or maybe ordering) works?



On Oct 8, 2012, at 10:23 AM, Alex Ogier <alex.og...@gmail.com> wrote:

> On Mon, Oct 8, 2012 at 1:06 PM, ptone <pres...@ptone.com> wrote:
> While git may be common, and your problem not unique - this is still a con=
dition of your dev environment rendering modification dates invalid. There m=
ight be other situations where this is the case (I've run into scripts that m=
uck with modification dates based on camera/jpeg metadata).
>=20
> So after some further discussion on IRC - it was determined that md5, whil=
e somewhat common, was far from a standard, and was likely not to be availab=
le as remote call for network based storage backends. And so the final resol=
ution is to wontfix the ticket.
>=20
> In the end - this lack of a universal fingerprint is just a limitation of o=
ur storage tools.
>=20
> -Preston
>=20
> Is there a reason this fingerprint must be universal? If you're dealing wi=
th a backend like S3, where network latency and expensive writes are a probl=
em, but md5 is a builtin remote call (available on any GET), why not just do=
 an md5 sum in the _save() method? Basically, just buffer the File object yo=
u receive, take an md5 in python, and then make a decision whether to upload=
 or not. In the common case of reading from local disk and writing to S3, th=
is is a big win, and doesn't require cooperation from any other backends, or=
 standardizing on md5 as a fingerprint method.
>=20
> Best,
> Alex Ogier=20
> --=20
> You received this message because you are subscribed to the Google Groups "=
Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to django-developers+unsubscrib=
e@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/djang=
o-developers?hl=3Den.

--Apple-Mail-F8790631-99F0-4A87-A6DD-80AB74D8152B
Content-Transfer-Encoding: 7bit
Content-Type: text/html;
	charset=utf-8

<html><head></head><body bgcolor="#FFFFFF"><div>Would it be reasonable to have a backend-specific hook to determine a fingerprint, where that could be mtime or md5 or whathaveyou as long as equality (or maybe ordering) works?<br><br><br></div><div><br>On Oct 8, 2012, at 10:23 AM, Alex Ogier &lt;<a href="mailto:alex.og...@gmail.com">alex.og...@gmail.com</a>&gt; wrote:<br><br></div><div></div><blockquote type="cite"><div>On Mon, Oct 8, 2012 at 1:06 PM, ptone <span dir="ltr">&lt;<a href="mailto:pres...@ptone.com" target="_blank">pres...@ptone.com</a>&gt;</span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>While git may be common, and your problem not unique - this is still a condition of your dev environment rendering modification dates invalid. There might be other situations where this is the case (I've run into scripts that muck with modification dates based on camera/jpeg metadata).</div>
<div><br></div><div>So after some further discussion on IRC - it was determined that md5, while somewhat common, was far from a standard, and was likely not to be available as remote call for network based storage backends. And so the final resolution is to wontfix the ticket.</div>
<div><br></div><div>In the end - this lack of a universal fingerprint is just a limitation of our storage tools.</div><div><br></div><div>-Preston</div></blockquote><div><br></div><div>Is there a reason this fingerprint must be universal? If you're dealing with a backend like S3, where network latency and expensive writes are a problem, but md5 is a builtin remote call (available on any GET), why not just do an md5 sum in the _save() method? Basically, just buffer the File object you receive, take an md5 in python, and then make a decision whether to upload or not. In the common case of reading from local disk and writing to S3, this is a big win, and doesn't require cooperation from any other backends, or standardizing on md5 as a fingerprint method.</div>
<div><br></div><div>Best,</div><div>Alex Ogier&nbsp;</div></div>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups "Django developers" group.<br>
To post to this group, send email to <a href="mailto:django-developers@googlegroups.com">django-developers@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href="mailto:django-developers+unsubscribe@googlegroups.com">django-developers+unsubscribe@googlegroups.com</a>.<br>

For more options, visit this group at <a href="http://groups.google.com/group/django-developers?hl=en">http://groups.google.com/group/django-developers?hl=en</a>.<br>


</div></blockquote></body></html>
--Apple-Mail-F8790631-99F0-4A87-A6DD-80AB74D8152B--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
