Upload with Gridedit

[brian]

Hello,

I've just been hacking around with getting file uploading working so
that it can be used with gridedit, but I have a few questions.

In order to upload the file, I need to post a form.
In order to update the gridedit, I need to tell it that it's changed
before it can render.

To do this, I install a per session pre-render handler on the first
file upload.
I then bind a handler for the file that I'm uploading into the
session, which is called by that pre-render handler.

This then goes and fills in a file-widget (derived from dataform),
marks the gridedit as dirty, and so on.
The page rendering now starts, and draws the updated gridedit as
expected.

This works, but it's kind of horrible -- is there a more sensible
approach?

Regards,
Brian.

[Vyacheslav Akhmechet]

On 2/14/08, brian <brian.s...@gmail.com> wrote:
> I've just been hacking around with getting file uploading working so
> that it can be used with gridedit, but I have a few questions.

Brian,

I've done very little with multipart requests (in weblocks or
otherwise), so this might be a silly question. Can't a multipart
request contain query/post style parameters? If it can, everything
that you've described can be done in the callback action, right?

Or perhaps I'm misunderstanding what you're doing?

[brian]

On Feb 15, 11:53 am, "Vyacheslav Akhmechet" <coffee...@gmail.com>
wrote:
> On 2/14/08, brian <brian.spilsb...@gmail.com> wrote:> I've just been hacking around with getting file uploading working so

On 15 Feb, 11:53, "Vyacheslav Akhmechet" <coffee...@gmail.com> wrote:
> On 2/14/08, brian <brian.spilsb...@gmail.com> wrote:> I've just been hacking around with getting file uploading working so

-- somewhat long adventure posted below --

Well, I started by using a custom form in order to get the enctype to
work.
I've gone back and attacked it from the other direction now --
producing a file-upload presentation.
However, it looks like there's an error in
http://www.defmacro.org/ramblings/ui-dsl.html.

(defview employee-form (:type form :method :post) ...)

This will cause an error, since :method isn't an initarg for form-
view.
Presumably it should be :default-method :post?

I then added support for :enctype "multipart/form-data".
Having done this, I run into the problem that initiateFormAction
always expects to use Ajax to submit.

I could build the request body here and send it through, but I don't
have access to the content of the file that I want to send by
javascript unless I enable a bunch of non-portable security bypasses.

So, I guess the answer is -- it looks like you could do it in a
callback handler if you can get the form to submit normally.
...
Ok, I've done this now, by giving with-html-form a :no-ajax option,
and the form view a use-ajax-p option which defaults to t (probably
should make those consistent).

When I submit the form all goes well, except that the page I get is
sometimes inconsistent with the navigation.
If I am in /file (for example), then submitting directly from the ajax
uploaded form will direct me to /.
On the other hand, if I press 'Add', then reload the page, then
submit, it will direct me to /file.

The reason seems to be that (request-uri-path) is / when the ajax
uploaded content is produced.
Is there something which can tell me where the current session thinks
it should be for navigation purposes?

Regards,
Brian,

[brian]

Ah, I think I've fixed it ...

function getActionUrl(actionCode, sessionString, isPure) {
var url = location.href + '?' + sessionString + '&action=' +
actionCode;

using location.href in the ajax request tells the server where we
currently are.

(when (ajax-request-p)
(setf *current-navigation-url* (request-uri-path)))

in request-handler.lisp tells the render where the ajax is running
from.

(:form :id ,id :class ,class :action #+nil (request-uri-path) #-nil
*current-navigation-url* :method (attributize-name ,method-
type) :enctype ,enctype

in html-utils.lisp so that forms generated by the ajax call will work
-- hmm, might not be necessary now.

There may be some subtle problems with this, but it seems to be
working.

If you like, I'll put together a patch since it seems to involve a
bunch of different files.

Regards,
Brian.

[Slava Akhmechet]

brian <brian.s...@gmail.com> writes:

> This will cause an error, since :method isn't an initarg for form-
> view. Presumably it should be :default-method :post?

Yes, thanks, just fixed that.

> If you like, I'll put together a patch since it seems to involve a
> bunch of different files.

That would be great, thanks! I've been aware of some of these bits (need
for :no-ajax and :enc-type in forms) but didn't get to fix them yet. I
also have a patch from someone adding :no-ajax that I didn't get around
to applying yet. A bigger patch that combines all of these fixes would
be most welcome.

--
Regards,
Slava Akhmechet.

[Leslie P. Polzer]

Brian,

could you post the guts of your file upload stuff?
Would save me some time.

Leslie

[Vyacheslav Akhmechet]

On 3/26/08, Leslie P. Polzer <leslie...@gmx.net> wrote:
> Brian,
>
> could you post the guts of your file upload stuff?

If someone posts their code for uploads I'll be happy to clean it up
and incorporate it into main repository.

[brian]

On 3월27일, 오전5시07분, "Vyacheslav Akhmechet" <coffee...@gmail.com> wrote:
> On 3/26/08, Leslie P. Polzer <leslie.pol...@gmx.net> wrote:> Brian,

>
> > could you post the guts of your file upload stuff?
>
> If someone posts their code for uploads I'll be happy to clean it up
> and incorporate it into main repository.

Well, this is the basic code.
I'm not entirely sure that it works as I've give it here, since I'm in
the middle of finishing up drag and drop support for it with the
clipboard, and my current code depends on list presentations and views
-- I've cut all of that code out.

If it doesn't work, it should hopefully be fairly straight-forward to
fix -- I've put in the eval-when so that you can deal with it as a
single file, but that should probably be spliced out.

If not, then I'll set up a small project to test the cut-down version
in.

I notice that the repository version doesn't seem to support the
gridedit add view yet -- if that were added, then the filegrid widget
would be completely redundant -- getting rid of that would be nice.

I use the file object id for the filename, so you will have pub/files/
1, pub/files/2, etc.
This has the advantage of not letting bad people do terrible things
with file-names,
and the disadvantage that you need to look up the file object in order
to know what it is
and how to deal with it.

I've included a little code to handle requests like /pub/files?id=3
but I intend to replace that with a more generic
/pub/data/class/id/view form

e.g.,

/pub/data/file/3/icon would produce a view of the file object with id
3 as an icon.

Anyhow, it should hopefully be illustrative.

--

(eval-when (:compile-toplevel :load-toplevel)
(defclass file ()
((id
:reader file-id)
(name
:initform ""
:accessor file-name
:initarg file-name
:type string)
(type
:initform ""
:accessor file-type
:initarg file-type
:type string)
(meta-data
:initform nil
:accessor file-meta-data
:initarg file-meta-data
:type list)))

(defwidget filegrid (gridedit)
((item-add-view
:initform nil
:initarg :item-add-view
:reader filegrid-item-add-view)))

;;; render file-upload implementation
(defun render-file-upload (name value &key id (class "file-upload")
maxlength)
"Renders a file-upload field in a form."
(with-html
(:input :type :file :name (attributize-name name) :id id :value
value :class class)))

(defclass file-upload-presentation (text-presentation input-
presentation)
()
(:documentation "A presentation for file-uploads."))

(defmethod render-view-field-value (value (presentation file-upload-
presentation)
(field form-view-field) (view
form-view)
widget obj &rest args)
(declare (ignore args))
(render-file-upload (view-field-slot-name field) nil))

;;; File Upload
(defclass file-upload-parser (parser)
()
(:documentation "A parser designed to parse file upload values."))

(defmethod parse-view-field-value ((parser file-upload-parser) value
obj
(view form-view) (field form-view-
field) &rest args)
(declare (ignore args))
(when (listp value) (setf value (namestring (first value))))
(values t (text-input-present-p value) value)))

(defmethod gridedit-create-new-item-widget ((grid filegrid))
(let ((data (make-instance (datagrid-data-class grid))))
(make-instance 'dataform
:data data
:ui-state :form
:on-cancel (lambda (obj)
(declare (ignore obj))
(gridedit-reset-state grid)
(throw 'annihilate-dataform nil))
:on-success (lambda (obj)
(handle-file (request-parameter
"location") data)
(gridedit-add-item grid (dataform-
data obj))
(gridedit-reset-state grid)
(mark-dirty grid))
:on-close (lambda (obj)
(declare (ignore obj))
(gridedit-reset-state grid))
:data-view (gridedit-item-data-view grid)
:form-view (filegrid-item-add-view grid)
:form-title "Adding")))

(defmethod gridedit-add-item ((grid filegrid) item)
(when (gridedit-on-add-item grid)
(funcall (gridedit-on-add-item grid) grid item))
; dataform persists this item for us, no need to do it again
(flash-message (datagrid-flash grid) "Item added."))

;;; Add View
(defview file-add-view (:type form :inherit-from '(:scaffold
file) :default-method :post :enctype "multipart/form-data" :use-ajax-p
nil)
(id :hidep t)
(name :hidep t)
(type :hidep t)
(location :reader (lambda (obj) "") :writer (lambda (&rest args)
nil) :parse-as file-upload :present-as file-upload)
(meta-data :hidep t))

;;; Form View
(defview file-form-view (:type form :inherit-from '(:scaffold file))
(id :hidep t))

;;; Data View
(defview file-data-view (:type data :inherit-from '(:scaffold file))
(id :hidep t))

(defmethod weblocks-memory:strictly-less-p ((a file) (b file))
(< (file-id a) (file-id b)))

(defun handle-file (post-parameter file)
(when (and post-parameter (listp post-parameter))
(destructuring-bind (path name ct) post-parameter
;; strip directory info sent by Windows browsers
(when (search "Windows" (hunchentoot:user-agent) :test #'char-
equal)
(setq name (cl-ppcre:regex-replace ".*\\\\" name "")))
(let ((new-path (merge-pathnames
(merge-pathnames
(make-pathname :directory `(:relative "pub"
"files"))
; (parse-namestring name)
; make sure they're unique
(make-pathname :name (format nil "~D" (file-
id file))))
(asdf-system-directory :x))))
(rename-file path (ensure-directories-exist new-path))
(setf (file-name file) (pathname-name (parse-namestring name))
(file-type file) ct)
(persist-object (object-store file) file)))))

; --- a little extra to retrieve files

(defun handle-file-request ()
(let* ((id (request-parameter "id"))
(file (find-persistent-object-by-id *prevalence-store* 'file
(parse-integer id :radix 10)))
(path (merge-pathnames
(make-pathname :directory `(:relative "pub"
"files") :name id)
(asdf-system-directory :x))))
(hunchentoot:handle-static-file path (file-type file))))

(push (hunchentoot:create-prefix-dispatcher "/pub/files" 'handle-
file-request) hunchentoot:*dispatch-table*)

[Evan Monroig]

On Fri, Mar 28, 2008 at 11:55 AM, brian <brian.s...@gmail.com> wrote:
>
> the middle of finishing up drag and drop support for it with the
> clipboard,

Do you mean that you're implementing drag and drop upload? I've
always wondered how to do this without extensions to the web browsers
and how it would be better to have than the upload applets et al. that
Flickr and other webapps use.

Evan

[brian]

On 3월28일, 오후12시16분, "Evan Monroig" <evan.monr...@gmail.com> wrote:

> On Fri, Mar 28, 2008 at 11:55 AM, brian <brian.spilsb...@gmail.com> wrote:
>
> > the middle of finishing up drag and drop support for it with the
> > clipboard,
>
> Do you mean that you're implementing drag and drop upload? I've
> always wondered how to do this without extensions to the web browsers
> and how it would be better to have than the upload applets et al. that
> Flickr and other webapps use.

Well, you can do it in mozilla if you're willing to allow the browser
to read all of your files and so on, which allows you to drag a bunch
of files in at once, and then upload them in a queue.
I've done that, but it isn't portable, and it's a real pain to work
around the security restrictions.

No, the drag-and-drop that I've been doing is for (a) list editing
[changing the order], and (b) clipboard support.
I can drag rows from the gridedit into a clipboard in order to keep a
reference to the files I want to use,
and then I can drag them from the clipboard to insert them into lists
I'm editing (and, shortly, vice versa).
Makes it a lot easier to build up objects which refer to other
objects.

I think that if you're not wanting to upload lots of files a once,
then the standard upload widget is appropriate.
If you really want to see how to do it with drag-and-drop from windows
in firefox, then I have code for it (but not weblockified).

Regards,
Brian.

[Philipp Meier]

On 28 Mrz., 09:06, brian <brian.spilsb...@gmail.com> wrote:
> On 3월28일, 오후12시16분, "Evan Monroig" <evan.monr...@gmail.com> wrote:
>
> > On Fri, Mar 28, 2008 at 11:55 AM, brian <brian.spilsb...@gmail.com> wrote:
>
> > > the middle of finishing up drag and drop support for it with the
> > > clipboard,
>
> > Do you mean that you're implementing drag and drop upload? I've
> > always wondered how to do this without extensions to the web browsers
> > and how it would be better to have than the upload applets et al. that
> > Flickr and other webapps use.

> No, the drag-and-drop that I've been doing is for (a) list editing
> [changing the order], and (b) clipboard support.
> I can drag rows from the gridedit into a clipboard in order to keep a
> reference to the files I want to use,
> and then I can drag them from the clipboard to insert them into lists
> I'm editing (and, shortly, vice versa).
> Makes it a lot easier to build up objects which refer to other
> objects.

Brian,

I'm going to implement list support soon but would you mind to share
your list support code? I think this would be appreciated as well.

-billy

[Vyacheslav Akhmechet]

Thanks, Brian. I think for now I'll take the presentation and parser
code and leave out the gridedit bits. I suppose most people are just
interested in these two.

What you're doing with grids of files seems very interesting. About
time someone came up with a better UI for multiple file upload!

[brian]

On 3월29일, 오전5시40분, "Vyacheslav Akhmechet" <coffee...@gmail.com> wrote:
> Thanks, Brian. I think for now I'll take the presentation and parser
> code and leave out the gridedit bits. I suppose most people are just
> interested in these two.

It's necessary to support the gridedit-create-new-item-widget which
allows the uploading.
If you add item-add-views to gridedit, then it should all follow
naturally. :)