This is why I think Loopback sucks

[Kristóf Mihály]

Here is the simplest webapp, which cannot be done on loopback without ugly ass hacks.

A simple user gallery app:

- User hasMany Galleries. 

- Gallery hasMany Images

- Image has a belongsTo relation to storage, and has the following meta data: createdBy, createdAt and file (the relation)

Task: create a backend with loopback, where frontend shows all galleries with all pictures. you can upload and delete pictures through /api/galleries/{gallery_id}/images/{file_id}. Uploaded pictures get saved to the hard disk with a uid filename.

It is ridiculous how difficult this is. I urge you to try it. 

Problems you'll face:

- storage cannot be a relation to models see ticket

- you cannot access posted data from remote hooks of file (upload won't work) see ticket

- GET -ting /api/galleries may include images, but won't include the file relation of images. see ticket

- applying createdBy on create is a pain in the ass also, because you might create a remote hook on image.beforeRemote, but that won't get called when someone calls POST /api/galleries/{id}/images. observers does not include access token. see ticket

- file renaming? hahahaha

I think we can agree that this is a super basic scenario. I find it hard to believe that strongloop would start building something on top of Express.js without these things in mind.

Until Loopback is not refactored, I don't have the trust to go forward with it in any of my projects. 

And I wouldn't advise it to you either.

I'd love to be convinced that Loopback is a great framework, because I liked it very much in the beginning. But now I feel like it just wasted a lot of my time with it. 

I'd love to hear your opinion.

M

[Raymond Feng]

Hi, Kristóf Mihály.

Thank you for pointing out some issues that you ran into with LoopBack. Honest feedback is  always welcome! As one of the maintainers on the project, I would like to share my thoughts:

1. File uploads to pluggable storage backends with metadata management is NOT a very job in reality. Think about the complexity to deal with multipart/form-data HTTP requests (they are hybrid JSON objects and streams) and connect the binary content with metadata management (some metadata are from the storage systems after the fact while others can be external).

2. loopback-component-storage is starting effort to simplify developers’ work for 1. There is definitely room for further enhancements. We do plan to add metadata management in the future.  It’s an open source project, and you are strongly encouraged to work with the community (including us) to fix issues or add features based on your use cases.

3. What modules/frameworks have you explored to provide the out-of-box support for file uploads with metadata? I personally tried Express and found the solution is fairly involved.  

4. We primarily use Github Issues (https://github.com/strongloop/loopback/issues) and Google Groups (https://groups.google.com/forum/#!forum/loopbackjs) to support the community and triage bugs. Most of issues you noted were on StackOverflow, which has been a challenge for us to track. Reaching us on the right channels will make things smoother.

5. As a framework, we try hard to balance conflicting interests, such as opinions vs flexibility, and the declarative versus programmatic approach. The framework needs to do enough to solve common problems. Meanwhile, it needs to keep the doors open for customization. Finding the right place to define extension points often requires multiple iterations with various use cases.

Thanks,

Raymond

[Van Carney]

I have to agree the files connector is appallingly bad and can best be described as a "black hole". I simply can not wrap my head around the thinking behind the loopback-component-storage. With file uploads, there are other things that may need to be done and in 99.9999999% of the practical use cases, that file will need to be associated with a record elsewhere in the API.

This is not a new or even particularly challenging problem, look over to ruby on rails for Paperclip or Carrierwave. There is also Papercut for NodeJS.

These modules allow you to attach a file or files to a Multi-Part POST and then assign attributions as to how to handle it. In a nutshell, an image may be sent to a drawing library for resizing, put over on S3 and then a reference to it inserted into the appropriate model record. Very straight forward and painless.

What do we get with Loopback's loopback-component-storage? A silent dump of the file someplace with no hooks or any means at all to put in any sort of application logic. I mean -- there is no way to even know a file has been handled by the API. That's a serious WTF right there. And I find myself having to concoct some Rube Golberg-esque routine using AWS Pipelines to process the file when it hits S3 and a secondary post from the UI to a standard Persisted model to save the user provided info. Honestly, if I had known about this before starting my present project it's most likely I would NOT have implemented Loopback -- this is really basic stuff we see handled elegantly in all other frameworks and this failing really makes Loopback look not at all ready for production use.

Please -- at the very least add Observers or some way to work with the POST Data and conduct business logic behind the scenes. This black box scenario is rubbish and rather startling to see provided as part of a "Robust API platform".

[EthraZa]

When I got to that point in the project I'm involved right now. I got really scared with this apparent lack of vision about file handling in Loopback. So I decided to not loose my time with that since in this project my files are always just 1 photo per data POST and resized in the client to a lower resolution and good compression JPG. So I just send the base64 encode of the image in the POST stream and let it be handled as any other field in the backend, so it ends in a field in a MongoDB document as any other data.

But yes, this is not optimal and is possible in my case just because the images are not that important and can be of a lower resolution, meaning it gets small (about 160k). But I'm starting to think about the pain will be backuping this DB when I reach some millions records! 

So I upvote to someone take a look at it and rethink how Loopback handle file uploads.

Thanks

[Raymond Feng]

Folks,

Thank you for the candid feedbacks. We acknowledge that there is room to improve the programming model in LoopBack to how to deal with files (binary and metadata). This has been always on our roadmap. 

To clarify, the issue is not about lack of visions. I personally have a few ideas that we can make file handling better. But we are constrained in resources as a startup. As I said before, we welcome contributions. To get it started, I can outline some of the ideas and hopefully each of you can chime in and help to make it happen!

Thanks,

---

Raymond Feng

Co-Founder and Architect @ StrongLoop, Inc.

StrongLoop makes it easy to develop APIs in Node, plus get DevOps capabilities like monitoring, debugging and clustering.

--
You received this message because you are subscribed to the Google Groups "LoopbackJS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to loopbackjs+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loopbackjs/CAJeGMDhnYXzaw35uWpd%2By%3D8MNhD%3D3Q5imVb%2BgYcsej44g5iwww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[ven...@betterfood.me]

Kristof,

This does seem like a serious shortcoming in the file handling capability of Loopback.

Did you ever get to figure out a way to solve this problem without having to kludge something together?

-Venkat

[Issac Roth]

Hi Kristof,

Thanks for filing the various tickets. We’ll work this into our development timeline as we go. Meanwhile if anyone wants to contribute some of the work, of course have at it!

Issac

--

You received this message because you are subscribed to the Google Groups "LoopbackJS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to loopbackjs+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/loopbackjs/45818610-3081-4a0b-b1a1-7e710e0a3a18%40googlegroups.com.

[Kristóf Mihály]

Yes, I found a workaround: http://stackoverflow.com/questions/28885282/how-to-store-files-with-meta-data-in-loopback

[ven...@betterfood.me]

Thanks Kristof. 

Thanks Issac for the note. I hope this gets prioritized, along with the million other things your team is working on :-)

Kidding aside, this ask seems quite necessary for file management. Every app these days has to deal with some sort of user submitted file or image.