Web Interface Improvements

[Vikhyat Korrapati]

Hi everyone!

The current web interface is great but I feel there is room for improvements. I have listed some problems with the current situation below:

1. There is no build tool in place to handle concatenating and minifying assets. This will make things unwieldy as the number of assets included grows.
2. The web interface pulls data by making API requests and constructing HTML by manually concatenating strings. (Example: https://github.com/stratosphere/stratosphere/blob/master/stratosphere-runtime/resources/web-docs-infoserver/js/jobmanagerFrontend.js#L259-L267). This will cause maintainability issues as the web interface grows in complexity.

In my opinion the following can be done to improve the current situation:

1. Move the front-end part of the web interface into a separate repository. This would make it possible to treat the web interface as a proper project with a build system, tests etc. The resulting output after building, concatenating, minifying assets will be included in the same place they are right now (https://github.com/stratosphere/stratosphere/blob/master/stratosphere-runtime/resources/web-docs-infoserver/), so it will not really result in any additional complexity added to the main project.
2. Modify the API to make it more RESTful, possibly conforming to the JSON-API spec (http://jsonapi.org/). This isn't strictly necessary but I think it would be a good idea.
3. Make the web interface a single-page application.

If these changes are acceptable I can volunteer to convert the existing web interface into a single-page application implemented using Ember.js. I just wanted to make sure whether this is an agreeable direction before getting started.

Regards,
Vikhyat Korrapati

[Vikhyat Korrapati]

I just found this Github issue that proposes something similar: https://github.com/stratosphere/stratosphere/issues/297

[Fabian Hueske]

Hi Vikhyat,

welcome on the Stratosphere mailing list!

Improving the web interface is a great idea and we definitely appreciate any help there :-)

I am not sure, if I would move the interface into a completely separate repository. Wouldn't it be sufficient to move it into a dedicated Maven module?

I also had a (very brief) look at Ember.js. Looks really good. One concern I have is that it seems to have a custom open source license. Not sure how well this plays together with our Apache V2 license. Do you have some information about this?

Best, Fabian

2014-05-27 18:51 GMT+02:00 Vikhyat Korrapati <vikh...@gmail.com>:

I just found this Github issue that proposes something similar: https://github.com/stratosphere/stratosphere/issues/297

--
You received this message because you are subscribed to the Google Groups "stratosphere-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stratosphere-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/stratosphere-dev.
For more options, visit https://groups.google.com/d/optout.

[Vikhyat Korrapati]

Hi Fabian,

Thanks for the reply!

To be honest I don't have much experience working with Maven so I'm not 100% sure but I think that would also work. Will give it a shot.

Regarding licensing issues Ember is licensed under the MIT license (see: https://github.com/emberjs/ember.js/blob/master/LICENSE). On this page (http://apache.org/legal/3party.html) the MIT/X11 is included in the list of licenses that may be included with Apache projects so I do not believe it will be a problem.

Regards,
Vikhyat

[Fabian Hueske]

Ah, great! I saw the license file as well, but did not identify it as MIT. Very good!

Thanks :-)

[Robert Metzger]

Hi,

I also think the license is not a problem, if it is MIT.

We should look into integrating the minification etc. into the maven build process. I can help you with Maven if you have problems.

But the goal of our build system is to create a runnable binary with one call.

If we would have two separate projects, the whole build process would be much more complicated. So I'm also against placing the interface in a different repository.

The "stratosphere-client" project is actually pretty small so I don't see a problem with the current setup.

I'm not sure if we really need to minify the css / js files. The web interface is not being used by millions of visitors every second. Its usually only one visitor and the traffic is nothing compared to what the system actually produces.

I would suggest to focus on creating an awesome, nice and intuitive web interface.

Also what you've mentioned regarding the string concatenation makes sense. The current implementation is not really maintainable.

We are all Java developers and I think we don't know the latest developments in modern web-development. If you are interested in fixing and improving the web frontend, we would be more than happy!

Regards,

Robert

[Vikhyat Korrapati]

Hi,

I found this project
(https://github.com/eirslett/frontend-maven-plugin) which should allow
using the standard front-end tools with Maven. I'm going to try to get
it to work.

The primary reason for getting a build system in place for
concatenating and minifying JS/CSS isn't only for performance, it also
helps with maintainability. Especially when using Ember it is standard
practice to have each component in a separate file (for example, see
https://github.com/discourse/discourse/tree/master/app/assets/javascripts/discourse/controllers)
so you would end up with a very large number of Javascript files
included in the HTML document. For example in development mode (where
assets are not concatenated) Discourse serves 619 different files. If
this is managed manually it will be fairly annoying to keep the files
and HTML in sync. Another advantage is that it makes it possible to
use module systems for Javascript, SCSS for CSS etc.

I think what I'll do is first get a front-end build system working
with Maven. Once that is done I can start moving the implementation to
a single page Ember app. I will keep you posted!

Regards,
Vikhyat

> You received this message because you are subscribed to a topic in the
> Google Groups "stratosphere-dev" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/stratosphere-dev/GeXmDXF9DOY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Robert Metzger]

Cool. Thanks for the explanation.

Let me know if you have any troubles with the maven module. I don't
know the specifics of it, but I know the general concepts and terms of
Maven.


Sent from my iPhone

[Vikhyat Korrapati]

Currently the "Job Manager" and "Web Client" are exposed via two
different web interfaces. Would it make sense to expose the
functionality of both from the same interface?

[Robert Metzger]

Hi,

yes. Even worse, there are two different webservers running, to serve the two pages.

I would really like to have an extra section on the Job Manager web interface that allows to submit jobs there.

There is only one thing we need to take care of: It is actually possible to execute arbitrary (also malicious) code via the "web client". Therefore, we should have an option in the configuration to disable it.

Robert

[Ufuk Celebi]

On 29 May 2014, at 08:56, Robert Metzger <rmet...@apache.org> wrote:

Hi,

yes. Even worse, there are two different webservers running, to serve the two pages.

I would really like to have an extra section on the Job Manager web interface that allows to submit jobs there.

There is only one thing we need to take care of: It is actually possible to execute arbitrary (also malicious) code via the "web client". Therefore, we should have an option in the configuration to disable it.

+1

Robert

On Thu, May 29, 2014 at 6:31 AM, Vikhyat Korrapati <vikh...@gmail.com> wrote:

Currently the "Job Manager" and "Web Client" are exposed via two
different web interfaces. Would it make sense to expose the
functionality of both from the same interface?

Yes, this is definitely a good idea. :-)

[tobw...@gmail.com]

Hi Vikhyat,

currently I am working on some WebInterface issues. I think your improvements sound like a good way to go. Especially in terms of improving the maintainability.

I would like to contribute to improve specific aspects in the update routine.

Regards,
Tobias