Unit testing in T3 framework
Kevin Brown
Does anyone have thoughts on that? Is there some work already done I could build on?
Thanks! I like the framework a lot.
Kevin Brown
I thought maybe I should add full details as to what my top-level post request is about. I think this is on-topic for a thread on unit tests, because it is about (partially) refactoring the T3 codebase to enable testing.
Background
I’d like to outline what I’m planning to do with refactoring and extending the T3 code base and ask for input. I recently undertook a large project in Joomla that used a number of extensions (social extension, event extension, forum extension, user awards extension, registration extension, CCK extension, etc.) from the JED. I investigated the different template frameworks available before I started the project (T3 / Gantry / Warp / Helix / Wright) – their capabilities /l imitations and general code structure. I respected the remarks I saw from Hung Dinh on the value of consolidating efforts on a template framework. I decided not to use a template framework in the project. Now that I have a very clear idea of what I need in order to do such a project again, I want to create a generic entity (“SPA”) to do it.
I am at danger of over-describing “SPA” to explain how I want to employ T3’s code base in it, but I need to give an overview so that this makes sense as to what I’m trying to get T3 to do. It is a single page application with angularjs as a front-end javascript MVVC framework.
I am conceptually thinking of Joomla as a framework comparable to Rails, Zope, or Laravel instead of as a CMS comparable to Drupal, Plone, or OpenCMS. Angular is very opinionated and uses HTML templates - the directives that tell Angular what object to render in the browser (like a button), and what other objects/data to bind with, is included in a plain HTML element, and doesn’t use the class for a javascript hook:
<div>
Quantity: <input type="number" ng-model="qty">
</div>You can see that Angular will insert an input element with whatever the “qty” variable in the model holds (and automatically update it if it changes, for example if another element like a “clear” button is bound to the “qty” variable). So this is definitely a template – just served as HTML. This is unlike other front-end frameworks like backbone.js, ember.js, etc. that use different template engines like handlebars, moustache, etc. Rails has great built-in support for whatever template engine you want to use and makes it easy to manage on the front and back ends.
Angular web applications live happily within a web page with non-angular conventional HTML/CSS/Javascript.
tl;dr If I use angular with a supportive template framework, I should be able to drag-and-drop really cool UI elements that automatically wire themselves up in a GUI page, similar to how T3 now lets you dynamically control the column width and number of modules in a row, and the number of rows in the template. An end user could drag-and-drop a “clear” button to a form, and then all necessary javascript, CSS is applied and data binding is done automagically. This implies a comprehensive technology stack underlying the template framework.
Observations About the T3 Codebase
I installed, used, and looked through the code of some other template engines: Gantry / Warp / Helix / Wright. It seemed to me there were some minor feature differences, but nothing fundamental. Each is a pretty substantial codebase, both PHP and Javascript. I didn’t really map out the different ways they each handle building a request. I’ve read through every line of T3 code, and am finishing class-responsibility-collaboration cards on the PHP portion tonight.
PHP Codebase Observations and Refactoring
Since I’m trying to fit the T3 code into a more comprehensive stack, it makes sense to me to use some more formal tools to analyze it and plan for a refactoring. I plan to do collaborator graphs (to analyze T3 caching design), construction graphs (for refactoring class hierarchies), and call graphs (to visualize cohesion within the T3 classes) this week on the PHP portion of the codebase. The code strikes me as pretty solid. I see the following problems for my use of it:
· The Joomla core now has a dependency injection container, but is still full of singleton factory methods for important classes. I need to refactor all of the static classes out of the T3 codebase for my use.
· T3’s integration with microformats is awesome, and essential for angular apps. I intend to use the new JLayout class in Joomla core to provide “widgets” to SPA-enabled extensions – HTML template snippets that create a particular HTML element with rich features, for example a combo box with filtering and sorting parameters. One area IMO that Joomla is critically weak in is support for WAI-ARIA, and this is a deal-killer for a lot of people considering Joomla. Drupal and Plone are strong in this area. If an extension uses HTML elements that incorporate intelligent microformat / WAI-ARIA inclusions, and so does the template framework, the problem is solved well. So this is to say I’d add a WAI-ARIA inclusion method similar to what been done for microformats to T3 (and probably building on the microformats GSOC work).
· Some of the current T3 classes have low cohesion (unrelated functions in the same class) and non-semantic class names (can’t tell from looking at a CRC card what a method does and to what). I would refactor to fix.
· Implement unified installer to avoid installing plugin and template separately. I would implement a library class and move a lot of the PHP code into that class hierarchy. With Composer in the core now it is easy to resolve template dependencies on the T3 codebase when different templates are installed.
· “Add-on” structure of T3
· All of the frameworks I’ve looked at, and a couple of optimization-specific extensions, operate by getting the header object on a late system hook and then using regular expressions to manipulate what <script> and <link rel=”stylesheet”> tags are included in the header. Joomla’s primitive asset pipeline concept is a big weakness for me compared to other frameworks ( (i.e. Joomla setters for bespoke JS/CSS files, access to the head object, and a few utility classes to load Bootstrap, JQuery UI, and core assets only once). I would completely separate the asset pipeline classes and options (Optimization, Template Extended Styles) to a plugin system organized under ./plugin/pipeline/. I have a functional asset pipline based on Pipe (PHP port of Sprockets) that does what I need, and using something like it is necessary for my project. Using observables gives developers the opportunity to layer the optimization services. This way I could plug in a SASS compiler instead of the lessc compiler, if I wanted to use SASS, or I could use both in the same project and CSS build. Another application important to me is to be able to wrap javascript assets in AMD closures so that I can dynamically load them with require.js on the client-side; this is easy to achieve with a plugin asset pipeline architecture, but most users aren’t going to want this. Also, I need a CoffeeScript compiler in the pipeline. I think the best approach is an interface class that the template framework can depend on – so the T3 GUI interface to it wouldn’t change. Asset pipline plugins could be required to make available a configuration UI as an add-on to T3 (or anything else that wanted to use them) based on the interface.
· The framework needs unit tests written with some coverage goal in mind.
· PHP codebase needs completion of PHPDoc comments on all declarations, methods, and classes, and some formatting changes to conform with Joomla coding standards.
· Error handling is inconsistent. Some classes simply echo error messages. Others use a mapper that provides legacy JError or exceptions. Less_Cache throws exceptions, but not in try/catch blocks. Some code that seem like it should have error-checking does not.
· The template framework does not make use of Joomla’s function caching. I would try to identify opportunities for function caching during the refactoring of the code base. Also, I put the issue of caching the T3 Block outputs in unresolved below.
· I would remove all SQL queries from their current classes, and move them to model classes in the library. They could then be dynamically injected as dependencies to classes that need them. This would improve visibility for the SQL (instead of scattered around in classes, it would be centrally located) and thus the likelihood of optimizations being made. It would simplify caching for database fetches; for example, it seems like T3:: getDefaultTemplate() can always be cached. The only time its data changes is when an admin makes a relevant change in the back-end, and that code already flushes the com_template –namespaced cache after changes (but doesn’t take advantage of that cache for storage in any way that I can see). Moving the query to a library model class would let the T3:: getDefaultTemplate() method check if it has a cache and call the SQL query only if it needs to, and it would allow keeping all of the CRUD methods in one class to share with the backend.
· I would put the code to implement CCK-like functionality for com_content in its own separate namespace (currently class JATemplateHelper), and same with the megamenu functionality. This is including “built-in” modules within a template framework without a clear reason to do so that I can see.
Javascript Codebase Observations
· The T3 javascript codebase consists of about 5,600 LOC, organized in JQuery modules.
· The code does not use JSDoc comments, so automated generation of an API browser is not possible.
· The Javascript codebase is tightly coupled to the HTML in the areas where the template framework exerts influence – for example, a very particular HTML structure for megamenu is required.
· The JQuery modules do everything – presentation, state, data binding, AJAX, etc.
· The PHP T3Ajax classintroduces an extra point of entry into the Joomla framework needlessly. There’s no current standard in Joomla on how to handle Ajax calls, but there are some mature proposals with code for how it could be handled (com_ajax, for example).
Unresolved
“Block” Caching
The T3 “Block” loading system does not use caching. There is no reason not to do so, since the <jdoc:include> elements are not parsed until after the blocks are buffered. The ./tpls/blocks/mainbody directory structure in the template directory is confusing to me. I think it should be reorganized to ./tpls/blocks/typeofblockdir/typeofblock.php, with typeofblock.php being an HTML file (i.e. move spotlight-1, spotlight-2, etc. into a spotlight folder).
Then, I would have simple configuration files in ./tpls/blocks with a list of the blocks required by this particular view: corporate, mainbody, etc. I would add a library class to parse the configuration file and buffer the required blocks. This way, the template framework could use Joomla function buffering to avoid the template rendering computation. The standard HTML5 boilerplate is safe to bury in the library with a way to override it from the template, perhaps with a configuration option to select alternate boilerplate (HTML6?). Template assets (css/js) can be added in code instead of with HTML elements. As dynamic content in the template itself expands (microformats, WAI-ARIA tags, dynamic CSS classes, etc.), the value of caching it increases.
This doesn’t solve how to wrap the function in a wrapper <div> class, for example for the off-canvas feature. How other developers in Joomla are handling this seems all over the board. By letting a block call another block with <?php $this->loadBlock('someblock') ?>, the template framework is implementing a HV (hierarchical view) pattern.
Jomsocial does this by creating a ./templates directory within their component and adding all of their layout files in the root of it with some namespacing (i.e. activities.apps.php, activities.events.php, etc.). Then, they use the view.feedtype.php file in the view directory to call the layout files from code. There is no /tmpl directory under the view to override. They did this (I would guess) because their views (groups, events, wall, etc.) don’t match well with the layouts they need (no activities view, but the groups view might need the activities.groups.join form). The disadvantages to this method are: (1) you have to create your own vendor-specific template file to override their HTML output, and possibly have support issues with using a non-standard 3rd party vendor installer for your template, and (2) it leads to burying HTML in class/method code.
Kunena Forum uses their own ./template directory, but retains the view ./tmpl directories in their component for backwards compatibility. The Kunena template is organized with layouts in their view folders (same as Joomla core) so they can be overridden from a system template’s ./html folder. They include a common view to make reusable layouts available to other views in a fashion similar to how the T3 framework is doing it (i.e. a “common” view with layouts like footer, whosonline, etc.).
It’s not clear to me how to handle this. My thought at present is to use an extended JLayoutFile object to load the block files, since it doesn’t matter where they’re located in the file system with that library and it has nice methods in the library for overrides, etc. , and to abandon the T3:: loadBlock method. This would happen from a library class that parses the configuration file in ./tpls/blocks I mention in the second paragraph of this section. I would build some sort of cyclical dependency detection into the library class, so that users could call blocks from within blocks safely (i.e. with an alternate loadBlock method).
If I were doing this in a component context, I would definitely do it through the new FOF RAD framework in the core and take advantage of the HMVC features of that framework. In FOF, models aren’t required, so I could get the flexibility of JLayout and have a controller (and the option of passing it some data) to boot. Taking advantage of this in a template framework would require creating an accompanying component to house the MVC files, since FOF is pretty opinionated (convention over configuration) as to how it will autoload although I’m not absolutely sure that is a true statement. Since the com_template namespace is already taken by the Joomla core, it might make sense to use it as a “home” for FOF MVC files, and do an in-memory sleight-of-hand to copy the classes from the template’s ./tpls/blocks/typeofblockdir/typeofblock.php file over to the com_template MVC directories (i.e. see Julio Pontes’ Joomla Override Plugin ). This will take some work to figure out how to organize the MVC code for an HTML layout file. In most cases, the controller won’t do anything (i.e. working just like loadBlock() does now), but it would be nice to be able to extend it and also to add some model data to a layout if necessary.
Javascript Refactoring
I think I will go a completely different route on refactoring the T3 framework’s javascript codebase. Using Angular with HTML templates will significantly reduce the total number of LOC. Angular uses JQuery for selection and a few other functions, so the non-data-binding code can be reused as necessary. However, in most cases it makes sense to convert the JQuery code to an Angular directive (which is pretty straightforward) and use the directives instead. This really helps with reusability in a functional module organization like the T3 framework currently has. It also makes writing unit tests for the front-end framework easy; I prefer Jasmine and would use it for the testing harness.
Another advantage of this approach is that it makes it very easy to add touch event handling broadly to the front-end. As I mentioned, you can reuse JQuery plugins in Angular, but all of the Bootstrap plugins have been ported to Angular (AngularUI). If developers are using the Joomla core API properly, it would be easy to switch from the bootstrap components to the angularUI components - i.e., developers use JHTML::_('some.bootstrap.component') instead of copypasting the HTML snippet from the Bootstrap docs. And the AngularUI components automatically support the touch events, for example swipe on the Carousel, if the appropriate library is loaded (ng-touch) and the JHTML snippets are overridden (again probably with Julio Pontes’ override plugin).
Megamenu / CCK Features
I need a megamenu for my purposes also, I just don’t think it belongs in a template framework. I will demo an angular megamenu module for the refactored framework codebase since I think it’s probably pretty important for a lot of users, and not easy to implement cleanly as a module without extending the core data scheme or adding a vendor-specific table to the database. It makes sense to follow the core menu-association scheme to provide multilingual support, and to use the core override plugin method to add administrative configuration for the megamenu to mod_menu (in addition to adding the xml configuration options – the megamenu configuration in mod_menu needs a model and access to the database). I think it’s fine for a template framework to include a default mod_menu HTML template override to provide the megamenu configuration options in mod_menu. The configuration option to turn it on/off should be in mod_menu.
As far as the CCK features, the rest of my project provides a solution but isn’t relevant to the template framework refactoring.
Unrelated
I mentioned the project I am working on and want to pull the T3 framework into (or have a dependency on), SPA. This is one of the big pieces of the project. I put this at the bottom, in case anyone’s curious, but I’m not trying to promote it in this Google Group so unless a question about it is related to refactoring the T3 framework, I’ll ask to defer. Other pieces are:
RESTful API - based on the Joomla web services API specification (2nd draft) and implemented in FOF RAD
Intelligent Asset Pipeline – described earlier to handle javascript/css files and source code
Object-Relation Mapping (ORM) Service – an opinionated ORM to provide CCK functionality to end users through a WYSIWYG UI. “Components” in SPA are code-less, and just configuration files: in the case of the model, just XML for the ORM to define data models. Those models are constrained to and optimized for a particular social-network data scheme (RSN) and easily moved to a NoSQL backend like MongoDB.
An opinionated for organizing Angular directives, building more complex widgets with them, and using them in components/modules.
The Downside of Angular
In my opinion, the success of Joomla depends heavily on the
ability of users to install it on low-end shared hosts without problems.
Angular web pages are gibberish, since the HTML is a template and the data to
fill it out with is in an (unseen) JSON file, until the Angular compiles the
template and renders it. The major search engines (Baidu, Yandex, Google, Bing)
support adding an html tag to the page to tell the search spider that the page
has to be rendered in a browser before indexing. Then, the spider will come
back with a configurable shebang (like http://example.com/#!/about)
in the URL to tell the server that it wants the rendered version. Then, you use
whatever rewrite engine the web server has to redirect to the webpage served by
PhantomJS on node.js, but that violates the “shared host with sftp” rule.
The above isn’t so relevant to what I propose for refactoring the T3 framework. Most of the angular stuff would be on the admin side, which isn’t crawled. The front-end T3 javascript would work without change (responsive.js, script.js, menu.js, etc.). The angular megamenu demo wouldn’t show to spiders and that would be a big SEO hit (without the PhantomJS solution), but if I use the existing classes and structure the existing JQuery module (mega.js) would work as normal in a non-angular version (the value add would be moving configuration to mod_menu).
Close
I am out on vacation for a week and won’t have good internet access. I hope some people might offer opinions. Thank you!