Tarbell refactoring for better admin tools

[Joe Germuska]

Howdy, folks:

As we discussed at NICAR, Northwestern University Knight Lab is interested in contributing to Tarbell development, and specifically, contributing development resources to make the tool easier for people to start using. To achieve this, we need to work around some historic design choices in Tarbell, and we wanted to have community input so that we make those choices intelligently.

Our general goal is to minimize the degree to which users need to interact with Tarbell on the command line, without removing any of Tarbell's flexibility. The general idea is to add a command, "tarbell admin", which would start a local web server through which users could manage configuration settings, create and run projects, and install blueprints. A second order goal is to create a double-clickable "app" which is easily installed and which provides a way to launch the admin/management interface from the OS GUI.

Unfortunately, providing management of projects and blueprints is complicated by the fact that the metadata for those things is stored in python code which is currently handled with import statements. If the admin tool is to go over all of the current projects or blueprints, there's a lot of unpredictability involved in importing module after module. It seems like it would be a lot simpler if basic metadata -- at least the project/blueprint name, maybe some similar string or simple values -- was stored in YAML or JSON.

We wanted to raise this with the community so that we can find a solution which is 

a) backwards compatible

b) preserves Tarbell's flexibility and extensibility

c) delivers on a more user friendly management experience

We would love some input from folks about risks or considerations as we try to come up with a proposal that balances these goals.

[David Eads]

Hey Joe!

Thanks for this.

The loading step with python based config files is a surmountable problem -- there's even code to do it associated with the little used `tarbell list` command. There is some unpredictability because of imports, but that's going to be an issue for the GUI generally.

However, changing the configuration and writing it back is, if it is even possible, going to be really hairy if we don't adopt some version of this proposal.

I'm inclined to say that we should keep the old loader as a fallback for backward compatibility, and move to a model where there's a JSON or YAML configuration file per project and an optional python module that acts as a Flask blueprint. This also has the advantage that Tarbell could automatically register the Flask blueprint if it is present, eliminating some boilerplate code.

Potentially the GUI could incorporate the fallback loader for backward compatibility and use try/except to trap any errors and simply report them to the user. If the fallback loader is used, the GUI could disable configuration write-ability and offer an upgrade path for older projects.

Anybody else have thoughts about this? I think Joe's proposal will be a huge step forward for the project.

David

--
Learn more about Tarbell at http://tarbell.tribapps.com
---
You received this message because you are subscribed to the Google Groups "Tarbell project discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tarbell-proje...@googlegroups.com.
To post to this group, send email to tarbell...@googlegroups.com.
Visit this group at http://groups.google.com/group/tarbell-project.
To view this discussion on the web visit https://groups.google.com/d/msgid/tarbell-project/b972984c-998a-459b-80c6-8188d445a9a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Geoffrey Hing]

I think per-project config in YAML/JSON seems like a good proposal.  Thinking about this makes me think about how Flask handles configuration (http://flask.pocoo.org/docs/0.10/config/):

Independent of how you load your config, there is a config object available which holds the loaded configuration values: The config attribute of the Flask object. This is the place where Flask itself puts certain configuration values and also where extensions can put their configuration values. But this is also where you can have your own configuration.

I think Tarbell would be well served to follow how Flask handles configuration and create a configuration object that can be loaded from a variety of sources (Python module, YAML file, either as specified by an environment variable, etc).

Best,
Geoff

To view this discussion on the web visit https://groups.google.com/d/msgid/tarbell-project/CAFQdJjsqac9wWKc1RPaFDGsBzJT9RXcuSdw3_%3D%2Be2WQWK1xUnA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Geoffrey Hing
geof...@gmail.com
773.969.6436
http://geoff.terrorware.com/
Twitter: @geoffhing

[Matt Waite]

I have no opinion on how configuration is handled, I just want to make a small plea on behalf of a certain user base: beginners. The reason I love Tarbell is that I see it as a gateway drug into making things on the interwebs. I teach it to students who have rarely ever used the command line before and are jumping into the world of many-moving-parts-makes-things-on-the-web for the first time. As such, the current configuration setup is just complicated enough to pose a challenge, but not so bad that it scares people off. Whatever it is replaced with, I pray you keep that beginner in mind. My students are a use case, the lone coder in a newsroom trying to make stuff with zero backup and resources is another. Keeping things simple or at least transparent for them is an admirable goal.

Matt

—
Matt Waite
Professor of Journalism, University of Nebraska-Lincoln
Founder, Drone Journalism Lab
Co-founder, Hot Type Consulting

To view this discussion on the web visit https://groups.google.com/d/msgid/tarbell-project/CAMTySv6nGSYxDRCkoxsFJF1QCWz0cS-Zm78vbu8So8Q-v8Y27g%40mail.gmail.com.

[Miguel Paz]

I agree with Matt. As a beginner, what I would like to have is a better and more simple explanation of Tarbell setup and the wonders you can do with it. David Eads was kind enough to take some time and explain to me and another person in my team - in a screencast - how Tarbell is setup. After that we understood it very well. 

As a beginner user case, for me it would be great if something is done to make it understandable enough, that David does not have to use his time explaining others about it (it`s not that I don`t like you David hehehe).

To view this discussion on the web visit https://groups.google.com/d/msgid/tarbell-project/1432747082912.5576f7be%40Nodemailer.

For more options, visit https://groups.google.com/d/optout.

--

Miguel Paz @miguelpaz 

Nieman-Berkman Fellow 2014-2015

Poderomedia.org @poderomedia - Poderopedia.org @poderopedia
PoderomediaAcademy #PAcademy - Manual Periodismo de Datos @periodismodata
Hacks/Hackers Chile @hackshackersSCL
US mobile 1-857-263-1535
Skype elmiguelpaz  

[Heather Billings]

Hopefully, the changes we're proposing will only make Tarbell easier for beginners to use. We are hoping to release a lightweight GUI that will help new users set up and configure their projects without the command line if they want. But we are trying to do this in such a way that the current workflow remains and is undisturbed. :)

To view this discussion on the web visit https://groups.google.com/d/msgid/tarbell-project/CAMbNd-L7VzFO2O50ZP9CzJWNsXH%3DW5h1cpSOGYAe%2ByjPzKaKgA%40mail.gmail.com.

[David Eads]

To reiterate what Heather said, the big initiatives NWU are working on involved better project administration via a GUI interface as well as a graphical installer. I've learned a lot implementing the new Authomatic library at NPR and will be integrating it into Tarbell which should also significantly reduce the friction of setting up the Oauth bits.

To view this discussion on the web visit https://groups.google.com/d/msgid/tarbell-project/CAMFjkAO6HMk05LmGEp3fbGgzvix%3Dw%2Bgo-twGhwHw8bbVcoTmbw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

David Eads | http://recoveredfactory.net

"Medical statistics will be our standard of measurement: we will weigh life for life and see where the dead lie thicker, among the workers or among the privileged." -- Rudolf Virchow

[David Eads]

Whoops, hit send too soon.

The #1 priority is making Tarbell easier to install and use, especially for people who are new to web production.