User documentation/workflow guides?

[Steve Kersley]

Firstly, thanks for creating what looks like a very useful system.  I can see many uses for it here, but initially we're hoping to try it out for:

* maintaining sets of documents for committee meetings (agendas, minutes and submitted papers, which may come as electronic or paper documents scanned in)

* a digital archive of electronic files with metadata for searching and indexing.

However, while I would install/maintain the system, I would not be the main user of it.  The documentation is generally good for installing and configuring, but is there any documentation intended for the end user, as to what the workflow is?

i.e., I just feel I'm not understanding the process of creating metadata, metadata sets, indexes, folders etc.  What order should these be done, and before or after importing files?  Are there any other guides on how to actually use the system day-to-day?  I'm happy to help to create such a guide if necessary based on our experiences, but would first need to figure it out!

As for the installation documents - like I say these were generally good, there were some areas where things were maybe missing or maybe not up-to-date, and only searching on the google groups found some information that with hindsight was obvious but that I'd clearly missed.

In particular (and this is intended as constructive, not negative):

The 'requirements' page doesn't make any note that LibreOffice is required, but it seems to be in order to convert Office files.  Pretty obvious now I look back, but as the page lists other software such as Tesseract, ImageMagick and so on as being required/optional, perhaps it needs to list it?

No mention in the install step-by-step guide on creating the document-storage folder, or setting permissions on either that or on image-cache so that the www-data (for debian/ubuntu) user that apache runs as can write to those folders.

Also still having some issues that I've not been able to resolve but will create another post on that!

Perhaps the fabric installer does take care of these, but I was following the step-by-step instructions?

Again, please don't anyone take these comments as critical - the intent is to help by suggesting improvements or omissions.

Cheers,

Steve.

[Roberto Rosario]

Hi Steve,

Thanks for trying out Mayan and for the criticism it is very much welcomed!  Your use scenario is very well into what Mayan was meant to do, so it should work well for your needs.

Oh yes documentation, I don't hide the facts that: it is lacking and that I'm very bad at it :)  I take some time before each release to work of documentation, but I'm very slow at it and manage to improve only slightly that it is way behind the capabilities of the current code base.  And true also, the documentation is mainly intended to the installer and administrator, not so much at the end user.  This is an area I could really use the community's help, so thanks for anything you can contribute.

• Metadata type - it is a definition of a specific kind data that relates to documents and the user doing the setup defines them, they are intended to be filled out by users during document upload, but user are not meant to define/create new metadata types, just enter data about them regarding to the document being uploaded: import date, approval date, cabinet number, client address.
• Metadata sets - are a group unit of metadata types.  During setup you may decide that legal documents should have specific metadata: upload date, client name, case number, and so on, so you create a metadata set called 'legal metaset' and include those in the set.  When an user is going to upload a legal document just selecting the 'legal metaset' is the same as selecting all the metadata type specified before.  Why selecting the metadata or metadataset is important?  It tells Mayan what to ask for in the metadata value data entry step during document uploads.  Which brings us to:
• Document types - When your organization has just a few kind of document or if the metadata for those document is not clearly established selecting the metadata by hand on every upload might make sense, but when your metadata requirement are very well defined or you have many kinds of document types or classes, setting which metadata are required by what document type is very handy.  During upload the user selects the document type and on the next step the required metadata types or set are already preselected so the user just presses 'Next' and goes to the metadata values data entry form.
• Indexes - Most existing document management software are good at importing and exporting documents, but very few are good at keeping those documents organized.  When you have tens or hundreds of documents, organizing them into hierarchical units by hand is not much of a chore, but when you have more than a few thousand documents, organizing them by hand is not even possible any more.  To tackle this on Mayan, you create a skeleton hierarchical structure (an index template) and tell it how documents are to be organized depending on their metadata.  As documents are uploaded, deleted or their metadata updated, Mayan fills out your skeleton index template and gives you a populated tree with document links and how they related to each other in terms of organizational/hierarchical structure.  You then give read only access to your users to the indexes and they can navigate the documents contained with the proper contextual knowledge of how they related to each other.
  
• Folders - Indexes are defined once and then they are handled automatically for you.  Folders are the opposite, they are manual organizational units, that you or your users can create and add what ever documents they want in them.
• Smart links - Imagine you have two directory trees on your computer: one for medical patients organized by alphabetical order and another directory tree for chemical medicine information organized by vendors.  On Mayan a good way to duplicate that is by using two indexes: one for patients and one for medicines, and let the software fill out the tree based of the patients name metadata and the medicine vendor name metadata.  Now suppose you prescribe patient A a certain medicine, say medicine X, it would be very beneficial to link medicine X to the documents of patient A for quick reference, but without altering the hierarchical structure since a medicine document has no place in the patient index, this is what smart links do.  Smart links allow you to links/connect documents based on a pre programmed relevance logic but without affecting they respective organizational structures.

If branching is the killer feature of Git, indexes and smart links are the killer features of Mayan.  They are powerful but a bit hard to understand at first.

On the topic of deployments, since there are so many ways to deploy Django projects and many schools of thought about how to do it, I tend to avoid siding with any particular deployment strategy only implementing Ubuntu+Apache+MySQL as an initial strategy on the fabfile (which as you are pondering does takes care as much as possible of it) because it is the one I know best.  If users could provide simple or if complex but well explained and duplicable deployment setups for Mayan I would devote an entire chapter to them.

--Roberto

[Lachlan Musicman]

On Thu, Dec 13, 2012 at 2:28 PM, Roberto Rosario
<roberto.rosa...@gmail.com> wrote:

> On the topic of deployments, since there are so many ways to deploy Django
> projects and many schools of thought about how to do it, I tend to avoid
> siding with any particular deployment strategy only implementing
> Ubuntu+Apache+MySQL as an initial strategy on the fabfile (which as you are
> pondering does takes care as much as possible of it) because it is the one I
> know best. If users could provide simple or if complex but well explained
> and duplicable deployment setups for Mayan I would devote an entire chapter
> to them.

I think that the fabfile method is the best for deployment of Mayan.

It's not the method I'd choose, but the method I've chosen means I
still don't have a working install.

The reality is that Mayan is so large and has so many dependencies
that keeping up is difficult. With the fabfile deployment you are
getting a working snapshot that can be up and running with a page of
instructions - this is fantastic.

When I say keeping up is difficult, take a look at the current stable
git branch - Django is behind in both major and minor version numbers;
South is one minor version behind, djangorestframework is
significantly behind; Pillow is a couple of minor versions behind; etc
etc.

This is not a criticism - I'm sure that Mayan is great (I can't wait
to have it up and running to test), but with so many dependencies it's
hard to keep them up to date. And it means that a generic installation
of each dependency individually results in lots of wrong versions and
no obvious way forward.

At the moment I'm getting "No module named djangorestframework"
errors, which I can only presume is because Mayan expects v 0.2.3 and
I have v 2.1.8

I would recommend the fabfile as the only sensible deployment strategy.

cheers
L.

--
...we look at the present day through a rear-view mirror. This is
something Marshall McLuhan said back in the Sixties, when the world
was in the grip of authentic-seeming future narratives. He said, “We
look at the present through a rear-view mirror. We march backwards
into the future.”

http://www.warrenellis.com/?p=14314

[Roberto Rosario]

Hi Lachian,

A requirements/production.txt file is included so you can get the correct versions installed very easily by issuing:

pip -r requirements/production.txt

I always recommend against installing the Python requirements by hand because you can end up very easily in dependency hell :) (http://en.wikipedia.org/wiki/Dependency_hell).

--Roberto

[Lachlan Musicman]

Thanks Roberto - yeah, I groked that once I started the process. I'm
so used to building django projects by hand that I forgot that
sometimes it's better to forget that Django is behind it and just
think of Mayan as a web delivered service

Cheers
L.

On Thu, Dec 13, 2012 at 3:58 PM, Roberto Rosario

> --