Summary Thursday (1/19) Developer Call
Damon Regan
## Individual Developer Status
Walt
1. Dynamo DB review (from Amazon) for data services
2. Closing pull requests and associated issues
3. ADL support on other internal development activities
Lou
1. Linux install guide updates
2. Wiki page for all the AMI instructions and documents
3. Will be meeting with NARA about direct publishing
4. CPALMS is ready
Joe H.
1. Playing with the VM node (see related stale view email)
2. Playing with how to get suggestions based on standards alignment
data. Prototype next week to show.
3. Watching new developer and plan to document observations
John P.
1. Finished publish multiple versions
2. Windows monitor to work
3. Distribute updates based on conversations with Dan
Jim
1. Reviewed celery and the repos stuff. Got frustrated with implementing it.
2. Began looking at what amazon simple db. It is basically hadoop
infrastructure.
3. Made a java view server for couch (hadoop is based on Java).
4. Plan to use Joe's code to do signature validation in Java.
Suraiya
1. Yesterday reviewed bootstrapping and had conversations. Heard
about the proposal to split the database into distributable and
non-distributable databases. Jim mentioned issues with watermarks.
Suraiya plans to call Jim to understand watermark issue and approach
to resolve. Pickup the database from s3 server. Filtering to reduce
dataset afterwards.
Alan
1. Getting the triggers for Jenkins deployment working. Need to setup
OAuth authentication for GitHub.
2. Have an idea of what is necessary to get email notifications
working. Need to open ports. Working to get notifications working.
Austin
1. Announced the LR .NET library
2. Working on a GUI app using GTK# to publish documents. Updating
.NET library to sign using X509 certificate.
## Discussion Summary
### Status Discussion
Walt will do split the resource_data into distributable and
non-distributable databases
### GitHub Pull Requests
#170 - Obtain returns invalid json if documents have empty resource
locators. Walt is working on a fix.
### Discussion of Joe's "stale views" email and data services
https://groups.google.com/group/learningreg-dev/browse_thread/thread/c532d06071a4a415
1. Config files from the plugfest VM are likely the culprit. Updated
configuration are in a gist linked in the document.
2. On the hourly update is not on the VM. It is only on the
production nodes. We need documentation for setting up cron jobs to
update views.
3. There is a known issue The node doesn't crash, the error only
appears at startup can't make route for xyz. nosetests all work,
there are just specific issues. If I publish more than 100 records, I
should see the new records, but I don't. The learningregistry uwsgi
log configuration should fix this. production.ini the gist contains
complete contents of the file. Joe will test and update the email
list.
4. To get the most updated code, you could do a git pull. How to
needs to be documented in a FAQ. Also need to update the config.
Always safe to go through the config (setup_node.py). These two
things need to be documented in the release notes. In the config
directory there is a script called couch_utils which is used by
setup_node.py. If you run it standalone, it asks for url and it does
everything else. It pushes all the design docs but it doesn't change
the configuration.
5. Data services won't necessarily solve the issue of getting the data
you need (title, related objects). Paradata doesn't have a title.
Joe's approach is to add this information to the envelope on ingest.
Hope of data services is to be able to do finer filtering. This is
what prompted Jim to enable views to be written in Java to pull titles
out of metadata or go hit an HTML document to pull the title from the
web page. Views that may provide different results at different times
can be written. e.g., resource locator points to wikipedia. Today it
is blacking out everything. So titles can't be retrieved. But
tomorrow, the titles can be retrieved. This is not what the couch
community wants.
Zope workflow was frustrating because it lacked good documentation.
Amazon / Hadoop is on hold.
Jim to propose shutting down development nodes to lower AWS costs.