[rfdoc] 4 new revisions pushed by janne.piironen@gmail.com on 2014-03-14 08:07 GMT
1 view
Skip to first unread message
codesite...@google.com
Mar 14, 2014, 4:10:28 AM3/14/14
to rfdoc-...@googlegroups.com
4 new revisions:
Revision: ea4bda47b167
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Tue Feb 25 11:54:12 2014 UTC
Log: Added possibility to filter searches by version...
http://code.google.com/p/rfdoc/source/detail?r=ea4bda47b167
Revision: facd8d649159
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Thu Feb 27 12:11:32 2014 UTC
Log: Added version override possibility to web upload....
http://code.google.com/p/rfdoc/source/detail?r=facd8d649159
Revision: d3cc4a1276cb
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 09:07:25 2014 UTC
Log: Modified invalid file upload messages to contain more
information....
http://code.google.com/p/rfdoc/source/detail?r=d3cc4a1276cb
Revision: 994a7a77bc0b
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 11:01:17 2014 UTC
Log: Moved uploader.py to rfdoc.upload...
http://code.google.com/p/rfdoc/source/detail?r=994a7a77bc0b
==============================================================================
Revision: ea4bda47b167
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Tue Feb 25 11:54:12 2014 UTC
Log: Added possibility to filter searches by version
Update issue 24
Since it is now possible to store multiple version of libraries, a version
filter was implemented for search. The filter is non required and works with
wildcards ? (single character) and * (multiple characters).
http://code.google.com/p/rfdoc/source/detail?r=ea4bda47b167
Modified:
/atest/resources/search.txt
/atest/search_functionality.txt
/atest/search_interface.txt
/src/rfdoc/rfdocapp/templates/search.html
/src/rfdoc/rfdocapp/templates/search_form.html
/src/rfdoc/rfdocapp/views/search.py
=======================================
--- /atest/resources/search.txt Thu Feb 20 09:43:29 2014 UTC
+++ /atest/resources/search.txt Tue Feb 25 11:54:12 2014 UTC
@@ -18,6 +18,10 @@
Unselect Checkbox case_insensitive
User searches with "${search term}" without documentation
+User searches "${search term}" with version "${search version}"
+ Input text search_version ${search version}
+ User searches with "${search term}"
+
"${keywords}" keywords are displayed
@{keywords} = Get names ${keywords}
:FOR ${name} IN @{keywords}
@@ -26,7 +30,8 @@
Search Result Should Contain ${name}
Page should contain element //td/a[text()='${name}']
-Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" and with documentation
- Page Should Contain Element
//td/a[@href='/lib/SeleniumLibrary#GetTitle' and text()='Get Title']
- Page Should Contain Element //td/a[text()='SeleniumLibrary']
+Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" with version and with documentation
+ Page Should Contain Element
//td/a[@href='/lib/SeleniumLibrary/2.2#GetTitle' and text()='Get Title']
+ Page Should Contain Element //td/a[@href='/lib/SeleniumLibrary/2.2'
and text()='SeleniumLibrary']
+ Page Should Contain Element //td[text()='2.2']
Page Should Contain Returns title of current page.
=======================================
--- /atest/search_functionality.txt Tue Oct 15 14:41:52 2013 UTC
+++ /atest/search_functionality.txt Tue Feb 25 11:54:12 2014 UTC
@@ -1,6 +1,6 @@
*** Settings ***
Resource resources/search.txt
-Suite Setup Given SeleniumLibrary exists in RFDoc
+Suite Setup Given SeleniumLibrary ExampleLibrary 1 and ExampleLibrary
2 exist in RFDoc
*** Test Cases ***
User Can Search Using Keyword Name And Documentation
@@ -27,6 +27,12 @@
Then "Close All Browsers" keywords are displayed
and notification "1 matching keyword found." is shown
+User Can Search Using Keyword Name And Version Info
+ "search" page is open
+ When user searches "First" with version "1"
+ Then "First Keyword" keywords are displayed
+ and notification "1 matching keyword found." is shown
+
Message Is Displayed When Search Matches No Keywords
"search" page is open
When user searches with "non-matching"
=======================================
--- /atest/search_interface.txt Tue Oct 15 14:41:52 2013 UTC
+++ /atest/search_interface.txt Tue Feb 25 11:54:12 2014 UTC
@@ -18,4 +18,4 @@
Search Results Contain Relevant Information
"search" page is open
And user searches with "title"
- Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" and with documentation
+ Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" with version and with documentation
=======================================
--- /src/rfdoc/rfdocapp/templates/search.html Thu Feb 20 09:43:29 2014 UTC
+++ /src/rfdoc/rfdocapp/templates/search.html Tue Feb 25 11:54:12 2014 UTC
@@ -10,10 +10,17 @@
{% if not kws %}No{% else %}{{ kws|length }}{% endif %} matching
keyword{{ kws|length|pluralize }} found.
</p>{% if kws %}
<table class="keywords results">
+ <tr>
+ <th>Keyword</th>
+ <th>Library</th>
+ <th>Version</th>
+ <th>Doc</th>
+ </tr>
{% for kw in kws %}
<tr>
- <td><a href="{% url 'library' kw.library.name|urlencode %}#{{
kw.name|nospaces }}">{{ kw.name }}</a></td>
- <td><a href="{% url 'library' kw.library.name|urlencode %}">{{
kw.library.name }}</a></td>
+ <td><a href="{% url 'version' kw.library.name|urlencode
kw.library.version|urlencode %}#{{ kw.name|nospaces }}">{{ kw.name
}}</a></td>
+ <td><a href="{% url 'version' kw.library.name|urlencode
kw.library.version|urlencode %}">{{ kw.library.name }}</a></td>
+ <td>{{ kw.library.version }}</td>
<td>{{ kw.doc|first_line }}</td>
</tr>
{% endfor %}
=======================================
--- /src/rfdoc/rfdocapp/templates/search_form.html Fri Dec 27 12:10:40 2013
UTC
+++ /src/rfdoc/rfdocapp/templates/search_form.html Tue Feb 25 11:54:12 2014
UTC
@@ -2,6 +2,7 @@
<form enctype="multipart/form-data" action="{% url 'search' %}"
method="POST">
{{ form.search_term.errors }}<table id="search">
<tr>
+ <td><label for="id_search_term">Search term</label></td>
<td>{{ form.search_term }}</td>
<td>
{{ form.include_doc }}
@@ -11,6 +12,13 @@
{{ form.case_insensitive }}
<label for="id_case_insensitive">Case-insensitive</label>
</td>
+ </tr>
+ <tr>
+ <td><label for="id_search_version">Filter version</label></td>
+ <td>{{ form.search_version }}</td>
+ <td></td>
+ <td></td>
+ </tr>
</table>
<p>
<input type="submit" name="search_button" value="Search" />
=======================================
--- /src/rfdoc/rfdocapp/views/search.py Wed Oct 16 15:50:40 2013 UTC
+++ /src/rfdoc/rfdocapp/views/search.py Tue Feb 25 11:54:12 2014 UTC
@@ -27,6 +27,7 @@
form = SearchForm(request.POST)
if form.is_valid():
term = form.cleaned_data['search_term']
+ version = form.cleaned_data['search_version']
query = Q(name__icontains = term)
if form.cleaned_data['include_doc']:
query = query | Q(doc__icontains = term)
@@ -39,6 +40,9 @@
if form.cleaned_data['include_doc']:
query = query | Q(doc__regex = r'.*%s.*' %
re.escape(term))
kws = Keyword.objects.filter(query)
+ if version:
+ version =
re.escape(version).replace('\?','.').replace('\*','.*')
+ kws = kws.filter(library__version__regex=r'^%s$' % version)
search_performed = True
else:
form = SearchForm()
@@ -52,6 +56,7 @@
class SearchForm(forms.Form):
search_term = forms.CharField()
+ search_version = forms.CharField(required=False)
include_doc = forms.BooleanField(required=False, initial=True)
case_insensitive = forms.BooleanField(required=False, initial=True)
==============================================================================
Revision: facd8d649159
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Thu Feb 27 12:11:32 2014 UTC
Log: Added version override possibility to web upload.
Update issue 25
Now it is possible to overwrite the library version when uploading from the
web UI.
http://code.google.com/p/rfdoc/source/detail?r=facd8d649159
Modified:
/atest/resources/rfdoc.txt
/atest/upload_documentation.txt
/src/rfdoc/rfdocapp/static/default.css
/src/rfdoc/rfdocapp/templates/upload.html
/src/rfdoc/rfdocapp/views/upload.py
=======================================
--- /atest/resources/rfdoc.txt Wed Feb 19 12:21:01 2014 UTC
+++ /atest/resources/rfdoc.txt Thu Feb 27 12:11:32 2014 UTC
@@ -13,7 +13,7 @@
@{BUILTIN KEYWORDS} Convert To Integer item Converts the given item to
an integer number. No Operation ${EMPTY} Does absolutely nothing.
@{EXAMPLELIBRARY VERSION 1 KEYWORDS} First Keyword first,
second=default, *args First KW doc My Second Keyword some Second KW doc
@{EXAMPLELIBRARY VERSION 2 KEYWORDS} First Keyword Updated first,
second=default updated, *args Updated KW doc My Second Keyword some
Second KW doc Third Keyword third Third KW doc
-
+@{EXAMPLELIBRARY VERSION 5 KEYWORDS} @{EXAMPLELIBRARY VERSION 1 KEYWORDS}
*** Keywords ***
Given no libraries exist in RFDoc
@@ -44,6 +44,9 @@
Notification contains a link to ${library}
Page should contain link /lib/${library} ${library}
+
+Notification contains a version "${version}" link to "${library}"
+ Page should contain link /lib/${library}/${version} ${library}
Navigate to library "${name}"
Go to ${BASE URL}/lib/${name}
@@ -56,6 +59,11 @@
User uploads ${library}
${file} = Set Variable ${library.replace(' ', '_')}.xml
Upload ${file}
+
+User overrides version with "${version}" and uploads "${library}"
+ ${file} = Set Variable ${library.replace(' ', '_')}.xml
+ Input text override_version ${version}
+ Upload ${file}
Upload ${file}
Choose File file ${TESTDATA PATH}${/}xmls${/}${file}
=======================================
--- /atest/upload_documentation.txt Wed Feb 19 12:21:01 2014 UTC
+++ /atest/upload_documentation.txt Thu Feb 27 12:11:32 2014 UTC
@@ -1,11 +1,18 @@
*** Settings ***
Resource resources/rfdoc.txt
+Test Setup Given no libraries exist in RFDoc
*** Test Cases ***
User Can Upload Library Documentation
- [Setup] Given no libraries exist in RFDoc
"upload" page is open
When user uploads BuiltIn
Then notification "Successfully uploaded library BuiltIn" is shown
- and notification contains a link to BuiltIn
+ and notification contains a version "2.1" link to "BuiltIn"
and RFDoc contains library BuiltIn
+
+User Can Upload Library Documentation And Override Version
+ "upload" page is open
+ When user overrides version with "5" and uploads "ExampleLibrary
version 1"
+ Then notification "Successfully uploaded library ExampleLibrary" is
shown
+ and notification contains a version "5" link to "ExampleLibrary"
+ and RFDoc contains versioned library ExampleLibrary version 5
=======================================
--- /src/rfdoc/rfdocapp/static/default.css Wed Feb 19 12:21:01 2014 UTC
+++ /src/rfdoc/rfdocapp/static/default.css Thu Feb 27 12:11:32 2014 UTC
@@ -127,6 +127,10 @@
list-style-type: none;
color: red;
}
+
+.form-options {
+ border: 1px dashed #eee;
+}
/* Search */
table#search {
=======================================
--- /src/rfdoc/rfdocapp/templates/upload.html Fri Dec 27 12:10:40 2013 UTC
+++ /src/rfdoc/rfdocapp/templates/upload.html Thu Feb 27 12:11:32 2014 UTC
@@ -12,16 +12,24 @@
<div class="hr"><hr /></div>
- <form enctype="multipart/form-data" action="{% url 'upload' %}"
method="POST">{% if libname %}
+ <form enctype="multipart/form-data" action="{% url 'upload' %}"
method="POST">{% if lib %}
<div class="success">
- Successfully uploaded library <a href="{% url 'library'
libname %}">{{ libname }}</a>.
+ Successfully uploaded library <a href="{% url 'version' lib.name|
urlencode lib.version|urlencode %}">{{ lib.name }}</a>.
</div>{% endif %}{{ form.file.errors }}
<div>
{{ form.file }}
</div>
- <div>
- {{ form.override }}
- <label for="id_override">Overwrite existing documentation with
same name</label>
+ <div class="form-options">
+ <table>
+ <tr>
+ <td><label for="id_override_version">Overwrite library
version with:</label></td>
+ <td>{{ form.override_version }}</td>
+ </tr>
+ <tr>
+ <td><label for="id_override">Overwrite existing
documentation with same name</label></td>
+ <td>{{ form.override }}</td>
+ </tr>
+ </table>
</div>
<p>
<input type="submit" value="Upload" />
=======================================
--- /src/rfdoc/rfdocapp/views/upload.py Wed Feb 19 12:21:01 2014 UTC
+++ /src/rfdoc/rfdocapp/views/upload.py Thu Feb 27 12:11:32 2014 UTC
@@ -21,17 +21,18 @@
def upload(request):
- libname = None
+ lib = None
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
- libname = form.parse_kw_spec(request.FILES['file'],
- form.cleaned_data['override'])
+ lib = form.parse_kw_spec(request.FILES['file'],
+ form.cleaned_data['override'],
+
form.cleaned_data['override_version'].strip())
else:
form = UploadFileForm()
return render_to_response('upload.html', {
'form': form,
- 'libname': libname
+ 'lib': lib
}
)
@@ -40,10 +41,14 @@
file = forms.FileField()
file.widget.attrs['size'] = 40
override = forms.BooleanField(required=False)
+ override_version = forms.CharField(required=False)
+ override_version.widget.attrs['size'] = 10
- def parse_kw_spec(self, fileobj, override):
+ def parse_kw_spec(self, fileobj, override, override_version):
try:
libdata = LibraryData(fileobj)
+ if override_version:
+ libdata.version=override_version
if
Library.objects.filter(name=libdata.name).filter(version=libdata.version):
if not override:
raise InvalidXmlError("Library %s version %s already
exists." % (libdata.name, libdata.version))
@@ -58,7 +63,7 @@
except InvalidXmlError, err:
self._errors['file'] = ErrorList([str(err)])
return None
- return lib.name
+ return lib
class LibraryData(object):
==============================================================================
Revision: d3cc4a1276cb
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 09:07:25 2014 UTC
Log: Modified invalid file upload messages to contain more information.
Update issue 22
All invalid file upload messages now contain name of the violating file
and some additional information on the (parsing) exception.
http://code.google.com/p/rfdoc/source/detail?r=d3cc4a1276cb
Modified:
/atest/invalid_file_upload.txt
/atest/uploader.txt
/src/rfdoc/rfdocapp/views/upload.py
/tools/uploader.py
=======================================
--- /atest/invalid_file_upload.txt Tue Oct 15 14:24:32 2013 UTC
+++ /atest/invalid_file_upload.txt Fri Feb 28 09:07:25 2014 UTC
@@ -3,24 +3,21 @@
Test Setup "Upload" page is open
*** Variables ***
-${NON XML FILE} Given file is not XML.
-${INVALID XML FILE} Given file contains invalid XML.
${NON EXISTING FILE} The submitted file is empty.
-${MISSING KEYWORDS} Given test library contains no keywords.
*** Test Cases *** *** Uploaded File *** ***
Error Message ***
-Non XML file Uploading fails text_file.txt ${NON
XML FILE}
-Wrong XML content Uploading fails invalid_content.xml
${INVALID XML FILE}
+Non XML file Uploading fails text_file.txt Given
file text_file.txt is not XML: syntax error: line 1, column 0
+Wrong XML content Uploading fails invalid_content.xml Given
file invalid_content.xml contains invalid XML: Root tag must be keywordspec
Non existing file Uploading fails non_existing_file.xml ${NON
EXISTING FILE}
-Empty library name Uploading fails empty_library_name.xml
${INVALID XML FILE}
-Missing library name Uploading fails missing_library_name.xml
${INVALID XML FILE}
-Missing library doc Uploading fails missing_library_doc.xml
${INVALID XML FILE}
-Empty keyword name Uploading fails empty_keyword_name.xml
${INVALID XML FILE}
-Missing keyword name Uploading fails missing_keyword_name.xml
${INVALID XML FILE}
-Missing keyword doc Uploading fails missing_keyword_doc.xml
${INVALID XML FILE}
-Empty keyword Uploading fails empty_keyword.xml
${INVALID XML FILE}
-Empty keywords Uploading fails empty_keywords.xml
${MISSING KEYWORDS}
-Missing keywords Uploading fails missing_keywords.xml
${MISSING KEYWORDS}
+Empty library name Uploading fails empty_library_name.xml
Library parsing error. Given file empty_library_name.xml contains invalid
XML: Attribute name not found
+Missing library name Uploading fails missing_library_name.xml
Library parsing error. Given file missing_library_name.xml contains invalid
XML: Attribute name not found
+Missing library doc Uploading fails missing_library_doc.xml
Library parsing error. Given file missing_library_doc.xml contains invalid
XML: Child element doc not found
+Empty keyword name Uploading fails empty_keyword_name.xml
Keyword parsing error. Given file empty_keyword_name.xml contains invalid
XML: Attribute name not found
+Missing keyword name Uploading fails missing_keyword_name.xml
Keyword parsing error. Given file missing_keyword_name.xml contains invalid
XML: Attribute name not found
+Missing keyword doc Uploading fails missing_keyword_doc.xml
Keyword parsing error. Given file missing_keyword_doc.xml contains invalid
XML: Child element doc not found
+Empty keyword Uploading fails empty_keyword.xml
Keyword parsing error. Given file empty_keyword.xml contains invalid XML:
Child element doc not found
+Empty keywords Uploading fails empty_keywords.xml Given
test library file empty_keywords.xml contains no keywords.
+Missing keywords Uploading fails missing_keywords.xml Given
test library file missing_keywords.xml contains no keywords.
*** Keywords ***
Uploading fails [Arguments] ${file} ${message}
=======================================
--- /atest/uploader.txt Mon Feb 24 10:15:07 2014 UTC
+++ /atest/uploader.txt Fri Feb 28 09:07:25 2014 UTC
@@ -132,4 +132,4 @@
Should Contain ${output} Try '${SCRIPT_NAME} --help' for more
information
Should Print ${host} Is Invalid Host
- Should Contain ${output} ${SCRIPT_NAME}: error: connection refused
to '${host}', check that the host responds and is reachable.
+ Should Contain ${output} ${SCRIPT_NAME}: Remote error: connection
refused to '${host}', check that the host responds and is reachable.
=======================================
--- /src/rfdoc/rfdocapp/views/upload.py Thu Feb 27 12:11:32 2014 UTC
+++ /src/rfdoc/rfdocapp/views/upload.py Fri Feb 28 09:07:25 2014 UTC
@@ -69,23 +69,24 @@
class LibraryData(object):
def __init__(self, fileobj):
+ self.filename = fileobj.name
root = self._get_root(fileobj)
try:
self.name = self._get_name(root)
self.version = self._get_version(root)
self.doc = self._get_doc(root)
- except InvalidXmlError:
- raise InvalidXmlError('Given file contains invalid XML.')
- self.inits = [ InitData(data) for data in self._get_inits(root) ]
- self.kws = [ KeywordData(data) for data in
self._get_keywords(root) ]
+ except InvalidXmlError as e:
+ raise InvalidXmlError('Library parsing error. Given file %s
contains invalid XML: %s' % (self.filename, e))
+ self.inits = [ InitData(data, self.filename) for data in
self._get_inits(root) ]
+ self.kws = [ KeywordData(data, self.filename) for data in
self._get_keywords(root) ]
def _get_root(self, fileobj):
try:
root = ET.parse(fileobj).getroot()
- except SyntaxError:
- raise InvalidXmlError('Given file is not XML.')
+ except SyntaxError as e:
+ raise InvalidXmlError('Given file %s is not XML: %s' %
(self.filename, e))
if root.tag != 'keywordspec':
- raise InvalidXmlError('Given file contains invalid XML.')
+ raise InvalidXmlError('Given file %s contains invalid XML:
Root tag must be keywordspec' % self.filename)
return root
def _get_name(self, elem):
@@ -111,19 +112,19 @@
# 'keywords/kw' is backwards compatibility for libdoc.py 2.1 and
earlier
kws = elem.findall('keywords/kw') + elem.findall('kw')
if not kws:
- raise InvalidXmlError('Given test library contains no
keywords.')
+ raise InvalidXmlError('Given test library file %s contains no
keywords.' % self.filename)
return kws
class KeywordData(object):
- def __init__(self, elem):
+ def __init__(self, elem, filename):
try:
self.name = self._get_name(elem)
self.doc = self._get_doc(elem)
self.args = ', '.join(arg.text for arg in self._get_args(elem))
- except InvalidXmlError:
- raise InvalidXmlError('Given file contains invalid XML.')
+ except InvalidXmlError as e:
+ raise InvalidXmlError('Keyword parsing error. Given file %s
contains invalid XML: %s' % (filename, e))
def _get_name(self, elem):
return get_attr(elem, 'name')
@@ -144,13 +145,13 @@
def get_attr(elem, attr_name):
attr = elem.get(attr_name)
if not attr:
- raise InvalidXmlError
+ raise InvalidXmlError('Attribute %s not found' % attr_name)
return attr
def get_child_element(elem, child_name):
child = elem.find(child_name)
if child is None:
- raise InvalidXmlError
+ raise InvalidXmlError('Child element %s not found' % child_name)
return child
=======================================
--- /tools/uploader.py Mon Feb 24 10:15:07 2014 UTC
+++ /tools/uploader.py Fri Feb 28 09:07:25 2014 UTC
@@ -68,7 +68,7 @@
finally:
xml_doc.original_close()
except DataError, e:
- sys.stderr.write('%s: error: %s\n' %
(os.path.basename(__file__),
+ sys.stderr.write('%s: Remote error: %s\n' %
(os.path.basename(__file__),
e.message))
exit(1)
==============================================================================
Revision: 994a7a77bc0b
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 11:01:17 2014 UTC
Log: Moved uploader.py to rfdoc.upload
Update issue 23
Uploader is now part of RFDoc in module rfdoc.upload and can be called
like defined in the issue.
http://code.google.com/p/rfdoc/source/detail?r=994a7a77bc0b
Added:
/src/rfdoc/upload.py
Deleted:
/tools/uploader.py
Modified:
/MANIFEST.in
/atest/uploader.txt
/src/rfdoc/settings.py
=======================================
--- /dev/null
+++ /src/rfdoc/upload.py Fri Feb 28 11:01:17 2014 UTC
@@ -0,0 +1,263 @@
+#!/usr/bin/env python
+
+# Copyright 2008-2013 Nokia Siemens Networks Oyj
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import with_statement
+
+import os
+import sys
+
+from contextlib import closing
+from HTMLParser import HTMLParser
+from httplib import HTTPConnection
+from optparse import OptionParser
+from re import match
+from StringIO import StringIO
+from urlparse import urlsplit
+
+from robot.errors import DataError
+from robot.libdocpkg import LibraryDocumentation
+from robot.parsing.populators import READERS
+
+
+class Uploader(object):
+
+ def __init__(self):
+ self._options = CommandlineUI()
+ self._uploader = XmlUploader(self._options.target_url)
+
+ def run(self):
+ try:
+ for library in self._options.libraries:
+ xml_doc = StringIO()
+ # LibraryDocumentation().save() calls close() for the
underlying
+ # file but closing StringIO object discards its data.
+ # This is why close() is overridden below.
+ xml_doc.original_close = xml_doc.close
+ try:
+ try:
+ if library.endswith('.xml'):
+ with open(library) as xml_file:
+ xml_doc.write(xml_file.read())
+ else:
+ xml_doc.close = lambda: None
+ if self._options.lib_version:
+ LibraryDocumentation(library,
version=self._options.lib_version).save(xml_doc, 'xml')
+ else:
+
LibraryDocumentation(library).save(xml_doc, 'xml')
+ except DataError, e:
+ message = "Library not found" if 'ImportError' in
e.message else e.message
+ sys.stderr.write("Skipping '%s' due to an
error: %s.\n" %
+ (library, message))
+ continue
+ xml_doc.name = library
+ self._uploader.upload_file(xml_doc)
+ sys.stdout.write("Updated documentation for '%s'.\n" %
library)
+ finally:
+ xml_doc.original_close()
+ except DataError, e:
+ sys.stderr.write('%s: Remote error: %s\n' %
(os.path.basename(__file__),
+ e.message))
+ exit(1)
+
+
+class ImprovedOptionParser(OptionParser):
+
+ # This prevents newlines from getting discarded from the help text.
+ def format_description(self, formatter):
+ return self.description
+
+ # This adds "try --help for information" message.
+ def error(self, message):
+ progname = os.path.basename(sys.argv[0])
+ sys.stderr.write('%s: error: %s\n'% (progname, message))
+ sys.stderr.write("Try '%s --help' for more information.\n" %
progname)
+ exit(2)
+
+
+class CommandlineUI(object):
+
+ valid_lib_exts = tuple(READERS.keys() + ['py', 'java', 'xml'])
+ default_url = 'localhost:8000'
+ usage_text = 'usage: %prog [options] PATH ...'
+ help_text = """
+This script updates the documentation at Robot Framework RFDoc server.
+
+PATH is one of the following (multiple can be given, separated by a space):
+1) A path to a library source file (e.g. src/libraries/example_lib.py)
+2) A path to a resource file (e.g. atest/resources/utils.txt)
+3) A path to a library XML, generated using LibDoc (e.g.
libdoc/BuiltIn.xml)
+4) A path to a directory, in which case it's recursively traversed for any
of
+ files mentioned in 1-3.
+"""[1:]
+ epilog_text = """
+The script is intended to be used as part of the CI pipeline or as an SCM
+post-change hook to update the documentation in RFDoc automatically.
+"""[1:]
+
+ def __init__(self):
+ self._parser = ImprovedOptionParser(
+ usage=self.usage_text,
+ description=self.help_text,
+ epilog=self.epilog_text
+ )
+ self._add_commandline_options()
+ self._options = self._get_validated_options()
+
+ @property
+ def target_url(self):
+ return self._options.target_url
+
+ @property
+ def libraries(self):
+ return self._options.libraries
+
+ @property
+ def lib_version(self):
+ return self._options.lib_version
+
+ def _add_commandline_options(self):
+ self._parser.add_option(
+ '-u', '--url',
+ dest='target_url',
+ default=self.default_url,
+ help="""Target RFDoc URL to update, e.g. '192.168.1.100:8000'
or
+'my.server.com/rfdoc'. If this option is not given, '%s' is assumed
+as target.""" % self.default_url
+ )
+ self._parser.add_option(
+ '-v', '--version',
+ dest='lib_version',
+ help="""Override library version""")
+
+ def _get_validated_options(self):
+ options, targets = self._parser.parse_args()
+ if len(targets) < 1:
+ self._exit_with_help()
+ options.libraries = self._traverse_targets_for_libraries(targets)
+ options.target_url = self._host_from_url(options.target_url)
+ return options
+
+ def _traverse_targets_for_libraries(self, targets):
+ libraries = targets
+ for directory in self._only_directories(targets):
+ libraries.remove(directory)
+ for path, _, filenames in os.walk(directory):
+ for filename in self._only_library_files(filenames):
+ libraries.append(os.path.join(path, filename))
+ return libraries
+
+ def _only_directories(self, targets):
+ for target in targets:
+ if os.path.isdir(target):
+ yield target
+
+ def _only_library_files(self, filenames):
+ for filename in filenames:
+ if self._is_library_file(filename):
+ yield filename
+
+ def _is_library_file(self, filename):
+ return any(filename.endswith(ext) for ext in self.valid_lib_exts)
+
+ def _host_from_url(self, url):
+ if not match(r'http(s?)\:', url):
+ url = 'http://' + url
+ return url
+
+ def _exit_with_help(self):
+ self._parser.print_help()
+ exit(1)
+
+
+class XmlUploader(object):
+ default_endpoint = 'upload'
+ body_template = """--%(boundary)s
+Content-Disposition: form-data; name="override"
+
+on
+--%(boundary)s
+Content-Disposition: form-data; name="file"; filename="%(filename)s"
+Content-Type: text/xml
+
+%(content)s
+--%(boundary)s--
+"""
+
+ def __init__(self, target_url):
+ self.url = urlsplit(target_url)
+
+ def upload_file(self, xml_doc):
+ with closing(HTTPConnection(self.url.netloc)) as connection:
+ try:
+ response = self._post_multipart(connection, xml_doc)
+ except Exception, message:
+ if 'Connection refused' in message:
+ message = "connection refused to '%s', " %
self.url.netloc
+ message += 'check that the host responds and is
reachable.'
+ raise DataError(message)
+ self._validate_success(response)
+
+ def _post_multipart(self, connection, xml_doc):
+ connection.connect()
+ content_type, body = self._encode_multipart_formdata(xml_doc)
+ headers = {'User-Agent': 'RFDoc uploader', 'Content-Type':
content_type}
+ connection.request('POST', self._get_endpoint(), body, headers)
+ return connection.getresponse()
+
+ def _get_endpoint(self):
+ return self.url.path + '/' + self.default_endpoint
+
+ def _encode_multipart_formdata(self, xml_doc):
+ boundary = '----------ThIs_Is_tHe_bouNdaRY_$'
+ body = self.body_template % {
+ 'boundary': boundary,
+ 'filename': xml_doc.name,
+ 'content': xml_doc.getvalue()
+ }
+ content_type = 'multipart/form-data; boundary=%s' % boundary
+ return content_type, body.replace('\n', '\r\n')
+
+ def _validate_success(self, response):
+ if response.status != 200:
+ raise DataError(response.reason.strip())
+ html = response.read()
+ if 'Successfully uploaded library' not in html:
+ raise DataError('\n'.join(self._ErrorParser(html).errors))
+
+ class _ErrorParser(HTMLParser):
+
+ def __init__(self, html):
+ HTMLParser.__init__(self)
+ self._inside_errors = False
+ self.errors = []
+ self.feed(html)
+ self.close()
+
+ def handle_starttag(self, tag, attributes):
+ if ('class', 'errorlist') in attributes:
+ self._inside_errors = True
+
+ def handle_endtag(self, tag):
+ if tag == 'ul':
+ self._inside_errors = False
+
+ def handle_data(self, data):
+ if self._inside_errors and data.strip():
+ self.errors.append(data)
+
+
+if __name__ == '__main__':
+ Uploader().run()
=======================================
--- /tools/uploader.py Fri Feb 28 09:07:25 2014 UTC
+++ /dev/null
@@ -1,263 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2008-2013 Nokia Siemens Networks Oyj
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from __future__ import with_statement
-
-import os
-import sys
-
-from contextlib import closing
-from HTMLParser import HTMLParser
-from httplib import HTTPConnection
-from optparse import OptionParser
-from re import match
-from StringIO import StringIO
-from urlparse import urlsplit
-
-from robot.errors import DataError
-from robot.libdocpkg import LibraryDocumentation
-from robot.parsing.populators import READERS
-
-
-class Uploader(object):
-
- def __init__(self):
- self._options = CommandlineUI()
- self._uploader = XmlUploader(self._options.target_url)
-
- def run(self):
- try:
- for library in self._options.libraries:
- xml_doc = StringIO()
- # LibraryDocumentation().save() calls close() for the
underlying
- # file but closing StringIO object discards its data.
- # This is why close() is overridden below.
- xml_doc.original_close = xml_doc.close
- try:
- try:
- if library.endswith('.xml'):
- with open(library) as xml_file:
- xml_doc.write(xml_file.read())
- else:
- xml_doc.close = lambda: None
- if self._options.lib_version:
- LibraryDocumentation(library,
version=self._options.lib_version).save(xml_doc, 'xml')
- else:
-
LibraryDocumentation(library).save(xml_doc, 'xml')
- except DataError, e:
- message = "Library not found" if 'ImportError' in
e.message else e.message
- sys.stderr.write("Skipping '%s' due to an
error: %s.\n" %
- (library, message))
- continue
- xml_doc.name = library
- self._uploader.upload_file(xml_doc)
- sys.stdout.write("Updated documentation for '%s'.\n" %
library)
- finally:
- xml_doc.original_close()
- except DataError, e:
- sys.stderr.write('%s: Remote error: %s\n' %
(os.path.basename(__file__),
- e.message))
- exit(1)
-
-
-class ImprovedOptionParser(OptionParser):
-
- # This prevents newlines from getting discarded from the help text.
- def format_description(self, formatter):
- return self.description
-
- # This adds "try --help for information" message.
- def error(self, message):
- progname = os.path.basename(sys.argv[0])
- sys.stderr.write('%s: error: %s\n'% (progname, message))
- sys.stderr.write("Try '%s --help' for more information.\n" %
progname)
- exit(2)
-
-
-class CommandlineUI(object):
-
- valid_lib_exts = tuple(READERS.keys() + ['py', 'java', 'xml'])
- default_url = 'localhost:8000'
- usage_text = 'usage: %prog [options] PATH ...'
- help_text = """
-This script updates the documentation at Robot Framework RFDoc server.
-
-PATH is one of the following (multiple can be given, separated by a space):
-1) A path to a library source file (e.g. src/libraries/example_lib.py)
-2) A path to a resource file (e.g. atest/resources/utils.txt)
-3) A path to a library XML, generated using LibDoc (e.g.
libdoc/BuiltIn.xml)
-4) A path to a directory, in which case it's recursively traversed for any
of
- files mentioned in 1-3.
-"""[1:]
- epilog_text = """
-The script is intended to be used as part of the CI pipeline or as an SCM
-post-change hook to update the documentation in RFDoc automatically.
-"""[1:]
-
- def __init__(self):
- self._parser = ImprovedOptionParser(
- usage=self.usage_text,
- description=self.help_text,
- epilog=self.epilog_text
- )
- self._add_commandline_options()
- self._options = self._get_validated_options()
-
- @property
- def target_url(self):
- return self._options.target_url
-
- @property
- def libraries(self):
- return self._options.libraries
-
- @property
- def lib_version(self):
- return self._options.lib_version
-
- def _add_commandline_options(self):
- self._parser.add_option(
- '-u', '--url',
- dest='target_url',
- default=self.default_url,
- help="""Target RFDoc URL to update, e.g. '192.168.1.100:8000'
or
-'my.server.com/rfdoc'. If this option is not given, '%s' is assumed
-as target.""" % self.default_url
- )
- self._parser.add_option(
- '-v', '--version',
- dest='lib_version',
- help="""Override library version""")
-
- def _get_validated_options(self):
- options, targets = self._parser.parse_args()
- if len(targets) < 1:
- self._exit_with_help()
- options.libraries = self._traverse_targets_for_libraries(targets)
- options.target_url = self._host_from_url(options.target_url)
- return options
-
- def _traverse_targets_for_libraries(self, targets):
- libraries = targets
- for directory in self._only_directories(targets):
- libraries.remove(directory)
- for path, _, filenames in os.walk(directory):
- for filename in self._only_library_files(filenames):
- libraries.append(os.path.join(path, filename))
- return libraries
-
- def _only_directories(self, targets):
- for target in targets:
- if os.path.isdir(target):
- yield target
-
- def _only_library_files(self, filenames):
- for filename in filenames:
- if self._is_library_file(filename):
- yield filename
-
- def _is_library_file(self, filename):
- return any(filename.endswith(ext) for ext in self.valid_lib_exts)
-
- def _host_from_url(self, url):
- if not match(r'http(s?)\:', url):
- url = 'http://' + url
- return url
-
- def _exit_with_help(self):
- self._parser.print_help()
- exit(1)
-
-
-class XmlUploader(object):
- default_endpoint = 'upload'
- body_template = """--%(boundary)s
-Content-Disposition: form-data; name="override"
-
-on
---%(boundary)s
-Content-Disposition: form-data; name="file"; filename="%(filename)s"
-Content-Type: text/xml
-
-%(content)s
---%(boundary)s--
-"""
-
- def __init__(self, target_url):
- self.url = urlsplit(target_url)
-
- def upload_file(self, xml_doc):
- with closing(HTTPConnection(self.url.netloc)) as connection:
- try:
- response = self._post_multipart(connection, xml_doc)
- except Exception, message:
- if 'Connection refused' in message:
- message = "connection refused to '%s', " %
self.url.netloc
- message += 'check that the host responds and is
reachable.'
- raise DataError(message)
- self._validate_success(response)
-
- def _post_multipart(self, connection, xml_doc):
- connection.connect()
- content_type, body = self._encode_multipart_formdata(xml_doc)
- headers = {'User-Agent': 'RFDoc uploader', 'Content-Type':
content_type}
- connection.request('POST', self._get_endpoint(), body, headers)
- return connection.getresponse()
-
- def _get_endpoint(self):
- return self.url.path + '/' + self.default_endpoint
-
- def _encode_multipart_formdata(self, xml_doc):
- boundary = '----------ThIs_Is_tHe_bouNdaRY_$'
- body = self.body_template % {
- 'boundary': boundary,
- 'filename': xml_doc.name,
- 'content': xml_doc.getvalue()
- }
- content_type = 'multipart/form-data; boundary=%s' % boundary
- return content_type, body.replace('\n', '\r\n')
-
- def _validate_success(self, response):
- if response.status != 200:
- raise DataError(response.reason.strip())
- html = response.read()
- if 'Successfully uploaded library' not in html:
- raise DataError('\n'.join(self._ErrorParser(html).errors))
-
- class _ErrorParser(HTMLParser):
-
- def __init__(self, html):
- HTMLParser.__init__(self)
- self._inside_errors = False
- self.errors = []
- self.feed(html)
- self.close()
-
- def handle_starttag(self, tag, attributes):
- if ('class', 'errorlist') in attributes:
- self._inside_errors = True
-
- def handle_endtag(self, tag):
- if tag == 'ul':
- self._inside_errors = False
-
- def handle_data(self, data):
- if self._inside_errors and data.strip():
- self.errors.append(data)
-
-
-if __name__ == '__main__':
- Uploader().run()
=======================================
--- /MANIFEST.in Tue Oct 29 13:22:20 2013 UTC
+++ /MANIFEST.in Fri Feb 28 11:01:17 2014 UTC
@@ -1,5 +1,4 @@
prune atest
-graft tools
include src/rfdoc/*.tmpl
include src/rfdoc/rfdocapp/templates/*.html
include src/rfdoc/rfdocapp/static/*.css
=======================================
--- /atest/uploader.txt Fri Feb 28 09:07:25 2014 UTC
+++ /atest/uploader.txt Fri Feb 28 11:01:17 2014 UTC
@@ -3,8 +3,8 @@
*** Variables ***
${INVALID URL} localhost:1
-${SCRIPT NAME} uploader.py
-${SCRIPT PATH} ${CURDIR}${/}..${/}tools${/}${SCRIPT NAME}
+${SCRIPT NAME} upload.py
+${SCRIPT PATH} ${CURDIR}${/}..${/}src${/}rfdoc${/}${SCRIPT
NAME}
${EXAMPLE LIBRARY SOURCE} ${TESTDATA PATH}${/}sources${/}example_lib.py
${EXAMPLE LIBRARY XML V1} ${TESTDATA
PATH}${/}xmls${/}ExampleLibrary_version_1.xml
${EXAMPLE LIBRARY XML V2} ${TESTDATA
PATH}${/}xmls${/}ExampleLibrary_version_2.xml
=======================================
--- /src/rfdoc/settings.py Mon Dec 23 09:50:35 2013 UTC
+++ /src/rfdoc/settings.py Fri Feb 28 11:01:17 2014 UTC
@@ -57,7 +57,7 @@
'rfdoc.rfdocapp'
)
-# CSRF removed from the defaults due to tools/uploader.py
+# CSRF removed from the defaults due to src/rfdoc/upload.py
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
Revision: ea4bda47b167
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Tue Feb 25 11:54:12 2014 UTC
Log: Added possibility to filter searches by version...
http://code.google.com/p/rfdoc/source/detail?r=ea4bda47b167
Revision: facd8d649159
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Thu Feb 27 12:11:32 2014 UTC
Log: Added version override possibility to web upload....
http://code.google.com/p/rfdoc/source/detail?r=facd8d649159
Revision: d3cc4a1276cb
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 09:07:25 2014 UTC
Log: Modified invalid file upload messages to contain more
information....
http://code.google.com/p/rfdoc/source/detail?r=d3cc4a1276cb
Revision: 994a7a77bc0b
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 11:01:17 2014 UTC
Log: Moved uploader.py to rfdoc.upload...
http://code.google.com/p/rfdoc/source/detail?r=994a7a77bc0b
==============================================================================
Revision: ea4bda47b167
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Tue Feb 25 11:54:12 2014 UTC
Log: Added possibility to filter searches by version
Update issue 24
Since it is now possible to store multiple version of libraries, a version
filter was implemented for search. The filter is non required and works with
wildcards ? (single character) and * (multiple characters).
http://code.google.com/p/rfdoc/source/detail?r=ea4bda47b167
Modified:
/atest/resources/search.txt
/atest/search_functionality.txt
/atest/search_interface.txt
/src/rfdoc/rfdocapp/templates/search.html
/src/rfdoc/rfdocapp/templates/search_form.html
/src/rfdoc/rfdocapp/views/search.py
=======================================
--- /atest/resources/search.txt Thu Feb 20 09:43:29 2014 UTC
+++ /atest/resources/search.txt Tue Feb 25 11:54:12 2014 UTC
@@ -18,6 +18,10 @@
Unselect Checkbox case_insensitive
User searches with "${search term}" without documentation
+User searches "${search term}" with version "${search version}"
+ Input text search_version ${search version}
+ User searches with "${search term}"
+
"${keywords}" keywords are displayed
@{keywords} = Get names ${keywords}
:FOR ${name} IN @{keywords}
@@ -26,7 +30,8 @@
Search Result Should Contain ${name}
Page should contain element //td/a[text()='${name}']
-Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" and with documentation
- Page Should Contain Element
//td/a[@href='/lib/SeleniumLibrary#GetTitle' and text()='Get Title']
- Page Should Contain Element //td/a[text()='SeleniumLibrary']
+Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" with version and with documentation
+ Page Should Contain Element
//td/a[@href='/lib/SeleniumLibrary/2.2#GetTitle' and text()='Get Title']
+ Page Should Contain Element //td/a[@href='/lib/SeleniumLibrary/2.2'
and text()='SeleniumLibrary']
+ Page Should Contain Element //td[text()='2.2']
Page Should Contain Returns title of current page.
=======================================
--- /atest/search_functionality.txt Tue Oct 15 14:41:52 2013 UTC
+++ /atest/search_functionality.txt Tue Feb 25 11:54:12 2014 UTC
@@ -1,6 +1,6 @@
*** Settings ***
Resource resources/search.txt
-Suite Setup Given SeleniumLibrary exists in RFDoc
+Suite Setup Given SeleniumLibrary ExampleLibrary 1 and ExampleLibrary
2 exist in RFDoc
*** Test Cases ***
User Can Search Using Keyword Name And Documentation
@@ -27,6 +27,12 @@
Then "Close All Browsers" keywords are displayed
and notification "1 matching keyword found." is shown
+User Can Search Using Keyword Name And Version Info
+ "search" page is open
+ When user searches "First" with version "1"
+ Then "First Keyword" keywords are displayed
+ and notification "1 matching keyword found." is shown
+
Message Is Displayed When Search Matches No Keywords
"search" page is open
When user searches with "non-matching"
=======================================
--- /atest/search_interface.txt Tue Oct 15 14:41:52 2013 UTC
+++ /atest/search_interface.txt Tue Feb 25 11:54:12 2014 UTC
@@ -18,4 +18,4 @@
Search Results Contain Relevant Information
"search" page is open
And user searches with "title"
- Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" and with documentation
+ Then search results contain keyword "Get Title" with link
to "SeleniumLibrary" with version and with documentation
=======================================
--- /src/rfdoc/rfdocapp/templates/search.html Thu Feb 20 09:43:29 2014 UTC
+++ /src/rfdoc/rfdocapp/templates/search.html Tue Feb 25 11:54:12 2014 UTC
@@ -10,10 +10,17 @@
{% if not kws %}No{% else %}{{ kws|length }}{% endif %} matching
keyword{{ kws|length|pluralize }} found.
</p>{% if kws %}
<table class="keywords results">
+ <tr>
+ <th>Keyword</th>
+ <th>Library</th>
+ <th>Version</th>
+ <th>Doc</th>
+ </tr>
{% for kw in kws %}
<tr>
- <td><a href="{% url 'library' kw.library.name|urlencode %}#{{
kw.name|nospaces }}">{{ kw.name }}</a></td>
- <td><a href="{% url 'library' kw.library.name|urlencode %}">{{
kw.library.name }}</a></td>
+ <td><a href="{% url 'version' kw.library.name|urlencode
kw.library.version|urlencode %}#{{ kw.name|nospaces }}">{{ kw.name
}}</a></td>
+ <td><a href="{% url 'version' kw.library.name|urlencode
kw.library.version|urlencode %}">{{ kw.library.name }}</a></td>
+ <td>{{ kw.library.version }}</td>
<td>{{ kw.doc|first_line }}</td>
</tr>
{% endfor %}
=======================================
--- /src/rfdoc/rfdocapp/templates/search_form.html Fri Dec 27 12:10:40 2013
UTC
+++ /src/rfdoc/rfdocapp/templates/search_form.html Tue Feb 25 11:54:12 2014
UTC
@@ -2,6 +2,7 @@
<form enctype="multipart/form-data" action="{% url 'search' %}"
method="POST">
{{ form.search_term.errors }}<table id="search">
<tr>
+ <td><label for="id_search_term">Search term</label></td>
<td>{{ form.search_term }}</td>
<td>
{{ form.include_doc }}
@@ -11,6 +12,13 @@
{{ form.case_insensitive }}
<label for="id_case_insensitive">Case-insensitive</label>
</td>
+ </tr>
+ <tr>
+ <td><label for="id_search_version">Filter version</label></td>
+ <td>{{ form.search_version }}</td>
+ <td></td>
+ <td></td>
+ </tr>
</table>
<p>
<input type="submit" name="search_button" value="Search" />
=======================================
--- /src/rfdoc/rfdocapp/views/search.py Wed Oct 16 15:50:40 2013 UTC
+++ /src/rfdoc/rfdocapp/views/search.py Tue Feb 25 11:54:12 2014 UTC
@@ -27,6 +27,7 @@
form = SearchForm(request.POST)
if form.is_valid():
term = form.cleaned_data['search_term']
+ version = form.cleaned_data['search_version']
query = Q(name__icontains = term)
if form.cleaned_data['include_doc']:
query = query | Q(doc__icontains = term)
@@ -39,6 +40,9 @@
if form.cleaned_data['include_doc']:
query = query | Q(doc__regex = r'.*%s.*' %
re.escape(term))
kws = Keyword.objects.filter(query)
+ if version:
+ version =
re.escape(version).replace('\?','.').replace('\*','.*')
+ kws = kws.filter(library__version__regex=r'^%s$' % version)
search_performed = True
else:
form = SearchForm()
@@ -52,6 +56,7 @@
class SearchForm(forms.Form):
search_term = forms.CharField()
+ search_version = forms.CharField(required=False)
include_doc = forms.BooleanField(required=False, initial=True)
case_insensitive = forms.BooleanField(required=False, initial=True)
==============================================================================
Revision: facd8d649159
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Thu Feb 27 12:11:32 2014 UTC
Log: Added version override possibility to web upload.
Update issue 25
Now it is possible to overwrite the library version when uploading from the
web UI.
http://code.google.com/p/rfdoc/source/detail?r=facd8d649159
Modified:
/atest/resources/rfdoc.txt
/atest/upload_documentation.txt
/src/rfdoc/rfdocapp/static/default.css
/src/rfdoc/rfdocapp/templates/upload.html
/src/rfdoc/rfdocapp/views/upload.py
=======================================
--- /atest/resources/rfdoc.txt Wed Feb 19 12:21:01 2014 UTC
+++ /atest/resources/rfdoc.txt Thu Feb 27 12:11:32 2014 UTC
@@ -13,7 +13,7 @@
@{BUILTIN KEYWORDS} Convert To Integer item Converts the given item to
an integer number. No Operation ${EMPTY} Does absolutely nothing.
@{EXAMPLELIBRARY VERSION 1 KEYWORDS} First Keyword first,
second=default, *args First KW doc My Second Keyword some Second KW doc
@{EXAMPLELIBRARY VERSION 2 KEYWORDS} First Keyword Updated first,
second=default updated, *args Updated KW doc My Second Keyword some
Second KW doc Third Keyword third Third KW doc
-
+@{EXAMPLELIBRARY VERSION 5 KEYWORDS} @{EXAMPLELIBRARY VERSION 1 KEYWORDS}
*** Keywords ***
Given no libraries exist in RFDoc
@@ -44,6 +44,9 @@
Notification contains a link to ${library}
Page should contain link /lib/${library} ${library}
+
+Notification contains a version "${version}" link to "${library}"
+ Page should contain link /lib/${library}/${version} ${library}
Navigate to library "${name}"
Go to ${BASE URL}/lib/${name}
@@ -56,6 +59,11 @@
User uploads ${library}
${file} = Set Variable ${library.replace(' ', '_')}.xml
Upload ${file}
+
+User overrides version with "${version}" and uploads "${library}"
+ ${file} = Set Variable ${library.replace(' ', '_')}.xml
+ Input text override_version ${version}
+ Upload ${file}
Upload ${file}
Choose File file ${TESTDATA PATH}${/}xmls${/}${file}
=======================================
--- /atest/upload_documentation.txt Wed Feb 19 12:21:01 2014 UTC
+++ /atest/upload_documentation.txt Thu Feb 27 12:11:32 2014 UTC
@@ -1,11 +1,18 @@
*** Settings ***
Resource resources/rfdoc.txt
+Test Setup Given no libraries exist in RFDoc
*** Test Cases ***
User Can Upload Library Documentation
- [Setup] Given no libraries exist in RFDoc
"upload" page is open
When user uploads BuiltIn
Then notification "Successfully uploaded library BuiltIn" is shown
- and notification contains a link to BuiltIn
+ and notification contains a version "2.1" link to "BuiltIn"
and RFDoc contains library BuiltIn
+
+User Can Upload Library Documentation And Override Version
+ "upload" page is open
+ When user overrides version with "5" and uploads "ExampleLibrary
version 1"
+ Then notification "Successfully uploaded library ExampleLibrary" is
shown
+ and notification contains a version "5" link to "ExampleLibrary"
+ and RFDoc contains versioned library ExampleLibrary version 5
=======================================
--- /src/rfdoc/rfdocapp/static/default.css Wed Feb 19 12:21:01 2014 UTC
+++ /src/rfdoc/rfdocapp/static/default.css Thu Feb 27 12:11:32 2014 UTC
@@ -127,6 +127,10 @@
list-style-type: none;
color: red;
}
+
+.form-options {
+ border: 1px dashed #eee;
+}
/* Search */
table#search {
=======================================
--- /src/rfdoc/rfdocapp/templates/upload.html Fri Dec 27 12:10:40 2013 UTC
+++ /src/rfdoc/rfdocapp/templates/upload.html Thu Feb 27 12:11:32 2014 UTC
@@ -12,16 +12,24 @@
<div class="hr"><hr /></div>
- <form enctype="multipart/form-data" action="{% url 'upload' %}"
method="POST">{% if libname %}
+ <form enctype="multipart/form-data" action="{% url 'upload' %}"
method="POST">{% if lib %}
<div class="success">
- Successfully uploaded library <a href="{% url 'library'
libname %}">{{ libname }}</a>.
+ Successfully uploaded library <a href="{% url 'version' lib.name|
urlencode lib.version|urlencode %}">{{ lib.name }}</a>.
</div>{% endif %}{{ form.file.errors }}
<div>
{{ form.file }}
</div>
- <div>
- {{ form.override }}
- <label for="id_override">Overwrite existing documentation with
same name</label>
+ <div class="form-options">
+ <table>
+ <tr>
+ <td><label for="id_override_version">Overwrite library
version with:</label></td>
+ <td>{{ form.override_version }}</td>
+ </tr>
+ <tr>
+ <td><label for="id_override">Overwrite existing
documentation with same name</label></td>
+ <td>{{ form.override }}</td>
+ </tr>
+ </table>
</div>
<p>
<input type="submit" value="Upload" />
=======================================
--- /src/rfdoc/rfdocapp/views/upload.py Wed Feb 19 12:21:01 2014 UTC
+++ /src/rfdoc/rfdocapp/views/upload.py Thu Feb 27 12:11:32 2014 UTC
@@ -21,17 +21,18 @@
def upload(request):
- libname = None
+ lib = None
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
- libname = form.parse_kw_spec(request.FILES['file'],
- form.cleaned_data['override'])
+ lib = form.parse_kw_spec(request.FILES['file'],
+ form.cleaned_data['override'],
+
form.cleaned_data['override_version'].strip())
else:
form = UploadFileForm()
return render_to_response('upload.html', {
'form': form,
- 'libname': libname
+ 'lib': lib
}
)
@@ -40,10 +41,14 @@
file = forms.FileField()
file.widget.attrs['size'] = 40
override = forms.BooleanField(required=False)
+ override_version = forms.CharField(required=False)
+ override_version.widget.attrs['size'] = 10
- def parse_kw_spec(self, fileobj, override):
+ def parse_kw_spec(self, fileobj, override, override_version):
try:
libdata = LibraryData(fileobj)
+ if override_version:
+ libdata.version=override_version
if
Library.objects.filter(name=libdata.name).filter(version=libdata.version):
if not override:
raise InvalidXmlError("Library %s version %s already
exists." % (libdata.name, libdata.version))
@@ -58,7 +63,7 @@
except InvalidXmlError, err:
self._errors['file'] = ErrorList([str(err)])
return None
- return lib.name
+ return lib
class LibraryData(object):
==============================================================================
Revision: d3cc4a1276cb
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 09:07:25 2014 UTC
Log: Modified invalid file upload messages to contain more information.
Update issue 22
All invalid file upload messages now contain name of the violating file
and some additional information on the (parsing) exception.
http://code.google.com/p/rfdoc/source/detail?r=d3cc4a1276cb
Modified:
/atest/invalid_file_upload.txt
/atest/uploader.txt
/src/rfdoc/rfdocapp/views/upload.py
/tools/uploader.py
=======================================
--- /atest/invalid_file_upload.txt Tue Oct 15 14:24:32 2013 UTC
+++ /atest/invalid_file_upload.txt Fri Feb 28 09:07:25 2014 UTC
@@ -3,24 +3,21 @@
Test Setup "Upload" page is open
*** Variables ***
-${NON XML FILE} Given file is not XML.
-${INVALID XML FILE} Given file contains invalid XML.
${NON EXISTING FILE} The submitted file is empty.
-${MISSING KEYWORDS} Given test library contains no keywords.
*** Test Cases *** *** Uploaded File *** ***
Error Message ***
-Non XML file Uploading fails text_file.txt ${NON
XML FILE}
-Wrong XML content Uploading fails invalid_content.xml
${INVALID XML FILE}
+Non XML file Uploading fails text_file.txt Given
file text_file.txt is not XML: syntax error: line 1, column 0
+Wrong XML content Uploading fails invalid_content.xml Given
file invalid_content.xml contains invalid XML: Root tag must be keywordspec
Non existing file Uploading fails non_existing_file.xml ${NON
EXISTING FILE}
-Empty library name Uploading fails empty_library_name.xml
${INVALID XML FILE}
-Missing library name Uploading fails missing_library_name.xml
${INVALID XML FILE}
-Missing library doc Uploading fails missing_library_doc.xml
${INVALID XML FILE}
-Empty keyword name Uploading fails empty_keyword_name.xml
${INVALID XML FILE}
-Missing keyword name Uploading fails missing_keyword_name.xml
${INVALID XML FILE}
-Missing keyword doc Uploading fails missing_keyword_doc.xml
${INVALID XML FILE}
-Empty keyword Uploading fails empty_keyword.xml
${INVALID XML FILE}
-Empty keywords Uploading fails empty_keywords.xml
${MISSING KEYWORDS}
-Missing keywords Uploading fails missing_keywords.xml
${MISSING KEYWORDS}
+Empty library name Uploading fails empty_library_name.xml
Library parsing error. Given file empty_library_name.xml contains invalid
XML: Attribute name not found
+Missing library name Uploading fails missing_library_name.xml
Library parsing error. Given file missing_library_name.xml contains invalid
XML: Attribute name not found
+Missing library doc Uploading fails missing_library_doc.xml
Library parsing error. Given file missing_library_doc.xml contains invalid
XML: Child element doc not found
+Empty keyword name Uploading fails empty_keyword_name.xml
Keyword parsing error. Given file empty_keyword_name.xml contains invalid
XML: Attribute name not found
+Missing keyword name Uploading fails missing_keyword_name.xml
Keyword parsing error. Given file missing_keyword_name.xml contains invalid
XML: Attribute name not found
+Missing keyword doc Uploading fails missing_keyword_doc.xml
Keyword parsing error. Given file missing_keyword_doc.xml contains invalid
XML: Child element doc not found
+Empty keyword Uploading fails empty_keyword.xml
Keyword parsing error. Given file empty_keyword.xml contains invalid XML:
Child element doc not found
+Empty keywords Uploading fails empty_keywords.xml Given
test library file empty_keywords.xml contains no keywords.
+Missing keywords Uploading fails missing_keywords.xml Given
test library file missing_keywords.xml contains no keywords.
*** Keywords ***
Uploading fails [Arguments] ${file} ${message}
=======================================
--- /atest/uploader.txt Mon Feb 24 10:15:07 2014 UTC
+++ /atest/uploader.txt Fri Feb 28 09:07:25 2014 UTC
@@ -132,4 +132,4 @@
Should Contain ${output} Try '${SCRIPT_NAME} --help' for more
information
Should Print ${host} Is Invalid Host
- Should Contain ${output} ${SCRIPT_NAME}: error: connection refused
to '${host}', check that the host responds and is reachable.
+ Should Contain ${output} ${SCRIPT_NAME}: Remote error: connection
refused to '${host}', check that the host responds and is reachable.
=======================================
--- /src/rfdoc/rfdocapp/views/upload.py Thu Feb 27 12:11:32 2014 UTC
+++ /src/rfdoc/rfdocapp/views/upload.py Fri Feb 28 09:07:25 2014 UTC
@@ -69,23 +69,24 @@
class LibraryData(object):
def __init__(self, fileobj):
+ self.filename = fileobj.name
root = self._get_root(fileobj)
try:
self.name = self._get_name(root)
self.version = self._get_version(root)
self.doc = self._get_doc(root)
- except InvalidXmlError:
- raise InvalidXmlError('Given file contains invalid XML.')
- self.inits = [ InitData(data) for data in self._get_inits(root) ]
- self.kws = [ KeywordData(data) for data in
self._get_keywords(root) ]
+ except InvalidXmlError as e:
+ raise InvalidXmlError('Library parsing error. Given file %s
contains invalid XML: %s' % (self.filename, e))
+ self.inits = [ InitData(data, self.filename) for data in
self._get_inits(root) ]
+ self.kws = [ KeywordData(data, self.filename) for data in
self._get_keywords(root) ]
def _get_root(self, fileobj):
try:
root = ET.parse(fileobj).getroot()
- except SyntaxError:
- raise InvalidXmlError('Given file is not XML.')
+ except SyntaxError as e:
+ raise InvalidXmlError('Given file %s is not XML: %s' %
(self.filename, e))
if root.tag != 'keywordspec':
- raise InvalidXmlError('Given file contains invalid XML.')
+ raise InvalidXmlError('Given file %s contains invalid XML:
Root tag must be keywordspec' % self.filename)
return root
def _get_name(self, elem):
@@ -111,19 +112,19 @@
# 'keywords/kw' is backwards compatibility for libdoc.py 2.1 and
earlier
kws = elem.findall('keywords/kw') + elem.findall('kw')
if not kws:
- raise InvalidXmlError('Given test library contains no
keywords.')
+ raise InvalidXmlError('Given test library file %s contains no
keywords.' % self.filename)
return kws
class KeywordData(object):
- def __init__(self, elem):
+ def __init__(self, elem, filename):
try:
self.name = self._get_name(elem)
self.doc = self._get_doc(elem)
self.args = ', '.join(arg.text for arg in self._get_args(elem))
- except InvalidXmlError:
- raise InvalidXmlError('Given file contains invalid XML.')
+ except InvalidXmlError as e:
+ raise InvalidXmlError('Keyword parsing error. Given file %s
contains invalid XML: %s' % (filename, e))
def _get_name(self, elem):
return get_attr(elem, 'name')
@@ -144,13 +145,13 @@
def get_attr(elem, attr_name):
attr = elem.get(attr_name)
if not attr:
- raise InvalidXmlError
+ raise InvalidXmlError('Attribute %s not found' % attr_name)
return attr
def get_child_element(elem, child_name):
child = elem.find(child_name)
if child is None:
- raise InvalidXmlError
+ raise InvalidXmlError('Child element %s not found' % child_name)
return child
=======================================
--- /tools/uploader.py Mon Feb 24 10:15:07 2014 UTC
+++ /tools/uploader.py Fri Feb 28 09:07:25 2014 UTC
@@ -68,7 +68,7 @@
finally:
xml_doc.original_close()
except DataError, e:
- sys.stderr.write('%s: error: %s\n' %
(os.path.basename(__file__),
+ sys.stderr.write('%s: Remote error: %s\n' %
(os.path.basename(__file__),
e.message))
exit(1)
==============================================================================
Revision: 994a7a77bc0b
Branch: default
Author: Janne Piironen <janne.p...@gmail.com>
Date: Fri Feb 28 11:01:17 2014 UTC
Log: Moved uploader.py to rfdoc.upload
Update issue 23
Uploader is now part of RFDoc in module rfdoc.upload and can be called
like defined in the issue.
http://code.google.com/p/rfdoc/source/detail?r=994a7a77bc0b
Added:
/src/rfdoc/upload.py
Deleted:
/tools/uploader.py
Modified:
/MANIFEST.in
/atest/uploader.txt
/src/rfdoc/settings.py
=======================================
--- /dev/null
+++ /src/rfdoc/upload.py Fri Feb 28 11:01:17 2014 UTC
@@ -0,0 +1,263 @@
+#!/usr/bin/env python
+
+# Copyright 2008-2013 Nokia Siemens Networks Oyj
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import with_statement
+
+import os
+import sys
+
+from contextlib import closing
+from HTMLParser import HTMLParser
+from httplib import HTTPConnection
+from optparse import OptionParser
+from re import match
+from StringIO import StringIO
+from urlparse import urlsplit
+
+from robot.errors import DataError
+from robot.libdocpkg import LibraryDocumentation
+from robot.parsing.populators import READERS
+
+
+class Uploader(object):
+
+ def __init__(self):
+ self._options = CommandlineUI()
+ self._uploader = XmlUploader(self._options.target_url)
+
+ def run(self):
+ try:
+ for library in self._options.libraries:
+ xml_doc = StringIO()
+ # LibraryDocumentation().save() calls close() for the
underlying
+ # file but closing StringIO object discards its data.
+ # This is why close() is overridden below.
+ xml_doc.original_close = xml_doc.close
+ try:
+ try:
+ if library.endswith('.xml'):
+ with open(library) as xml_file:
+ xml_doc.write(xml_file.read())
+ else:
+ xml_doc.close = lambda: None
+ if self._options.lib_version:
+ LibraryDocumentation(library,
version=self._options.lib_version).save(xml_doc, 'xml')
+ else:
+
LibraryDocumentation(library).save(xml_doc, 'xml')
+ except DataError, e:
+ message = "Library not found" if 'ImportError' in
e.message else e.message
+ sys.stderr.write("Skipping '%s' due to an
error: %s.\n" %
+ (library, message))
+ continue
+ xml_doc.name = library
+ self._uploader.upload_file(xml_doc)
+ sys.stdout.write("Updated documentation for '%s'.\n" %
library)
+ finally:
+ xml_doc.original_close()
+ except DataError, e:
+ sys.stderr.write('%s: Remote error: %s\n' %
(os.path.basename(__file__),
+ e.message))
+ exit(1)
+
+
+class ImprovedOptionParser(OptionParser):
+
+ # This prevents newlines from getting discarded from the help text.
+ def format_description(self, formatter):
+ return self.description
+
+ # This adds "try --help for information" message.
+ def error(self, message):
+ progname = os.path.basename(sys.argv[0])
+ sys.stderr.write('%s: error: %s\n'% (progname, message))
+ sys.stderr.write("Try '%s --help' for more information.\n" %
progname)
+ exit(2)
+
+
+class CommandlineUI(object):
+
+ valid_lib_exts = tuple(READERS.keys() + ['py', 'java', 'xml'])
+ default_url = 'localhost:8000'
+ usage_text = 'usage: %prog [options] PATH ...'
+ help_text = """
+This script updates the documentation at Robot Framework RFDoc server.
+
+PATH is one of the following (multiple can be given, separated by a space):
+1) A path to a library source file (e.g. src/libraries/example_lib.py)
+2) A path to a resource file (e.g. atest/resources/utils.txt)
+3) A path to a library XML, generated using LibDoc (e.g.
libdoc/BuiltIn.xml)
+4) A path to a directory, in which case it's recursively traversed for any
of
+ files mentioned in 1-3.
+"""[1:]
+ epilog_text = """
+The script is intended to be used as part of the CI pipeline or as an SCM
+post-change hook to update the documentation in RFDoc automatically.
+"""[1:]
+
+ def __init__(self):
+ self._parser = ImprovedOptionParser(
+ usage=self.usage_text,
+ description=self.help_text,
+ epilog=self.epilog_text
+ )
+ self._add_commandline_options()
+ self._options = self._get_validated_options()
+
+ @property
+ def target_url(self):
+ return self._options.target_url
+
+ @property
+ def libraries(self):
+ return self._options.libraries
+
+ @property
+ def lib_version(self):
+ return self._options.lib_version
+
+ def _add_commandline_options(self):
+ self._parser.add_option(
+ '-u', '--url',
+ dest='target_url',
+ default=self.default_url,
+ help="""Target RFDoc URL to update, e.g. '192.168.1.100:8000'
or
+'my.server.com/rfdoc'. If this option is not given, '%s' is assumed
+as target.""" % self.default_url
+ )
+ self._parser.add_option(
+ '-v', '--version',
+ dest='lib_version',
+ help="""Override library version""")
+
+ def _get_validated_options(self):
+ options, targets = self._parser.parse_args()
+ if len(targets) < 1:
+ self._exit_with_help()
+ options.libraries = self._traverse_targets_for_libraries(targets)
+ options.target_url = self._host_from_url(options.target_url)
+ return options
+
+ def _traverse_targets_for_libraries(self, targets):
+ libraries = targets
+ for directory in self._only_directories(targets):
+ libraries.remove(directory)
+ for path, _, filenames in os.walk(directory):
+ for filename in self._only_library_files(filenames):
+ libraries.append(os.path.join(path, filename))
+ return libraries
+
+ def _only_directories(self, targets):
+ for target in targets:
+ if os.path.isdir(target):
+ yield target
+
+ def _only_library_files(self, filenames):
+ for filename in filenames:
+ if self._is_library_file(filename):
+ yield filename
+
+ def _is_library_file(self, filename):
+ return any(filename.endswith(ext) for ext in self.valid_lib_exts)
+
+ def _host_from_url(self, url):
+ if not match(r'http(s?)\:', url):
+ url = 'http://' + url
+ return url
+
+ def _exit_with_help(self):
+ self._parser.print_help()
+ exit(1)
+
+
+class XmlUploader(object):
+ default_endpoint = 'upload'
+ body_template = """--%(boundary)s
+Content-Disposition: form-data; name="override"
+
+on
+--%(boundary)s
+Content-Disposition: form-data; name="file"; filename="%(filename)s"
+Content-Type: text/xml
+
+%(content)s
+--%(boundary)s--
+"""
+
+ def __init__(self, target_url):
+ self.url = urlsplit(target_url)
+
+ def upload_file(self, xml_doc):
+ with closing(HTTPConnection(self.url.netloc)) as connection:
+ try:
+ response = self._post_multipart(connection, xml_doc)
+ except Exception, message:
+ if 'Connection refused' in message:
+ message = "connection refused to '%s', " %
self.url.netloc
+ message += 'check that the host responds and is
reachable.'
+ raise DataError(message)
+ self._validate_success(response)
+
+ def _post_multipart(self, connection, xml_doc):
+ connection.connect()
+ content_type, body = self._encode_multipart_formdata(xml_doc)
+ headers = {'User-Agent': 'RFDoc uploader', 'Content-Type':
content_type}
+ connection.request('POST', self._get_endpoint(), body, headers)
+ return connection.getresponse()
+
+ def _get_endpoint(self):
+ return self.url.path + '/' + self.default_endpoint
+
+ def _encode_multipart_formdata(self, xml_doc):
+ boundary = '----------ThIs_Is_tHe_bouNdaRY_$'
+ body = self.body_template % {
+ 'boundary': boundary,
+ 'filename': xml_doc.name,
+ 'content': xml_doc.getvalue()
+ }
+ content_type = 'multipart/form-data; boundary=%s' % boundary
+ return content_type, body.replace('\n', '\r\n')
+
+ def _validate_success(self, response):
+ if response.status != 200:
+ raise DataError(response.reason.strip())
+ html = response.read()
+ if 'Successfully uploaded library' not in html:
+ raise DataError('\n'.join(self._ErrorParser(html).errors))
+
+ class _ErrorParser(HTMLParser):
+
+ def __init__(self, html):
+ HTMLParser.__init__(self)
+ self._inside_errors = False
+ self.errors = []
+ self.feed(html)
+ self.close()
+
+ def handle_starttag(self, tag, attributes):
+ if ('class', 'errorlist') in attributes:
+ self._inside_errors = True
+
+ def handle_endtag(self, tag):
+ if tag == 'ul':
+ self._inside_errors = False
+
+ def handle_data(self, data):
+ if self._inside_errors and data.strip():
+ self.errors.append(data)
+
+
+if __name__ == '__main__':
+ Uploader().run()
=======================================
--- /tools/uploader.py Fri Feb 28 09:07:25 2014 UTC
+++ /dev/null
@@ -1,263 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2008-2013 Nokia Siemens Networks Oyj
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from __future__ import with_statement
-
-import os
-import sys
-
-from contextlib import closing
-from HTMLParser import HTMLParser
-from httplib import HTTPConnection
-from optparse import OptionParser
-from re import match
-from StringIO import StringIO
-from urlparse import urlsplit
-
-from robot.errors import DataError
-from robot.libdocpkg import LibraryDocumentation
-from robot.parsing.populators import READERS
-
-
-class Uploader(object):
-
- def __init__(self):
- self._options = CommandlineUI()
- self._uploader = XmlUploader(self._options.target_url)
-
- def run(self):
- try:
- for library in self._options.libraries:
- xml_doc = StringIO()
- # LibraryDocumentation().save() calls close() for the
underlying
- # file but closing StringIO object discards its data.
- # This is why close() is overridden below.
- xml_doc.original_close = xml_doc.close
- try:
- try:
- if library.endswith('.xml'):
- with open(library) as xml_file:
- xml_doc.write(xml_file.read())
- else:
- xml_doc.close = lambda: None
- if self._options.lib_version:
- LibraryDocumentation(library,
version=self._options.lib_version).save(xml_doc, 'xml')
- else:
-
LibraryDocumentation(library).save(xml_doc, 'xml')
- except DataError, e:
- message = "Library not found" if 'ImportError' in
e.message else e.message
- sys.stderr.write("Skipping '%s' due to an
error: %s.\n" %
- (library, message))
- continue
- xml_doc.name = library
- self._uploader.upload_file(xml_doc)
- sys.stdout.write("Updated documentation for '%s'.\n" %
library)
- finally:
- xml_doc.original_close()
- except DataError, e:
- sys.stderr.write('%s: Remote error: %s\n' %
(os.path.basename(__file__),
- e.message))
- exit(1)
-
-
-class ImprovedOptionParser(OptionParser):
-
- # This prevents newlines from getting discarded from the help text.
- def format_description(self, formatter):
- return self.description
-
- # This adds "try --help for information" message.
- def error(self, message):
- progname = os.path.basename(sys.argv[0])
- sys.stderr.write('%s: error: %s\n'% (progname, message))
- sys.stderr.write("Try '%s --help' for more information.\n" %
progname)
- exit(2)
-
-
-class CommandlineUI(object):
-
- valid_lib_exts = tuple(READERS.keys() + ['py', 'java', 'xml'])
- default_url = 'localhost:8000'
- usage_text = 'usage: %prog [options] PATH ...'
- help_text = """
-This script updates the documentation at Robot Framework RFDoc server.
-
-PATH is one of the following (multiple can be given, separated by a space):
-1) A path to a library source file (e.g. src/libraries/example_lib.py)
-2) A path to a resource file (e.g. atest/resources/utils.txt)
-3) A path to a library XML, generated using LibDoc (e.g.
libdoc/BuiltIn.xml)
-4) A path to a directory, in which case it's recursively traversed for any
of
- files mentioned in 1-3.
-"""[1:]
- epilog_text = """
-The script is intended to be used as part of the CI pipeline or as an SCM
-post-change hook to update the documentation in RFDoc automatically.
-"""[1:]
-
- def __init__(self):
- self._parser = ImprovedOptionParser(
- usage=self.usage_text,
- description=self.help_text,
- epilog=self.epilog_text
- )
- self._add_commandline_options()
- self._options = self._get_validated_options()
-
- @property
- def target_url(self):
- return self._options.target_url
-
- @property
- def libraries(self):
- return self._options.libraries
-
- @property
- def lib_version(self):
- return self._options.lib_version
-
- def _add_commandline_options(self):
- self._parser.add_option(
- '-u', '--url',
- dest='target_url',
- default=self.default_url,
- help="""Target RFDoc URL to update, e.g. '192.168.1.100:8000'
or
-'my.server.com/rfdoc'. If this option is not given, '%s' is assumed
-as target.""" % self.default_url
- )
- self._parser.add_option(
- '-v', '--version',
- dest='lib_version',
- help="""Override library version""")
-
- def _get_validated_options(self):
- options, targets = self._parser.parse_args()
- if len(targets) < 1:
- self._exit_with_help()
- options.libraries = self._traverse_targets_for_libraries(targets)
- options.target_url = self._host_from_url(options.target_url)
- return options
-
- def _traverse_targets_for_libraries(self, targets):
- libraries = targets
- for directory in self._only_directories(targets):
- libraries.remove(directory)
- for path, _, filenames in os.walk(directory):
- for filename in self._only_library_files(filenames):
- libraries.append(os.path.join(path, filename))
- return libraries
-
- def _only_directories(self, targets):
- for target in targets:
- if os.path.isdir(target):
- yield target
-
- def _only_library_files(self, filenames):
- for filename in filenames:
- if self._is_library_file(filename):
- yield filename
-
- def _is_library_file(self, filename):
- return any(filename.endswith(ext) for ext in self.valid_lib_exts)
-
- def _host_from_url(self, url):
- if not match(r'http(s?)\:', url):
- url = 'http://' + url
- return url
-
- def _exit_with_help(self):
- self._parser.print_help()
- exit(1)
-
-
-class XmlUploader(object):
- default_endpoint = 'upload'
- body_template = """--%(boundary)s
-Content-Disposition: form-data; name="override"
-
-on
---%(boundary)s
-Content-Disposition: form-data; name="file"; filename="%(filename)s"
-Content-Type: text/xml
-
-%(content)s
---%(boundary)s--
-"""
-
- def __init__(self, target_url):
- self.url = urlsplit(target_url)
-
- def upload_file(self, xml_doc):
- with closing(HTTPConnection(self.url.netloc)) as connection:
- try:
- response = self._post_multipart(connection, xml_doc)
- except Exception, message:
- if 'Connection refused' in message:
- message = "connection refused to '%s', " %
self.url.netloc
- message += 'check that the host responds and is
reachable.'
- raise DataError(message)
- self._validate_success(response)
-
- def _post_multipart(self, connection, xml_doc):
- connection.connect()
- content_type, body = self._encode_multipart_formdata(xml_doc)
- headers = {'User-Agent': 'RFDoc uploader', 'Content-Type':
content_type}
- connection.request('POST', self._get_endpoint(), body, headers)
- return connection.getresponse()
-
- def _get_endpoint(self):
- return self.url.path + '/' + self.default_endpoint
-
- def _encode_multipart_formdata(self, xml_doc):
- boundary = '----------ThIs_Is_tHe_bouNdaRY_$'
- body = self.body_template % {
- 'boundary': boundary,
- 'filename': xml_doc.name,
- 'content': xml_doc.getvalue()
- }
- content_type = 'multipart/form-data; boundary=%s' % boundary
- return content_type, body.replace('\n', '\r\n')
-
- def _validate_success(self, response):
- if response.status != 200:
- raise DataError(response.reason.strip())
- html = response.read()
- if 'Successfully uploaded library' not in html:
- raise DataError('\n'.join(self._ErrorParser(html).errors))
-
- class _ErrorParser(HTMLParser):
-
- def __init__(self, html):
- HTMLParser.__init__(self)
- self._inside_errors = False
- self.errors = []
- self.feed(html)
- self.close()
-
- def handle_starttag(self, tag, attributes):
- if ('class', 'errorlist') in attributes:
- self._inside_errors = True
-
- def handle_endtag(self, tag):
- if tag == 'ul':
- self._inside_errors = False
-
- def handle_data(self, data):
- if self._inside_errors and data.strip():
- self.errors.append(data)
-
-
-if __name__ == '__main__':
- Uploader().run()
=======================================
--- /MANIFEST.in Tue Oct 29 13:22:20 2013 UTC
+++ /MANIFEST.in Fri Feb 28 11:01:17 2014 UTC
@@ -1,5 +1,4 @@
prune atest
-graft tools
include src/rfdoc/*.tmpl
include src/rfdoc/rfdocapp/templates/*.html
include src/rfdoc/rfdocapp/static/*.css
=======================================
--- /atest/uploader.txt Fri Feb 28 09:07:25 2014 UTC
+++ /atest/uploader.txt Fri Feb 28 11:01:17 2014 UTC
@@ -3,8 +3,8 @@
*** Variables ***
${INVALID URL} localhost:1
-${SCRIPT NAME} uploader.py
-${SCRIPT PATH} ${CURDIR}${/}..${/}tools${/}${SCRIPT NAME}
+${SCRIPT NAME} upload.py
+${SCRIPT PATH} ${CURDIR}${/}..${/}src${/}rfdoc${/}${SCRIPT
NAME}
${EXAMPLE LIBRARY SOURCE} ${TESTDATA PATH}${/}sources${/}example_lib.py
${EXAMPLE LIBRARY XML V1} ${TESTDATA
PATH}${/}xmls${/}ExampleLibrary_version_1.xml
${EXAMPLE LIBRARY XML V2} ${TESTDATA
PATH}${/}xmls${/}ExampleLibrary_version_2.xml
=======================================
--- /src/rfdoc/settings.py Mon Dec 23 09:50:35 2013 UTC
+++ /src/rfdoc/settings.py Fri Feb 28 11:01:17 2014 UTC
@@ -57,7 +57,7 @@
'rfdoc.rfdocapp'
)
-# CSRF removed from the defaults due to tools/uploader.py
+# CSRF removed from the defaults due to src/rfdoc/upload.py
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
Reply all
Reply to author
Forward
0 new messages