So, we need a project registry...

[Steve Bennett]

Hi everyone,
I'm at ANDS, the Australian National Data Service, which has a
similar need to JISC: we fund and otherwise facilitate software
projects at research institutions (mostly universities). And now we'd
like to have a public, searchable, updateable project registry to
better expose the software they produce, and to help projects find
collaborators. By the end of next year, we'd anticipate something like
150 projects either running or completed.

So, after a bit of research, a DOAP-based registry like SIMAL seems
like an obvious fit. We'd probably provide an authoring form tightly
coupled with it rather than harvesting lots of doap.xml files.

Now, I have a few questions about how to go about this.

First, and most important, what is the development status of SIMAL? I
see there was a new release not long ago, but this list is pretty
quiet, and the version number is pretty low. We don't need many
features, but our development resources are very limited, so we kind
of need something that will run without too much hand-holding and
tweaking.

Second, the PROD interface is very pretty and seems to work very well.
How much of that functionality is built into SIMAL, and how much is
PROD-specific? And is it possible to get the PROD source code? The
Trac requires username/password, and appears not to be public?

Lastly, are there alternatives I should be looking at. From the DOAP
primer at oss-watch, I'm aware of Apache Forrest. Does anyone have any
experience with this or other tools?

Thanks in advance for any pointers, advice etc!

Steve

[Ross Gardler]

On 07/01/2011 05:51, Steve Bennett wrote:
> So, after a bit of research, a DOAP-based registry like SIMAL seems
> like an obvious fit. We'd probably provide an authoring form tightly
> coupled with it rather than harvesting lots of doap.xml files.

We already have an authoring form. The form is provided as a Widget and
thus can be hosted on any web page. See
http://registry.oss-watch.ac.uk/widgets/widgetTitle/doapcreator

> Now, I have a few questions about how to go about this.
>
> First, and most important, what is the development status of SIMAL? I
> see there was a new release not long ago, but this list is pretty
> quiet, and the version number is pretty low. We don't need many
> features, but our development resources are very limited, so we kind
> of need something that will run without too much hand-holding and
> tweaking.

Simal is still actively developed. We hope to push a new release out
pretty soon.

This list is quiet because at present we are the only active developers
here and our own resources are limited. We would be very happy to
combine our own resources with yours in order to move things on a little
quicker in key areas.

We believe the system to be pretty stable, but limited in features. Our
focus has been on stability rather than features at this stage. So the
version number represents feature completeness, rather than utility. We
do use the system for our support work here in the UK.

It is not perfect. There are bugs, but it does work and we consider the
data to be safe.

You can see out version of the registry at
http://registry.oss-watch.ac.uk/ it currently has 1664 projects in 147
categories with 1405 people.

We believe it is easy to get up and running, but until someone tests
that for us we can't be certain. I can assure you that ease of set-up is
something that we have focussed on.

> Second, the PROD interface is very pretty and seems to work very well.
> How much of that functionality is built into SIMAL, and how much is
> PROD-specific? And is it possible to get the PROD source code? The
> Trac requires username/password, and appears not to be public?

A long time ago we tried to get the PROD team to use SIMAL as their
back-end. Our focus has always been to provide a solid back end and
functional API. Unfortunately that never happened and the developer
working on PROD left CETIS.

PROD uses a relational database for its back end. This means it is much
more limited with respect to capturing the data you actually need and
relating it to other data out there. However, if all you want is a
database of projects then PROD might be OK for you.

I have no idea if CETIS are willing to make the code available.

From a Simal UI point of view we have recently implemented a widget
framework within Simal and intend to replace the key functionality with
widgets. What this means is that each functional unit of the UI will be
a separate component that can be hosted in any location. we already
(through Apache Wookie) have connectors for Moodle, Drupal, Wordpress
and many others. It's simple to include in pretty much any HTML page,
for example if you open in your http://oss.ly/doap browser you will see
the DOAP form that will submit to our version of the registry.

However, we only have the DOAP form as a widget at this point.

So you can modify the interface we already have (it is highly
configurable, but does require work) or you can build new widgets using
the API and embed them in a new or existing platform (which requires
even more work and is largely untested).

We would be happy to support you on either of these two paths.

> Lastly, are there alternatives I should be looking at. From the DOAP
> primer at oss-watch, I'm aware of Apache Forrest. Does anyone have any
> experience with this or other tools?

I was one of the originators of Forrest and wrote the DOAP features it
has. Simal was created because Forrest was not really up to the job.
However, if you just want a static representation of a set of DOAP
records then Forrest can do the job.

I'm not aware of anything else. None of the solutions out there at
present are ideal. However, be assured that Simal is under active
development (with limited resources) and we will be happy to help you
apply your resources most productively here if this is the way you
choose to go.

In the meantime, I'd be happy to help your developer get started on
testing locally.

Ross

[Steve Bennett]

On 8 January 2011 00:24, Ross Gardler <rgar...@apache.org> wrote:

We already have an authoring form. The form is provided as a Widget and thus can be hosted on any web page. See http://registry.oss-watch.ac.uk/widgets/widgetTitle/doapcreator

Oh, good. That looks very close to the same set of fields we would want, too. 

This list is quiet because at present we are the only active developers here and our own resources are limited. We would be very happy to combine our own resources with yours in order to move things on a little quicker in key areas.

Yes, I didn't notice the number of checkins in Google Code before posting. Our resources are perhaps even more limited than yours :) But I won't know for sure until after I do this initial investigation and report back to directors etc. It's likely that the only resources will be about a quarter or a half of me.

 

We believe it is easy to get up and running, but until someone tests that for us we can't be certain. I can assure you that ease of set-up is something that we have focussed on.

I'm having a go installing it on my laptop (XP) atm. Should I report issues here? For example when compiling the trunk, I get this dependency error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.

3.2:compile (default-compile) on project simal-core: Compilation failure: Compil

ation failure:

[ERROR] \simal\simal-trunk\uk.ac.osswatch.simal.core\src\main\java\uk\ac\osswatc

h\simal\service\jena\JenaRepositoryService.java:[22,31] package org.codehaus.ple

xus.util does not exist

[ERROR] \simal\simal-trunk\uk.ac.osswatch.simal.core\src\main\java\uk\ac\osswatc

h\simal\service\jena\JenaRepositoryService.java:[85,8] cannot find symbol

[ERROR] symbol  : variable StringUtils

[ERROR] location: class uk.ac.osswatch.simal.service.jena.JenaRepositoryService

I tried adding a dependency to the pom.xml, but no luck. (It looked like it was trying to download a non-existent .jar).

Is there perhaps an earlier stable version that I should work from?

From a Simal UI point of view we have recently implemented a widget framework within Simal and intend to replace the key functionality with widgets. What this means is that each functional unit of the UI will be a separate component that can be hosted in any location. we already (through Apache Wookie) have connectors for Moodle, Drupal, Wordpress and many others. It's simple to include in pretty much any HTML page, for example if you open in your http://oss.ly/doap browser you will see the DOAP form that will submit to our version of the registry.

Cool. Our website is not widget-friendly, but presumably we will work something out when we get to that point.

 

So you can modify the interface we already have (it is highly configurable, but does require work) or you can build new widgets using the API and embed them in a new or existing platform (which requires even more work and is largely untested).

We would be happy to support you on either of these two paths.

Cool, will investigate the former path.

Incidentally, if you couple the doap authoring form to the registry, how do you allow people to maintain their record later on? Do you issue them some kind of token or user/pass, or simply let anyone edit and personally approve all changes?

 

I'm not aware of anything else. None of the solutions out there at present are ideal. However, be assured that Simal is under active development (with limited resources) and we will be happy to help you apply your resources most productively here if this is the way you choose to go.

Well, given the close alignment of JISC and ANDS, and what I see so far, SIMAL looks like a pretty good avenue to explore further.

 

In the meantime, I'd be happy to help your developer get started on testing locally.

That would be me. :) To explain my role a bit, I'm a business analyst, and my main role is working with our funded institutions to define their projects, get them off the ground, then monitor them. However, I also have a development background (though not much in the last few years), and have some experience configuring and deploying Java webapps. I'm very comfortable with the Java language, but not so much the modern frameworks like Spring, Hibernate, Plexus (which I hadn't heard of until now), etc.

Btw, if installing on XP is untested, then it's probably not worth my time trying as that wouldn't be our final deployment environment. In that case I'll have to get access to a linux test environment.

Steve

[Sander W G van der Waal]

> From: simal...@googlegroups.com [mailto:simal...@googlegroups.com] On
> Behalf Of Steve Bennett
> Sent: 11 January 2011 01:41
> To: simal...@googlegroups.com
> Subject: Re: [simal-users] So, we need a project registry...

>
> On 8 January 2011 00:24, Ross Gardler <rgar...@apache.org> wrote:
>> We already have an authoring form. The form is provided as a Widget and
>> thus can be hosted on any web page. See http://registry.oss-
>> watch.ac.uk/widgets/widgetTitle/doapcreator
>
> Oh, good. That looks very close to the same set of fields we would want,
> too.

Note that some changes have been made to the form after the version
linked to above was released. This is partly to separate pure DOAP fields
(eg. roles like Maintainer, Developer, Tester) from custom fields we
introduced (eg. roles like the Project Director). Because we use RDF it's
really easy to add any information to the backend you want so any custom
field you add to the DOAP file is persisted (including any other non-DOAP
fields, as long as the file is valid RDF/XML). The UI however currently does
not show all these custom fields, but is limited to (mainly) fields from the
DOAP spec. Of course this can be extended.

>> This list is quiet because at present we are the only active developers
>> here and our own resources are limited. We would be very happy to combine
>> our own resources with yours in order to move things on a little quicker in
>> key areas.
>
> Yes, I didn't notice the number of checkins in Google Code before posting.
> Our resources are perhaps even more limited than yours :) But I won't know
> for sure until after I do this initial investigation and report back to
> directors etc. It's likely that the only resources will be about a quarter
> or a half of me.
>
>> We believe it is easy to get up and running, but until someone tests that
>> for us we can't be certain. I can assure you that ease of set-up is
>> something that we have focussed on.
>
> I'm having a go installing it on my laptop (XP) atm. Should I report issues
> here? For example when compiling the trunk, I get this dependency error:

It would be great if you could create an issue for any bug you find in the
tracker on our Google Code website [1]. However the dependency error you
mention below has (hopefully) been resolved today.

> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-

<snip>

>
> I tried adding a dependency to the pom.xml, but no luck. (It looked like it
> was trying to download a non-existent .jar).
>
> Is there perhaps an earlier stable version that I should work from?

In general it should be safe to work from the trunk. We're using continuous
integration with a Hudson server (which turns out was not running which is
why this error was not caught). Commits should therefore not break the
build or let any of the unit tests fail.

>> So you can modify the interface we already have (it is highly configurable,
>> but does require work) or you can build new widgets using the API and embed
>> them in a new or existing platform (which requires even more work and is
>> largely untested).
>>
>> We would be happy to support you on either of these two paths.
>
> Cool, will investigate the former path.

Excellent, and I'd like to second Ross's comment that we'd be happy to support
you.

> Incidentally, if you couple the doap authoring form to the registry, how do
> you allow people to maintain their record later on? Do you issue them some
> kind of token or user/pass, or simply let anyone edit and personally
> approve all changes?

I'm currently working on this functionality (Issue 335 [2]). As it stands,
anybody can press the edit button to edit project details on the project detail
page. There's also a very basic login facility. Nothing has yet been done to
merge the two, although I'm building the editing stuff in a way that makes it
easy to switch it on and off depending on whether you're logged in or not.
Approving changes, as some sort of workflow step, is not currently on our
roadmap.

>> I'm not aware of anything else. None of the solutions out there at present
>> are ideal. However, be assured that Simal is under active development (with
>> limited resources) and we will be happy to help you apply your resources
>> most productively here if this is the way you choose to go.
>
> Well, given the close alignment of JISC and ANDS, and what I see so far,
> SIMAL looks like a pretty good avenue to explore further.
>
>
>> In the meantime, I'd be happy to help your developer get started on testing
>> locally.
>
> That would be me. :) To explain my role a bit, I'm a business analyst, and
> my main role is working with our funded institutions to define their
> projects, get them off the ground, then monitor them. However, I also have
> a development background (though not much in the last few years), and have
> some experience configuring and deploying Java webapps. I'm very
> comfortable with the Java language, but not so much the modern frameworks
> like Spring, Hibernate, Plexus (which I hadn't heard of until now), etc.
>
> Btw, if installing on XP is untested, then it's probably not worth my time
> trying as that wouldn't be our final deployment environment. In that case
> I'll have to get access to a linux test environment.

I'm working on WinXP myself so it's probably the best supported platform
at the moment.. ;)

Thanks for your interest in Simal!

Sander

[1] http://code.google.com/p/simal/issues/list
[2] http://code.google.com/p/simal/issues/detail?id=335

> Steve
>
> --
> You received this message because you are subscribed to the Google Groups
> "simal-users" group.
> To post to this group, send email to simal...@googlegroups.com.
> To unsubscribe from this group, send email to simal-
> users+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/simal-
> users?hl=en.

[Steve Bennett]

On 12 January 2011 00:39, Sander W G van der Waal <sander.v...@oucs.ox.ac.uk> wrote:

It would be great if you could create an issue for any bug you find in the
tracker on our Google Code website [1]. However the dependency error you
mention below has (hopefully) been resolved today.

Ok. It can be hard to tell whether it's a bug or user error though.

I've now got core and REST installed, and can access REST through Jetty. However, no webapp. I'm a bit confused because the instructions don't actually say to build in the simal.web directory. And if I try anyway (mvn install):

[INFO] Building Simal Web Application 0.2.5-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[WARNING] The POM for org.joseki:joseki:jar:3.4.1 is missing, no dependency info

rmation available

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 1.312s

[INFO] Finished at: Wed Jan 12 11:23:53 EST 2011

[INFO] Final Memory: 3M/15M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal on project simal-webapp: Could not resolve depend

encies for project uk.ac.osswatch:simal-webapp:war:0.2.5-SNAPSHOT: Failure to fi

nd org.joseki:joseki:jar:3.4.1 in http://www.16degrees.com.au/maven was cached i

n the local repository, resolution will not be reattempted until the update inte

rval of simal has elapsed or updates are forced -> [Help 1]

Starting Jetty with Maven shows that the REST component is running but not the webapp:

Error 404 - Not Found.

No context on this server matched or handled this request.
Contexts known to this server are:

• /simal-rest ---> org.mortbay.jetty.plugin.Jetty6PluginWebAppContext@1a7c484{/simal-rest,C:\simal\simal-trunk\uk.ac.osswatch.simal.rest\src\main\webapp}

(Also, the local.simal.properties file I created is not being used).

 

I'm currently working on this functionality (Issue 335 [2]). As it stands,

anybody can press the edit button to edit project details on the project detail
page. There's also a very basic login facility. Nothing has yet been done to
merge the two, although I'm building the editing stuff in a way that makes it
easy to switch it on and off depending on whether you're logged in or not.
Approving changes, as some sort of workflow step, is not currently on our
roadmap.

Ok. The reason I wonder is because the basic use case is likely to be that someone creates and registers the project, then completely forgets about us until 6 months later or something, then wants to update it. Usernames and passwords didn't seem like a good fit. But I guess we'll cross that bridge when we come to it.

Thanks, 

Steve

[Ross Gardler]

On 12 Jan 2011, at 05:27, Steve Bennett <ste...@gmail.com> wrote:

On 12 January 2011 00:39, Sander W G van der Waal <sander.v...@oucs.ox.ac.uk> wrote:

It would be great if you could create an issue for any bug you find in the
tracker on our Google Code website [1]. However the dependency error you
mention below has (hopefully) been resolved today.

Ok. It can be hard to tell whether it's a bug or user error though.

Don't worry about that. We'll cope and you'll soon know the boundaries of what Simal can do. Feel free to discuss here first if unsure. 

I've now got core and REST installed, and can access REST through Jetty.

I'll let Sander address that as I'm not actually working on the code right now. 

I'm currently working on this functionality (Issue 335 [2]). As it stands,

anybody can press the edit button to edit project details on the project detail
page. There's also a very basic login facility. Nothing has yet been done to
merge the two, although I'm building the editing stuff in a way that makes it
easy to switch it on and off depending on whether you're logged in or not.
Approving changes, as some sort of workflow step, is not currently on our
roadmap.

Ok. The reason I wonder is because the basic use case is likely to be that someone creates and registers the project, then completely forgets about us until 6 months later or something, then wants to update it. Usernames and passwords didn't seem like a good fit. But I guess we'll cross that bridge when we come to it.

The goal is to have Simal automatically update DOAP records from a remotely hosted file, that is the project can host it on their own website. 

Most of the basic building blocks for this are in place, but we need to build the infrastructure to automate the process. 

It's not a huge amount of work, just like everything else ;-)

Ross

Thanks, 

Steve

--
You received this message because you are subscribed to the Google Groups "simal-users" group.
To post to this group, send email to simal...@googlegroups.com.

To unsubscribe from this group, send email to simal-users...@googlegroups.com.

[rgardler]

On Jan 12, 9:12 am, Ross Gardler <rgard...@apache.org> wrote:
> On 12 Jan 2011, at 05:27, Steve Bennett <stev...@gmail.com> wrote:

>
> > On 12 January 2011 00:39, Sander W G van der Waal <sander.vanderw...@oucs.ox.ac.uk> wrote:
> > It would be great if you could create an issue for any bug you find in the
> > tracker on our Google Code website [1]. However the dependency error you
> > mention below has (hopefully) been resolved today.
>
> > Ok. It can be hard to tell whether it's a bug or user error though.
>
> Don't worry about that. We'll cope and you'll soon know the boundaries of what Simal can do. Feel free to discuss here first if unsure.

Although you may want to subscribe to the developer list, since that
is where you'll be notified of work in progress.

http://groups.google.com/group/simal-contributors

Ross

[rgardler]

On Jan 12, 9:12 am, Ross Gardler <rgard...@apache.org> wrote:

> On 12 Jan 2011, at 05:27, Steve Bennett <stev...@gmail.com> wrote:

>
> > On 12 January 2011 00:39, Sander W G van der Waal <sander.vanderw...@oucs.ox.ac.uk> wrote:
> > It would be great if you could create an issue for any bug you find in the
> > tracker on our Google Code website [1]. However the dependency error you
> > mention below has (hopefully) been resolved today.
>

...

> > I've now got core and REST installed, and can access REST through Jetty.
>
> I'll let Sander address that as I'm not actually working on the code right now.

Thanks to you reporting these issues we have discuovered the
continuous integration server has not been running and indeed the
build is broken. This will be fixed as a matter of priority and Sander
will let you know when things are sorted.

Ross

[Sander W G van der Waal]

> From: simal...@googlegroups.com [mailto:simal...@googlegroups.com] On
> Behalf Of Steve Bennett

> Sent: 12 January 2011 05:28

> To: simal...@googlegroups.com
> Subject: Re: [simal-users] So, we need a project registry...
>

> On 12 January 2011 00:39, Sander W G van der Waal

> <sander.v...@oucs.ox.ac.uk> wrote:
>> It would be great if you could create an issue for any bug you find in the
>> tracker on our Google Code website [1]. However the dependency error you
>> mention below has (hopefully) been resolved today.
>

> Ok. It can be hard to tell whether it's a bug or user error though.

That's not a problem at all. It's very useful to have issues in there that
in the end turn out to be user errors. If you're hitting it, someone else
may as well so having it documented is useful.

> I've now got core and REST installed, and can access REST through Jetty.

> However, no webapp. I'm a bit confused because the instructions don't
> actually say to build in the simal.web directory.

Thanks for mentioning this, you're right that this instruction should have
been there. I've updated the GettingStarted wiki page.

> And if I try anyway (mvn install):
>
> [INFO] Building Simal Web Application 0.2.5-SNAPSHOT
> [INFO] --------------------------------------------------------------------
> ----
> [WARNING] The POM for org.joseki:joseki:jar:3.4.1 is missing, no dependency
> information available

The Maven repository we depended on for this jar (at 16degrees.com.au) is
offline. I changed the pom of the web project to use the regular openjena
repo [1], this resolved the issue on my machine. However, I'm not sure about
this message in your report:

> n the local repository, resolution will not be reattempted until the update

> interval of simal has elapsed or updates are forced -> [Help 1]

'simal' here refers to the repo that's offline. It might mean that you will
have to force Maven to check all the listed repos, eg. by manually removing
the Joseki folders/files in your local repo.

> Starting Jetty with Maven shows that the REST component is running but not
> the webapp:
>
> Error 404 - Not Found.
> No context on this server matched or handled this request.
> Contexts known to this server are:

> . /simal-rest ---
> > org.mortbay.jetty.plugin.Jetty6PluginWebAppContext@1a7c484{/simal-

> rest,C:\simal\simal-trunk\uk.ac.osswatch.simal.rest\src\main\webapp}

I suspect this is because the webapp project could not compile.

> (Also, the local.simal.properties file I created is not being used).

The instructions on this were unfortunately out of date following the
changes from Issue 347 [2]. I updated the getting started instructions
to reflect these changes. Note that a local properties file is not
required. If you choose not to specify local properties the webapp will
run on http://localhost:8080.

I also added a bit more explanation about using a non-default port number
for your the webapp because I didn't find that intuitive myself when
reading it. Bottom line is you will have to specify the non-standard port
number in the custom properties file as well as in the runtime parameter
jetty.port.

Sander

[1] http://code.google.com/p/simal/source/detail?r=2098
[2] http://code.google.com/p/simal/issues/detail?id=347

>
> I'm currently working on this functionality (Issue 335 [2]). As it stands,
> anybody can press the edit button to edit project details on the project
> detail
> page. There's also a very basic login facility. Nothing has yet been done
> to
> merge the two, although I'm building the editing stuff in a way that makes
> it
> easy to switch it on and off depending on whether you're logged in or not.
> Approving changes, as some sort of workflow step, is not currently on our
> roadmap.
>
> Ok. The reason I wonder is because the basic use case is likely to be that
> someone creates and registers the project, then completely forgets about us
> until 6 months later or something, then wants to update it. Usernames and
> passwords didn't seem like a good fit. But I guess we'll cross that bridge
> when we come to it.
>

> Thanks,
> Steve
> --
> You received this message because you are subscribed to the Google Groups
> "simal-users" group.
> To post to this group, send email to simal...@googlegroups.com.
> To unsubscribe from this group, send email to simal-

> users+un...@googlegroups.com.

[Steve Bennett]

On 13 January 2011 04:32, Sander W G van der Waal

<sander.v...@oucs.ox.ac.uk> wrote:
> The Maven repository we depended on for this jar (at 16degrees.com.au) is
> offline. I changed the pom of the web project to use the regular openjena
> repo [1], this resolved the issue on my machine. However, I'm not sure about
> this message in your report:

Thanks. I've done a fresh checkout and that built fine. So now I have
a working local web app. Yay :)

So, after a bit of fumbling around, I see I need to install Wookie to
get the DOAP authoring form. (Suggestion: link to DoapCreatorWidget as
an optional step at the end of the GettingStarted)

Installing Wookie, run into the annoying issue with Ant 1.8.x (which
they helpfully mention *after* telling you to simply install Ant). You
might want to mention this up front: download Ant 1.7.1 if using XP.

Just a comment at this point: is the Wookie dependency absolutely
necessary? It's a bit of a painful hurdle, downloading, building and
installing this immature software just to get...a form. I understand
that it's a whole widget framework, but for our purposes at least, we
could probably manage without? Maybe there could be a widgetless
version?

Moving on, a step in DoapCreatorWidget explaining how to deploy the
built widget would be good. (Answer appears to be go to
localhost:8888/wookie/admin, login java/java, select the file and push
'publish')

Shock, horror - that worked. DOAP Form is now visible.

Even more amazingly - it works. I entered some test data, it generated
RDF for me, and also saved it to the database. Ok, the user experience
is somewhat clunky, but it definitely works :)

Well, anyway, that's a very important first step. I guess next I will
have a play with the code and think about what we would want to
customise to make it most appropriate for our needs.

Steve