[fossil-users] Make a CGI request WITHOUT a script file

[Guillermo Estrada]

Hi, I've been trying to implement a webservice that hosts fossil repositories. Kinda like chiselapp.com . I have achieved running my own server app written in golang to make the CGI requests in a routed handler (i.e /user/repository ), but the problem I'm facing is that the only way I can make it work is if a cgi script exists to pass as argument to fossil.

Is there any way around this? Like setting the repository or directory in an environment variable or something? You can understand that I don't want having lots of CGI scripts around when I could just ask the database and call fossil through cgi direclty, reading through 1 or 2 line cgi scripts dosn't seem much, but it is unecessary for a service like this (creating the scripts for each user/repo setting file permissions, etc...) when I can just point fossil where the directory or repository is.

I do know CGI conventions and standards do not work like that, but is there something I don't know about fossil? Is there a way to request a feature to do this? This will boost a lot the potential of running fossil repositories behind dynamic applications.

Thanks to everyone for your wisdom and guidance in this matter.

Sincerely,
Guillermo Estrada

--
"No le digas a Dios lo grande de tu tormenta, mejor dile a tu tormenta lo grande que es tu Dios."

[Stephan Beal]

On Sat, Mar 3, 2012 at 12:04 AM, Guillermo Estrada <phro...@gmail.com> wrote:

I do know CGI conventions and standards do not work like that, but is there something I don't know about fossil? Is there a way to request a feature to do this? This will boost a lot the potential of running fossil repositories behind dynamic applications.

i don't think fossil can do this out of the box, but one option might be to generate the .cgi script wrappers on the fly (perhaps as temp files), and then run:

fossil cgi /path/to/the/script

-- 

----- stephan beal
http://wanderinghorse.net/home/stephan/

http://gplus.to/sgbeal

[Richard Hipp]

On Fri, Mar 2, 2012 at 6:04 PM, Guillermo Estrada <phro...@gmail.com> wrote:

Hi, I've been trying to implement a webservice that hosts fossil repositories. Kinda like chiselapp.com . I have achieved running my own server app written in golang to make the CGI requests in a routed handler (i.e /user/repository ), but the problem I'm facing is that the only way I can make it work is if a cgi script exists to pass as argument to fossil.

Is there any way around this? Like setting the repository or directory in an environment variable or something? You can understand that I don't want having lots of CGI scripts around when I could just ask the database and call fossil through cgi direclty, reading through 1 or 2 line cgi scripts dosn't seem much, but it is unecessary for a service like this (creating the scripts for each user/repo setting file permissions, etc...) when I can just point fossil where the directory or repository is.

I do know CGI conventions and standards do not work like that, but is there something I don't know about fossil? Is there a way to request a feature to do this? This will boost a lot the potential of running fossil repositories behind dynamic applications.

I experimented some with adding the capability to specify the repository on the CGI command-line.  But this seems like a relatively obscure feature.  And it looks like that once I add it, it will soon be followed by a cascade of related requests for options to specify a repository directory, a debug log, options to specify the home directory, etc, etc.

You already have to have a separate repository file for each client in your service:  "xxxxx.fossil".  How much more trouble is it, really, to have a separate CGI file in the same directory:  "xxxxx.cgi". 

Thank you for the suggestion, but I think this is something that is best left to the ingenuity of the web service.
 

Thanks to everyone for your wisdom and guidance in this matter.

Sincerely,
Guillermo Estrada

--
"No le digas a Dios lo grande de tu tormenta, mejor dile a tu tormenta lo grande que es tu Dios."

_______________________________________________
fossil-users mailing list
fossil...@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

--
D. Richard Hipp
d...@sqlite.org

[Dmitry Chestnykh]

On 03/03/2012 12:04 AM, Guillermo Estrada wrote:
> Hi, I've been trying to implement a webservice that hosts fossil

> repositories. Kinda like chiselapp.com <http://chiselapp.com> . I have

> achieved running my own server app written in golang to make the CGI
> requests in a routed handler (i.e /user/repository ), but the problem
> I'm facing is that the only way I can make it work is if a cgi script
> exists to pass as argument to fossil.

Interesting! Will it be open source?

> Is there any way around this?

I assume your Go program launches Fossil. When you open a CGI process,
can't you just write to its standard input:

repository: /path/to/repo.fossil

?

--
Dmitry Chestnykh
http://www.codingrobots.com

[Stephan Beal]

On Sat, Mar 3, 2012 at 3:30 PM, Dmitry Chestnykh <dmi...@codingrobots.com> wrote:

I assume your Go program launches Fossil. When you open a CGI process, can't you just write to its standard input:

repository: /path/to/repo.fossil

That won't work: the stdin for a CGI process is the HTTP request headers and POST data.

[Dmitry Chestnykh]

On 03/03/2012 03:34 PM, Stephan Beal wrote:
> On Sat, Mar 3, 2012 at 3:30 PM, Dmitry Chestnykh
> <dmi...@codingrobots.com <mailto:dmi...@codingrobots.com>> wrote:
>
> I assume your Go program launches Fossil. When you open a CGI
> process, can't you just write to its standard input:
>
> repository: /path/to/repo.fossil
>
>
> That won't work: the stdin for a CGI process is the HTTP request headers
> and POST data.

Of course! Pardon my moment of stupidity.

[Ashish SHUKLA]

Guillermo Estrada writes:
> Hi, I've been trying to implement a webservice that hosts fossil
> repositories. Kinda like chiselapp.com . I have achieved running my own
> server app written in golang to make the CGI requests in a routed handler
> (i.e /user/repository ), but the problem I'm facing is that the only way I
> can make it work is if a cgi script exists to pass as argument to fossil.

> Is there any way around this? Like setting the repository or directory in
> an environment variable or something? You can understand that I don't want
> having lots of CGI scripts around when I could just ask the database and
> call fossil through cgi direclty, reading through 1 or 2 line cgi scripts
> dosn't seem much, but it is unecessary for a service like this (creating
> the scripts for each user/repo setting file permissions, etc...) when I can
> just point fossil where the directory or repository is.

> I do know CGI conventions and standards do not work like that, but is there
> something I don't know about fossil? Is there a way to request a feature to
> do this? This will boost a lot the potential of running fossil repositories
> behind dynamic applications.

> Thanks to everyone for your wisdom and guidance in this matter.

Did you look at reverse-proxying to fossil server? I've improved
reverse-proxying support in Fossil[1] to do something like what you're trying
to do. Latest changes are available from 'ashish-ipv6' branch.

References:
[1] http://www.mail-archive.com/fossil...@lists.fossil-scm.org/msg06298.html

HTH
--
Ashish SHUKLA

“There are two major products that come out of Berkeley: LSD and
UNIX. We don't believe this to be a coincidence.” (Jeremy S. Anderson)

[Guillermo Estrada]

First of all pardon my stupidity by replying to the digest!!!!

I don't have any trouble generating cgi files temporal or static ones. It
is not a matter of ingenuity, but generating cgi files for tons of repos
(launchpad, bitbucket, github, googlecode, I think fossil can have one too,
that will boost his adoption) generates not only disk latency while reading
(though small) but also security issues and a lot more of administration
tasks (like backups and such). I do think that "If there is a Environment
Variable called Directory or repository, use that one instead" is not a
huge feature that will affect anyone, only benefit.

> Did you look at reverse-proxying to fossil server? I've improved
> reverse-proxying support in Fossil[1] to do something like what you're trying
> to do. Latest changes are available from 'ashish-ipv6' branch.

References:
[1]  http://www.mail-archive.com/fossil...@lists.fossil-scm.org/msg06298.html

As far as I know the proxy server way can't serve multiple repos, that
truly would be the most optimal way to host such a service (agan, disk
latency) starting with pretty small hardware. But then again, not having to
rely on lots of cgi scripts to serve the repositories it's a nice start to
efficient hosting.

> Interesting! Will it be open source?

I actually thought of it as a free service mostly using it for personal and
some friends in the community, but then again I would have no trouble open
sourcing it for others to use. It would have an admin interface and
avatars, comments and such, a friend of mine was thinking of a theme/skin
gallery too. I love how fossil works, and my goal was to enable "go install
service.com/user/repository" style of installing golang libraries with
fossil.

> I assume your Go program launches Fossil. When you open a CGI process,
> can't you just write to its standard input:

That could be, but the fossil expects the location of the file through the
arguments in the command line, am i correct? As someone else pointed out
everything else is expected through the environment variables.

> i don't think fossil can do this out of the box, but one option might be to
> generate the .cgi script wrappers on the fly (perhaps as temp files)

Again, more secure, but twice the latency.

Thank you for all your support and answers, I appreciate them a lot. If
anyone knows a way around it, or if it's easy to implement a env check for
the route it would be awesome.

Cheers,
Guillermo Estrada

--
"No le digas a Dios lo grande de tu tormenta, mejor dile a tu tormenta lo grande de tu Dios."

[Remigiusz Modrzejewski]

On Mar 4, 2012, at 10:43 , Guillermo Estrada wrote:

> As far as I know the proxy server way can't serve multiple repos, that
> truly would be the most optimal way to host such a service (agan, disk
> latency) starting with pretty small hardware. But then again, not having to
> rely on lots of cgi scripts to serve the repositories it's a nice start to
> efficient hosting.

Why? The server itself supports multiple repositories. I'm reverse-proxying to a single Fossil instance serving over 20 repos at http://dev.lrem.net/ and it works like a charm...

Kind regards,
Remigiusz Modrzejewski

[Guillermo Estrada]

> Why? The server itself supports multiple repositories. I'm reverse-proxying to a single Fossil instance serving over 20 repos at http://dev.lrem.net/ and it works like a charm...

From here:
http://lists.fossil-scm.org:8080/pipermail/fossil-users/2010-January/001997.html

I read a lot of useful information about the reverse proxy option. I have already tested it and it works with subdirectories too! (ie. /user/repository) in the url, so it's going to be the way to go. Thank you for pointing it to me. Just a question, is there any way to serve and index or do a redirect? The reverse proxy always need a repository when accessed by the url. But when I go to "/" it just sends a 404 Not Found as expected. I guess that is not gonna matter cause I intend to catch all routing with my app before serving them with the reverse proxy, but anyway it would be nice to know.

Thank you for all your knowledge and guidance.

Sincerely,