pgxn_client should has an option to create an extension skeleton
Dickson S. Guedes
i was very tired to create all the skeleton all the time..
so, how i couldn't find the source off the pgxnclient
i created a bash script [1] to create an extension skeleton
for me.
It was made quicklly and i didn't much test with it, so it is
in alpha or some kind of a Prof of concept.
What do you think about?
[1] https://github.com/guedes/pgxn-utils/blob/master/pgxn_create_extension
regards,
Guedes
--
Dickson S. Guedes
mail/xmpp: gue...@guedesoft.net - skype: guediz
http://guedesoft.net - http://www.postgresql.org.br
Jonathan "Duke" Leto
This script looks really useful!
My suggestions:
1) Don't try to detect the current username
2) Add a command-line flag to specify the Name+Email to use
Duke
--
Jonathan "Duke" Leto <jona...@leto.net>
209.691.DUKE // http://leto.net
NOTE: Personal email is only checked twice a day at 10am/2pm PST,
please call/text for time-sensitive matters.
David E. Wheeler
> This script looks really useful!
>
> My suggestions:
>
> 1) Don't try to detect the current username
> 2) Add a command-line flag to specify the Name+Email to use
Also version and perhaps long description, tags. And maybe a template for the documentation, doc/$extension.md.
Looks great, thanks!
David
Dickson S. Guedes
> Howdy,
>
> This script looks really useful!
>
> My suggestions:
>
> 1) Don't try to detect the current username
> 2) Add a command-line flag to specify the Name+Email to use
It is good to change this script or integrate this functionality
within pgxnclient?
Should I convert it to python and pull a patch to pgxnclient? If
'yes', is this [1] the correct repository?
[1] https://github.com/dvarrazzo/pgxnclient
--
Dickson S. Guedes
e-mail/gtalk: gue...@guedesof.net - skype: guediz
David E. Wheeler
> It is good to change this script or integrate this functionality
> within pgxnclient?
>
> Should I convert it to python and pull a patch to pgxnclient? If
> 'yes', is this [1] the correct repository?
>
> [1] https://github.com/dvarrazzo/pgxnclient
That's the right one, yes. My only question is whether we want to keep tools for developers, like this, separate from tools for end-users?
Best,
David
Dickson S. Guedes
So could be a pgxnclient and a pgxnutils, the first one for end-user
and the last with options for developers.
This last one could provide some options like:
- create extension skeleton: a start point for create a new cool
extension with some templates for docs, sql, tests, README, etc.
- run tests: shortcuts to run tests and generate its outputs in other
formats (?)
- manage the semver of extension: to update version in the files and
tag in version control as an option?
- upload the extension: the api has this feature ?
- ...
What do you think people? Is this useful too or just the extension
skeleton? One tool with many shortcuts and options or a set of tools
made separated? I lost something?
Best,
David E. Wheeler
> This last one could provide some options like:
>
> - create extension skeleton: a start point for create a new cool
> extension with some templates for docs, sql, tests, README, etc.
> - run tests: shortcuts to run tests and generate its outputs in other
> formats (?)
> - manage the semver of extension: to update version in the files and
> tag in version control as an option?
> - upload the extension: the api has this feature ?
Yes, although I need to do a bit of tweaking. But it's there, yes.
> - ...
>
> What do you think people? Is this useful too or just the extension
> skeleton? One tool with many shortcuts and options or a set of tools
> made separated? I lost something?
I kind of like the git model, where there's a single executable (git) that, depending on what command you pass to it, executes some other executable. So `git log` executes `git-log` and `git svn` executes `git-svn`. If you don't have git-svn (maybe you don't have Perl installed), then that particular command just doesn't work.
Best,
David
Dickson S. Guedes
>
>> This last one could provide some options like:
>>
>> - create extension skeleton: a start point for create a new cool
>> extension with some templates for docs, sql, tests, README, etc.
>> - run tests: shortcuts to run tests and generate its outputs in other
>> formats (?)
>> - manage the semver of extension: to update version in the files and
>> tag in version control as an option?
>> - upload the extension: the api has this feature ?
>
> Yes, although I need to do a bit of tweaking. But it's there, yes.
Nice. And I suppose that pgxnclient only download the extensions,
doesn't upload one created. I do it by packaging it and uploading on
site, right?
>> - ...
>>
>> What do you think people? Is this useful too or just the extension
>> skeleton? One tool with many shortcuts and options or a set of tools
>> made separated? I lost something?
>
> I kind of like the git model, where there's a single executable (git) that, depending on what command you pass to it, executes some other executable. So `git log` executes `git-log` and `git svn` executes `git-svn`. If you don't have git-svn (maybe you don't have Perl installed), then that particular command just doesn't work.
You read my mind David.
Just wondering in what language the "main" program should be created.
I'm thinking in portability to but I work better with bash or ruby
than python ":/
I'm thinking to start a prototype based on all that cool suggestions
and push to github.
What you think people?
Best
David E. Wheeler
>> Yes, although I need to do a bit of tweaking. But it's there, yes.
>
> Nice. And I suppose that pgxnclient only download the extensions,
> doesn't upload one created. I do it by packaging it and uploading on
> site, right?
Right.
> You read my mind David.
>
> Just wondering in what language the "main" program should be created.
> I'm thinking in portability to but I work better with bash or ruby
> than python ":/
Sure. The stuff I wrote is Perl, Daniele's is Python, we clearly need some Ruby stuff now. ;-P
> I'm thinking to start a prototype based on all that cool suggestions
> and push to github.
>
> What you think people?
+1, but see what Daniele says about his plans…
Best,
David
Daniele Varrazzo
Hello!
Reading now the thread... interesting!
I see the script is really easy: mostly two templates and variables
replacement. I don't think it would be a problem to integrate it in
the main client. While I started writing the client, I've thought
about an extension mechanism, but because pgxn is relatively simple -
more than mercurial for instance (an example of extendible script with
subcommand in python, allowing e.g. the "graph" plugin to add the
option "-g" to the "log" command etc.), I've preferred not
overengineering it. So:
1. pgxnclient is implicitly extendible: there is a class Command and,
if a subclass of it is created, it is automatically "known" by the
program, so pgxn --help lists it and pgxn <command> --help gives its
help and it receives the global extensions etc. All the client's
current commands are implemented this way, none is somehow
"privileged". The internal protocol is very simple (example: the
download command is here: http://tinyurl.com/66vsk4r): a command is a
Command subclass defining:
- a name to be called with (if it's not set it means that the subclass
is not a real command but another base class, so it's not added to the
command line),
- a description to be shown as one-line help,
- a customize_parser() method if it needs to add options to the command line,
- a run() method called to make the work.
2. extending it via external commands (so that "pgxn create" would
look for and invoke pgxn-create) I think requires to establish a
protocol between the main script and the external ones, for instance
to get an help line, and then the subcommands should reimplement
option parsing, which I find a duplication of effort. For instance
"pgxn upload" would need a --mirror option, and this should be
implemented by an eventual external command and manually kept
consistent with the other commands, whereas with a new python command
it would come for free as a global option. Another example: a command
requiring access to the database, if written in Python, can inherit
from the WithDatabase mixin class and have all the relevant options
parsed (-U/--username etc.). Similarly there are mixins WIthMake,
WithSpec, all providing option parsing and services that would
otherwise require re-implementation.
For this reason I think the easiest way to extend the client would be
to add a subpackage to it and a plugin could be implemented as module
in that package (e.g. the file pgxnclient/commands/create.py) defining
one or more command subclass, plus whatever else code it takes to make
them work.
If bigger use cases arise, we may think about option 2, but I really
think that option 1. would be the easiest way to go.
So possible followups are:
a. Dickson, you may implement the command in Python, I may write the
extension mechanism in 1. and we can put the things together (I
actually have to refactor commands.py: it's dangerously close to the
1k lines. I want to have a module (or more) for the base classes and
mixins and another module (or more) for the concrete command classes).
Of course I have no problem in having your work in the main code, it
doesn't have to be an external project.
b. Dickson, you implement the command in whatever language you want
and I can convert the implementation to python into pgxnclient, with
your credits.
c. We can decide a way to make pgxn operate as in 2; in this case you
should explain what should I present to the user when it asks for pgxn
--help.
These are in order of my preference, but I'm happy to hear your
opinion. About the timing, I think I will have the entire weekend for
hacking.
-- Daniele
Dickson S. Guedes
> On Fri, May 6, 2011 at 12:34 AM, David E. Wheeler <da...@kineticode.com> wrote:
>> On May 5, 2011, at 4:17 PM, Dickson S. Guedes wrote:
>>> I'm thinking to start a prototype based on all that cool suggestions
>>> and push to github.
>>>
>>> What you think people?
>>
>> +1, but see what Daniele says about his plans…
>
> Hello!
>
> Reading now the thread... interesting!
[... cut ...]
> So possible followups are:
>
> a. Dickson, you may implement the command in Python, I may write the
> extension mechanism in 1. and we can put the things together (I
> actually have to refactor commands.py: it's dangerously close to the
> 1k lines. I want to have a module (or more) for the base classes and
> mixins and another module (or more) for the concrete command classes).
> Of course I have no problem in having your work in the main code, it
> doesn't have to be an external project.
>
> b. Dickson, you implement the command in whatever language you want
> and I can convert the implementation to python into pgxnclient, with
> your credits.
>
> c. We can decide a way to make pgxn operate as in 2; in this case you
> should explain what should I present to the user when it asks for pgxn
> --help.
>
> These are in order of my preference, but I'm happy to hear your
> opinion.
My python-fu is very low and not enough to do the 'a' in a better way,
i'll use the 'b' alternative for now.
David suggests something that i like: client tools and a developer
tools. If pgxn walks to that approach, I think that tools could be
centralized on 'pgxn' utility so the 'c' alternative will be a way to
implement it, but all called tools should implements the same
protocol, as Daniele sad, there will be some duplications, so i'll try
to implement something here, in ruby initially... and If this starts
to be bad, I help to translate it to python.
> About the timing, I think I will have the entire weekend for hacking.
Nice! I will post a prof-of-concept tonight to github and to the list,
because i'll visit my parents this weekend, i'll off until Monday.
Thanks all!
Dickson S. Guedes
[... cut ...]
> (I actually have to refactor commands.py: it's dangerously close to the
> 1k lines. I want to have a module (or more) for the base classes and
> mixins and another module (or more) for the concrete command classes).
Daniele, did you know about 'plac' [1] or 'commandant' [2] for
python? I've found they when searching for something like Thor [3] for
python.
[1] http://pypi.python.org/pypi/plac/0.8.1
[2] https://launchpad.net/commandant
[3] https://github.com/wycats/thor/wiki/Getting-Started
my $ 0.02 cents...
Best..
Daniele Varrazzo
> 2011/5/6 Daniele Varrazzo <daniele....@gmail.com>:
> [... cut ...]
>> (I actually have to refactor commands.py: it's dangerously close to the
>> 1k lines. I want to have a module (or more) for the base classes and
>> mixins and another module (or more) for the concrete command classes).
>
> Daniele, did you know about 'plac' [1] or 'commandant' [2] for
> python? I've found they when searching for something like Thor [3] for
> python.
I've used argparse, as it's available for python 2.4, the oldest
version I want to support (as it is still shipped on CentOS) up to the
foreseeable future, as it's in the stdlib now.
The 1k line of code is not the command line parsing :) it's the
commands implementation.
Plac is a wrapper around argparse to somehow dumb it down, a need I
don't feel; I didn't know commandant, but it seems very easy to
implement the same functionality without requiring bzrlib. I don't
like the implementation: you are supposed to provide a text file
containing the help: it seems a step backwards respect to commands
generating the help themselves. I'd rather call all the subcommands
with --help and get the first line from there.
-- Daniele
Dickson S. Guedes
> 2011/5/6 Daniele Varrazzo <daniele....@gmail.com>:
>> About the timing, I think I will have the entire weekend for hacking.
>
> Nice! I will post a prof-of-concept tonight to github and to the list,
> because i'll visit my parents this weekend, i'll off until Monday.
Hi all,
In this thread, Daniele suggested some alternatives: use python or
some language and so translated to fit the pgxn needs. How my
python-fu isn't enough to express what i'm trying/like to do, I've
started something in Ruby, and put it in a branch on github [1].
Yes, it is greater than the bash version, sorry... But have test, a
template structure used by Thor [2] and, in really, all the code to
create the skeleton [3] in the destination directory of extension is
in one-line basically [4,5].
Yes, it is a draft yet, but functional, as you can see in the screencast [6].
At this moment i'm in doubt: should I continue putting other features
in it and try to make it play with the already existent pgxn command
line protocol based on listed appointments by Daniele or stop now and
try to translate it to python to merge it on pgxn?
I'd like to hear your opinions.
[1] https://github.com/guedes/pgxn-utils/tree/pgxn_utils
[2] https://github.com/wycats/thor, https://github.com/wycats/thor/wiki
[3] https://github.com/guedes/pgxn-utils/tree/pgxn_utils/lib/pgxn_utils/templates/root
[4] https://github.com/guedes/pgxn-utils/blob/pgxn_utils/lib/pgxn_utils/cli.rb#L17-18
[5] http://rdoc.info/github/wycats/thor/master/Thor/Actions#directory-instance_method
[6] http://pgcasts.com/media/pgxn_utils-usage-example.mpeg
Best
Daniele Varrazzo
> 2011/5/6 Dickson S. Guedes <gue...@guedesoft.net>:
>> 2011/5/6 Daniele Varrazzo <daniele....@gmail.com>:
>>> About the timing, I think I will have the entire weekend for hacking.
>>
>> Nice! I will post a prof-of-concept tonight to github and to the list,
>> because i'll visit my parents this weekend, i'll off until Monday.
>
> Hi all,
>
> In this thread, Daniele suggested some alternatives: use python or
> some language and so translated to fit the pgxn needs. How my
> python-fu isn't enough to express what i'm trying/like to do, I've
> started something in Ruby, and put it in a branch on github [1].
>
> Yes, it is greater than the bash version, sorry... But have test, a
> template structure used by Thor [2] and, in really, all the code to
> create the skeleton [3] in the destination directory of extension is
> in one-line basically [4,5].
I think better in Ruby than in Bash: allows you to grow much better.
> Yes, it is a draft yet, but functional, as you can see in the screencast [6].
Very nice! I like very much the idea of the diff on conflict. Is there
also the possibility to fire an external merge program?
> At this moment i'm in doubt: should I continue putting other features
> in it and try to make it play with the already existent pgxn command
> line protocol based on listed appointments by Daniele or stop now and
> try to translate it to python to merge it on pgxn?
I think you should keep working with the language that makes you more
productive now that seems you have ideas to develop.
> I'd like to hear your opinions.
One thing about the interface: I think the -e option shouldn't be an
option but a mandatory parameter: I would have called pgxn_utils
create_extension [options] NAME, with a --target option to specify the
directory, defaulting to ".".
Nice work :) Cheers
-- Daniele
Dickson S. Guedes
> On Wed, May 11, 2011 at 5:22 AM, Dickson S. Guedes <gue...@guedesoft.net> wrote:
>> Yes, it is a draft yet, but functional, as you can see in the screencast [6].
>
> Very nice! I like very much the idea of the diff on conflict. Is there
> also the possibility to fire an external merge program?
Yes, in fact it uses "diff -u" by default, but reading some code from
Thor [1] i've found that it expects the environments variables
THOR_DIFF or RAILS_DIFF. I'll wrapper this in another name, maybe a
DIFFTOOL environment variable? (internally i'll read it from evn and
set THOR_DIFF)
>> At this moment i'm in doubt: should I continue putting other features
>> in it and try to make it play with the already existent pgxn command
>> line protocol based on listed appointments by Daniele or stop now and
>> try to translate it to python to merge it on pgxn?
>
> I think you should keep working with the language that makes you more
> productive now that seems you have ideas to develop.
Thanks!
>> I'd like to hear your opinions.
>
> One thing about the interface: I think the -e option shouldn't be an
> option but a mandatory parameter: I would have called pgxn_utils
> create_extension [options] NAME, with a --target option to specify the
> directory, defaulting to ".".
I agree, in fact i feel this too when recording that screencast.
> Nice work :) Cheers
Thanks! I'll consider all you suggestions, make some changes and poke
you again ":)
[1] https://github.com/wycats/thor/blob/master/lib/thor/shell/basic.rb#L249
Daniele Varrazzo
> 2011/5/11 Daniele Varrazzo <daniele....@gmail.com>:
>> Very nice! I like very much the idea of the diff on conflict. Is there
>> also the possibility to fire an external merge program?
>
> Yes, in fact it uses "diff -u" by default, but reading some code from
> Thor [1] i've found that it expects the environments variables
> THOR_DIFF or RAILS_DIFF. I'll wrapper this in another name, maybe a
> DIFFTOOL environment variable? (internally i'll read it from evn and
> set THOR_DIFF)
I've not understood yet what thor is :) I thought it was a command
line parser, not something dealing with diffs or spawning external
programs.
-- Daniele
David E. Wheeler
>> Yes, it is a draft yet, but functional, as you can see in the screencast [6].
>
> Very nice! I like very much the idea of the diff on conflict. Is there
> also the possibility to fire an external merge program?
Huh, the screencast doesn't play for me in QuickTime. Ah…works in Movist. Good.
/me watches it…
Oh, WOW. I'm blown away. This is way more awesome than anything I envisioned. Thanks Dickson, this is *amazing!* I'm going to use this a lot over the next couple of years, I can tell.
> I think you should keep working with the language that makes you more
> productive now that seems you have ideas to develop.
+1
The more languages the better, IMHO.
Best,
David
Dickson S. Guedes
Hum... sorry, each time I cite Thor I present different characteristics :D.
Well I'll try to explain.
Thor [1] is about tasks, working as a "scripting framework" to build
self-documenting command line utilities, but it have built-in
facilities [2] that short some jobs, create directory structures,
inject text in some files, change a value of some key in a hashstore
file or based on a regex and many other things [3]. The syntax is
Rake-like, so it should be familiar to most Rake users.
I'm changing many shell scripts to use it instead and I'm happy with,
I'm clearing code and documenting them better, IMHO.
[1] https://github.com/wycats/thor
[2] http://rdoc.info/github/wycats/thor/master/Thor/Actions
[3] https://github.com/wycats/thor/wiki
Dickson S. Guedes
> On May 11, 2011, at 2:15 AM, Daniele Varrazzo wrote:
>
>>> Yes, it is a draft yet, but functional, as you can see in the screencast [6].
>>
>> Very nice! I like very much the idea of the diff on conflict. Is there
>> also the possibility to fire an external merge program?
>
> Huh, the screencast doesn't play for me in QuickTime. Ah…works in Movist. Good.
>
> /me watches it…
>
> Oh, WOW. I'm blown away. This is way more awesome than anything I envisioned. Thanks Dickson, this is *amazing!* I'm going to use this a lot over the next couple of years, I can tell.
I'm trying my best to do it and I'm very happy to read this, thanks
David and Daniele! :D
In time, I'll create more two tasks, one for package the extension in
some format supported by pgxn and another to push the package to PGXN.
I'm wondering if "package" and "push" are good names. Any opinion?
regards,
Guedes
David E. Wheeler
> I'm trying my best to do it and I'm very happy to read this, thanks
> David and Daniele! :D
>
> In time, I'll create more two tasks, one for package the extension in
> some format supported by pgxn and another to push the package to PGXN.
> I'm wondering if "package" and "push" are good names. Any opinion?
"package" isn't bad. "bundle" maybe? I dunno. I think "release" is better than "push", though.
Best,
David
Daniele Varrazzo
I would say "package" and "upload". By the way "skeleton" looks
prettier than create_extension for me... but it's just bikeshedding of
course.
-- Daniele
Dickson S. Guedes
In the site I see "Release It" and "Upload distribution", i think
"release" is better too, since it could change version internally and
in archive filename.
I'm in doubt between "package" and "bundle".
David E. Wheeler
> I would say "package" and "upload".
Yea, but the tagline for PGXN, appearing in my talk and on the t-shirts, is "Release it!" :-)
> By the way "skeleton" looks
> prettier than create_extension for me... but it's just bikeshedding of
> course.
+1
David
David E. Wheeler
>> would say "package" and "upload".
>
> Yea, but the tagline for PGXN, appearing in my talk and on the t-shirts, is "Release it!" :-)
BTW, I'm updating the HOWTO right now to add details on 9.1-style extension stuff and remembered something I wanted as part of the "package" task: control file generation. I think that the control file can and should be generated from the META.json, if there is no specific control file. This is because there's quite a bit of duplicate information.
Thoughts?
Best,
David
Dickson S. Guedes
> 2011/5/6 Dickson S. Guedes <gue...@guedesoft.net>:
>> 2011/5/6 Daniele Varrazzo <daniele....@gmail.com>:
>>> About the timing, I think I will have the entire weekend for hacking.
>>
>> Nice! I will post a prof-of-concept tonight to github and to the list,
>> because i'll visit my parents this weekend, i'll off until Monday.
I've pushed my work-in-progress to master [1] (yes, I feel it is still
WIP). I'm open to suggestion, opinions, tomatoes.. ":)
Thanks all that made this possible by very nice and cool opinions in
this thread and on IRC!
[1] https://github.com/guedes/pgxn-utils
regards,
Guedes
David E. Wheeler
> I've pushed my work-in-progress to master [1] (yes, I feel it is still
> WIP). I'm open to suggestion, opinions, tomatoes.. ":)
>
> Thanks all that made this possible by very nice and cool opinions in
> this thread and on IRC!
>
> [1] https://github.com/guedes/pgxn-utils
Awesome, thanks!
Now, how do I get RubyGems upgraded…
Best,
David