edit - test cycle in package development taking a long long time ...

[Thomas Lynch]

Hello, I must be doing this wrong.

I created a package  (now at github.com/Antigonus/liquid.git).  It is broken into liquid,  liquid-lib, and liquid-doc like other packages I see in the contributions.

There are many local function tests hooked into a list.  These were used mostly before the installing the package, and are stable.  That is nice.

I am writing some examples using the locally installed package. Invariably I'm running into features and change issues for the package, and bugs.   So here is the problem.  To make a change, this is what I am doing:

1. raco pkg remove liquid  -- I have to do this because otherwise there are dependency problems.

2. raco pkg remove liquid-lib 

3. make changes (takes very little time)

4. cd to liquid-lib and raco pkg install   ... etc.

5. test, find things, or introduce the next change item on the list, and repeat

Going through this loop takes a gawdawful long time.  About 1/8 cup of sipped coffee or two facebook posts (these are official developer time units ;-).  Ten little changes and runs through this loop can be an hour.  (If I run into those cases of errors without line numbers then a change can take an afternoon).  I can reduce the time a little bit by removing the liquid dependency.   But still the remove and replace of liquid-lib takes a long time.

At one point I added a symbolic link from the example directory to the liquid-lib directory, and named the link 'liquid' .. but this doesn't actually work because the naming convention for installed modules is different than that for locally required modules, hence I also had to add '.rkt'  and put quotes around the module names.   This helps when changes are isolated and one spends time in a one or two files so that munging of names is an incremental cost ...

Surely there is a better way to bring up examples and a library together???  I would very much appreciate learning how to do this correctly!  Thanks!

[Matthew Flatt]

Can you say more about why you uninstall "liquid-lib"?

If you're uninstalling so you can use `raco pkg install` to rebuild the
package's contents, then you can use `raco setup liquid` instead of
uninstalling and re-installing.

> --
> You received this message because you are subscribed to the Google Groups
> "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-dev/44a58e04-45be-43be-8215-9427eed8f4
> ec%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Thomas Lynch]

On Fri, Jul 17, 2015 at 5:19 PM, Matthew Flatt <mfl...@cs.utah.edu> wrote:

Can you say more about why you uninstall "liquid-lib"?

I installed the package so I could use it.

I am going through the pain of working with an unstable library so others won't have to so much.  Plus, I am writing an application of my own using the library.  There is nothing like putting something to use to find out what you really need from it.

So in examples/early applications I have,  (require liquid/red-lib)    instead of (require "path/red-lib.rkt"), just like a user would have.

If you're uninstalling so you can use `raco pkg install` to rebuild the
package's contents, then you can use `raco setup liquid` instead of
uninstalling and re-installing.

Yeah, I figured that out for the documents.  It works out well for them because they aren't called out in from any nascent application modules.

.. I wish I could set a search path or something during this early use phase so as to be able to modify the library without having the uninstall and compile reinstall steps on every trip through the modification-compile-test cycle..  You know,  just like it was lisp code or something ;-)   If I could remove those steps things would go more smoothly at this stage.  

[Thomas Lynch]

oh excuse me Matt,  why I "uninstall"

If I don't uninstall I get this:

1.  edit some module in the collection

2.  then type:

> raco pkg install

  raco pkg install: package is already installed

  package: liquid-lib

3.  oops  so try:

> raco pkg update liquid-lib

raco pkg update: cannot update linked packages without `--link'

  package name: liquid-lib

  package source: /home/deep/liquid-lib

ok earlier I was getting a more serious error with update that I thought implied I couldn't use it, perhaps something with the code at that point, that doesn't happen every time, so update is probably the answer.  What is the deal with link?   If this is the default, what am I typing here?

> raco pkg update --help

/ --link : Link a directory package source in place (default for a directory)

| --static-link : Link in place, promising collections do not change

[Thomas Lynch]

Alright, of course,  raco pkg update is  looking for new versions to install, it is not a make like utility just touching only modified parts.  Is there a make like option for install that only updates modified parts?

[Jay McCarthy]

Hi Thomas,

I think the thing you want to do is install the package with the
"link" setting. This won't copy any files but will basically install
symlinks in the right place. This means that any change to any module
will happen immediately. If you ever need to build the docs or test
the setup process, then you can run "raco setup".

Jay

> --
> You received this message because you are subscribed to the Google Groups
> "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-dev/44a58e04-45be-43be-8215-9427eed8f4ec%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Jay McCarthy
http://jeapostrophe.github.io

"Wherefore, be not weary in well-doing,
for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
- D&C 64:33

[Thomas Lynch]

On Fri, Jul 17, 2015 at 8:22 PM, Jay McCarthy <jay.mc...@gmail.com> wrote:

Hi Thomas,

I think the thing you want to do is install the package with the
"link" setting. This won't copy any files but will basically install
symlinks in the right place. This means that any change to any module
will happen immediately. If you ever need to build the docs or test
the setup process, then you can run "raco setup".

Jay

Ah Ok,  thanks!  whew  been drinking way too much coffee ;-)  ..ah this is what I thought already happened by virtue of raco pkg install from my development directory.  Is something else happening?  

I'll look for the explicit option.  The update option notes that --link is the default for directories.

Now another point of confusion, setup made a directory called 'compiled'  so I gather raco setup is compiling modules into the compiled directory.  Will raco setup do this in an incremental fashion.  Say if I mod one module then run raco setup, will it just compile the one?  Or will all the modules in the collection by compiled again?

[Greg Hendershott]

> Now another point of confusion, setup made a directory called 'compiled' so
> I gather raco setup is compiling modules into the compiled directory. Will
> raco setup do this in an incremental fashion. Say if I mod one module then
> run raco setup, will it just compile the one? Or will all the modules in
> the collection by compiled again?

`raco setup` compiles only as necessary. (The `compiled` dir has both
.zo and .dep files. The latter help determine "necessary".)


p.s. This discussion is useful for understanding how to do a project
with separate foo, foo-lib, and foo-doc packages, like in the Racket
distribution. But -- is that split valuable enough to be worth the
hassle, for most packages?

[Thomas Lynch]

The thing I was doing wrong is racket install from a directory after each change, when instead, apparently, the files can be left local in the directory and just updated incrementally with racket setup.  The setup instead of install flow will save time, and avoid the dependency problem I had due to the split up.

I'm not familiar enough with this to speak to whether the split is useful.  I just chose a similar complexity package install to emulate after looking in the distribution.  It does seem like a natural partitioning,  lib, docs, examples...

 

[Greg Hendershott]

I don't mean to discourage you from splitting.

I'm just not sure it's worthwhile for most packages. As a result, I
wouldn't want people to feel it's generally expected.

(I wouldn't want that to discourage someone from making a package at
all. Or to divert too much time from other ways to make the package
better. Or be an excuse to say, well docs are already hard, and
putting them in a separate -doc package is the last straw, so I won't
bother with docs at all. And so on.)

[Thomas Lynch]

for the benefit of those who may come later ...

I have been developing code for a package that will eventually be 'contributed'  at github.com/Antigonus/liquid.  It is broken into multiple dependent packages,  liquid  (top level), liquid-lib (has the modules that users make use of),  liquid-doc (has the scribble docs in it).  See the git hub for the "info.rkt" files that glue it all into one package.

The local install for testing and example development  had to be done only once.  Because the install was from a directory the code was linked by name into the catalog  rather than moved into the /usr/share/racket/pkg hierarchy.  This is the default behavior when installing from a directory.  Because of dependencies, I had to do the local install like this:

cd /top/liquid-doc

raco pkg install

cd /top/liquid-lib

raco pkg install

etc. for other split out parts .. and then

cd /top/liquid

raco pkg install

The install only has to be done once.   Once installed, my library modules are (require liquid/module)  instead of (require "a-rel-path/module.rkt").   As they haven't moved, I can make modifications in the original files.  There are no copies to deal with.

After the install, modifications to files are compiled and thus included in the installation by using a single command:

raco setup liquid

It is pretty quick.  And my gosh much much better than what I was doing before (reinstalling).

Thanks Matt, Greg.