openwrap: Apache repository and separate packaging of projects
18 views
Skip to first unread message
Reinhold Rumberger
Aug 24, 2012, 10:44:57 AM8/24/12
to openevery...@googlegroups.com
Hi!
I've only recently begun using openwrap, so I hope I haven't missed anything
obvious.
I have two things left I'm trying to do, and can't seem to figure out how to.
The first is that I would like to set up my own private repository using
Apache as server. I'd like to be able to both publish to this repository via
HTTP and use that repository for dependency resolution.
So far my workaround is to publish to a local directory and mirror that to the
server, having Apache handle the serving. This is not an acceptable solution,
though, since publishing and retrieval is done via two different remotes.
Also, it doesn't work once more people need to push to the HTTP repository.
My second problem is that my solution is split into several interdependent
projects. Since other (external) projects want to use as few assemblies as
possible, they should be able to depend on individual projects, not just the
entire solution.
How would I best go about implementing this? My attempt to create several
.wrapdescs for each of the projects in the main solution directory doesn't
work, and if I understand the code correctly, placing them in individual
sub-directories with the solution .wrapdesc still in the main directory won't
work, either. Also, that workaround would mean several "wraps/" directories,
which I would like to avoid.
Sincerely,
Reinhold Rumberger
I've only recently begun using openwrap, so I hope I haven't missed anything
obvious.
I have two things left I'm trying to do, and can't seem to figure out how to.
The first is that I would like to set up my own private repository using
Apache as server. I'd like to be able to both publish to this repository via
HTTP and use that repository for dependency resolution.
So far my workaround is to publish to a local directory and mirror that to the
server, having Apache handle the serving. This is not an acceptable solution,
though, since publishing and retrieval is done via two different remotes.
Also, it doesn't work once more people need to push to the HTTP repository.
My second problem is that my solution is split into several interdependent
projects. Since other (external) projects want to use as few assemblies as
possible, they should be able to depend on individual projects, not just the
entire solution.
How would I best go about implementing this? My attempt to create several
.wrapdescs for each of the projects in the main solution directory doesn't
work, and if I understand the code correctly, placing them in individual
sub-directories with the solution .wrapdesc still in the main directory won't
work, either. Also, that workaround would mean several "wraps/" directories,
which I would like to avoid.
Sincerely,
Reinhold Rumberger
Sebastien Lambla
Aug 25, 2012, 6:49:09 AM8/25/12
to openevery...@googlegroups.com
Hi,
For the first part, we don't have a server as it stands. If you want to build your own server, you only need a URI that takes a POST with the package file and add that to the publish uri in the index.wraplist file. It's very little code that you can write in any language you want and it'll work with the existing client.
I never released any server code because the demand has been so low...
To your second question: OpenWrap tries to push you into having smaller packages, so I usually recommend splitting your solution in smaller packages (for example, contracts in a WCF system ought to be a separate package from the actual solution). OpenWrap is optimized to make building package-managed smaller solutions.
If you want to generate multiple pacakges from your solution, at that stage I'd highly recommend doing it manually with a script. The layout of packages, the files in it and the descriptors you want are simple enough.
That said, if you want to have a -descriptor param to the commands, I'd be happy to take that pull request.
Thanks
Seb
For the first part, we don't have a server as it stands. If you want to build your own server, you only need a URI that takes a POST with the package file and add that to the publish uri in the index.wraplist file. It's very little code that you can write in any language you want and it'll work with the existing client.
I never released any server code because the demand has been so low...
To your second question: OpenWrap tries to push you into having smaller packages, so I usually recommend splitting your solution in smaller packages (for example, contracts in a WCF system ought to be a separate package from the actual solution). OpenWrap is optimized to make building package-managed smaller solutions.
If you want to generate multiple pacakges from your solution, at that stage I'd highly recommend doing it manually with a script. The layout of packages, the files in it and the descriptors you want are simple enough.
That said, if you want to have a -descriptor param to the commands, I'd be happy to take that pull request.
Thanks
Seb
Reinhold Rumberger
Aug 25, 2012, 7:37:30 AM8/25/12
to openevery...@googlegroups.com
Am Samstag 25.08.2012, 12:49:09 schrieb Sebastien Lambla:
> Hi,
>
> For the first part, we don't have a server as it stands. If you want to
> build your own server, you only need a URI that takes a POST with the
> package file and add that to the publish uri in the index.wraplist file.
> It's very little code that you can write in any language you want and
> it'll work with the existing client.
Great, thanks. I assume openwrap supports HTTP Basic Authentication?
> Hi,
>
> For the first part, we don't have a server as it stands. If you want to
> build your own server, you only need a URI that takes a POST with the
> package file and add that to the publish uri in the index.wraplist file.
> It's very little code that you can write in any language you want and
> it'll work with the existing client.
https://github.com/openrasta/openwrap/wiki/Openwrap-publishing-protocol isn't
clear on authentication method support.
> I never released any server code because the demand has been so low...
>
> To your second question: OpenWrap tries to push you into having smaller
> packages, so I usually recommend splitting your solution in smaller
> packages (for example, contracts in a WCF system ought to be a separate
> package from the actual solution). OpenWrap is optimized to make building
> package-managed smaller solutions.
openwrap having to maintain a separate wraps/ directory for every package.
Ideally, all packages in a solution should be able to share one wraps/
directory to reduce redundancy.
There wouldn't happen to be a way to specify which wraps/ directory to use
that I have missed so far?
The big wrap of the whole solution is supposed to make things easier for
projects that need all subprojects, it isn't something I absolutely have to
have. In fact, I'm not completely sure I actually want that - I still need to
think about this.
> If you want to generate multiple pacakges from your solution, at that stage
> I'd highly recommend doing it manually with a script. The layout of
> packages, the files in it and the descriptors you want are simple enough.
>
> That said, if you want to have a -descriptor param to the commands, I'd be
> happy to take that pull request.
code: would this parameter potentially conflict with the scoping support?
If so, how would you like that conflict handled?
Thanks for you help.
--Reinhold
Marcin Mikołajczak
Aug 25, 2012, 7:28:09 AM8/25/12
to openevery...@googlegroups.com
Hi,
Just wanted to remind everyone that SymbolSource can handle pushing/serving OpenWrap packages, both as a hosted solution and on-premise with SymbolSource.Server.Basic :)
For anyone interested have a look at these posts:
http://www.symbolsource.org/Public/Blog/View/2011-10-08/Using_SymbolSource_as_an_OpenWrap_repository
If you put symbols and sources into you wraps, you'll get the added benefit of having a symbol and source server for source-stepping in your debugging sessions :) You don't event need to prepare 2 separate packages, we strip symbols and sources automatically on upload and only serve the rest to OpenWrap (actually it's the same with NuGet).
Instructions tell you to upgrade to beta OpenWrap, which is 2.0.2 in the repo I think, but if you're a pro like Seb you'll surely manage to get it working on the project level or with some other build. This is just what we tested with.
Cheers,
Marcin
Ruben Vandeginste
Aug 26, 2012, 9:58:34 AM8/26/12
to openevery...@googlegroups.com
On Fri, Aug 24, 2012 at 4:44 PM, Reinhold Rumberger
<rumbe...@googlemail.com> wrote:
> [...]
We currently use a webdav server and we mount the repository as a
local drive (in Windows: net use x: https://domain/folder). We've been
using OpenWrap that way for several months, and it's working fine.
Since it is based on webdav, it is also easy to give specific
developers specific permissions.
> My second problem is that my solution is split into several interdependent
> projects. Since other (external) projects want to use as few assemblies as
> possible, they should be able to depend on individual projects, not just the
> entire solution.
> How would I best go about implementing this? My attempt to create several
> .wrapdescs for each of the projects in the main solution directory doesn't
> work, and if I understand the code correctly, placing them in individual
> sub-directories with the solution .wrapdesc still in the main directory won't
> work, either. Also, that workaround would mean several "wraps/" directories,
> which I would like to avoid.
I've stumbled on that too. In the end, we chose to have a 1-on-1
mapping between packages and solutions. That means that we end up with
a lot of small solutions. The advantage of this approach is that
everything is cleanly separated. The granularity of the packages also
depends on which parts of a solution you want to be able to use
independently.
An example of this is a service that consists of the implementation of
that service, and the api that is offered to other systems. The other
systems only need a dependency on that api, but not on the service
itself.
By splitting up into different solutions, you are forced to use
package dependencies instead of project dependencies.
Back to the example: you would have one solution with a project
dependency of the implementation service on the api project. If you
split it up in separate solutions, you will have a package dependency
from the service solution on the api pacakge.
I think this is good in a way because it enforces clean separation. On
the other hand, this approach definitely has drawbacks. A big drawback
is that you can no longer do refactorings over multiple
projects/packages, since they are no longer part of the same solution.
Back to the example: you want to rename a method in the api. In one
solution this is very easy with a good refactor tool. With separate
solutions, you will first rename the method in the api, create a
package from that, and use that package in the service solution, and
then refactor the service. So it is more work. On the other hand, it
might make you think twice about changing the api... this might not be
a bad thing, since one has to realise that every existing system that
uses that api will also need to be modified.
I also develop in Java, and I've used Maven as a dependency manager in
Java. A big difference bewteen java IDEs and Visual Studio, is that
you don't really have the concept of a "solution" in a java ide. You
just open projects in the ide. You can open as many projects as you
want, whether they are related or not. In combination with maven, it
also allows you to override a "package dependency" with a "project
dependency". With that kind of setup you don't have the problems as I
mentioned before, but I'm afraid that this is not possible in .NET
with Visual Studio.
Either way... like I said, we have a solution per package and it works for us.
Ruben
<rumbe...@googlemail.com> wrote:
> [...]
> The first is that I would like to set up my own private repository using
> Apache as server. I'd like to be able to both publish to this repository via
> HTTP and use that repository for dependency resolution.
> So far my workaround is to publish to a local directory and mirror that to the
> server, having Apache handle the serving. This is not an acceptable solution,
> though, since publishing and retrieval is done via two different remotes.
> Also, it doesn't work once more people need to push to the HTTP repository.
Hi, just telling you my own experience...
> Apache as server. I'd like to be able to both publish to this repository via
> HTTP and use that repository for dependency resolution.
> So far my workaround is to publish to a local directory and mirror that to the
> server, having Apache handle the serving. This is not an acceptable solution,
> though, since publishing and retrieval is done via two different remotes.
> Also, it doesn't work once more people need to push to the HTTP repository.
We currently use a webdav server and we mount the repository as a
local drive (in Windows: net use x: https://domain/folder). We've been
using OpenWrap that way for several months, and it's working fine.
Since it is based on webdav, it is also easy to give specific
developers specific permissions.
> My second problem is that my solution is split into several interdependent
> projects. Since other (external) projects want to use as few assemblies as
> possible, they should be able to depend on individual projects, not just the
> entire solution.
> How would I best go about implementing this? My attempt to create several
> .wrapdescs for each of the projects in the main solution directory doesn't
> work, and if I understand the code correctly, placing them in individual
> sub-directories with the solution .wrapdesc still in the main directory won't
> work, either. Also, that workaround would mean several "wraps/" directories,
> which I would like to avoid.
mapping between packages and solutions. That means that we end up with
a lot of small solutions. The advantage of this approach is that
everything is cleanly separated. The granularity of the packages also
depends on which parts of a solution you want to be able to use
independently.
An example of this is a service that consists of the implementation of
that service, and the api that is offered to other systems. The other
systems only need a dependency on that api, but not on the service
itself.
By splitting up into different solutions, you are forced to use
package dependencies instead of project dependencies.
Back to the example: you would have one solution with a project
dependency of the implementation service on the api project. If you
split it up in separate solutions, you will have a package dependency
from the service solution on the api pacakge.
I think this is good in a way because it enforces clean separation. On
the other hand, this approach definitely has drawbacks. A big drawback
is that you can no longer do refactorings over multiple
projects/packages, since they are no longer part of the same solution.
Back to the example: you want to rename a method in the api. In one
solution this is very easy with a good refactor tool. With separate
solutions, you will first rename the method in the api, create a
package from that, and use that package in the service solution, and
then refactor the service. So it is more work. On the other hand, it
might make you think twice about changing the api... this might not be
a bad thing, since one has to realise that every existing system that
uses that api will also need to be modified.
I also develop in Java, and I've used Maven as a dependency manager in
Java. A big difference bewteen java IDEs and Visual Studio, is that
you don't really have the concept of a "solution" in a java ide. You
just open projects in the ide. You can open as many projects as you
want, whether they are related or not. In combination with maven, it
also allows you to override a "package dependency" with a "project
dependency". With that kind of setup you don't have the problems as I
mentioned before, but I'm afraid that this is not possible in .NET
with Visual Studio.
Either way... like I said, we have a solution per package and it works for us.
Ruben
Reinhold Rumberger
Aug 26, 2012, 1:12:05 PM8/26/12
to openevery...@googlegroups.com
Am Sonntag 26.08.2012, 15:58:34 schrieb Ruben Vandeginste:
> On Fri, Aug 24, 2012 at 4:44 PM, Reinhold Rumberger
> <rumbe...@googlemail.com> wrote:
> [...]
>
> On Fri, Aug 24, 2012 at 4:44 PM, Reinhold Rumberger
> <rumbe...@googlemail.com> wrote:
> [...]
>
> > My second problem is that my solution is split into several
> > interdependent projects. Since other (external) projects want to use as
> > few assemblies as possible, they should be able to depend on individual
> > projects, not just the entire solution.
> > How would I best go about implementing this? My attempt to create several
> > .wrapdescs for each of the projects in the main solution directory
> > doesn't work, and if I understand the code correctly, placing them in
> > individual sub-directories with the solution .wrapdesc still in the main
> > directory won't work, either. Also, that workaround would mean several
> > "wraps/" directories, which I would like to avoid.
>
> I've stumbled on that too. In the end, we chose to have a 1-on-1
> mapping between packages and solutions.
This isn't totally clear, but as I mentioned in my answer to Sebastien's mail,
> > interdependent projects. Since other (external) projects want to use as
> > few assemblies as possible, they should be able to depend on individual
> > projects, not just the entire solution.
> > How would I best go about implementing this? My attempt to create several
> > .wrapdescs for each of the projects in the main solution directory
> > doesn't work, and if I understand the code correctly, placing them in
> > individual sub-directories with the solution .wrapdesc still in the main
> > directory won't work, either. Also, that workaround would mean several
> > "wraps/" directories, which I would like to avoid.
>
> I've stumbled on that too. In the end, we chose to have a 1-on-1
> mapping between packages and solutions.
I'd really like to avoid having a billion wraps/ directories. There's already
enough bloat in the SVN.
I'm pretty much getting comfortable with the idea of doing the packaging and
publishing manually on the build server. Most of the necessary infrastructure
is already in place, and it would allow me to automatically set the proper
dependency versions (intra-solution dependencies need to be restricted to make
sure all packages from one build depend on each other and not earlier or later
versions). I need this because the internal APIs are way too unstable to allow
different builds to be mixed and openwrap doesn't seem to support this use
case.
This would also simplify publishing, since I don't need to account for quite
so many variables.
Anyway, thanks for the ideas and for making me think; I now have a concrete
idea how to manage the publishing and availability side of the problem. Now I
just need to figure out how to best handle the consumer side (which,
thankfully, is only partially my problem).
--Reinhold
Ruben Vandeginste
Aug 27, 2012, 4:32:07 AM8/27/12
to openevery...@googlegroups.com
On Sun, Aug 26, 2012 at 7:12 PM, Reinhold Rumberger
<rumbe...@googlemail.com> wrote:
> Am Sonntag 26.08.2012, 15:58:34 schrieb Ruben Vandeginste:
>> I've stumbled on that too. In the end, we chose to have a 1-on-1
>> mapping between packages and solutions.
>
> This isn't totally clear, but as I mentioned in my answer to Sebastien's mail,
> I'd really like to avoid having a billion wraps/ directories. There's already
> enough bloat in the SVN.
I left that out of my explanation. We decided to not commit any
<rumbe...@googlemail.com> wrote:
> Am Sonntag 26.08.2012, 15:58:34 schrieb Ruben Vandeginste:
>> I've stumbled on that too. In the end, we chose to have a 1-on-1
>> mapping between packages and solutions.
>
> This isn't totally clear, but as I mentioned in my answer to Sebastien's mail,
> I'd really like to avoid having a billion wraps/ directories. There's already
> enough bloat in the SVN.
"wraps" folders to subversion. Whenever we check out code, the first
thing that we do, is "o update-wrap" to fetch the dependencies. Are
there any drawbacks to that? In principle yes: it is possible that one
developer has more recent packages in the dependencies as another
developer. In practice we've not experienced any problems with that.
(Note that when the "wraps" folder already exists, you have to use the
command "o update-wrap -usesystem")
When you tag a released version in subversion (Every version that is
packaged and published, is also tagged in our subversion repository),
you have the choice to add the wraps folder to subversion at that
point. We considered that, but we do not do that at this point. Next
to that, you also have the option to use the command "lock-wrap". This
will record the exact version of the dependencies at that time in a
file called "packages.lock". If you commit that file on the tag in
subversion, you get a record of the exact version of the dependencies
that were in use when the tag was created. We do that for our "end
products" so that afterwards we can still completely reconstruct the
exact version of all dependencies that were use for a specific release
of a program or service. (Even though we record the packages.lock
file, we actually never needed to reconstruct a specific version.)
> I'm pretty much getting comfortable with the idea of doing the packaging and
> publishing manually on the build server. Most of the necessary infrastructure
> is already in place, and it would allow me to automatically set the proper
> dependency versions (intra-solution dependencies need to be restricted to make
> sure all packages from one build depend on each other and not earlier or later
> versions). I need this because the internal APIs are way too unstable to allow
> different builds to be mixed and openwrap doesn't seem to support this use
> case.
> This would also simplify publishing, since I don't need to account for quite
> so many variables.
This works pretty good so far, but we are at a point that most apis
are fairly stable. I do know that this is more difficult when there is
still heavy development going on and apis are very unstable.
> Anyway, thanks for the ideas and for making me think; I now have a concrete
> idea how to manage the publishing and availability side of the problem. Now I
> just need to figure out how to best handle the consumer side (which,
> thankfully, is only partially my problem).
Ruben
Reply all
Reply to author
Forward
0 new messages