Why is there lots of disk space used compared to Apple server?

[Ratti]

Hi,

I am running an Apple server with 10.5 on it. I have manually patched
the config to offer updates for 10.6 also. Used disc space is 27 GB.

Now, with some lion clients in the company I tried to switch to
reposado. Starting with the default configuration I now have
downloaded 75 GB, completely filled the virtual machine and not done
yet. This seems not to be what I want - one more OS version should not
eat up 400% disc space.

There seems to download lots of stuff that looks like language/speech
files? I unfortunately just deleted my logfiles, but the names where
similar to "Natasha_RU_ru.pkg", "Andre_FR_fr.pkg“,… when looking at
them, they are deprecated. Why are deprecated files downloaded? This
differs a lot from the behaviour of the original Apple server.

To be honest, even after reading a lot of documentation I still don´t
get this "deprecated" stuff. If something is deprecated, why do we
download them at all? For example, why is there a 10.7.2 updater? The
10.7.2 users only need 10.7.3, and everyone else needs 10.7.3Combo.
There is no use for a 10.7.2-Update, except you plan to hold back
updates. Thats something I don´t want.

So, what is the best way to get the OS X Server behaviour:

- Download everything that´s new
- Don´t download old stuff
- Auto-delete old stuff when newer version are available

[Nate]

I am fairly certain you could script at least 2 of those 3 desired behaviors and set it to launch on a schedule via Launchd.

As for only downloading current updates, I was under the impression that after you removed deprecated items, it no longer re-downloaded them.  This could be incorrect though.

Nate

[Nate]

After looking at this further, it seems there is only a mechanism to remove Deprecated updates from branches but not from the repo itself.  So this won't help with the disk space dilemma.  I'm going to see if I can't figure out a good way to get rid of them (perhaps manually if need be).

Nate

[Greg Neagle]

On Feb 8, 2012, at 9:07 AM, Ratti wrote:

> Hi,
>
> I am running an Apple server with 10.5 on it. I have manually patched
> the config to offer updates for 10.6 also. Used disc space is 27 GB.
>
> Now, with some lion clients in the company I tried to switch to
> reposado. Starting with the default configuration I now have
> downloaded 75 GB, completely filled the virtual machine and not done
> yet. This seems not to be what I want - one more OS version should not
> eat up 400% disc space.

Yet it does. Lion moves a lot of software out of the initial install and to the "cloud" for on-demand installation: printer drivers and high-quality speech voices.
Yesterday, Apple updated 54 "Multi-lingual Voices" for Lion for around 10GB of data. (Some voices are 500MB in size!)

I haven't done the analysis to see how much space the Lion-only updates are taking on my reposado server, but I would not be surprised to find out that they were taking up 30-40GB.

>
> There seems to download lots of stuff that looks like language/speech
> files? I unfortunately just deleted my logfiles, but the names where
> similar to "Natasha_RU_ru.pkg", "Andre_FR_fr.pkg“,… when looking at
> them, they are deprecated. Why are deprecated files downloaded?

They aren't.
"Deprecated" really means that an item is no longer listed in any sucatalog. Since reposado uses the sucatalogs to download updates for itself, it cannot download deprecated updates. But items it has already downloaded can _become_ deprecated.

Your reposado server downloaded the original Multilingual Voice files. Yesterday, Apple updated them all, so the older ones are now deprecated.

> This
> differs a lot from the behaviour of the original Apple server.
>
> To be honest, even after reading a lot of documentation I still don´t
> get this "deprecated" stuff. If something is deprecated, why do we
> download them at all?

We don't. We can't download deprecated updates. But updates we've already downloaded can _become_ deprecated.

> For example, why is there a 10.7.2 updater? The
> 10.7.2 users only need 10.7.3, and everyone else needs 10.7.3Combo.
> There is no use for a 10.7.2-Update, except you plan to hold back
> updates. Thats something I don´t want.

But many admins do -- in fact that's a major reason to run your own SUS server. You release the 10.7.3 update for your testing clients, while leaving the bulk of the organization on 10.7.2 until your testing is complete. At that point, you can release the 10.7.3 update to everyone in your organization. But until then you are still offering the "deprecated" 10.7.2 update.

>
> So, what is the best way to get the OS X Server behaviour:
>
> - Download everything that´s new

It does that now.

> - Don´t download old stuff

Depends on how you define "old stuff" -- reposado won't download anything that's no longer offered by Apple.

> - Auto-delete old stuff when newer version are available

It doesn't do that and never will. That is not behavior I want.

In the future, admins will be able to selectively purge updates; you could then write a script that auto-deleted old stuff (however you want to define that).

-Greg

[Greg Neagle]

I'm currently working on a --purge-product option to allow an admin to remove a product from the ProductInfo database and also (if you are replicating products) remove it from disk.

I think the option should warn you if the product you want to purge is not deprecated or is in any branch. A --force option could override this.

Without further work, non-deprecated products that were purged would just be re-downloaded the next time repo_sync runs.

-Greg

[Mike Solin]

I would love this - working with whatever spare HDs I can scrounge up, so keeping things as slim as possible is really appreciated.  I generally don't need to offer deprecated updates here.

Would it break anything to delete deprecated updates from disk without removing them from Reposado?  As long as they're not in any catalogs, that should be okay, right?

Thank you so much, Greg!

Mike

mi...@mikesolin.com / @flammable
http://mikesolin.com

[Ratti]

On 8 Feb., 23:26, Mike Solin <m...@mikesolin.com> wrote:
> I would love this - working with whatever spare HDs I can scrounge up, so keeping things as slim as possible is really appreciated.  I generally don't need to offer deprecated updates here.
>
> Would it break anything to delete deprecated updates from disk without removing them from Reposado?  As long as they're not in any catalogs, that should be okay, right?
>
> Thank you so much, Greg!
>
> Mike

Hello everybody,

Thanks for your explanations - I understand now thats correct to have
zillions of gigabyte occupied on your drive when offering lion
support. :-)

However, I am not happy having hosted older versions. While there is
no "right" or "wrong" in different scenarios, _we_ just need a
"proxy" (or whatever one may call that) to avoid 25 machines
downloading the huge apple updates over small bandwith.

Our Mac OS X Server 10.5 has 3 options that my be combined:

[X] Automatically copy (Dropdown: ALL | all new) updates from apple
[X] Automatically enable copied updates
[X] Purge unused/legacy software update packages automatically

Combining these options should meet all scenarios, and for me it looks
like the last one is not implemented in reposado?
Or, maybe someone can turn me in the right direction?

My problem is:
While normal disc space is not too expensive nowadays, it still is in
some virtualization szenarios. ATM I move 80 Gigs from a drive to a
second (bigger) one since my old-but-stable kvm doesn´t support
resizing compressed filesystems. Hell.

Thanks a lot,
bye, Jörg

[Nate]

There was an update made to Reposado that allows you to purge deprecated updates now.  See: http://groups.google.com/group/reposado/browse_thread/thread/f1f32d4a6834823d

Nate

[Ratti]

On 9 Feb., 11:57, Nate <nate.wa...@gmail.com> wrote:
> There was an update made to Reposado that allows you to purge deprecated

> updates now.  See:http://groups.google.com/group/reposado/browse_thread/thread/f1f32d4a...
>
> Nate

That´s great! Thanks for pointing me there.

I´ll wait until all the language packs have downloaded and then commit
the overlay file of the virtual machine. That way I can try out
purging without really losing files.

You guys rock. I´m happy about every task I can move from Mac OS to
Linux.

Bye,
Jörg