Runtime Shadow Copy Download
Arvilla Hardan
In the following sample app I create a new AppDomain and I execute it with shadow copy enabled. From the new AppDomain I then try to delete (replace) the original main exe. However I get an "access is denied error". Interestingly, after launching the program, from Windows Explorer it is possible to rename the main exe (but not to delete it).
You can achive self updating application within a single application with multiple AppDomains. The trick is move application executable to a temporary directory and copy back to your directory, then load the copied executable in a new AppDomain.
You can easily copy the content of an entire hard disk to another disk. If you copy, for example, C:\ to D:\, you will even be able to boot from that drive. Since ShadowCopy is capable of tranferring all the system files you will have an almost exact clone.
I am trying to stop my application locking DLLs in my MEF plugin directory so that I can overwrite the assemblies at runtime (note I'm not actually trying to have MEF reload them on the fly, at the next app start is fine, i just dont want to have to stop the app to do the copy)
and then access my MEF component catalog via the CompositionContainer on this class. However the composition container seems to only be initialised inside the shadowcopy domain (which makes sense) and this means that its null in my application domain. I was just wondering if theres a better way to do this or some way to cross domain query to get my MEF components
Do you have the option of not using a DirectoryCatalog and use AssemblyCatalog to load up all the assemblies in the directory ? You could even go to code plex and copy the same code from the DirectoryCatalog class that reads through the directory and loads up the assemblies.
Shadow copying enables assemblies that are used in an application domain to be updated without unloading the application domain. This is particularly useful for applications that must be available continuously, such as ASP.NET sites.
The common language runtime locks an assembly file when the assembly is loaded, so the file cannot be updated until the assembly is unloaded. The only way to unload an assembly from an application domain is by unloading the application domain, so under normal circumstances, an assembly cannot be updated on disk until all the application domains that are using it have been unloaded.
When an application domain is configured to shadow copy files, assemblies from the application path are copied to another location and loaded from that location. The copy is locked, but the original assembly file is unlocked and can be updated.
The only assemblies that can be shadow copied are those stored in the application directory or its subdirectories, specified by the ApplicationBase and PrivateBinPath properties when the application domain is configured. Assemblies stored in the global assembly cache are not shadow copied.
By default, this setting causes all assemblies in the application path to be copied to a download cache before they are loaded. This is the same cache maintained by the common language runtime to store files downloaded from other computers, and the common language runtime automatically deletes the files when they are no longer needed.
The base path for the location is formed by concatenating the ApplicationName property to the CachePath property as a subdirectory. Assemblies are shadow copied to subdirectories of this path, not to the base path itself.
There are a few reasons why you might want to set a custom location for shadow copied files. You might want to set a custom location for shadow copied files if your application generates a large number of copies. The download cache is limited by size, not by lifetime, so it is possible that the common language runtime will attempt to delete a file that is still in use. Another reason to set a custom location is when users running your application do not have write access to the directory location the common language runtime uses for the download cache.
When an application domain that uses shadow copying starts, there is a delay while assemblies in the application directory are copied to the shadow copy directory, or verified if they are already in that location. Before the .NET Framework 4, all assemblies were copied to a temporary directory. Each assembly was opened to verify the assembly name, and the strong name was validated. Each assembly was checked to see whether it had been updated more recently than the copy in the shadow copy directory. If so, it was copied to the shadow copy directory. Finally, the temporary copies were discarded.
Beginning with the .NET Framework 4, the default startup behavior is to directly compare the file date and time of each assembly in the application directory with the file date and time of the copy in the shadow copy directory. If the assembly has been updated, it is copied by using the same procedure as in earlier versions of the .NET Framework; otherwise, the copy in the shadow copy directory is loaded.
The AppDomain class has several methods, such as SetShadowCopyFiles and ClearShadowCopyPath, that can be used to control shadow copying on an application domain, but these have been marked obsolete in .NET Framework version 2.0. The recommended way to configure an application domain for shadow copying is to use the properties of the AppDomainSetup class.
I decided to try out C# runtime compilation. Really, C# is the best scripting language I could ask for. If I succeeded, I could keep writing the same code I've been writing, but I could recompile it and see it in action with a keystroke instead of restarting the game!
Now the problem is, once we load the DLL, the .NET runtime locks the file until the program exits. There's no way to unload the library and unlock the file, unless we put it in a different AppDomain, which means we have to marshal data back and forth between the script and game, which defeats the purpose.
Since one of our requirements is being able to recompile the script without restarting the game, we need to find a way to load the DLL without locking the file. Enter shadow copying. You can tell the .NET runtime to make a copy of every assembly it loads, and load the "shadow copy" instead of the real file. That leaves the original DLL free for us to rewrite. Great, so let's just enable shadow copying and we're done.
Unfortunately, life is never that easy. You can't enable shadow copying for an AppDomain once the domain has been created. You have to create a new AppDomain with the settings you want. Solution: create a tiny launcher executable that creates an AppDomain with shadow copying enabled, and then executes our game in that AppDomain. Here's the code:
Note: the AppDomainSetup.ShadowCopyDirectories property lets you provide a list of directories to limit the use of shadow copying. I tried to use it to specify that only the script DLLs be shadow copied, but it didn't work. YMMV.
I did some reflecting and found out the C# runtime compiler is kinda janky. It isn't, like I assumed, a .NET wrapper around some kind of compiler library. Instead, it actually invokes the C# compiler executable, then loads the resulting DLL. If you specify GenerateInMemory = true, it invokes the compiler with a temporary DLL file path and loads the temp file into memory.
Furthermore, it doesn't shadow copy the DLL when it loads it, even if the AppDomain is configured to do so. I'm guessing this is because it uses a different Assembly.Load function... don't quote me but I believe Assembly.LoadFile does not shadow copy files, while Assembly.LoadFrom does.
Anyway, to solve this, we can take advantage of the compiler's quirky nature here by not specifying an output assembly path. That will cause the compiler to generate a temporary file path for the output assembly. So it will still lock the DLL, but we don't care, because it's a temp file. Luckily, we can still read the DLL even though it's locked, so we can copy it to the path we originally wanted for later use.
ASP.NET Core runs in a separate process and manages the runtime. ASP.NET Core doesn't rely on loading the desktop CLR (.NET CLR). The Core Common Language Runtime (CoreCLR) for .NET Core is booted to host the app in the worker process. Setting the .NET CLR version to No Managed Code is optional but recommended.
When an ASP.NET Core app is running on Windows, the binaries are locked so that they can't be modified or replaced. Shadow copying enables the app assemblies to be updated while the app is running by making a copy of the assemblies.
Shadow copy isn't intended to enable zero-downtime deployment, so its expected that IIS will still recycle the app, and some requests may get an 503 Service Unavailable response. We recommend using a pattern like blue-green deployments or Azure deployment slots for zero-downtime deployments. Shadow copy helps minimize downtime on deployments, but can't completely eliminate it.
When I first started using the .NET Framework, I was somewhat confused about how applications actually ran considering an assembly is a PE format with some CLR headers and CLR data. What this means (and it makes a lot of sense) is that the CLR is really just a COM application hosting a runtime environment (the CLR).
When shadow copying is enabled, ASP.NET copies certain resources (all assemblies) from the application directory into a temporary folder and those are the resources which remain in use during the life of an AppDomain. This cache is the same one used for downloaded assemblies and is cleaned by the CLR when the assemblies are no longer in use (akin to Garbage Collection of runtime resources). By default, the location for these cached assemblies will be something like
The shadow copying process is kicked off any time an AppDomain recycle occurs. Tess Ferrandez (who has a killer blog, you should check it out) lists some reasons why the AppDomain may be recycled in her post, ASP.NET Case Study: Lost session variables and appdomain recycles:
760c119bf3