I want to delete the data in the remote computer, is there a data copy in local computer?

[Witty Yonng]

Hi everyone,

I run external codes in the remote computer, and when job is done, I want to delete the data in the remote computer. So I want to ask that whether there is a data copy in my local computer? Or I need to do something else before deleting data in the remote computer?

Thanks.

Best,

Witty

[Jonathan Chico]

Hi!

As far as I know it depends, AiiDA stores data in two ways

1. The database, these are things like Str, Int, Dict, etc

2. The repository; think about it as a little filesystem where things like SingleFileData, Retrieved, ArrayData, etc are stored in some way.

If the data that you are thinking about is in one of those two categories you should be able to delete it from the remote without problem, since they should be accessible in the repository or database (and both of those can be accessed via the verdi shell interface). If on the other hand whatever data you are trying to gather is not stored there you should store them somewhere before deleting them in the remote. This is far from ideal since keeping provenance is then going to be quite difficult. I guess it should be possible to attach a node to a calculation after it finishes, but I have never done that before.

The best way to check this is by looking into the outputs of your calculation and checking what is available as normal output ports (Ints, StructureData, Floats, etc) and what is inside the retrieved folder since that should show you all the raw files that were marked for storage in the repository by the calculation itself.

Cheers

--
AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).
 
Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
---
You received this message because you are subscribed to the Google Groups "aiidausers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/f12789b0-7f5a-402d-9d52-2fa6d612d13an%40googlegroups.com.

--

Jonathan Chico

[Sebastiaan Huber]

Hi Witty,

I will add to Jonathan's excellent answer.

AiiDA does not explicitly track or maintain all files in the remote folder of calculations.
It is the plugin that decides which files are retrieved and permanently stored by AiiDA in the `retrieved_folder`.
If all the files you need are in the `retrieved_folder` you can safely delete the remote folder contents.
Note the easiest way to do this is

calculation = load_node(PK)
calculation.outputs.remote_folder._clean()

Note that this will need to connect to the remote computer, so if it is remote, you need to have internet connection.
Note also that many of the base workchains do this automatically, unless you specify `clean_workdir = False` in the inputs.

Finally, many plugins will not retrieve all files (such as `PwCalculation` for example) because not all files are necessary for reproducibility.
For example they will delete temporary files, but these might be necessary for restarts.
This is the case of `PwCalculation` for example: if you clean its `remote_folder` you can no longer use it for restarts.
Again, all of this is decided by the plugin and so is plugin dependent so I cannot give a general answer.

Hope that helps,

Sebastiaan

To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/CAO8p0kXQMQ6ZPZ1aOCxvBwYAQhPTOP%3DeJhXSuC28s%2BMhrL%2B5Bg%40mail.gmail.com.