Storage expand problem.

[Dr. Filipiuc Ciprian]

Hello.

Maybe you heard a 100 times this problem... but i just don t know what is the next step to it...

So... i am handling with orthanc since 06.2019 all pacients from a IRM , a CT and a Mamograph.  i have made a storage (in raid) of 4 TB. At the moment we are at 68% ussage.

/dev/sdb1                3.6T  2.4T  1.3T  66% /pac

So... my question is... how i expand storage?  i need to put another hard disks on server with another partition bigger? for example 8 TB ? and copy all 4 tb on the 8 tb and continue using Orthanc? We use ESXI with Virtual Machines on it. One of them is Orthanc ( with 2 storage drives - one ssd for os and database and 2 sas drives for storage - in raid).

There is a posibily to add another destination on storage? Or it is possible to run another Orthanc process on other port with other destination on storage and the same data base? 

ATM i am using PostGresql and my DB is 12 GB. I storage on the main SSD ( like the OS). space for data base we still have... but how it best to handle storage?

Thank you

[Benjamin Golinvaux]

Hello,

Orthanc only supports a single storage directory (this is by design, because of the very availability of the workaround I suggest below)

Furthermore, the Orthanc database cannot be shared by multiple Orthanc instances (this is an actual limitation that could/should be lifted, in the future)

However, the correct way to expand your storage, in my opinion, would be to use a system-level abstraction layer like LVM, that allows you to have several physical disks under a same logical volume (since I assume you are using Linux, from your /dev/sdb1 device path, where LVM is supported in all the main distributions I am aware of, and I guess that Windows and/or macOS supply the same feature)

You can easily Google for tutorials on how to create new volumes, attach new HDD etc... For an overview : https://www.digitalocean.com/community/tutorials/an-introduction-to-lvm-concepts-terminology-and-operations

Best,

[Bryan Dearlove]

If your Orthanc server is a VM, then you can expand storage on the host, then shut down Orthanc, and expand storage on the guest. Depending on your server/storage configuration usually you can add an additional hard drive and expand RAID on the fly. So:

1. Add Drive(s) to server.

2. Add Drive(s) to RAID/Expand RAID. 

3. Expand ESXi storage. 

4. Expand VM Guest Storage in ESXi. (Expand the VMDK file)

5. Expand storage within the Guest VM. 

[William Sanchez]

Hi

The solution to your problem is simple (although laborious). Add disks to the server, create a new partition, move the data from the old disk to the new disk and redirect the location of the data in the Orthanc configuration file to the new partition in the new disk.

I had the same problem a few months ago, I had a server with 1TB and when I run out of space, I added more disks, moved the data from the 1TB disk to the new 8TB array and modified the search for the data in the configuration file from Orthanc to the new hard drive partition. It works perfect and does not lose the stored information.

Regards and good luck

[Diego De la Fuente]

There may be a lot of options to expand volumes in the Orthanc environment. But in my case, when we have to expand the storage, I see it as a very tedious task. 

We currently have a 28TB volume to store Dicom files. We are growing at a rate of 20TB per year. 

We have been using orthanc since March 2019, so I estimate that by the middle of this year, we will run out of space. 

Expanding the volume is the first option I can think of. 

But what will happen if I do this in 5 or 6 years, when I have hdds of such age that are part of a huge volume? 

I will have to establish a mechanism to remove the first Raid from the volume, and so on. 

In the previous Pacs we had, only one thing looked better than Orthanc. 

I could define storage areas, and each study had that area associated. 

Thus, when growing up, it simply generated a new volume and marked it as an asset for new studies. 

All the other studies had the mark of their storage container, which made it easy to lift it. 

In general, when I had a volume of many years, the studies associated with it were also many years old. 

With which we made a backup to a DAT and removed the volume, or simply left it while not having failures. 

On the other hand, each volume (and group of hdds) had its technology according to its time, and each time we added a new volume (which is independent of the rest) we improved in performance. 

Orthanc has storage compatibility with Postgres and MySql, which complicates this a bit

My point of view, with this long comment, in my experience it is better for us to have storage containers and not delegate the issue of growth to something external.

Regards

Diego

[Sébastien Jodogne]

Hello,

First of all, you should really learn about LVM. LVM allows you to increase the size of a "logical volume" (LV) by adding new "physical volumes" (PV) as need be, in a process that is fully transparent to Orthanc. This is not "something external", as it is implemented directly by the Linux kernel:

https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)

Extending a logical volume is a very simple task (check out the "lvextend" command), and many tutorials exist on Internet, e.g.:

https://www.tecmint.com/create-lvm-storage-in-linux/

https://www.tecmint.com/extend-and-reduce-lvms-in-linux/

If you still want to avoid LVM, you could consider implementing a custom storage area plugin that would hash the "uuid" to map the files to different mount points, similarly to a hash table:

https://book.orthanc-server.com/developers/creating-plugins.html

https://sdk.orthanc-server.com/group__Callbacks.html#ga1a63a48ff8db57b8d7e0238bdf8d487c

https://en.wikipedia.org/wiki/Hash_table

Obviously, the resizing of the hash table would imply moving files across the different mount points. This is a maintenance task that is definitely not trivial to implement.

As an alternative to hash tables, you could develop a custom storage area plugin that would maintain a separate database mapping UUIDs to their mount point. The plugin would select an available mount point in which there is enough free space to store the new file.

HTH,

Sébastien-

[bobIQ morane]

On Tuesday, January 7, 2020 at 11:19:29 PM UTC+1, Diego De la Fuente wrote:

There may be a lot of options to expand volumes in the Orthanc environment. But in my case, when we have to expand the storage, I see it as a very tedious task. 

We currently have a 28TB volume to store Dicom files. We are growing at a rate of 20TB per year. 

We have been using orthanc since March 2019, so I estimate that by the middle of this year, we will run out of space. 

I had a similar problem: store 30+TB with the possibility to expand to even more in the next years.

I solved this by purchasing a 16-bay NAS server (ts-1683xu-rp from QNAP), and running Orthanc in a VM on the NAS. The storage is handled by the NAS through a RAID 6 volume. That way you can add storage, replace disks and even tolerate disk failures, and it is transparent for Orthanc and the VM it is in. All you have to do is rebuild the RAID array, that's easily done through the NAS GUI.

Note that reliable 8TB disks are commonplace (for instance I use Seagate Exos 7E8), with a 16-bay server in RAID 6 you can go to 112 TB of storage easily, and after that you can plug in NAS extensions to add HDD bays. I think the limit is for a single volume 200TB (limited by the NAS sofware)

I'll see with my boss if I can post more details on my solution.

[John Hearns]

These days, with Software Defined Storage it is easy to expand storage pools online.

I would look at

BeeGFS https://www.beegfs.io/content/  Very easy to set up and the storage just expands when you add a new pool

CEPH https://ceph.io/  there are several companies selling CEPH appliances. Look at Softiron and Ambedded

Gluster https://www.gluster.org/  Very easy to set up

There are many commercial SDS setups of course too.

For high performance look at Weka and Panasas.

If anyone wants some help in setting up any opf the above ping me offline.

--
You received this message because you are subscribed to the Google Groups "Orthanc Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orthanc-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/orthanc-users/8ec82708-f7ae-4cf0-b468-fba629abe2fe%40googlegroups.com.

Kheiron Medical Technologies

kheironmed.com | supporting radiologists with deep learning

Kheiron Medical Technologies Ltd. is a registered company in England and Wales. This e-mail and its attachment(s) are intended for the above named only and are confidential. If they have come to you in error then you must take no action based upon them but contact us immediately. Any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. Although this e-mail and its attachments are believed to be free of any virus, it is the responsibility of the recipient to ensure that they are virus free. If you contact us by e-mail then we will store your name and address to facilitate communications. Any statements contained herein are those of the individual and not the organisation.

Registered number: 10184103. Registered office: 2nd Floor Stylus Building, 116 Old Street, London, England, EC1V 9BG

[Dr. Filipiuc Ciprian]

Hello again. As my problem was extending storage, atm i have installed a NAS in my network with 24 TB. My question is : there is a possibility to point the nas storage directly to orthanc? or i need to mount the drive into the system and normaly point to directory to storage (eg /orthanc-storage - where the nas drive will be mounted)?

[Adrian Schiopu]

Hello, if your NAS is Synology I had put here  few mount ago a guide to install Orthanc directly on NAS. 

[Sébastien Jodogne]

Hello, if Orthanc is not being run directly by your NAS, you'll most probably have to mount it as a network drive into your system.