InsightIQ 4.1.1-increase capacity on local datastore

[Shanmuga S]

Hello All, 

Is there a possibility / procedure to upgrade the capacity of local datastore instead of adding a nfs datastore. Thank you 

[John Beranek - PA]

On Monday, 20 March 2017 16:59:27 UTC, Shanmuga S wrote:

Hello All, 

Is there a possibility / procedure to upgrade the capacity of local datastore instead of adding a nfs datastore. Thank you 

I'm not sure if you're talking about the InsightIQ Linux appliance, and if so which version of the appliance you started with.

Here we started with an older version of the appliance and have upgraded a number of times, and have now moved the datastore to Isilon NFS.

What I remember of our appliance was that it had the datastore mounted on a "raw" partition (i.e. not configured with LVM) so resizing it is a little bit painful...

If that's still the way the current appliance is, then you should be able to shutdown the appliance, increase the datastore disk size, then:

1) Boot the appliance with either a Linux Live CD, or in single-user mode[*]

2) Increase the size of the datastore partition

3) Expand the datastore filesystem

If you're not comfortable with Linux administration, leave this to someone that is!

Cheers,

John

[*] There are a bunch of other ways to do this, including ways to do it without rebooting the appliance, but that's even harder to explain...!

[Andrew Stack]

One important caveat not mentioned, backup your database beforehand...which is also a convenient way to redeploy with a greater partition size and import back in if you so wish.  But I echo John's comments and we've done the same.   Put the datastore on an external resource and grow at will (not the same cluster you are monitoring for obvious reasons).

Oh, another thing....your datastore size is a function of your record retention so pay attention to these areas on each cluster under settings.

FSA config - Result set retention

Data Set Config - what to monitor

Cheers,

- Andrew

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Andrew Stack

Sr. Storage Administrator, Storage SSF Data Center Services

Pharma Informatics

Genentech/F. Hoffmann-La Roche Ltd.

Mobile - 650.867.5524

stack....@gene.com

[John Beranek - PA]

On Tuesday, 21 March 2017 14:44:03 UTC, Andrew Stack wrote:

One important caveat not mentioned, backup your database beforehand...which is also a convenient way to redeploy with a greater partition size and import back in if you so wish.  But I echo John's comments and we've done the same.   Put the datastore on an external resource and grow at will (not the same cluster you are monitoring for obvious reasons).

Oh, another thing....your datastore size is a function of your record retention so pay attention to these areas on each cluster under settings.

FSA config - Result set retention

Data Set Config - what to monitor

Another point I just picked up reading the documentation on the latest InsightIQ[1] is that it will discard old data if the datastore does fill up. This is something we'd likely be happy with, because I've got to wonder if we do really need 3 years worth of Isilon stats, using up the 230GiB of space which is currently being used...

As we've put the InsightIQ datastore on an Isilon though, the available space seen by InsightIQ is many TiBs, so InsightIQ will never see the datastore as full, and start throwing away old data. I might experiment with setting a hard quota on the InsightIQ datastore, as then InsightIQ should see a smaller datastore size, and then utilise its "old data expiry" behaviour...

The same documentation states:

Do not apply a quota to the InsightIQ data store through the SmartQuotas module. If you limit the size of the InsightIQ data store through a quota, InsightIQ cannot detect the available space. The data store might become full before InsightIQ can delete older data to make space available for newer data.

Not sure I agree with this, because if you set a hard quota and enable the "Show available space as size of the hard threshold", the NFS client should see the "correct" size.

John

[1] https://www.emc.com/collateral/TechnicalDocument/docu81827.pdf

[Peter Serocka]

> On 2017 Mar 22 Wed, at 07:21, John Beranek - PA <john.b...@pressassociation.com> wrote:
[…]

> The same documentation states:
>
> Do not apply a quota to the InsightIQ data store through the SmartQuotas module. If you limit the size of the InsightIQ data store through a quota, InsightIQ cannot detect the available space. The data store might become full before InsightIQ can delete older data to make space available for newer data.
>
> Not sure I agree with this, because if you set a hard quota and enable the "Show available space as size of the hard threshold", the NFS client should see the "correct" size.
>

Problem is, the shown size of the available space will become incorrect
when the Isilon cluster is nearly full.

Say your datastore size (hard quota limit, as “container”)
is 5TB and the datastore uses 1TB, so 4TB will be shown as available.

Is only say 2TB are left available on the cluster,
a df for the datastore will still show 4TB.

It would make some sense if df reported min(quota-remaining, cluster-available),
which of course means OneFS would need to provide this
computed value to df.

If one wants to use a hard quota limit for an InsightIQ datastore,
one “just” needs to make sure that at any time there is enough
capacity available on the Isilon ;)

Cheers
— Peter

[Shan]

Hello Everyone, Thanks for your response. 

I have a newly installed InsightIQ 4.1.1 (latest version as of now). Its VMware edition. This VM has a local datastore of 65GB. I am looking to increase this local datastore from 65GB to 200 GB as i have three cluster for InsightIQ to generate reports. 

I am aware of the procedure for increasing datastore size for linux VM's. I assume the same procedure works for InsightIQ. I was looking to see if there were anything specific to InsightIQ to consider for increasing capacity of local datastore.

Thanks 

Shan

Emperor Shan
http://emperorshan.blogspot.com/
http://picasaweb.google.co.uk/u.shanmugam

--
You received this message because you are subscribed to a topic in the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/isilon-user-group/TPXMYbstuRA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to isilon-user-group+unsubscribe@googlegroups.com.

[saurabh chaudhary]

Hello All,

Yes it is possible to expand the local data storage size on the InsightIQ application server. Please follow the steps for the InsightIQ v3.2 or later new version:

Open an SSH connection to the InsightIQ VM and log in using the "administrator" account.

1) ShutDown the server by command : sudo shutdown -P now

2) Added new Devices\disk to VM server from ESX.

3) PowerON the VM from ESX.

4) Login to the server by root access.

5) Find new disk on server by command: fdisk -l

Disk /dev/sdc: 440.2 GB, 440234147840 bytes
255 heads, 63 sectors/track, 53522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

6) Create a new partition on the new drive:

[root@IQServerX ~]# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x1bc0b567.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-53522, default 1): 1
Last cylinder, +cylinders or +size{K,M,G} (1-53522, default 53522):
Using default value 53522

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@IQServerX ~]#

7) Check if new Partition created or not: fdisk -l

Disk /dev/sdc: 440.2 GB, 440234147840 bytes
255 heads, 63 sectors/track, 53522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1bc0b567

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       53522   429915433+  83  Linux

8) Format & create new FS on the new datastore by running the following command:

[root@IQServerX ~]# mkfs -t ext3 /dev/sdc1
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
26869760 inodes, 107478858 blocks
5373942 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
3280 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
[root@IQServerX ~]#

9) Create a temporary directory mount location, Stop the InsightIQ application and DataBase Service:

[root@IQServerX ~]# mkdir /datastore_tmp
[root@IQServerX ~]# iiq_stop
Stopping insightiq:                                        [  OK  ]
[root@IQServerX ~]# sudo service iiq_db stop
Stopping iiq_db service:                                   [  OK  ]
[root@IQServerX ~]#

10) Mount the new datastore in a temporary location

[root@IQServerX ~]# mount /dev/sdc1 /datastore_tmp
[root@IQServerX ~]#
[root@IQServerX ~]# df -hP
Filesystem                              Size  Used Avail Use% Mounted on
/dev/sda3                               4.3G  1.9G  2.3G  45% /
tmpfs                                   3.9G     0  3.9G   0% /dev/shm
/dev/sda1                               243M   36M  195M  16% /boot
/dev/sdb1                                63G   51G  8.9G  86% /datastore
/dev/sdc1                               404G  199M  383G   1% /datastore_tmp
[root@IQServerX ~]#

11) Copy the data from the original datastore to the new datastore by command: cp -rpv /datastore/. /datastore_tmp/

[root@IQServerX ~]# cp -rp /datastore/. /datastore_tmp/

12) Compare the two directories to confirm they are identical: diff -rq /datastore /datastore_tmp

[root@IQServerX ~]# diff -rq /datastore /datastore_tmp
[root@IQServerX ~]#

13) Take backup of /etc/fstab file:

[root@IQServerX ~]#
[root@IQServerX ~]# cp -v /etc/fstab /etc/fstab.bak
`/etc/fstab' -> `/etc/fstab.bak'
[root@IQServerX ~]#

14) Determine the UUID of the new filesystem. This is necessary to complete the next step:

[root@IQServerX ~]#
[root@IQServerX ~]# dumpe2fs /dev/sdc1 | grep UUID

dumpe2fs 1.41.12 (17-May-2010)
Filesystem UUID:          65d0b7c6-7f2e-46b0-8961-38af49aaed9a
[root@IQServerX ~]#

65d0b7c6-7f2e-46b0-8961-38af49aaed9a

15) Edit the /etc/fstab file to configure InsightIQ to use the new datastore

UUID=65d0b7c6-7f2e-46b0-8961-38af49aaed9a /datastore ext3 defaults 0 0

16) reboot the InsightIQ VM:

reboot

17) Check the new size and mount by : df -hP

18A) If Rollback required: Reverting to the previous datastore
If these changes need to be reverted, you can revert to the old /etc/fstab by running the following command and then
rebooting:
sudo cp -v /etc/fstab.bak /etc/fstab
reboot
Alternatively, you can shut down and revert to a snapshot if one was taken.

18B) If InsightIQ works perfect then delete the temp mount point : rmdir /datastore_tmp

Thanks and Regards,

Saurabh

On Monday, 20 March 2017 22:29:27 UTC+5:30, Shanmuga S wrote:

[John Beranek - PA]

On Wednesday, 22 March 2017 16:54:18 UTC, saurabh chaudhary wrote:

Hello All,

Yes it is possible to expand the local data storage size on the InsightIQ application server. Please follow the steps for the InsightIQ v3.2 or later new version:

Open an SSH connection to the InsightIQ VM and log in using the "administrator" account.

If you go to the trouble of adding a new disk and copying data across, I'd put the new filesystem on top of LVM, so you can resize it easily in the future should you ever want to...

For VMware, my experience tells me it's best to place the PV on the whole disk (e.g. /dev/sdc) because then if you need to add space you just online grow the disk in VMware, rescan the SCSI device, then pvresize->lvextend->resize2fs.

John