Server specs

[Kim Hyde - NOAA Federal]

Hello,

We are looking to purchase new hardware for our ERDDAP and IT was asking what the recommended specs are for the server. We were looking to start with several TB of solid state drives that can be expanded to meet our data storage needs. It is my understanding that the memory and processing needs are rather low, but I wasn't sure of the specifics. If anyone could share their recommendations I would greatly appreciate it.

Thanks,

Kim

[Roy Mendelssohn - NOAA Federal]

Hi Kim:

The solid state drives will make for faster access to the data if you will be storing the data on the server, rather than continue with storing on AWS, and if so you need to estimate how much data you will be storing (and will make your data access much faster) - but otherwise it is not a key factor. The key question is how much RAM you need. There is no set answer to that, it is a function of the number and size of the datasets, how often they are updated, and usage of your ERDDAP™. What would help is to get an idea of your present usage, first the java settings in the setenv.sh file, and second over-all memory usage. I believe you are running in a Docker container, you can use "docker stats" or "docker inspect" along with the container-id. This will give an idea of your memory usage to get an idea of how much RAM you need. Several reading over different times would be helpful as we want the maximum memory usage you see. If someone there can run say jconsole from within the Docker image that would help.

HTH,

-Roy

> --
> You received this message because you are subscribed to the Google Groups "ERDDAP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to erddap+un...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/erddap/a9dfec69-3e62-4eec-8acb-ae9e4f787104n%40googlegroups.com.

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Men...@noaa.gov www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

[Kim Hyde - NOAA Federal]

Thanks Roy,

Our current ERDDAP really doesn't have much on it yet and usage is limited so it is a bit hard to gauge. I sent your reply to IT and this is what they suggested.

• Your current CPU is ok, I would upgrade it to maybe 16 Core instead of 4 core.
  
• Your RAM is extremely low, so I would upgrade that to 256 GB which should easily handle the data pulls that are required and would ensure that you have what you need for the next 5 years.
  
• The only thing I'm not sure of is the Storage you will need. Currently there is only 125GB of Storage on that Server. I would upgrade those to SSD's and possibly start with at least a few TB

I am working on the storage bit, but if you have any thoughts on the CPU and RAM (too little or overkill) that would be helpful.

Thanks again,

Kim

[Roy Mendelssohn - NOAA Federal]

Agree 16 core CPU, and 256GB RAM should more than adequate (we now recommend that total Java use at peak use - and not just heap as a lot of memory now goes into non-heap space, be under 50% of total RAM). We have over 3000 datasets with very heavy usage and run about 20GB-27GB actual memory usage. As for storage, I have no idea, you folks know your data and what you might be doing better that I do. The real question is will you be storing data on AWS or on the server. If AWS, ERDDAP itself doesn't take up all that much storage (at least usually relative to the datasets). If on the server, putting your most used datasets on SSDs will greatly improve access.

If they need to cut down on price, unless you see massive usage or large number of datasets in the future, you should be ok with 64GB or 128GB of RAM, unless you will be running a lot of the things on that machine.

HTH,

-Roy

> To view this discussion on the web, visit https://groups.google.com/d/msgid/erddap/cfd59ae6-80d0-460a-bc60-cc85ea906c0fn%40googlegroups.com.