Historical Humidity Data By Zip Code
Latisha Gervase
Daily summaries of past weather by location come from the Global Historical Climatology Network daily (GHCNd) database and are accessed through the Climate Data Online (CDO) interface, both of which are managed and maintained by NOAA NCEI.
GHCNd includes daily observations from automated and human-facilitated weather stations across the United States and around the world. Observations can include weather variables such as maximum and minimum temperatures, total precipitation, snowfall, and depth of snow on ground. However, not all stations record all variables; about half the stations only report precipitation. Searching by zip code will yield no results if there is no weather station within that zip code, but you can easily expand your search to a city or county.
The action will now move to your email inbox. First, you'll receive a notice that the request has been submitted. Usually, just a few minutes later, you'll receive an email stating that your order has been processed. The second email contains a link for you to download the data you requested, in a multi-page data table. Check all pages to see the full range of data.
LinkedIn and 3rd parties use essential and non-essential cookies to provide, secure, analyze and improve our Services, and to show you relevant ads (including professional and job ads) on and off LinkedIn. Learn more in our Cookie Policy.
b. During my PhD, I mainly used Matlab and I became quite comfortable with it and good at it too. Back in 2013, python was not as big as today and there was not as much interest/push for open source data or online tutorials. I remember how difficult it was for me in those days to analyze NetCDF data without making a mistake on the latlon or time coordinate (but now xarray has made it unbelievably easy!). It is always difficult to change, and my hope is to showcase the capabilities and convenience of python, and be a starting point that helps people navigate through their learning journey.
e. I use some of the functionalities explained in this tutorial on a daily basis, and gathering all the useful commands (e.g. how to adjust plots or format datasets) at one place makes it easier for me to locate and use them later on.
The goal of this tutorial is to exercise geospatial / climate data analysis (i.e. analyzing multidimensional datasets that have data corresponding to specific latitudes, longitudes, and time) using Python. My hope is that after this tutorial, you will have more understanding about various datasets such as:
The tutorial consists of three separate sections: basic, intermediate, and advanced. The sections are developed in a way that you do can start from each one separately (i.e. no need to start from the beginning, and all the necessary packages for each section are imported at the beginning of the sections). In all the sections, a primary library that we'll use is xarray; an open source Python package that makes working with labelled multi-dimensional arrays simple and efficient. A summary of each section is provided in the following:
This section focuses on some of the basic commands and essential functions of xarray. We will download and extract daily observed precipitation data (CPC-CONUS from NOAA) for 4 years, and we will practice working with functions such as groupby, concat, and sel & isel (to select data for specific dates or particular locations). In addition, we will learn how to handle leap years, and we will be saving our desired outputs as NetCDF files. Lastly, we will be making simple plots and save them as high quality figures. In this section, our main focus is on the 2012 precipitation across the CONUS (when an extreme drought caused massive damage to Midwest crops). Two of the plots that are developed in this section are presented here for your reference.
In this section, we go beyond the basics. We utilize observed air temperature data from two datasets, and we focus on the February 2021 cold storm that happened in Texas (more info here). We will be practicing interpolation, scaling datasets (i.e. converting from K to C), assigning new attributes (e.g. converting 0:360 degrees longitude to -180:180), timeseries analysis (working with datetime format coordinate), and rolling average (with specific time window). Two of the sample plots that are generated in this section are shown here.
In both parts, we will load the data directly in the memory (without downloading the data on the disk, thus skipping data download). The main functionalities that are explored in this section are timeseries analysis, anomaly calculation, working with zarr data format, and making a timelapse animation.
Here's another figure showing the standard deviation of forecasts (among ensemble members), which indicates the spread / uncertainty of forecasts for the entire globe as well as across North America only.
In this part, we will explore the available CMIP6 climate data on Google Cloud Storage (again skipping data download), and we will be focusing on one model (i.e. GFDL-CM4) during a particular historical period (i.e. 1980-2010) as well as SSP585 future projection in distant future (i.e. 2070-2099). Needless to say that this is just an example that focuses on an extreme future realization (which can be biassed) and it does not necessarily indicate how global warming in future will be. We will look at monthly temperature changes and generate figures such as this one:
Besides the topics that were analyzed in this tutorial, there are several other subjects that I think are useful to practice in order to become an expert climate data scientist. I may have mentioned some of these in the tutorial, but I gather them here just to have them all in one place:
Climate datasets are stored in different formats, and I think it is essential for a climate data scientist to be able to analyze various data formats. Here are a few data formats (along with a few examples for each case) that you can consider working with:
In addition to the data format, I think it is beneficial to experience working with data at different temporal resolutions (e.g. sub-hourly, daily, monthly, seasonal, annual, and decadal) since they each require unique skills. Similarly, it is beneficial to work with data at different spatial resolutions. Overall, you need to have a general understanding of how big you can define a project and what is practical given your available resources (e.g. is it practical for you to downscale CMIP6 climate projections for all models and ensemble members to 1km resolution?)
Dask is a customizable open source library that provides advanced parallelism for analytics. It can be employed for speeding up the analyses or to analyze data which is larger than the available memory (RAM). I tired to add an example with Dask to the tutorial, but I was getting errors on Google Colab and I did not want to spend too much time on debugging, so I excluded it. If you are interested to learn Dask and play around with it, I suggest two resources: one is the tutorials that are on Dask's website, and the other is a short course developed by the Coild team on TalkPython.
Cartopy is a Python package for geospatial data processing and plotting. You can utilize it to plot scalar and vector data (e.g. points, lines, vectors, polygons, and images) in various coordinate systems (i.e. map projection). The default plots generated by matplotlib or xarray do not offer coast lines or country / state borders. Whereas, you can leverage cartopy and add these features as well as background image or lakes and much more to your plots.
I mentioned this one in the tutorial as well; OpenCV is a powerful image processing tool that has many applications in computer vision analyses. However, it can be super useful for processing climate data too. In fact, if we consider one time step of any climate variable (at a certain elevation or pressure level), it can be viewed as an image with hundreds or thousands of pixels (i.e. grids). OpenCV can be utilized for geospatial analyses (e.g. edge detection for detecting clouds or detecting the boundary of a farm from satellite imagery, object tracking for tracking a storm or cold front through time, blurring and spatial smoothing, etc.).
Last but definitely not least, you surely need to know about (and have some experience with) machine learning too. Depending on the sector you are focusing on, you may need to have worked with one or more of the following:
Notably, there are numerous tutorials, blog posts, packages, and codes available to develop complicated machine learning models, and you can easily train a model with several million parameters on Google Colab or even your laptop. While it is necessary to know how to train and test such models, it is more important to learn the fundamentals and know what approach is suitable for your problem (e.g. the difference between logistic and linear regression, or when to choose parametric vs. non-parametric models, or how to validate/interpret model outputs, or how to work with imbalanced datasets, etc.). It is often easy to jump in and develop/implement predictive AI models, but being sure of the robustness of a model when new data is fed to it is crucial. Often times, depth is more important than breadth of knowledge.
If you are new to Python and you're trying to switch but don't want to necessarily work on Google Colab, you can have a similar working environment on your machine by installing Jupyter Lab and Anaconda. I am sure there are tons of step by step tutorials on Youtube and many blog posts that can walk you through the installation process. Believe me, it is really easy. Just do it!
One very useful geospatial tool that I did not mention about in this tutorial is GDAL, and the main reason is that it was not originally a Python package (though it is now available in Python). You can do a lot of data translation, clipping, and more using GDAL either on command line or through your Python scripts.
For professionals and specialists with middle sized project, we recommend our Professional collections, which included Current & Forecasts collection, Historical weather data collection, Weather Maps collection and other APIs.
b37509886e