Getting Started#
The Chalmers Hydrometor Inversion Product from the Arctic Weather Satellite (CHIP-AWS), provides global atmospheric ice mass estimates derived from passive microwave measurements. The data are available as NetCDF files, the following examples will show how to get started using them with python and xarray.
[1]:
from datetime import datetime
import requests
import xarray as xr
def file_urls_for_timeranges(timeranges: list[slice]) -> list[str]:
"""
Helper function to get a list of level2 urls for the requested timeranges
"""
base_url = "http://data.clouds-and-precip.group/chip-aws/chip-aws-v0.3/level2"
index = requests.get(f"{base_url}/index.json").json()
urls = [
f"{base_url}/{item['filepath']}#mode=bytes"
for item in index['items']
for timerange in timeranges
if datetime.fromisoformat(item['datetime_end']) >= datetime.fromisoformat(timerange.start.replace("Z", "+00:00"))
and datetime.fromisoformat(item['datetime_start']) <= datetime.fromisoformat(timerange.stop.replace("Z", "+00:00"))
]
return sorted(set(urls))
[3]:
# List some interesting scenes
timeranges = [
slice('2025-05-01T07:18:00', '2025-05-01T07:28:00'),
slice('2025-05-01T11:19:00', '2025-05-01T11:28:00'),
slice('2025-05-01T11:46:00', '2025-05-01T11:55:00'),
]
# Open the files as a single xarray dataset
ds = xr.open_mfdataset(
file_urls_for_timeranges(timeranges),
combine="nested",
concat_dim="scan",
data_vars="minimal",
)
# Each file is generated from a single source level 1 file for self-contained processing.
# Consecutive source files have short overlaps. To avoid duplicate data,
# we remove overlapping scans by keeping only the last occurrence of each duplicate.
# Sorting ensures scans are in chronological order from these overlaps,
# allowing selection by time slices.
ds = ds.sortby('scan').sel(scan=~ds.get_index('scan').duplicated(keep='last'))
Dataset structure#
The main variables are given in the along-track scan dimension and the across-track fov (field-of-view) dimension. The structure of the data is shown below:
[4]:
ds
[4]:
<xarray.Dataset> Size: 100MB
Dimensions: (scan: 6721, fov: 88, quantile: 5, surface_type: 6,
channel: 11)
Coordinates:
* surface_type (surface_type) <U7 168B 'ocean' ... 'glacier'
latitude (scan, fov) float32 2MB dask.array<chunksize=(3360, 88), meta=np.ndarray>
longitude (scan, fov) float32 2MB dask.array<chunksize=(3360, 88), meta=np.ndarray>
* quantile (quantile) float32 20B 0.02 0.16 0.5 0.84 0.98
* channel (channel) <U5 220B 'AWS21' 'AWS31' ... 'AWS44'
* scan (scan) datetime64[ns] 54kB 2025-05-01T06:05:36.77...
Dimensions without coordinates: fov
Data variables: (12/16)
fwp_mean (scan, fov) float32 2MB dask.array<chunksize=(3360, 88), meta=np.ndarray>
fwp_most_prob (scan, fov) float32 2MB dask.array<chunksize=(3360, 88), meta=np.ndarray>
fwp_quantiles (scan, fov, quantile) float32 12MB dask.array<chunksize=(3360, 88, 5), meta=np.ndarray>
fwp_dm_mean (scan, fov) float32 2MB dask.array<chunksize=(3360, 88), meta=np.ndarray>
fwp_dm_most_prob (scan, fov) float32 2MB dask.array<chunksize=(3360, 88), meta=np.ndarray>
fwp_dm_quantiles (scan, fov, quantile) float32 12MB dask.array<chunksize=(3360, 88, 5), meta=np.ndarray>
... ...
surface_type_fractions (scan, fov, surface_type) float32 14MB dask.array<chunksize=(3360, 88, 6), meta=np.ndarray>
tb (scan, fov, channel) float32 26MB dask.array<chunksize=(3360, 88, 11), meta=np.ndarray>
l1b_index_scans (scan) uint32 27kB dask.array<chunksize=(3360,), meta=np.ndarray>
l1b_index_fovs (fov) uint32 352B dask.array<chunksize=(88,), meta=np.ndarray>
flag_bad_data (scan, fov) uint8 591kB dask.array<chunksize=(3360, 88), meta=np.ndarray>
fwp_ccic (scan, fov) float32 2MB dask.array<chunksize=(3360, 88), meta=np.ndarray>
Attributes:
title: CHIP-AWS: Chalmers Hydrometeor Inversion Product from t...
institution: Chalmers University of Technology
references: https://clouds-and-precip.group/datasets/chip-aws/
history: Retrieval processing
source_file: W_NO-KSAT-Tromso,SAT,AWS1-MWR-1B-RAD_C_OHB__20250501074...
source_file_md5: d569c33690327feb2f3b925f3172a208
source: chip-aws-v0.4.4Plotting FWP#
[5]:
import matplotlib
import matplotlib.pyplot as plt
# Plot the mean Frozen Water Path (FWP) for our time ranges
fig, axs = plt.subplots(1, 3, figsize=(12, 5), dpi=200, subplot_kw={'aspect': 'equal'})
for ax, timerange in zip(axs, timeranges):
ds_scene = ds.sel(scan=slice(timerange.start, timerange.stop))
# Some footprints can be marked as bad.
# We can filter to keep only footprints without any bad data flag.
da_fwp_mean = ds_scene.fwp_mean.where(ds_scene.flag_bad_data == 0)
# Scatter plot is most robust to NaNs in data.
# NaNs can occur even if data has been filtered or if part of the scan
# extends outside the supported +/- 60 degree latitude region.
s = ax.scatter(
da_fwp_mean.longitude,
da_fwp_mean.latitude,
c=da_fwp_mean,
s=0.5,
norm=matplotlib.colors.LogNorm(vmin=0.025) # Recommended
)
ax.set_xlabel(ds_scene.longitude.attrs['units'])
ax.set_ylabel(ds_scene.latitude.attrs['units'])
plt.colorbar(s, ax=ax, label=ds_scene.fwp_mean.attrs['long_name'] + f" ({ds_scene.fwp_mean.attrs['units']})")
plt.tight_layout()
[6]:
# Another plotting option is to plot with pcolormesh, wich will interpolate between points.
# This can look better, but requires scans with NaNs to be filtered out.
# Load scene and filter out scans with NaNs
ds_scene = ds.sel(scan=slice(timeranges[0].start, timeranges[0].stop))
validscans = (~ds_scene.latitude.isnull() & (ds_scene.flag_bad_data == 0)).all(['fov']).values
da_fwp_mean = ds_scene.fwp_mean.isel(scan=validscans)
_, ax = plt.subplots(1, 1, figsize=(4, 5), subplot_kw={'aspect': 'equal'})
da_fwp_mean.plot.pcolormesh(
x='longitude',
y='latitude',
norm=matplotlib.colors.LogNorm(vmin=0.025), # Recommended
ax=ax
)
[6]:
<matplotlib.collections.QuadMesh at 0x78948d8a5190>
Quantiles#
The retrieval process estimates a cumulative distriubtion function (CDF) for each variable. We’ve so-far seen the mean computed from this CDF. Each variable in the dataset also have corresponding quantile values. We can plot these quantiles to get a sense of the estimated distibution of values.
[7]:
fig = plt.figure(figsize=(12, 5), dpi=300)
gs = fig.add_gridspec(1, 2, width_ratios=[1, 3])
# Select a scene and then select a scan within that scene
ds_scene = ds.sel(scan=slice(timeranges[0].start, timeranges[0].stop))
validscans = (~ds_scene.latitude.isnull() & (ds_scene.flag_bad_data == 0)).all(['fov']).values
ds_scene = ds_scene.isel(scan=validscans)
ds_scan = ds_scene.isel(scan=220)
# Draw map and highlight scan
ax_map = fig.add_subplot(gs[0], aspect='equal') # Smaller subplot with map projection
ds_scene.fwp_mean.plot.pcolormesh(x='longitude', y='latitude', norm=matplotlib.colors.LogNorm(vmin=0.025), ax=ax_map)
ax_map.plot(ds_scan.longitude, ds_scan.latitude, color='red', alpha=0.5)
# Plot quantiles for the scan
ax_quantiles = fig.add_subplot(gs[1], xlabel='Field of view index', ylabel='Frozen Water Path (kg/m²)', yscale='log', ylim=(1e-4, 1e1))
ax_quantiles.plot(ds_scan.fwp_mean, label='Mean')
ax_quantiles.plot(ds_scan.fwp_quantiles.sel(quantile=0.5), label='Quantile 0.5')
for i in range(ds_scan['quantile'].size // 2):
lower = ds_scan.fwp_quantiles.sel(quantile=ds_scan['quantile'][i])
upper = ds_scan.fwp_quantiles.sel(quantile=ds_scan['quantile'][-(i + 1)])
ax_quantiles.fill_between(
x=range(len(lower)),
y1=lower,
y2=upper,
alpha=0.3,
label=f'Quantiles {ds_scan["quantile"][i].item():.2f}-{ds_scan["quantile"][-(i + 1)].item():.2f}'
)
ax_quantiles.legend()
plt.tight_layout()
Extra Data#
In addition to retrieved variables, the dataset includes the level1B antenna temperatures remapped to the AWS3X footprint grid and co-located FWP estimates from the Chalmers Cloud Ice Climatology. These can be useful for comparison and for identifying cases based on channel Ta values.
[8]:
fig, axs = plt.subplots(1, 3, figsize=(12, 5), dpi=200, subplot_kw={'aspect': 'equal'})
ds_scene = ds.sel(scan=slice('2025-05-01T11:19:00', '2025-05-01T11:28:00'))
ds_scene.tb.sel(channel='AWS44').plot.pcolormesh(x='longitude', y='latitude', ax=axs[0])
ds_scene.fwp_mean.plot.pcolormesh(x='longitude', y='latitude', norm=matplotlib.colors.LogNorm(vmin=0.025), ax=axs[1])
ds_scene.fwp_ccic.plot.pcolormesh(x='longitude', y='latitude', norm=matplotlib.colors.LogNorm(vmin=0.025), ax=axs[2])
plt.tight_layout()
