A Tutorial on GeoAI: Designing Footprint Extraction from NAIP Imagery Using U-Net, Grounding DINO, SAM, and Mask R-CNN
In this tutorial, we design a whole GeoAI workflow for extracting constructing footprints from high-resolution NAIP aerial imagery. We start by configuring the geospatial deep studying setting, downloading raster imagery and vector labels, and inspecting their spatial properties earlier than producing georeferenced picture chips and segmentation masks. We then prepare a U-Net mannequin with a ResNet-34 encoder, consider its studying conduct, and apply sliding-window inference to an unseen scene. Beyond semantic segmentation, we convert predicted masks into cleaned and regularized constructing polygons, calculate IoU and F1 metrics, discover zero-shot segmentation with Grounding DINO and SAM, and examine the outcomes with a pretrained Mask R-CNN occasion segmentation mannequin. We additionally display how the identical pipeline extends to real-world areas utilizing NAIP imagery from Microsoft Planetary Computer and constructing labels from Overture Maps.
import os
import subprocess
import sys
import time
import warnings
warnings.filterwarnings("ignore")
IN_COLAB = "google.colab" in sys.modules
def pip_install(packages, quiet=True):
"""Install packages with pip from contained in the pocket book course of."""
cmd = [sys.executable, "-m", "pip", "install", "--upgrade"]
if quiet:
cmd.append("-q")
subprocess.run(cmd + record(packages), test=False)
attempt:
import geoai
besides ImportError:
print(">>> Installing geoai-py and associates (takes ~2-4 minutes on Colab)...")
pip_install(
[
"geoai-py",
"segmentation-models-pytorch",
"buildingregulariser",
]
)
attempt:
import geoai
besides Exception as e:
increase SystemExit(
f"Import failed after set up ({e}).n"
"=> Runtime > Restart session, then re-run this cell. "
"The set up is cached, so will probably be quick the second time."
)
import geopandas as gpd
import matplotlib.pyplot as plt
import numpy as np
import rasterio
import torch
from rasterio.plot import plotting_extent
from IPython.show import show
print(f"geoai : {geoai.__version__}")
print(f"torch : {torch.__version__}")
print(f"CUDA obtainable: {torch.cuda.is_available()}")
if torch.cuda.is_available():
print(f"GPU : {torch.cuda.get_device_name(0)}")
else:
print("!! No GPU detected. Training will nonetheless run however be a lot slower.")
print(" Colab: Runtime > Change runtime kind > Hardware accelerator > T4 GPU")
DEVICE = geoai.get_device()
print(f"geoai gadget : {DEVICE}")
CFG = {
"tile_size": 512,
"stride": 256,
"buffer_radius": 0,
"structure": "unet",
"encoder": "resnet34",
"encoder_weights": "imagenet",
"num_channels": 3,
"num_classes": 2,
"batch_size": 8,
"num_epochs": 12,
"learning_rate": 1e-3,
"val_split": 0.2,
"window_size": 512,
"overlap": 256,
"run_zero_shot": True,
"run_pretrained": True,
"run_real_aoi": False,
}
WORK = "/content material/geoai_tutorial" if IN_COLAB else os.path.abspath("geoai_tutorial")
os.makedirs(WORK, exist_ok=True)
os.chdir(WORK)
print(f"working dir : {WORK}")
def banner(textual content):
print("n" + "=" * 92 + f"n {textual content}n" + "=" * 92)
def timed(fn, label):
"""Run fn(), report wall time, by no means let one step kill the pocket book."""
banner(label)
t0 = time.time()
attempt:
out = fn()
print(f"n[OK] {label} — {time.time() - t0:.1f}s")
return out
besides Exception as exc:
import traceback
print(f"n[SKIPPED] {label}n{kind(exc).__name__}: {exc}")
traceback.print_exc(restrict=3)
return None
HF = "https://huggingface.co/datasets/giswqs/geospatial/resolve/foremost"
train_raster_url = f"{HF}/naip_rgb_train.tif"
train_vector_url = f"{HF}/naip_train_buildings.geojson"
test_raster_url = f"{HF}/naip_test.tif"
def step1():
train_raster = geoai.download_file(train_raster_url)
train_vector = geoai.download_file(train_vector_url)
test_raster = geoai.download_file(test_raster_url)
for p in (train_raster, train_vector, test_raster):
print(f" {os.path.getsize(p) / 1e6:8.2f} MB {p}")
return train_raster, train_vector, test_raster
paths = timed(step1, "STEP 1 — Downloading pattern NAIP imagery and constructing labels")
TRAIN_RASTER, TRAIN_VECTOR, TEST_RASTER = paths
We configure the setting, set up the required GeoAI and deep studying libraries, and confirm GPU availability. We outline the central configuration parameters for dataset creation, mannequin coaching, inference, and non-compulsory processing phases. We then create the working listing, outline reusable execution utilities, and obtain the NAIP imagery and constructing footprint labels.
def step2():
data = geoai.get_raster_info(TRAIN_RASTER)
for okay, v in data.gadgets():
print(f" {okay:<16}: {v}")
print("n--- per-band statistics ---")
print(geoai.get_raster_stats(TRAIN_RASTER))
print("n--- vector data ---")
vinfo = geoai.get_vector_info(TRAIN_VECTOR)
for okay, v in vinfo.gadgets():
print(f" {okay:<16}: {v}")
gdf = gpd.read_file(TRAIN_VECTOR)
print(f"n {len(gdf)} coaching buildings | CRS {gdf.crs}")
print(gdf.head(3))
geoai.view_vector(
gdf,
raster_path=TRAIN_RASTER,
outline_only=True,
edge_color="yellow",
outline_linewidth=0.8,
figsize=(11, 11),
title="NAIP coaching scene + constructing footprints",
)
attempt:
show(geoai.view_vector_interactive(TRAIN_VECTOR, layer_name="Buildings"))
besides Exception as e:
print(f" (interactive map unavailable right here: {e})")
return gdf
LABELS_GDF = timed(step2, "STEP 2 — Inspecting raster + vector information")
TILES_DIR = os.path.be a part of(WORK, "tiles")
def step3():
stats = geoai.export_geotiff_tiles(
in_raster=TRAIN_RASTER,
out_folder=TILES_DIR,
in_class_data=TRAIN_VECTOR,
tile_size=CFG["tile_size"],
stride=CFG["stride"],
buffer_radius=CFG["buffer_radius"],
all_touched=True,
skip_empty_tiles=False,
quiet=False,
)
n_img = len(os.listdir(f"{TILES_DIR}/photographs"))
n_lbl = len(os.listdir(f"{TILES_DIR}/labels"))
print(f"n chips: {n_img} photographs / {n_lbl} masks")
if isinstance(stats, dict):
tot = max(stats.get("total_tiles", n_img), 1)
print(f" tiles containing buildings: {stats.get('tiles_with_features')} "
f"({100 * stats.get('tiles_with_features', 0) / tot:.1f}%)")
print(f" foreground pixels: {stats.get('feature_pixels'):,}")
geoai.display_training_tiles(TILES_DIR, num_tiles=6, figsize=(18, 6))
return stats
TILE_STATS = timed(step3, "STEP 3 — Exporting picture chips and label masks")
We examine the raster and vector datasets to grasp their coordinate methods, dimensions, statistics, and characteristic constructions. We visualize the constructing labels over the aerial imagery and generate an interactive map for spatial exploration. We then divide the supply imagery into overlapping georeferenced chips and create matching raster masks for mannequin coaching.
MODEL_DIR = os.path.be a part of(WORK, "models_unet")
BEST_MODEL = os.path.be a part of(MODEL_DIR, "best_model.pth")
def step4():
geoai.train_segmentation_model(
images_dir=f"{TILES_DIR}/photographs",
labels_dir=f"{TILES_DIR}/labels",
output_dir=MODEL_DIR,
structure=CFG["architecture"],
encoder_name=CFG["encoder"],
encoder_weights=CFG["encoder_weights"],
num_channels=CFG["num_channels"],
num_classes=CFG["num_classes"],
batch_size=CFG["batch_size"],
num_epochs=CFG["num_epochs"],
learning_rate=CFG["learning_rate"],
val_split=CFG["val_split"],
save_best_only=True,
early_stopping_patience=5,
verbose=True,
)
print(f"n greatest checkpoint: {BEST_MODEL}")
print(f" dimension: {os.path.getsize(BEST_MODEL) / 1e6:.1f} MB")
return BEST_MODEL
timed(step4, f"STEP 4 — Training {CFG['architecture']}/{CFG['encoder']} "
f"for {CFG['num_epochs']} epochs")
def step5():
hist_path = os.path.be a part of(MODEL_DIR, "training_history.pth")
geoai.plot_performance_metrics(
history_path=hist_path,
figsize=(15, 5),
verbose=True,
save_path=os.path.be a part of(WORK, "training_curves.png"),
)
h = torch.load(hist_path, weights_only=False)
best_ep = int(np.argmax(h["val_iou"])) + 1
print(f"n greatest val IoU {max(h['val_iou']):.4f} at epoch {best_ep}")
print(" Reading the curves: val loss rising whereas prepare loss falls => overfitting;")
print(" each flat and excessive => underfitting (extra epochs, larger encoder, or extra chips).")
timed(step5, "STEP 5 — Training diagnostics")
We prepare a U-Net semantic segmentation mannequin with a ResNet-34 encoder utilizing the ready picture and masks tiles. We configure the coaching course of with validation splitting, early stopping, checkpoint saving, and efficiency monitoring. We then load the coaching historical past, plot the educational curves, and determine the epoch that produces the best validation IoU.
PRED_MASK = os.path.be a part of(WORK, "test_prediction.tif")
PRED_PROB = os.path.be a part of(WORK, "test_probability.tif")
def step6():
geoai.semantic_segmentation(
input_path=TEST_RASTER,
output_path=PRED_MASK,
model_path=BEST_MODEL,
structure=CFG["architecture"],
encoder_name=CFG["encoder"],
num_channels=CFG["num_channels"],
num_classes=CFG["num_classes"],
window_size=CFG["window_size"],
overlap=CFG["overlap"],
batch_size=4,
probability_path=PRED_PROB,
)
geoai.print_raster_info(PRED_MASK, show_preview=False)
geoai.plot_prediction_comparison(
original_image=TEST_RASTER,
prediction_image=PRED_MASK,
titles=["NAIP test scene", "Predicted building mask"],
figsize=(16, 8),
prediction_colormap="viridis",
save_path=os.path.be a part of(WORK, "prediction_comparison.png"),
)
with rasterio.open(PRED_MASK) as src:
m = src.learn(1)
px = float(abs(src.remodel.a) * abs(src.remodel.e))
print(f" predicted constructing pixels: {int((m > 0).sum()):,} "
f"({100 * (m > 0).imply():.2f}% of scene, ~{(m > 0).sum() * px:,.0f} m2)")
timed(step6, "STEP 6 — Sliding-window inference on the take a look at scene")
VEC_RAW = os.path.be a part of(WORK, "buildings_raw.geojson")
VEC_ORTHO = os.path.be a part of(WORK, "buildings_orthogonal.geojson")
VEC_FINAL = os.path.be a part of(WORK, "buildings_final.geojson")
def step7():
grouped = geoai.region_groups(
PRED_MASK,
connectivity=2,
min_size=50,
out_image=os.path.be a part of(WORK, "test_prediction_cleaned.tif"),
)
clean_mask = os.path.be a part of(WORK, "test_prediction_cleaned.tif")
uncooked = geoai.raster_to_vector(
clean_mask,
output_path=VEC_RAW,
threshold=0,
min_area=15,
simplify_tolerance=0.5,
)
print(f" uncooked polygons : {len(uncooked)}")
ortho = geoai.orthogonalize(
input_path=clean_mask,
output_path=VEC_ORTHO,
epsilon=1.5,
min_area=15,
)
print(f" orthogonalized : {len(ortho)}")
remaining = geoai.regularization(ortho, angle_tolerance=12, simplify_tolerance=0.4)
remaining = geoai.add_geometric_properties(
remaining, properties=["area", "perimeter", "solidity", "elongation", "orientation"]
)
remaining.to_file(VEC_FINAL, driver="GeoJSON")
print(f" remaining footprints : {len(remaining)}")
print(remaining.head())
if "space" in remaining.columns:
print("n footprint space stats (m2):")
print(remaining["area"].describe().spherical(1).to_string())
fig, axes = plt.subplots(1, 2, figsize=(16, 8))
with rasterio.open(TEST_RASTER) as src:
rgb = src.learn([1, 2, 3]).transpose(1, 2, 0)
rgb = np.clip(rgb / np.percentile(rgb, 99), 0, 1)
ext = plotting_extent(src)
for ax, g, t in zip(axes, [raw, final], ["Raw polygonization", "Orthogonalized + regularized"]):
ax.imshow(rgb, extent=ext)
g.plot(ax=ax, facecolor="none", edgecolor="crimson", linewidth=1.1)
ax.set_title(t)
ax.set_axis_off()
plt.tight_layout()
plt.present()
attempt:
show(geoai.view_vector_interactive(VEC_FINAL, layer_name="Predicted buildings"))
besides Exception:
cross
return remaining
FINAL_GDF = timed(step7, "STEP 7 — Vectorizing and regularizing the anticipated footprints")
We apply sliding-window inference to an unseen NAIP scene and generate each prediction and chance rasters. We take away small noisy areas, convert the anticipated masks into vector polygons, and regularize the footprint geometries to provide cleaner constructing boundaries. We additionally calculate geometric properties and examine the uncooked polygonized outcomes with the orthogonalized and regularized outputs.
def step8():
gt_raster = os.path.be a part of(WORK, "train_gt_mask.tif")
geoai.vector_to_raster(
vector_path=TRAIN_VECTOR,
output_path=gt_raster,
reference_raster=TRAIN_RASTER,
fill_value=0,
all_touched=True,
dtype=np.uint8,
)
train_pred = os.path.be a part of(WORK, "train_prediction.tif")
geoai.semantic_segmentation(
input_path=TRAIN_RASTER,
output_path=train_pred,
model_path=BEST_MODEL,
structure=CFG["architecture"],
encoder_name=CFG["encoder"],
num_channels=CFG["num_channels"],
num_classes=CFG["num_classes"],
window_size=CFG["window_size"],
overlap=CFG["overlap"],
quiet=True,
)
metrics = geoai.calc_segmentation_metrics(
ground_truth=gt_raster,
prediction=train_pred,
num_classes=2,
metrics=["iou", "f1"],
)
print("n --- pixel-wise metrics (class 0 = background, class 1 = constructing) ---")
for okay, v in metrics.gadgets():
print(f" {okay:<10}: {np.spherical(v, 4)}")
print("n Building-class IoU is the quantity that issues; background IoU is inflated")
print(" by the massive adverse class and at all times appears nice.")
geoai.plot_prediction_comparison(
original_image=TRAIN_RASTER,
prediction_image=train_pred,
ground_truth_image=gt_raster,
titles=["Imagery", "Prediction", "Ground truth"],
figsize=(18, 6),
save_path=os.path.be a part of(WORK, "accuracy_comparison.png"),
)
return metrics
METRICS = timed(step8, "STEP 8 — Quantitative accuracy evaluation")
def step9():
if not CFG["run_zero_shot"]:
print(" disabled in CFG"); return
chip = os.path.be a part of(WORK, "test_chip.tif")
with rasterio.open(TEST_RASTER) as src:
b = src.bounds
cx, cy = (b.left + b.proper) / 2, (b.backside + b.prime) / 2
half = min((b.proper - b.left), (b.prime - b.backside)) / 6
bbox = [cx - half, cy - half, cx + half, cy + half]
geoai.clip_raster_by_bbox(TEST_RASTER, chip, bbox=bbox, bbox_type="geo")
print(f" clipped chip: {chip}")
sam = geoai.GroundedSAM(
detector_id="IDEA-Research/grounding-dino-tiny",
segmenter_id="fb/sam-vit-base",
tile_size=1024,
overlap=128,
threshold=0.3,
)
out_mask = os.path.be a part of(WORK, "zeroshot_mask.tif")
gdf = sam.segment_image(
input_path=chip,
output_path=out_mask,
text_prompts=["building", "house", "rooftop"],
polygon_refinement=True,
export_polygons=True,
min_polygon_area=30,
simplify_tolerance=1.5,
)
geoai.plot_prediction_comparison(
original_image=chip,
prediction_image=out_mask,
titles=["Chip", "Zero-shot: 'building / house / rooftop'"],
figsize=(14, 7),
prediction_colormap="viridis",
)
print(f" zero-shot objects discovered: {len(gdf) if gdf is just not None else 0}")
geoai.empty_cache()
timed(step9, "STEP 9 — Zero-shot text-prompted segmentation (Grounding DINO + SAM)")
We consider the segmentation mannequin by evaluating its predictions with rasterized ground-truth constructing labels. We calculate pixel-level IoU and F1 metrics and visualize the imagery, predictions, and reference masks collectively. We then apply Grounding DINO and SAM to carry out zero-shot constructing segmentation utilizing textual content prompts with out further mannequin coaching.
def step10():
if not CFG["run_pretrained"]:
print(" disabled in CFG"); return
extractor = geoai.BuildingFootprintExtractor(model_path="building_footprints_usa.pth")
gdf = extractor.process_raster(
TEST_RASTER,
output_path=os.path.be a part of(WORK, "buildings_maskrcnn.geojson"),
batch_size=4,
confidence_threshold=0.5,
overlap=0.25,
mask_threshold=0.5,
min_object_area=100,
filter_edges=True,
)
if gdf is None or len(gdf) == 0:
print(" no cases returned"); return
print(f" constructing cases: {len(gdf)}")
reg = extractor.regularize_buildings(gdf, min_area=20, angle_threshold=15)
reg.to_file(os.path.be a part of(WORK, "buildings_maskrcnn_regularized.geojson"), driver="GeoJSON")
extractor.visualize_results(TEST_RASTER, gdf=reg, figsize=(12, 12))
if FINAL_GDF is just not None:
print(f"n your U-Net : {len(FINAL_GDF)} polygons")
print(f" pretrained R-CNN: {len(reg)} polygons")
print(" Different counts are anticipated: U-Net merges adjoining roofs, Mask R-CNN splits")
print(" them into cases. Pick the paradigm that matches your downstream query.")
geoai.empty_cache()
timed(step10, "STEP 10 — Pretrained Mask R-CNN occasion segmentation")
def step11():
if not CFG["run_real_aoi"]:
print(" disabled (set CFG['run_real_aoi'] = True to run; wants open web)")
return
bbox = (-83.9400, 35.9500, -83.9250, 35.9600)
gadgets = geoai.pc_stac_search(
assortment="naip",
bbox=record(bbox),
time_range="2021-01-01/2023-12-31",
max_items=3,
)
print(f" STAC gadgets discovered: {len(gadgets)}")
aoi_dir = os.path.be a part of(WORK, "aoi")
tif = geoai.download_naip(bbox=bbox, output_dir=aoi_dir, max_items=1, preview=False)
print(f" NAIP: {tif}")
ovt = os.path.be a part of(aoi_dir, "overture_buildings.geojson")
geoai.download_overture_buildings(bbox=bbox, output=ovt, overture_type="constructing")
print(f" Overture buildings: {ovt}")
print(geoai.extract_building_stats(ovt))
raster = tif[0] if isinstance(tif, (record, tuple)) else tif
geoai.export_geotiff_tiles(
in_raster=raster,
out_folder=os.path.be a part of(aoi_dir, "tiles"),
in_class_data=ovt,
tile_size=512,
stride=256,
)
print(" AOI dataset prepared — feed it to train_segmentation_model() precisely as in STEP 4.")
timed(step11, "STEP 11 — (non-compulsory) Real AOI: Planetary Computer NAIP + Overture Maps labels")
def step12():
outputs = [f for f in sorted(os.listdir(WORK))
if f.endswith((".tif", ".geojson", ".png", ".pth"))]
print(" artifacts produced:")
for f in outputs:
print(f" {os.path.getsize(os.path.be a part of(WORK, f)) / 1e6:8.2f} MB {f}")
zip_path = os.path.be a part of(WORK, "geoai_results.zip")
subprocess.run(
["zip", "-qr", zip_path, ".", "-i", "*.geojson", "*.png", "*.tif", "-x", "*tiles*"],
cwd=WORK, test=False,
)
print(f"n bundle: {zip_path}")
if IN_COLAB:
print(" Download it with: from google.colab import recordsdata; "
"recordsdata.obtain('%s')" % zip_path)
timed(step12, "STEP 12 — Results abstract")
banner("DONE")
print("""
Where to go subsequent
----------------
* Swap the top, maintain the code: structure="deeplabv3plus", encoder_name="efficientnet-b3"
(or any timm encoder) in train_segmentation_model().
* 4-band NAIP (RGB+NIR): num_channels=4 in all places; the primary conv is auto-adapted.
* Multi-class land cowl: geoai.train_segmentation_landcover() + geoai.export_landcover_tiles(),
with DiceLoss / FocalLoss / TverskyLoss from geoai.landcover_train for sophistication imbalance.
* Instance segmentation you prepare your self: geoai.train_MaskRCNN_model() then
geoai.instance_segmentation(..., vectorize=True).
* Object detection with georeferenced containers: geoai.prepare.object_detection() /
geoai.object_detection(textual content="automobiles") for open-vocabulary Grounding DINO.
* Foundation fashions: geoai.prithvi_inference (NASA/IBM Prithvi), geoai.universat_inference,
geoai.DINOv3GeoProcessor for embeddings and similarity maps.
* Change detection: geoai.change_detection (torchange backends).
* Deploy: geoai.export_to_onnx() + geoai.onnx_semantic_segmentation(), or the QGIS plugin.
Docs and notebooks: https://opengeoai.org | Book: https://guide.opengeoai.org
""")
We use a pretrained Mask R-CNN mannequin to extract particular person constructing cases and examine them with the U-Net outcomes. We optionally create a real-world dataset by downloading NAIP imagery from Microsoft Planetary Computer and matching constructing labels from Overture Maps. We lastly acquire the generated rasters, vectors, plots, and mannequin outputs right into a compressed outcomes bundle for additional evaluation or obtain.
In conclusion, we accomplished an end-to-end geospatial deep studying pipeline that transforms uncooked aerial imagery into structured and analysis-ready constructing footprint information. We ready coaching samples, skilled and evaluated a semantic segmentation mannequin, generated seamless predictions, and refined raster outputs into orthogonalized vector geometries with helpful spatial attributes. We additionally examined various extraction approaches by way of zero-shot basis fashions and pretrained occasion segmentation, which helps us perceive the trade-offs between customized coaching, prompt-based detection, and ready-to-use fashions. By packaging the generated masks, chance rasters, analysis plots, skilled weights, and GeoJSON outputs, we created a reusable basis that we will adapt for land-cover mapping, infrastructure detection, change evaluation, and large-scale GeoAI functions.
Check out the Full Codes here. Also, be happy to observe us on Twitter and don’t neglect to hitch our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to companion with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so forth.? Connect with us
The put up A Tutorial on GeoAI: Designing Footprint Extraction from NAIP Imagery Using U-Net, Grounding DINO, SAM, and Mask R-CNN appeared first on MarkTechPost.
