Arizona

Overview

This notebook constructs a cleaned precinct geography for Arizona by combining 2020 Census blocks with 2024 election precincts.

The workflow performs several major tasks:

Load Census block and precinct shapefiles.
Standardize county names.
Reproject both datasets into a common projected coordinate system.
Assign every Census block to a voting precinct.
Clean small geometric discontinuities.
Merge disconnected precinct fragments.
Ensure every precinct is contiguous.
Detect and eliminate graph articulation points.
Attach 2024 presidential election results.
Export cleaned shapefiles

Required Local Libraries

Several custom libraries are used throughout the workflow.

These libraries provide functionality for:

county lookup tables
graph construction
shapefile I/O
hierarchy operations
topology cleaning

Most of the spatial processing relies on these modules rather than GeoPandas alone.

import geopandas as gpd
import numpy as np
import pandas as pd
import os
import shapely
import sys
sys.path.append("/Users/Ranthony/Downloads/kmeans/data/lib")

from tqdm import tqdm

############ local libraries #################
from importlib import reload

import censusBlockFunctions as cbF
import countyFunctions as cnF
import hierarchy
import reader
import shapefileToGraph
import writer
reload(cbF)
reload(cnF)
reload(hierarchy)
reload(reader)
reload(shapefileToGraph)
reload(writer)
##############################################

<module 'writer' from '/Users/Ranthony/Downloads/kmeans/data/lib/writer.py'>

Configuration

Specify the state abbreviation, FIPS code, and expected congressional district count.

These values are reused throughout the pipeline.

Changing them allows this workflow to be adapted for another state, provided the appropriate shapefiles are supplied.

stateAbv = "AZ"
stateFIPS = "04"
dists = 9

Load GIS Data

Load

election precinct boundaries
Census block boundaries

The precinct shapefile contains election attributes, while the Census blocks provide the smallest geographic units that will ultimately be aggregated into clean precinct polygons.

You will need to obtain precinct shape file and census block shapes files for this, example links are provided below:

https://redistrictingdatahub.org/data/download-data/
https://www2.census.gov/geo/tiger/TIGER2024/TABBLOCK20/

precinctShapefile = gpd.read_file("az_2024_gen_prec/az_2024_gen_prec.shp") # https://redistrictingdatahub.org/data/download-data/
cblockShapefile = gpd.read_file("tl_2025_04_tabblock20") # https://www2.census.gov/geo/tiger/TIGER2024/TABBLOCK20/

reload(cnF)
cnF.determineCountyIDMaps("data", path="/Users/Ranthony/Downloads/kmeans/")

inspect columns

cblockShapefile.columns

Index(['STATEFP20', 'COUNTYFP20', 'TRACTCE20', 'BLOCKCE20', 'GEOID20',
       'GEOIDFQ20', 'NAME20', 'MTFCC20', 'UR20', 'UACE20', 'FUNCSTAT20',
       'ALAND20', 'AWATER20', 'INTPTLAT20', 'INTPTLON20', 'HOUSING20', 'POP20',
       'geometry'],
      dtype='object')

precinctShapefile.columns

Index(['UNIQUE_ID', 'COUNTYFP', 'COUNTY_NAM', 'PRECINCTNA', 'CONG_DIST',
       'SLDL_DIST', 'SLDU_DIST', 'G24JUSBOLN', 'G24JUSBOLY', 'G24JUSKINN',
       ...
       'GSL28RPIN', 'GSL29DOCH', 'GSL29DTZI', 'GSL29OWRI', 'GSL29RMON',
       'GSL29RTAY', 'GSL30DTIM', 'GSL30RBIA', 'GSL30RGIL', 'geometry'],
      dtype='object', length=283)

Standardize County Names

Although both datasets contain county identifiers, they store them as FIPS codes.

The custom county lookup converts these numeric codes into standardized county names.

Using common county names allows later spatial joins to be restricted within county boundaries, preventing accidental assignments across county lines.

cblockShapefile["COUNTY"] = cblockShapefile["COUNTYFP20"].apply(lambda g: cnF.countyName(g, stateID=stateFIPS))
precinctShapefile["COUNTY"] = precinctShapefile["COUNTYFP"].apply(lambda g: cnF.countyName(g, stateID=stateFIPS))

Reproject into a Common Coordinate System

Spatial overlay operations require both datasets to share the same coordinate reference system (CRS), so we need the CRS of our two shape files to match!

Arizona State Plane (ESRI:103226) is used because it preserves distances and areas much better than geographic latitude/longitude coordinates.

You can find state CRS (or approximation) here: - https://spatialreference.org/ref/esri/

Note: Working in a projected CRS is particularly important for:

spatial joins
area calculations
topology cleaning
adjacency detection

cblockShapefile = cblockShapefile.to_crs("ESRI:103226") # find state CRS (or approximation) https://spatialreference.org/ref/esri/
precinctShapefile = precinctShapefile.to_crs("ESRI:103226")

Assign census blocks to precincts

len(set(list(precinctShapefile.COUNTY))), len(set(list(cblockShapefile.COUNTY)))

(15, 15)

# check naming convensions -- should be fine since both were sourced from fp codes above
precinctShapefile.loc[0, "COUNTY"], cblockShapefile.loc[0, "COUNTY"]

('APACHE COUNTY', 'MARICOPA COUNTY')

set(precinctShapefile.COUNTY) == set(cblockShapefile.COUNTY)

True

# check the right state
precinctShapefile.plot()

Standardize County Names

Although both datasets contain county identifiers, they store them as FIPS codes.

The custom county lookup converts these numeric codes into standardized county names.

Using common county names allows later spatial joins to be restricted within county boundaries, preventing accidental assignments across county lines.

precinctShapefile["NAME"] = precinctShapefile["UNIQUE_ID"]

precinctShapefile.COUNTY = precinctShapefile.COUNTY.map(lambda c: c.upper())

pctField = "NAME"

reload(hierarchy)
cblockFile = "cblocksWPct2020_" + stateAbv
filePath = os.path.join("..", "shapefiles", "derived", cblockFile + ".shp")
if os.path.exists(filePath):
    cblockShapefile = gpd.read_file(filePath)
else:
    cblockShapefile = hierarchy.smallUnitsToPartition(cblockShapefile, precinctShapefile,
                                                      pctField, "COUNTY")

Clean precinct noise/small discontinuities

Spatial overlay frequently introduces tiny disconnected polygons caused by

sliver polygons
precision errors
extremely small isolated fragments

These artifacts are generally not meaningful voting geography.

We want to remove these small geographic artifacts. The cleaning routine reassigns these fragments based on neighboring population, producing cleaner precinct geometries suitable for graph construction.

cblockShapefile = hierarchy.cleanSmallPartitionDiscontinuity(cblockShapefile, 
                                                             ["COUNTY", pctField], 
                                                             "POP20")

Merge or split multipolyginol precincts

Many election precinct datasets contain disconnected pieces, so we want to eliminate multiPolygon p.precincts.

Disconnected precincts create problems for graph algorithms because a single node would represent multiple disconnected regions.

This stage merges or reallocates disconnected components until every precinct consists of a single connected polygon.

reload(hierarchy)

cblockShapefile = hierarchy.mergeSignificantPartitionDiscontinuity(cblockShapefile, 
                                                                   ["COUNTY", pctField], 
                                                                   "POP20")

pctSF = cblockShapefile.dissolve(by=["COUNTY", pctField], aggfunc='sum')
pctSF = pctSF.reset_index()
split_regions = pctSF[pctSF.geometry.map(lambda g: isinstance(g, 
                                         shapely.geometry.multipolygon.MultiPolygon))]
assert len(split_regions) == 0

len(split_regions)
split_regions[[ "COUNTY", pctField ]].head()

	COUNTY	NAME

len(pctSF), len(precinctShapefile)

(1689, 1733)

Find biconnected components and merge

Detected articulation points are merged according to the hierarchy rules.

The objective is to create a graph where every precinct belongs to a biconnected component.

This improves robustness for downstream optimization algorithms that rely on stable connectivity.

#import pyximport; pyximport.install(reload_support=True, language_level=3)
from lib import biconnectedComponents, shapefileToGraph

adjacency = shapefileToGraph.findAdjacency(pctSF) 
adjacency_nobdr = [e for e in adjacency if -1 not in e]
nbrList = shapefileToGraph.neighborList(pctSF, adjacency_nobdr)
articulationPoints = biconnectedComponents.getArticulationPoints_cdef(nbrList)

reload(hierarchy)
pctSF = hierarchy.mergeArticulationDict(pctSF, adjacency, articulationPoints, 
                                                  ["COUNTY", pctField], 
                                                  "POP20")

pctSF["UACE20"] = pctSF["UACE20"].astype(str)

pctSF = pctSF.dissolve(by = ["COUNTY", pctField], aggfunc='sum')
pctSF = pctSF.reset_index()
len(pctSF)

adjacency_check = [e for e in shapefileToGraph.findAdjacency(pctSF) 
                   if -1 not in e]
nbrList_check = shapefileToGraph.neighborList(pctSF, adjacency_check)
articulationPoints = biconnectedComponents.getArticulationPoints_cdef(nbrList_check)
assert len(articulationPoints) == 0

Add Election Results

Election results are transferred from the original precinct shapefile to the cleaned precinct geometries.

Most precinct names match exactly.

However, some cleaned precincts are the result of merging multiple original precincts.

For merged precincts, vote totals are computed as the sum of the constituent precincts.

dr = dict(zip(precinctShapefile[pctField], precinctShapefile.G24PRERTRU))
dd = dict(zip(precinctShapefile[pctField], precinctShapefile.G24PREDHAR))




pctSF["G24PREDEM"] = pctSF[pctField].map(lambda n: getv(n, dd))
pctSF["G24PREREP"] = pctSF[pctField].map(lambda n: getv(n, dr))

Merged precinct identifiers are stored as comma-separated precinct names.

This helper function

1. checks for an exact match,
2. otherwise splits the merged name,
3. sums vote totals across all original precincts.

If necessary, missing precinct names are reported so they can be investigated manually.

def getv(name, votedict):

    # First try exact match
    if name in votedict:
        return votedict[name]

    # Otherwise assume merged precincts
    split_name = [n.strip() for n in name.split(",")]

    total = 0

    for n in split_name:

        if n not in votedict:
            print("Missing precinct:", repr(n))
            continue

        total += votedict[n]

    return total

pctSF.columns

Index(['COUNTY', 'NAME', 'geometry', 'STATEFP20', 'COUNTYFP20', 'TRACTCE20',
       'BLOCKCE20', 'GEOID20', 'GEOIDFQ20', 'NAME20', 'MTFCC20', 'UR20',
       'UACE20', 'FUNCSTAT20', 'ALAND20', 'AWATER20', 'INTPTLAT20',
       'INTPTLON20', 'HOUSING20', 'POP20', 'G24PREDEM', 'G24PREREP'],
      dtype='object')

Remove Unneeded Census Attributes

Once aggregation is complete, many Census block attributes are no longer needed.

Removing these fields substantially reduces file size while retaining

geometry
population
election results
precinct identifiers

pctSF.drop(["STATEFP20", "COUNTYFP20", "BLOCKCE20", "TRACTCE20", "GEOID20", 
            "GEOIDFQ20", "NAME20", "HOUSING20", 'MTFCC20', 'UR20',
            'UACE20', 'FUNCSTAT20', 'ALAND20', 'AWATER20', 'INTPTLAT20',
            'INTPTLON20', 'HOUSING20'], axis=1, inplace=True)

Export Cleaned Precinct Shapefile

# pctSF.to_file("../shapefiles/derived/pctSF_"+ stateAbv)
writer.writeShapefile(pctSF, "../shapefiles/derived/pctSF2_"+ stateAbv)

pctSF

	COUNTY	NAME	geometry	POP20	G24PREDEM	G24PREREP
row_id
0	APACHE COUNTY	001-:-02 ALPINE	POLYGON ((1479666.289 1002139.159, 1479666.637...	664	94	315
1	APACHE COUNTY	001-:-03 CANYON DE CHELLY	POLYGON ((1386793.007 1892441.636, 1386778.319...	4666	1467	320
2	APACHE COUNTY	001-:-05 CHINLE	POLYGON ((1359774.789 1846439.083, 1359573.952...	2949	821	224
3	APACHE COUNTY	001-:-09 CONCHO	POLYGON ((1324389.227 1223348.348, 1324343.898...	2945	348	1701
4	APACHE COUNTY	001-:-10 CORNFIELDS	POLYGON ((1325630.076 1659215.97, 1325671.599 ...	693	338	86
...	...	...	...	...	...	...
1680	YUMA COUNTY	027-:-5	POLYGON ((-122326.893 637655.049, -122729.574 ...	39	5	2
1681	YUMA COUNTY	027-:-6	POLYGON ((-124948.102 623062.023, -125333.564 ...	3063	295	540
1682	YUMA COUNTY	027-:-7	POLYGON ((-138217.859 628950.592, -138220.272 ...	5357	712	810
1683	YUMA COUNTY	027-:-8	POLYGON ((-146305.612 629566.726, -146286.187 ...	5124	675	1048
1684	YUMA COUNTY	027-:-9	POLYGON ((-150089.483 628872.966, -150256.291 ...	4453	618	692

1685 rows × 6 columns

precinctShapefile

	UNIQUE_ID	COUNTYFP	COUNTY_NAM	PRECINCTNA	CONG_DIST	SLDL_DIST	SLDU_DIST	G24JUSBOLN	G24JUSBOLY	G24JUSKINN	...	GSL29DTZI	GSL29OWRI	GSL29RMON	GSL29RTAY	GSL30DTIM	GSL30RBIA	GSL30RGIL	geometry	COUNTY	NAME
0	001-:-02 ALPINE	001	Apache	02 ALPINE	02	06	06	91	232	97	...	0	0	0	0	0	0	0	POLYGON ((1436817.379 973582.289, 1436816.931 ...	APACHE COUNTY	001-:-02 ALPINE
1	001-:-03 CANYON DE CHELLY	001	Apache	03 CANYON DE CHELLY	02	06	06	612	1052	559	...	0	0	0	0	0	0	0	POLYGON ((1365687.123 1941820.726, 1366088.848...	APACHE COUNTY	001-:-03 CANYON DE CHELLY
2	001-:-05 CHINLE	001	Apache	05 CHINLE	02	06	06	414	555	379	...	0	0	0	0	0	0	0	POLYGON ((1398392.825 1860525.917, 1398116.904...	APACHE COUNTY	001-:-05 CHINLE
3	001-:-09 CONCHO	001	Apache	09 CONCHO	02	06	06	556	1029	532	...	0	0	0	0	0	0	0	POLYGON ((1324605.175 1207457.412, 1324490.015...	APACHE COUNTY	001-:-09 CONCHO
4	001-:-10 CORNFIELDS	001	Apache	10 CORNFIELDS	02	06	06	139	245	122	...	0	0	0	0	0	0	0	POLYGON ((1321741.192 1657286.028, 1321175.096...	APACHE COUNTY	001-:-10 CORNFIELDS
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
1728	027-:-43	027	Yuma	43	09	25	25	73	323	70	...	0	0	0	0	0	0	0	POLYGON ((140080.761 869745.104, 140080.846 86...	YUMA COUNTY	027-:-43
1729	027-:-44	027	Yuma	44	09	25	25	384	1319	367	...	0	0	0	0	0	0	0	POLYGON ((267003.92 867811.918, 267004.974 867...	YUMA COUNTY	027-:-44
1730	027-:-45	027	Yuma	45	07	23	23	0	0	0	...	0	0	0	0	0	0	0	POLYGON ((-54407.318 600601.913, -54407.318 60...	YUMA COUNTY	027-:-45
1731	027-:-46	027	Yuma	46	07	23	23	0	0	0	...	0	0	0	0	0	0	0	POLYGON ((72942.721 611488.391, 74596.873 6114...	YUMA COUNTY	027-:-46
1732	027-:-47	027	Yuma	47	07	23	23	0	0	0	...	0	0	0	0	0	0	0	POLYGON ((263575.929 582687.712, 263575.257 58...	YUMA COUNTY	027-:-47

1733 rows × 285 columns

Export precinct graphs

import os

outDir = os.path.join("..", "graph")
os.makedirs(outDir, exist_ok=True)

outPath = os.path.join(outDir, stateAbv + "_pct20.json")

pctSF["area"] = pctSF.geometry.area
writer.saveGerrychainJSON(pctSF, outPath, stateAbv)

Inspect Shapefiles

pctSF.plot()

precinctShapefile.plot()

Summary

At the completion of this workflow, every precinct should satisfy the following properties:

All Census blocks are assigned to a precinct.
County assignments remain consistent.
Precinct geometries are contiguous.
MultiPolygon precincts have been eliminated.
The adjacency graph contains no articulation points.
Presidential vote totals have been attached.
Both precinct shapefile and precinct graph have been exported.

These outputs are suitable for use in generating ensembles using cyclewalk.