AtlasUtilities.jl

AtlasUtilities.jl is a Julia package that provides a command-line tool, atlas, for inspecting, transforming, and extracting data from Atlas files. It is built on top of AtlasIO.jl and is aimed at the everyday tasks that come up when working with large collections of maps produced by the sampling code, such as the Cycle Walk and Metropolized Forest ReCom algorithms.

The source code, installation instructions, and the full command reference are maintained in the AtlasUtilities.jl Git repository. This page gives a high-level tour of what the tool can do; for the complete list of subcommands, flags, and options, see the repository.

Installation

atlas is built with Comonicon. Installing the package runs a build step that drops an atlas launcher into ~/.julia/bin/ and compiles a system image so the command starts fast.

One dependency, StreamHistogram, isn’t registered in the Julia General registry — it’s pinned by URL via Project.toml’s [sources] table. That pin is only honored when AtlasUtilities is the active project being resolved, so Pkg.add(url=...) into some other environment won’t work; install from a local clone instead:

git clone https://github.com/jonmjonm/AtlasUtilities.jl
cd AtlasUtilities.jl
julia --project -e 'using Pkg; Pkg.instantiate()'
julia --project deps/build.jl        # installs the `atlas` launcher + sysimg

Then add ~/.julia/bin to your PATH (one time):

echo 'export PATH="$HOME/.julia/bin:$PATH"' >> ~/.zshrc && source ~/.zshrc

Now the command is available:

atlas info some_atlas.jsonl.gz

The first install compiles a system image and takes several minutes.

Inspecting Atlases

Before working with an Atlas file it is often useful to look at what it contains. The atlas info command prints the file’s metadata, run parameters, and field names:

atlas info examples/cycleWalk_ct_metadata.jsonl.gz

Some Atlases also embed a copy of the script that was used to generate them (and, when one was used, the TOML config that parameterized the run), so that a run can be traced back to the exact code and settings that produced it. When either is present, atlas info notes it, and both can be pulled out of the file with the --extract-script flag:

atlas info examples/cycleWalk_ct_metadata.jsonl.gz --extract-script

This writes the run script to the file named by the header’s script_name entry (falling back to extracted_script.jl) and, if a TOML config is embedded, writes it to the file named by toml_config_file (falling back to extracted_config.toml).

To see what data fields are attached to each map, use atlas list-map-data:

atlas list-map-data examples/cycleWalk_ct_metadata.jsonl.gz

And to list the node IDs used by a particular map in the Atlas, use atlas list-nodes:

atlas list-nodes examples/demo_grid_3x3.jsonl.gz --map 2

Relabeling Districts

Because district numbering is arbitrary, the same underlying districting plan can be labeled differently from one map to the next in a sequence, which makes it harder to track a district over time. The atlas relabel command canonicalizes district numbering across consecutive maps so that labels stay consistent:

atlas relabel input.jsonl.gz relabeled.jsonl.gz --first-map --quiet

It also supports hierarchical or multiscale Atlases, where relabeling is done with respect to a dual-graph specification:

atlas relabel examples/demo_multiscale.jsonl.gz out.jsonl.gz examples/demo_multiscale_graph.json

By default every node (or, for multiscale Atlases, every finest-resolution unit) counts equally toward the alignment. Passing --weight-population with a node-link JSON and --population-attr naming its population field instead weights the alignment by population, so districts stay consistent in terms of population moved rather than raw area/unit count:

atlas relabel examples/atlas_ordered.jsonl.gz out.jsonl.gz Data/NC_pct21.json \
  --weight-population Data/NC_pct21.json --population-attr pop2020cen

Adding Computed Data

The atlas add command evaluates one of the Cycle Walk “writer” functions, such as get_log_spanning_trees or get_isoperimetric_scores, on every map in an Atlas and stores the result alongside the map. This is useful for enriching an already-generated Atlas with statistics that weren’t computed during the original sampling run.

atlas add get_log_spanning_trees Atlas1.jsonl.gz Atlas2.jsonl.gz --config param.toml

A --config is read only for its [plans] table — the graph file and the column names a writer needs (see the configuration file). The measure the run sampled plays no part here: which writers to evaluate is given on the command line, so a configuration in either measure format serves equally, and the same information can be supplied with the explicit flags below instead.

Multiple writers can be requested in a single pass, and the full list of available writers can be printed with --list-writers:

atlas add "get_log_spanning_trees,get_isoperimetric_scores" Atlas1.jsonl.gz Atlas2.jsonl.gz
atlas add --list-writers

Writers that need geographic information accept the corresponding dual-graph and column names:

atlas add get_isoperimetric_scores Atlas1.jsonl.gz Atlas2.jsonl.gz \
  --graph NC_pct21.json --pop-col POP20 --node-col NAME \
  --area-col area --border-col border_length --edge-perimeter-col length

The partisan writers (get_partisan_margins, get_partisan_seats) additionally need --vote-cols, one or more votes1,votes2 column pairs separated by ; (quote the argument in your shell since it contains a ;). Each pair produces a field named writer_votes1_votes2:

atlas add get_partisan_margins Atlas1.jsonl.gz Atlas2.jsonl.gz \
  --graph NC_pct21.json --pop-col POP20 --node-col NAME \
  --vote-cols "G20_PR_D,G20_PR_R;G16_PR_D,G16_PR_R"

By default atlas add errors if a requested field already exists on a map; pass --overwrite to recompute it anyway. --quiet suppresses the progress bar.

Extracting to CSV

For downstream analysis outside of Julia, Atlas data can be exported to CSV. atlas extract-map-data writes each map-data field to its own CSV (one row per map) in a directory named after the atlas, along with an about.md describing the source atlas:

atlas extract-map-data cycleWalk_ct_slice.jsonl.gz

It can also compute and include writer functions on the fly, without needing a separate atlas add step first (accepting the same graph/column flags as atlas add, including --vote-cols for the partisan writers):

atlas extract-map-data run.jsonl.gz --add get_log_spanning_trees --config param.toml --force

By default output is gzip-compressed (.csv.gz); pass --no-compression for plain .csv. --force overwrites an existing output file (otherwise it’s skipped), and --max-maps <n> stops after extracting n maps — the output filenames get a -partial suffix so a partial run never collides with a full one in the same directory.

atlas extract-assignments instead produces a single wide CSV with the district assignment of every node in every map, with one row per node and one column per map:

atlas extract-assignments examples/demo_grid_3x3.jsonl.gz

It accepts the same --no-compression/--force/--quiet flags as extract-map-data. Note that extract-assignments currently does not support multiscale/hierarchical Atlases (unlike atlas relabel, which does); it expects a single-level dual graph.

Histograms and Summary Statistics

For large atlases, materializing every per-map value as a CSV row can be more data than you need. atlas extract-map-data-histogram instead accumulates each map-data field into a StreamHistogram — one histogram per scalar field, or one per index for a vector field — and writes a <field>-histogram.csv (the raw bin edges and counts) plus a <field>-histogram.json (summary statistics, moment errors, and ASH-density bin counts) for each field:

atlas extract-map-data-histogram run.jsonl.gz --burn-in 500 --bin-num 40

It accepts the same --add/graph/column flags as extract-map-data for computing writer functions on the fly, plus:

  • --burn-in <n>: skip the first n maps before accumulating.
  • --bin-range <lo,hi> / --bin-num <n> / --bins <e1,e2,...>: control the histogram bin edges, either a fixed range with a bin count, or explicit edges.
  • --moment-powers <p1,p2,...>: which moments to accumulate and report (default 1,2,4,8).
  • --integer true|false|auto: treat a field as integer-valued (one bin per integer, no ASH density); auto decides per-field/per-index from a learn-phase sample.
  • --no-sort: by default, a vector field is sorted ascending within each map before accumulating, so histogram j holds the j-th order statistic across maps rather than raw index j; --no-sort disables that.

atlas summarize-map-data prints the same statistics to the terminal instead of writing files — mean, standard deviation, and [min, 25% quartile, median, 75% quartile, max] for every map-data field, built over an internal StreamHistogram:

atlas summarize-map-data run.jsonl.gz --max-maps 2000

Both commands warn when a field’s histogram range or interpolation looks unreliable for the requested statistic (e.g. too much data falling outside the learned bin range).

Rendering Maps

A CycleWalk atlas stores only districtings (node id → district number), not geometry, so drawing a map needs a separate geometry source supplying each node’s shape. atlas render draws one map of an atlas as a filled-precinct choropleth, colored by district, and saves it to an image file:

atlas render examples/cycleWalk_ct_slice.jsonl.gz Data/shapefile_CT.zip --key NAME

The geometry source is either a shapefile — a .shp file, or (as above) a .zip bundling one with its .shx/.dbf companions, unzipped to a temp directory automatically — or a point-position node-link JSON graph, the same sort of dual-graph file the other subcommands accept. A point graph has no polygons to read, so each node’s cell is built as a Voronoi region from the graph’s own adjacency; because it uses the adjacency the graph actually encodes rather than re-triangulating the points, it stays faithful to that graph, and reproduces true hexagons and squares for the hex/grid example graphs as a side effect:

atlas render run.jsonl.gz Data/hex_graph_10_by_10.json --key node_name

Matching node ids to geometry

--key names the attribute whose values match the atlas’s node ids — a shapefile column, or a node attribute of a point graph (node_name for the hex/grid example graphs). For the CT/OH example data the node id is literally the shapefile’s NAME field. For atlases whose node id has more than one component (e.g. NC’s (county, prec_id)), pass a comma-separated list in the same order — they’re joined with :, the same way the atlas’s own node id is rendered as a string:

atlas render nc_run.jsonl.gz Data/northCarolinaPrecinct2021.zip --key county,prec_id

Use atlas list-nodes on the atlas first if you’re not sure of the exact node-id format a given atlas expects.

A shapefile and a dual graph can each merge several underlying precincts into a single record, and the two merges don’t always agree, so matching is done at the level of comma-split “atomic” precinct names rather than whole ids. Two flags cover the mismatches that come up in practice:

  • --strip-suffix <regex> strips a trailing regex match from each node id before matching. Graphs built for legislative-district consistency, for instance, can suffix node ids with __SD<n>__HD<n> tags that have no shapefile counterpart.
  • --county-column <col> names the county attribute. Some graphs keep precinct-level detail only for the counties a district line splits, collapsing an entire unsplit county into one node (e.g. NC’s ("GASTON",)). Such a node matches no precinct at all, and instead falls back to every precinct in that county, all drawn in its district color.

By default a node id with no matching geometry record is an error — usually a sign that --key names the wrong column. --skip-missing renders anyway, leaving the unmatched areas blank and printing a summary of what failed to match.

Other options

  • --map <k>: 1-based index of the map to draw (default 1).
  • --format png|svg|pdf: output format when --out is not given (default png).
  • --out <path>: output file path (default <atlas basename>-map<k>.<format>).
  • --county-lines: draw each county’s outline over the precinct fill (requires --county-column). For a shapefile the county’s rows are dissolved into its true polygon; for a point graph the exact shared border between differing-county neighbors is read straight off the adjacency.
  • --no-precinct-lines: omit the per-precinct stroke, which is drawn by default.
  • --show: open the saved image in the OS’s default viewer after writing it.
  • --quiet: suppress the confirmation message.

Performance

The relabel, add, extract-*, and summarize-map-data commands all process maps in parallel across the available Julia threads. The number of threads is controlled with the JULIA_NUM_THREADS environment variable:

JULIA_NUM_THREADS=8 atlas add get_log_spanning_trees Atlas1.jsonl.gz Atlas2.jsonl.gz

Using AtlasUtilities.jl as a Julia Library

Beyond the atlas command line tool, the district-relabeling and alignment logic behind atlas relabel — finding the label permutation that best matches one map to another, applying it, and computing Hamming distances between districtings (optionally population-weighted, and for multiscale Atlases) — is exported and can be called directly from other Julia code:

using AtlasIO
using AtlasUtilities

σ = findRelabeling(ref, cur, d)
aligned = relabelMap(cur, σ)

This is useful for aligning maps in memory, computing distances between districtings as part of an analysis, or driving a custom relabeling loop instead of atlas relabel’s atlas-to-atlas pipeline.

The plotting behind atlas render is exported as well. render_map builds the figure without saving or displaying it, so you can add to it, restyle it, or place it in a larger layout before deciding what to do with it; run_render is the thin driver that saves it to a file, and districtColors hands back the same district palette for your own plots:

using AtlasUtilities, CairoMakie

fig = render_map("run.jsonl.gz", "Data/shapefile_CT.zip"; key = "NAME", map = 12)
save("map12.png", fig)

render_map returns a plain Makie.Figure and is backend-agnostic — CairoMakie is the backend the package itself depends on, but any other Makie backend you install and activate works too.

See library.md in the repository for the full library API.

Full Reference

This page covers the main things AtlasUtilities.jl can do, but not every flag and option. For the complete and up-to-date command reference, configuration file format, and source code, see the AtlasUtilities.jl Git repository.