Posts Recap

An overview of each blog post in this folder, with a short description and a few ideas for possible future developments.

analyzing-tatoeba-dataset.md

This post analyzes the Tatoeba dataset, a crowdsourced collection of sentences translated across many languages, which as of February 2025 held over 12.5 million sentences in 421 languages. Using DuckDB, Parquet files, and a Jinja/JavaScript pipeline that auto-generates the article with interactive ECharts visualizations, the author explores data growth (about 1 million sentences per year), the 85% ratio of translations to original sentences, the tricky mechanics of sentence “linking” and clusters, contributor activity trends, the Gini coefficient of contribution inequality (0.972), and sentence-length distributions across scripts. The tooling and code are published on GitHub.

Possible future developments:

  • Fix the cluster-detection logic so that transitively equivalent clusters (like sentences 59 and 1157) are correctly merged even when a direct link is missing.
  • Add analysis of translation quality or accuracy, for example by cross-referencing flagged/corrected sentences or audio contributions.
  • Investigate the declining active-contributor trend more rigorously (e.g. cohort retention analysis) rather than relying on guesses.
  • Build a live, continuously updated dashboard that refreshes the charts as new dataset dumps are released.

calculate-sunbathing-time.md

This post (still marked as a draft) walks through a playful geometry and trigonometry problem: determining if and when sunlight reaches the author’s balcony. It frames the task as raycasting/shadow-projection, sets up a left-handed XYZ coordinate system aligned with compass directions, computes the sun’s position from azimuth and elevation angles using borrowed Python code, and projects obstacle shadows onto the ground plane. The implementation relies heavily on Shapely (backed by GEOS) for 2D geometry operations and SVG visualization, with the author stressing that constant visualization is key to catching subtle geometric mistakes, and finishing by rotating from a “Balcony Reference System” to the compass-aligned XYZ system.

Possible future developments:

  • Finish and publish the draft, including the promised BRS-to-XYZ rotation code and a complete worked example for the author’s actual balcony.
  • Add the unit tests the author acknowledges are missing, once the shadow logic is consolidated.
  • Support partial/fuzzy shadows and multiple candidate sunbathing points by reusing the computed shadow region, as hinted at in the alternative framing.
  • Integrate live sun-position data or a small web UI/animation so users can enter their own coordinates and obstacles interactively.

calculating-reachability-metro-milan.md

Inspired by a Reddit visualization of Milan Metro station catchment areas based on straight-line (geodesic) distance, this post builds a more realistic map using actual travel times. It combines OpenStreetMap road-network data (PBF, trimmed with osmium) and ATM/AMAT GTFS transit schedules, routes them with r5py (a Python wrapper over a Java engine) to compute a travel-time matrix from a 200k-point origin grid to ~120 stations (a ~6-hour computation). The author then derives station catchment regions two ways—first by finding grid edges with DuckDB, then more cleanly via SciPy Voronoi tessellation aggregated into Shapely polygons—and renders a travel-time heatmap with Pillow, publishing everything on a MapLibre-GL 3D-globe frontend. Code is on GitHub.

Possible future developments:

  • Parallelize the r5py travel-time computation, since the author noted the CPU was heavily underused during the 6-hour run.
  • Deduplicate stations shared by intersecting lines earlier in the pipeline instead of hardcoding the three Milan cases.
  • Overlay population density data to answer the author’s original question of how well-served areas are relative to who lives there.
  • Add time-of-day/day-of-week comparisons (weekend vs. weekday, night service) and possibly GTFS-rt real-time data for more realistic reachability.

chip8-python-emulator.md

This post recounts building a CHIP-8 emulator in Python (as a stepping stone toward a Game Boy emulator) using only the standard library and passing mypy in strict mode. Written in just 2-3 hours, it covers Python’s surprisingly capable bit-manipulation tools (bytearray, array, bitwise ops, match/case), endianness quirks discovered via hexdump, rendering approaches (terminal with Unicode block elements, then Tkinter Canvas with multiprocessing to work around Tk’s blocking, non-thread-safe main loop), keyboard scancode handling, using typing Protocols for structural subtyping of the display/control interfaces, the headache of differing CHIP-8 “dialects,” and the importance of test ROMs and debugging. The working emulator (minus sound) is on GitHub.

Possible future developments:

  • Implement the missing sound support to complete the emulator.
  • Add the suggested Debug protocol alongside Display and Control to inspect registers, view the current instruction, and step/pause execution.
  • Write a comprehensive unit-test suite, which the author repeatedly notes would be very valuable for this kind of code.
  • Compile the strict-typed code with mypyc for performance, and proceed to the originally intended Game Boy emulator.

correlate-sleep-and-flashcard-performance.md

This post investigates whether sleep duration correlates with next-day Anki flashcard performance. Unable to reliably export sleep data from a Mi Band 2 / Mi Fit / Google Fit chain, the author reconstructs sleep from Google MyActivity data (using phone-unlock and alarm events, exploiting the fact that the phone goes into airplane mode overnight), parsing the huge HTML export with Python’s streaming html.parser. Sleep hours are inferred heuristically, Anki review “ease” is pulled from its SQLite database (ignoring cards newer than 10 days), and a Pearson correlation is computed. The result is a coefficient of -0.04, indicating no correlation in the author’s personal data. Code is on GitHub.

Possible future developments:

  • Use a more reliable, higher-resolution sleep data source (e.g. a wearable with a proper data export or API) instead of inferring sleep from phone activity.
  • Explore cumulative/rolling sleep windows rather than only the single previous night, as the author suggests.
  • Expand the dataset across more people or a longer timeframe to reduce noise and improve statistical power.
  • Test additional variables (caffeine, exercise, time of day, card difficulty categories) and non-linear models rather than a single Pearson correlation.

delightful-user-experience-elixir-phoenix.md

This post is an unfinished draft (dated 2970 and containing bullet-point stubs) about solving Advent of Code 2019 as a full-stack web application, using Elixir/Phoenix for the backend and React+TypeScript for the frontend. The written portion covers the initial Phoenix + React setup and a Day 6 performance problem: computing a transitive closure over 1604 direct connections (expanding to 254,447 indirect ones) via a recursive function using maps of MapSets, which surprisingly took 45 seconds, prompting the need for a profiler. Remaining topics (profiler, dockerizing, React+TypeScript, hot-reloading vs. Vue, Material UI) are only listed as placeholders.

Possible future developments:

  • Complete the draft by writing out the stubbed sections (profiler, Dockerizing, React+TypeScript, hot-reloading comparison with Vue, Material UI) and fix the placeholder date and broken Markdown link.
  • Diagnose and optimize the 45-second Day 6 transitive-closure computation (e.g. better graph algorithm or data structures) and document the profiler findings.
  • Add the graphical/visual representations of problems the author found so valuable in the previous year’s Elixir-only attempt.
  • Reflect more concretely on the “delightful UX” thesis promised by the title, with specific Elixir/Phoenix ergonomics examples.

eliza-in-italiano.md

Written in Italian, this post reflects on ELIZA, the 1966 psychotherapist-imitating chatbot often considered the first of its kind, and the author’s long fascination with chatbots since childhood. It observes that despite decades of more powerful hardware and semantic databases, ELIZA still holds up well against systems like Cleverbot, largely because giving generic, deflecting “therapist” responses is far easier than genuinely intelligent replies. The author recounts earlier chatbot experiments (a GWBASIC school-representative simulator and an IRC bot using the Fleximatcher NLP parser) and the false-positive problems of rule-based NLP, then describes translating masswerk’s JSON-based JavaScript ELIZA port into Italian—requiring adaptations for gender agreement and the ambiguity of “perché” (why/because)—with the translated config published on GitHub, alongside sample dialogues.

Possible future developments:

  • Debug the Italian rule set, since the author noticed some questions and responses were being ignored entirely, likely due to errors in the translated patterns.
  • Add proper handling of Italian grammatical gender and formality (e.g. consistent use of “lei”), and disambiguate “perché” by detecting punctuation.
  • Expand the Fleximatcher-based Italian NLP grammar to recognize more structures and reduce false positives.
  • Compare or combine the classic rule-based ELIZA approach with a modern LLM to explore hybrid Italian conversational agents.

fleximatcher-parse-natural-language.md

This 2017 post introduces Fleximatcher, a Java 8 library the author built to help parse and extract information from natural language text. Inspired by Python’s pattern library, it lets users define matching expressions (PoS tags, hypernyms, regexes, custom annotators) and combine them into recursive, grammar-like tags defined via TSV rules that emit JSON annotations. It works like a simpler, embeddable alternative to annotation engines such as UIMA and GATE, using a top-down parser with caching and cycle-detection tricks, and ships with an Italian PoS tagger and a REST/web interface for testing patterns. The author notes it is now dated and suggests spaCy’s Matcher as a modern alternative.

Possible future developments:

  • Adapt the library to Apache Stanbol enhancement engines or the UIMA Ruta project, as the author himself suggests.
  • Add broader language coverage and richer Italian linguistic resources (hyperonymy, larger PoS tag sets) to reduce reliance on custom rules.
  • Provide a modern Python binding or bridge to spaCy so the flexible recursive-grammar approach can be used within contemporary NLP pipelines.
  • Improve usability with a rule-authoring GUI, validation/debugging tooling, and better performance benchmarking against the top-down parser.

generate-grammar-quiz.md

This 2021 post explains how the author populated the database behind grammarquiz.online, a cloze-deletion grammar quiz app built for Tatoeba’s Kodoeba initiative. Using the openly licensed Tatoeba sentence dataset (about 300 of 400+ languages, filtered by sentence count), the tool identifies stop words via token frequency and generates fill-in-the-gap cards focused on grammatical/syntactic words. Tokenization across scripts is handled with the ICU library (via PyICU) for non-destructive, language-agnostic splitting, and base forms for inflected words come from en.wiktionary data extracted with Wiktextract. Card generation is tuned by trial and error and made deterministic per sentence (seeded by sentence ID) so weekly dataset updates don’t invalidate the spaced-repetition schedule.

Possible future developments:

  • Replace purely frequency-based stopword detection with proper PoS tagging or lightweight ML models to select more pedagogically meaningful gaps per language.
  • Add difficulty grading and adaptive selection so cloze cards adjust to a learner’s demonstrated skill level.
  • Extend disambiguation of Wiktionary base forms (currently limited to unambiguous tokens) using context, expanding declension cards to more words and languages.
  • Introduce distractor-based multiple-choice variants and audio/pronunciation support to broaden the exercise types beyond text clozes.

gensim-generator-is-not-iterator.md

This short 2018 tip post addresses a TypeError raised by Gensim’s word2vec when passed a Python generator (“Try an iterator”) as the corpus. The author explains the cause: Gensim may need to iterate over the dataset more than once, but a generator is exhausted after a single pass. The solution is a small SentencesIterator wrapper class that stores the generator function and re-creates the generator each time __iter__ is called, giving an iterator interface backed by a generator. The author used it to stream a bigger-than-memory chat corpus from Postgres (via psycopg2 named cursors) to build FastText embeddings and LSI indexes.

Possible future developments:

  • Package the SentencesIterator wrapper as a small reusable utility/library with tests rather than an inline snippet.
  • Add error handling and logging (e.g., for DB connection resets on each re-iteration) to make the streaming approach robust for long training runs.
  • Benchmark and document performance/memory trade-offs of re-querying Postgres on every epoch versus caching to disk.
  • Update the example for current Gensim APIs and compare with modern embedding tooling (e.g., Hugging Face datasets streaming).

infrared-camera-raspberry-pi.md

This 2017 project post describes building a simple remote infrared webcam from a Raspberry Pi B+ and a Pi NoIR sensor, using raspistill to capture near-infrared images. The author explains why IR photos have unusual colors (the camera remaps the invisible spectrum) and why plants appear pink/yellow due to chlorophyll’s strong near-infrared reflectance, the same principle NASA and ESA use for vegetation-health monitoring. Using the Pi NoIR’s blue filter plus the Infragram tool enhances vegetation visibility. To view images conveniently, the author and a friend built a small Flask web application that triggers raspistill via an API, displays the resulting photo in the browser, refreshes every 60 seconds, and tolerates multiple simultaneous users.

Possible future developments:

  • Add live video streaming (e.g., MJPEG or WebRTC) instead of periodic still-image polling for a smoother remote viewing experience.
  • Compute and display vegetation indices such as NDVI directly in the web app, automating the Infragram-style analysis.
  • Add authentication, HTTPS, and capture scheduling/time-lapse features to make it a usable long-term monitoring station.
  • Support newer Raspberry Pi camera stacks (libcamera/Picamera2) and hardware, replacing the deprecated raspistill.

ingest-data-into-postgres-fast.md

This 2021 post, aimed at data engineers, surveys fast techniques for ingesting data into Postgres from Python, using a real case of loading 5 million Reddit comments from JSONL files with incremental upsert logic keyed on a retrieved_at timestamp. It covers INSERT ... ON CONFLICT (upsert), reproducible dockerized benchmarking via a Makefile, and compares libraries and methods: asyncpg prepared statements, psycopg2 execute_values and execute_batch, and psycopg3’s programmatic COPY (including binary mode) into a temporary table merged afterward, plus the speed benefit of UNLOGGED tables. The author deliberately avoids a benchmark chart because measured speeds (~3300-3780 rows/sec) are close and noisy, offering instead a star-rated summary table favoring asyncpg + prepared statements and psycopg2 + execute_values.

Possible future developments:

  • Explore Postgres 15’s MERGE statement (later covered in the follow-up post) as a cleaner alternative to ON CONFLICT.
  • Add parallel/partitioned ingestion benchmarks, since the post notes the temp-table COPY approach pairs well with partitioning for parallelism.
  • Revisit binary COPY once psycopg3 automates type serialization (e.g., timezone-aware timestamps) to reduce the current manual complexity.
  • Provide reproducible benchmark tooling and datasets so readers can measure results in their own environments given the high variance.

ingest-more-data-into-postgres.md

This 2023 follow-up post (marked as a draft) updates the earlier Postgres ingestion article to reflect the state of the art as of early 2023, ingesting roughly 16M Italian- and German-language Reddit comments and submissions. It highlights three developments: psycopg3 reaching a stable release (removing prior workarounds), Postgres 15’s new MERGE functionality as a more legible upsert alternative, and SQLAlchemy 2.0’s psycopg3 support (important for tools like Pandas and Apache Superset). It shows psycopg3’s automatic use of the binary protocol and automatic prepared statements (with an explicit-parameter preference), and details a careful dockerized Makefile setup that pre-creates the data directory and mounts /etc/passwd so file permissions match the host user. The post appears incomplete, ending after the DB-creation method without final benchmarks or conclusions.

Possible future developments:

  • Finish the draft: add the actual ingestion benchmarks, a MERGE vs ON CONFLICT comparison, and a concluding recommendation.
  • Demonstrate SQLAlchemy 2.0 + psycopg3 end to end, including integration with Pandas/Superset now that support has landed.
  • Scale-test the approach on the full ~16M-row dataset with partitioning and parallel workers, reporting throughput.
  • Add coverage of newer Postgres versions and features (e.g., improvements to MERGE, logical replication, or COPY enhancements) as they land.

location-data-heatmap.md

This 2019 post walks through an open-source Python script that turns a Google Location History Takeout JSON export into a static and animated heatmap of the places the author frequented in Berlin. Coordinates in Google’s E7 format are binned into a 2D numpy array where each cell accumulates minutes spent (derived from time differences between ordered data points), scaled by a configurable factor. Because dominant locations (home, office) overwhelm the raw map, the author normalizes values using percentile/quintile ranking and Gaussian blurring, then selects a matplotlib colormap (settling on Spectral). Animated versions filter points by minute-of-day and blend consecutive frames for a fading effect, with the final video produced via MoviePy after ffmpeg proved too heavy; the OpenStreetMap background is a manually matched screenshot.

Possible future developments:

  • Replace the manually captured OpenStreetMap background with an automated basemap (e.g., contextily or a local tile server) for perfectly aligned, reproducible maps.
  • Support additional/newer location data sources and formats (Google’s current Timeline JSON schema, GPX, or on-device loggers) as the E7 Takeout format evolves.
  • Add interactive web visualization (Leaflet/deck.gl) with zoom, time scrubbing, and filtering, complementing the static/video outputs.
  • Enrich analysis using the available vehicle/movement-type and accuracy metadata to distinguish commuting, cycling, and walking patterns.

postgres-timezone-shenanigans.md

This post documents the author’s practical struggles handling timezones across Python and Postgres, drawn from work at a bus company where local time at bus stops matters. It explains that Postgres TIMESTAMP WITH TIME ZONE does not actually store a timezone (it converts to UTC on input and uses the same 8-byte representation as TIMESTAMP), warns against Python’s pytz library (whose eager offset calculation can produce bizarre results like a 50-minute offset for Europe/Rome) in favor of the standard-library zoneinfo, and investigates where Postgres and Python source their tz database, noting that the official Debian-based Postgres Docker image ships stale tzdata while the Alpine flavor and RDS 14.6 bundle newer versions.

Possible future developments:

  • Add a companion piece with concrete migration/upgrade guidance: how to detect and safely update stale tzdata in RDS, Debian, and Alpine deployments before new timezone names break production.
  • Provide a small reusable Python utility or test suite that asserts a deployment’s tz database version and flags missing timezones (e.g. Europe/Kyiv) at startup.
  • Extend the discussion to cover DST-boundary edge cases (ambiguous and non-existent local times) with PEP 495 fold examples in both Postgres and zoneinfo.
  • Benchmark or document behavior across newer Postgres versions (15/16/17) and other managed providers (Neon, DigitalOcean) to see whether the bundled-tzdata approach has become more consistent.

pydata-2022-presentation.md

This is a very short announcement post for a talk the author gave at PyCon Berlin / PyData 2022 (on April 12th, 2022) about using Postgres in a data science project at Flixbus. It contains little prose beyond the announcement and provides a link to download the slide deck (a PDF) plus contact channels (conference Discord, Twitter, GitHub).

Possible future developments:

  • Convert the slide deck into a full written article so the content is accessible and searchable without downloading a PDF.
  • Add a recording or embedded video of the talk if one exists, along with a transcript for accessibility.
  • Publish a follow-up reflecting on how the Postgres-in-production practices have evolved at the company since 2022.
  • Add speaker notes or a summary of key takeaways inline, since the current post gives no substantive content to a reader who does not open the slides.

render-building-from-openstreetmap.md

This detailed technical post walks through generating a 3D triangle mesh of a building from OpenStreetMap data, aimed at rendering it in Blender or game engines like Godot. The author imports OSM extracts into PostGIS via pgOSM Flex, extracts Shapely geometries with Psycopg3, and uses Trimesh/Open3D to build and export meshes (in ply and glTF 2 formats). The core narrative is debugging a broken building-with-holes mesh: the initial result was garbled because of incorrect normals/winding order, which was fixed by controlling vertex ordering (counterclockwise = front-facing in OpenGL), while a persistent missing side was later attributed (in a 2024 edit) to Delaunay triangulation failing on concave shapes, requiring ear clipping instead. The author ultimately leans toward a voxel representation as simpler and more robust for handling anomalous geometries and downstream tasks like 3D printing.

Possible future developments:

  • Implement the ear-clipping triangulation fix mentioned in the 2024 edit and publish a corrected, watertight mesh pipeline for concave buildings with holes.
  • Add the deferred features the author lists as future work: roof shapes, per-feature colors, level-of-detail handling, and simple animation (e.g. cars moving in streets).
  • Build an interactive in-browser viewer (Three.js/Babylon.js) that renders generated glTF buildings directly from PostGIS, closing the loop from data to visualization.
  • Scale the pipeline from single buildings to whole city blocks and add automated validation (watertightness, Euler-number checks) to catch broken geometries before export.

seam-carving-in-pure-java.md

This 2016 post describes the author’s implementation of seam carving (content-aware image resizing) as a small, dependency-free Java library. It explains computing per-pixel importance via RGB gradient magnitude, then finding low-importance vertical seams using a minimum-cumulative-importance (MCI) dynamic-programming matrix analogous to the forward-backward algorithm used in hidden Markov models. The author shows that naive per-row removal and even multi-path selection produce ugly artifacts (paths bunch together and “jump”), and that the visually best result comes from removing a single seam and recomputing the MCI matrix each time, at significant computational cost, illustrated on the Milan skyline, Bosch’s Garden of Earthly Delights, and a street scene.

Possible future developments:

  • Add seam insertion (image enlargement) and vertical/horizontal resizing in both dimensions, not just width reduction.
  • Optimize the slow recompute-after-each-seam approach, e.g. incremental/local MCI updates, multithreading, or GPU acceleration, to make it usable on large images.
  • Experiment with better energy functions (e.g. Sobel/Scharr operators, entropy, or forward-energy seam carving) to reduce artifacts on line-heavy images.
  • Add object-protection and object-removal masks so users can preserve or delete specific regions during resizing.

skorch-punctuation-pytorch.md

This 2019 post is an experiment using Skorch (a library wrapping PyTorch models in a scikit-learn-compatible interface) to reconstruct punctuation in Italian text. The author frames punctuation restoration as predicting the punctuation following each token, using a fixed-size window of surrounding GloVe word embeddings (plus casing features) flattened into a 12000-dimensional feature vector so that a neural network can be compared directly against k-NN, decision tree, and random forest baselines (which reached ~85.75%, 77.9%, and 83.8% accuracy). The neural network trained on 400k utterances plateaued around 86.9% accuracy but essentially only learned to insert periods and apostrophes, rarely commas or quotes; the author attributes this to the limited window size and reflects on training mistakes (misusing partial_fit with many epochs, not checkpointing the model during a ~40-hour training run).

Possible future developments:

  • Replace the fixed-window feed-forward model with an LSTM (or Transformer) to capture longer sentence structure, which the author suspects is needed to learn commas.
  • Add character-level token modeling to handle proper nouns, emojis, and neologisms that fall back to a placeholder GloVe vector.
  • Enrich features with part-of-speech tags (using a PoS tagger that does not rely on punctuation as input) to give the model more syntactic signal.
  • Run systematic hyperparameter optimization via scikit-learn GridSearchCV/Skorch integration, and add proper checkpointing and a validation set for reproducible comparisons.

static-maps-in-2024.md

This is a short, still-draft (draft: true) post that revisits the author’s earlier static-map series and outlines an updated, simpler, and more robust pipeline. It recaps the previous approach (download OSM extract, Tilemaker for vector tiles with awkward SDF font generation, PyOsmium for a static search index, and a vector-tile frontend) and lists the pain points and intended improvements: moving to the emerging OSM Shortbread vector tile format (with Geofabrik prebuilt extracts), using MapLibre/Mapbox local-font options instead of SDF fonts, loading data via osm2pgsql (now with JSONB and update support), doing 3D mesh work interactively in Godot or Babylon.js rather than Python, and adopting Protomaps for simpler static data delivery via HTTP range requests.

Possible future developments:

  • Finish and publish the draft, delivering the promised updated end-to-end pipeline with working code and a live demo map.
  • Complete the migration to the Shortbread vector tile format using Geofabrik prebuilt extracts and document the style adaptation required.
  • Integrate Protomaps (single-file PMTiles + HTTP range requests) and benchmark its performance and simplicity against the previous tile-directory approach.
  • Demonstrate the Babylon.js + MapLibre 3D integration for interactive buildings, connecting it back to the earlier 3D-from-OSM experiment.

static-maps-part-1-qgis-raster.md

This is part 1 of a series on building a fully static interactive map, meaning a map served as plain files by nginx or a CDN with no backend or external service dependency. It walks step by step through downloading an OSM PBF extract (using Osmium to cut a bounding box around Milan), converting it to a SpatiaLite database with ogr2ogr, importing and styling it in QGIS (including using a ready-made American Red Cross style), generating raster XYZ PNG tiles via the QGIS toolbox, and wiring up a client-side Leaflet renderer with proper OpenStreetMap attribution. The author gives concrete size/time figures (25 MB PBF to 3.6 GB of tiles across zoom levels in ~25 minutes) and notes the trade-off that static maps must be prepared in advance and are not auto-updated, while flagging vector tiles and WebGL as topics for later articles.

Possible future developments:

  • Write the promised follow-up parts on vector tiles and WebGL-accelerated rendering, comparing them against the raster approach shown here.
  • Automate the whole raster pipeline (PBF to SpatiaLite to styled tiles) into a reproducible script or Docker container to remove the manual QGIS steps.
  • Address the large tile storage footprint with strategies like tile compression, on-the-fly generation, or the PMTiles/Protomaps single-file format.
  • Add a worked example of the historical-map capability (rendering OSM state at a past date via Osmium time-filtered extracts), which the post mentions but does not demonstrate.

static-maps-part-2-vector-tiles.md

This is the second post in a series on building fully static maps from OpenStreetMap data. It explains why vector tiles are superior to the raster tiles covered in part 1: they take far less disk space (Milan drops from ~3-4 GB to 55 MB), generate faster, allow dynamic styling without reprocessing, support feature interaction and 3D, and never pixelate when zoomed. The author walks through the Mapbox Vector Tile format, the distinction between a folder of .pbf tiles and a single SQLite-based MBTiles file, generating tiles with Tilemaker via Docker, and rendering them client-side with MapLibre GL JS using a JSON style definition that controls sources, layers, filters, zoom-dependent colors, text labels and SDF fonts.

Possible future developments:

  • Explore the 3D/perspective and extrusion capabilities the post mentions but does not implement, e.g. building height extrusion for the Milan map.
  • Automate the config.json manipulation and Docker build steps into the author’s static-osm-indexer tool so layer selection, compression and metadata are handled without manual editing.
  • Add tooling to validate the generated style JSON against the actual layers/tags present in metadata.json, catching mismatches before they silently fail to render.
  • Provide a guided workflow around Maputnik so non-technical users can theme their own static maps and export ready-to-use JSON.

static-maps-part-3-text-search.md

The third post in the static-maps series adds location text search (a lightweight geocoding substitute) to an otherwise backend-free static map. Using Pyosmium the author extracts named features (streets, shops, monuments, with per-language name tags) from an OSM PBF into a JSONL file, then builds a static full-text index. After finding Lunr.js and Pagefind unsuitable (Pagefind needed 30+ GB RAM for a small city), he implements a custom prefix-based index that splits entries into per-prefix JSON files (3 characters for Latin scripts, 1 for CJK ideograms), which the TypeScript frontend fetches on demand as the user types. The post also covers regex tokenization limits (Firefox lacking Intl.Segmenter) and an unfinished navigation/routing feature whose road-network graph proved too large to ship to the browser.

Possible future developments:

  • Solve the routing data-size problem by chunking the road-network graph into per-city/per-tile files with frontend logic to load only the relevant chunk, enabling in-browser shortest-path search via Graphology.
  • Improve tokenization for space-less languages (Thai, Chinese) by adopting Intl.Segmenter with a polyfill fallback for Firefox, rather than relying solely on the Unicode regex split.
  • Adopt flatbuffers or another compact binary format for the index files (the post notes san.json reaches 18 MB) to shrink download size and speed up filtering.
  • Add fuzzy/typo-tolerant matching and result ranking to move the exact-prefix search closer to real geocoding behavior for ambiguous queries.

visualize-the-functioning-of-supervised-learning-models.md

The opening article of a series that visually compares supervised learning models on a toy problem inspired by a ConvnetJS demo: predicting a pixel’s RGB color from its (x, y) coordinates, then regenerating the whole image (Grant Wood’s “American Gothic”) to see how each model generalizes or overfits. It starts with linear regression, which can only produce gradients (validated against a synthetic gradient image), then introduces non-linearity via a hand-written kernel/enricher function that appends Euclidean distances to a grid of reference points, letting a linear model reproduce distinguishable image regions. Quantization artifacts from casting to uint8 are noted throughout. All code lives in a shared Jupyter notebook.

Possible future developments:

  • Turn the static image comparisons into an interactive web demo (WASM or JS) where readers adjust sample count and model per channel and watch the reconstruction update live.
  • Systematically study how training-sample size and reference-grid density affect reconstruction quality, plotting a quantitative error metric rather than relying on visual inspection.
  • Properly handle the uint8 overflow/negative-value artifacts by clamping or scaling instead of casting, and document the color-space impact.
  • Extend the coordinate-to-color framing to animation or video frames to visualize how models cope with temporal sequences.

visualize-the-functioning-of-supervised-learning-models-part-2.md

Part 2 replaces the hand-written kernel from part 1 with scikit-learn’s built-in Support Vector Regression (using the native libsvm RBF kernel). It explains the roles of the gamma and C hyperparameters as controls over overfitting versus underfitting, notes that SVR predicts a single channel at a time (so three models are trained, later observing that MultiOutputRegressor would do this automatically), and shows an unexpectedly dark, flat reconstruction that had to be manually normalized. It then demonstrates GridSearchCV for automated hyperparameter search across kernels, C and gamma (1200 candidates per channel in ~10 minutes on 8 cores, optionally parallelized across a cluster with Dask), including a caution about the base-10 exponent semantics of np.logspace.

Possible future developments:

  • Investigate and document the cause of the dark/flat SVR output (likely feature scaling or epsilon) instead of correcting it post-hoc with min-max normalization.
  • Replace the three per-channel models with MultiOutputRegressor and benchmark accuracy and training time against the manual approach.
  • Add input feature standardization and compare the effect on optimal gamma/C ranges and on convergence speed.
  • Visualize the GridSearchCV results as a heatmap over the C/gamma space to make the overfitting/underfitting trade-off intuitive to readers.

visualize-the-functioning-of-supervised-learning-models-part-3-k-neighbours-and-decision-trees.md

Part 3 applies two more scikit-learn models to the coordinate-to-color problem. K-nearest-neighbors with K=1 produces a Voronoi-style mosaic of the sampled points, and increasing K averages neighbors to smooth the regions. The post uses this to explain the spatial indexing behind fast neighbor lookup: KD-trees work well in low dimensions but succumb to the curse of dimensionality, whereas ball trees index with nested hyperspheres to handle high-dimensional data at higher build cost. It then shows decision trees trained on 1000 and 3000 samples, praising them as fast, simple and interpretable (the full model can be printed), and admiring the blocky aesthetic of their reconstructions.

Possible future developments:

  • Visualize the KD-tree/ball-tree partitioning itself overlaid on the image to make the indexing explanation concrete.
  • Compare prediction and index-build timings of KD-tree versus ball tree as the number of samples and feature dimensions grow, backing the curse-of-dimensionality claim with data.
  • Render the actual decision-tree structure alongside the reconstructed image to connect split thresholds to the visible rectangular regions.
  • Add ensemble methods (random forests, gradient boosting) to show how averaging many trees smooths the blocky artifacts.

visualize-the-functioning-of-supervised-learning-models-part-4-neural-networks.md

The final part of the series applies a feed-forward neural network (built with Keras) to the pixel-color regression problem. It gives an accessible explanation of neurons, weights, activation functions and backpropagation, and offers an opinionated aside warning against deep-learning hype and a possible AI winter, advising practitioners to start with simpler, more interpretable models. The concrete network has three dense layers of 1000 selu neurons with light (1%) dropout for regularization, an Adam optimizer, MSE loss, and is trained for 400 epochs on 5000 samples, closing the series with the network’s reconstruction of “American Gothic.”

Possible future developments:

  • Systematically compare activation functions, layer widths/depths and dropout rates to replace the trial-and-error choices with a documented ablation.
  • Add convolutional or Fourier/positional-encoding input features (as in later SIREN/NeRF-style coordinate networks) to sharpen the notoriously blurry coordinate-to-color output.
  • Produce a single side-by-side summary across the whole series (linear, SVR, KNN, tree, NN) with a shared quantitative error metric to conclude the comparison.
  • Use the fit_generator approach the author mentions to stream training data, enabling experiments on much larger or higher-resolution images.

writing-a-tree-sitter-grammar.md

A hands-on account of learning parsing by writing a Tree-sitter grammar, motivated by the author’s Milan dashboard project and its many parsing/transpilation needs (Markdown, TypeScript, DuckDB SQL, Parquet, Vega-Lite). It introduces generative grammars, ASTs, lexers and the fragmented landscape of parser generators (ANTLR, Parsimonious, Peggy, Lark, Tree-sitter), noting there is no single mainstream grammar format. The author chooses Tree-sitter for its incremental parsing, documentation and community, and writes a grammar for Monicelli, an Italian esoteric language, in a few hours. The post praises the developer experience: easy npm install, grammar.js scripting, a built-in test command, AST queries, one-command builds into Python/JS/Swift/Go packages, and out-of-the-box syntax highlighting.

Possible future developments:

  • Follow through on the stated goal of writing an interpreter and a WebAssembly compiler for Monicelli on top of the grammar.
  • Integrate the generated parser into a browser/PWA (via WASM) to offer live, offline Monicelli syntax highlighting and validation in the Milan dashboard.
  • Exploit Tree-sitter’s query feature (barely used so far) to build linting, refactoring and “which values can be displayed” automations the author wants for his dataviz pipeline.
  • Publish the grammar to editor ecosystems (neovim, Emacs, Helix) with highlight/indent/fold queries so others exploring the language get full editor support.