Urban Operations Intelligence Platform
How does a city know where to send its plows after a snowfall?
expanding into Data Engineering
Hi, I'm James — a backend engineer with 9 years of experience building distributed systems where scale, performance, and data reliability matter.
Now based in Winnipeg, I'm extending that experience into Data Engineering and Applied AI through post-degree study and hands-on platform projects.
$ ▌A couple of projects showing how I take systems from raw data to production.
How does a city know where to send its plows after a snowfall?
What turns a stream of notes into a knowledge base a team can live in?
How do you get 24 production stages onto one system?
How do you make 1,783 icons searchable without writing a single keyword?
Nine years of backend engineering across e-commerce, SaaS, and manufacturing — now moving into data and AI.
Artificial Intelligence Post-Degree Diploma · Business Analysis & Transformation Post-Degree Program
End-to-end delivery: business problem framing, data pipelines, and ML engineering.
Bachelor of Engineering, Mechanical Design, Manufacturing and Automation
Full-Stack Developer (Contract) — Manufacturing Execution System
Steel pipe manufacturer. Sole developer on a greenfield MES, end to end.
Senior Java Software Engineer
Enterprise SaaS commerce platform. Owned search and promotion infrastructure for 20,000+ merchants.
Selected engineering work
A large campaign could fan out into millions of activity-by-store updates, while an upstream product service initially limited the pipeline to 3,000 QPS. I split update events by change type, isolated oversized workloads into dedicated processing paths, and negotiated a quota increase to 10,000 QPS after reducing unnecessary upstream reads. End-to-end publish latency fell from more than 30 minutes to under 10.
During a seven-month backfill, a data-heavy account representing more than a third of indexed data pushed one node past Lucene's 2.1-billion-document ceiling and stopped writes. I restored writes by separating large accounts into a dedicated cluster, then redesigned routing and query paths while the legacy system continued serving customers. The platform grew from roughly 65M to 90M indexed product records across 32 data nodes; hot/cold separation, filter-only queries, and index sorting kept query latency stable without adding nodes.
Backfill, reindex, and repair jobs ran for months against the same clusters and upstream quota as live traffic. I built a distributed task platform with five priority queues, backpressure, lock-based idempotency, emergency stops, alert controls, and dead-letter replay. It remained in use after the migration as the team's shared platform for backfill, repair, and index operations.
MySQL and Elasticsearch could drift apart without warning, so the first signal sometimes came from a customer. I built a configuration-driven, multi-tenant reconciliation service that compared data by field, scope, and document count, then supported alerts, retries, repair, and an audit trail. The work turned silent divergence into a detectable and recoverable condition, and was later incorporated into a company-wide reconciliation platform used by multiple internal teams.
Senior Java Software Engineer
Used-car trading SaaS platform for automotive dealers across China; listed on NASDAQ in 2026.
Self-directed study — IBM AI Engineering Professional Certificate.
Supply-chain and ERP maintenance, standalone-site integration, marketing automation.
B2C ticketing SaaS — ordering, refunds, validation, payments — and a group-tour ERP.
Points-based loyalty malls, and mobile field-data apps for infrastructure contractors.
Open to backend, data engineering, and AI platform roles in Manitoba.