Industry experience (consulting)

Data architecture and AI platform assessment

• Built a Python discovery toolkit that probes the compliance, admin and analytics surfaces of two major generative-AI vendors and reports coverage against the thirteen mandatory columns of a Silver-layer Snowflake model.
• Implemented credential-type detection by key prefix before the first API call, so a run of 401s is never misread as missing data. Access diagnosis dropped from days to minutes.
• Designed an evidence status scheme, available / partial / derived / documented / absent / untested, that separates what was verified live from what was inherited from earlier documentation, preventing a pipeline being built on unvalidated assumptions.
• Found session endpoints absent from the prior findings document that resolve entry channel as a direct source rather than an inferred one, along with the tool catalogue feeding the feature dimension.
• Reported nine fields available, three derived and one partial instead of an aggregate "thirteen of thirteen", making explicit that the derived columns need their own logic in the translator.
• Evaluated both vendors' analytics APIs against the value-measurement data model and identified the granularity mismatch that redefined the workstream's scope: daily per-user aggregates against an event-level fact table.
• Ran an independent pre-pilot assessment of nine high-priority capabilities in an enterprise workspace of roughly 3,500 members, scoring zero to four with documented evidence, and verified vendor claims through tests designed to refute them.
• Raised seven methodological inconsistencies in the evaluation instrument, including three scoring scales coexisting in a single file, and rebuilt the test protocol for the twenty-four unassigned rows.
• Built a proof-of-concept conversational intake agent on Salesforce Agentforce using a pro-code approach, with guided capture of missing requirements, standardised scope generation and PII guardrails, packaged as a deployable authoring bundle.
• Established the reusable implementation pattern for the eight remaining agents on the roadmap.

Linear TV audience research, Nielsen panel and big data

• Designed and built the prepared-table layer in Snowflake (telecast, half-hour, quarter-hour, competitive universe, fiscal calendar and date metadata) sitting between the unified Nielsen views and business queries, cutting per-question scans of tens of millions of rows down to a bounded, reusable dataset across a multi-network portfolio and the full competitive cable and broadcast universe.
• Set the design principle that classifications become columns while filters stay in the queries, so changing a business default never forces a table rebuild. Every table carries a declared grain, build order, reasoned clustering key and a uniqueness check.
• Implemented roughly twenty-five documented production queries covering overnight summaries, series performance snapshots and competitive rankers, each parameterised with dynamic defaults and its own numeric validation case, with zero pinned dates anywhere.
• Resolved Nielsen data-integrity problems that were silently doubling metrics: the panel and big-data overlap at the currency cutover, duplicate rows where the panel emits one record with VCR and one without for the same telecast, and double-registered ACM streams. Defined unique keys and permanent diagnostic cells that profile duplicates at the source before deduplication hides them.
• Implemented Nielsen broadcast-day chronology anchored at 6:00am so post-midnight telecasts sort within their own night, and established that a Nielsen month is not a calendar month and belongs in the planner layer rather than hardcoded in SQL.
• Validated results against the client's official rankers and overnight summaries as an acceptance criterion, reproducing their figures at telecast level and documenting every difference by its methodological cause rather than adjusting it. That surfaced a ranker averaging flat instead of weighting by minutes, another running on a different daypart boundary, and an inherited telecast floor that existed in no source and was dropping series silently.
• Built a resolution layer converting a natural-language question into verified parameters before any query runs, handling program names truncated to 25 characters by Nielsen and free-text season codes, with an explicit contract for when the answer is that the thing does not exist.
• Documented the limits of the data instead of approximating them: C3 and C7 streams do not exist at half-hour grain, so a dayparted network rank on C3 is not computable, reported and escalated with the vendor's own documentation as backing.

Research information management platform, higher education

• Lead an eight-person technical services team of data integration engineers, data migration engineers and specialists, owning the data-loading platform behind a research information management product used by more than 200 institutions.
• Led the end-to-end technical discovery of the platform, reverse-engineering three undocumented subsystems that all write to the same Postgres backend: 204 client-specific PHP autoload subclasses, 88 Matillion pipelines, and a Python desktop migration tool.
• Built and ran an SFTP scanner across 108 institution tenants, producing the first ground-truth inventory of the ingestion estate: 839,221 files and 1.13 TB, with 97 per cent of stored bytes older than two years. That surfaced roughly 1 TB of reclaimable storage and 37 institutions dormant for 180 days or more, whose pipelines were about to be migrated at zero business value.
• Surfaced and escalated production risks found in code review, including an unexplained 894 GB bulk load on a single tenant and a hardcoded database password in a delete pipeline, each converted into a tracked remediation item.
• Mapped the platform's three competing record identifiers and showed that the encrypted API id is never persisted by any consumer, a finding that directly reshaped the target architecture.
• Designed the target architecture: a batch-first FastAPI REST service on EKS accepting arrays of up to 500 records per call, with idempotency keys, threshold-protected batch deletes and audit logging, targeting at least a threefold throughput improvement over the existing per-record GraphQL path.
• Designed a config-driven Databricks ingestion pipeline to replace 88 hand-maintained pipelines, turning a two to five day bespoke onboarding into a single configuration row.
• Owned the platform and DevOps foundations: ECR repositories, EKS IAM roles, internal ALB ingress, per-environment Helm values and GitHub Actions promotion pipelines, delivered through Terraform and Terragrunt.
• Authored the ten-week modernisation plan, four epics and roughly forty tickets across five sprints, and defended the scope call that deferred orchestration work in order to protect a credible deadline.
• Built risk controls into every migration ticket: shadow-mode parallel running before cutover, equivalence QA sign-off and documented rollback runbooks.

Consumer research and audience analytics

• Modelling on survey data at scale in Snowflake: 8.6 billion observations, roughly 105 million users and about 3,600 syndicated questions.
• Documented the ten core warehouse tables and the publisher chain, verifying every relationship, cardinality and match rate against the information schema and the data itself. The resulting ERD and catalogue of known traps became the team's mandatory filter standard for all analysis.
• Built a Naive Bayes model predicting a user's most likely answer to any syndicated question from their known answers, with mutual-information feature selection and hold-out validation on accuracy, log-loss and Brier score.
• Implemented a KNN alternative over sparse one-hot vectors with cosine similarity and weighted voting, evaluated separately for single-select and checkbox questions.
• Attacked the 98.8 per cent sparsity of the user by question matrix with publisher-based imputation, blending the answer distribution across the publishers a user was seen on, then extending it with a Naive Bayes posterior over known answers.
• Designed the evaluation harness: two decoding strategies, two publisher levels and a global baseline, measured by total variation distance with breakdowns by question and by demographic.
• Built a brand lift pipeline treating the exposed segment as a KNN-imputed treatment, applying multiple imputation with Rubin's rules to combine imputations into a lift estimate with correct variance, p-values and confidence intervals, and audited it against the real-only lift.
• Built an end-to-end audience segmentation pipeline: user by top-300 question matrix, truncated SVD embeddings, KMeans with k chosen by silhouette, and population representativeness weights raked to census margins so every reported share describes adults nationally rather than the raw panel.
• Automated segment labelling with four LLMs running in parallel under strict JSON output, with side-by-side comparison and consensus selection, plus 24-month trends and an executive summary.
• Turned "is behavioural or demographic segmentation more useful" into a measurable experiment: three segmentations over the same cohort, method, weighting and k, and corrected a one-hot encoding bias that was destroying the order of ordinal demographics.

Online technical education

• Authored ungraded labs on agentic workflows, reflection and planning for a widely used AI course catalogue.
• Designed agent pipelines in FastAPI with planning, execution and reflection loops for complex task automation.
• Built and deployed agent applications including research assistants, email simulators and customer-segment builders, wired to tools for SQL execution and document retrieval.
• Developed a research agent that analyses and summarises scientific documents through natural language, using external tools and contextual memory.
• Integrated hosted LLMs into FastAPI services and deployed interactive demos for hands-on learner testing.

E-commerce and retail analytics

• Led the data engineering and data science function across multiple product lines.
• Enriched product catalogues with local LLMs for metadata and descriptions.
• Modelled product, customer and feature relationships in Neo4j for graph-based recommendation.
• Built AI agents for product recommendation and audience construction using local and hosted models.
• Developed LLM systems that personalise dialogue and lift conversion across audience segments.
• Near-real-time data lake enrichment with Kinesis Streams and Firehose, on S3 and Google Cloud Storage.
• Warehousing on Redshift and BigQuery, transformation layers in dbt, pipelines in AWS Glue, GCP Dataflow and Python.
• Image feature extraction via vision APIs to improve DeepFM, Apriori and graph neural network models.
• Customer lifetime value models with scikit-learn and Keras.

Freight and logistics analytics

• Route and path analysis for freight markets using mapping APIs, OpenStreetMap and OSRM.
• Logistics visualisation and optimisation tools built with Folium, Dash and Python.
• ML models and pipelines in Python, Keras, scikit-learn, pandas, NumPy and BigQuery ML, with dbt for transformation.
• Reporting in Looker Studio, delivery through Docker, GitHub, REST APIs and cloud functions.

Media and streaming measurement

• Ingestion and processing of S3-hosted data into Snowflake, with staging areas for reliable retrieval.
• Models predicting impressions per stream, trained and deployed through Snowflake stored procedures and tasks.
• Dashboards built directly in the warehouse environment with Streamlit.

Model evaluation and alignment data, two freelance contracts

• Evaluated and ranked model responses against defined criteria, with written justification comparing candidates and identifying failure modes.
• Generated alternative responses targeting specific loss categories, including deliberately flawed responses matching realistic failure scenarios.

Database migration, part-time three-month contract

• Data quality assessment and automated migration comparison between on-premise databases and Azure.
• Dimensionality reduction techniques applied to make real-time record comparison faster and more precise.

Payroll systems

• Python applications generating tailored HTML and Excel payroll reports, on Azure Functions and SQL Server.
• Delivery tracked through Azure DevOps with version control in GitHub.

Applied data science training

• Led 40 project teams averaging eight members each, on three-month delivery cycles.
• Built Python applications for varied markets including footwear retail and energy.
• Stack: Python, Docker, REST APIs, Dash, AWS RDS, EC2 and S3, with GitHub for versioning.