SENIOR DATA ENGINEER

SENIOR DATA ENGINEER

COUNTRY

Colombia, Mexico

FORMAT

Remote

Svitla Systems Inc. is looking for a Senior Data Engineer for a full-time position (40 hours per week) in Colombia, Mexico.

Our client helps organizations determine applicable regulatory requirements, identify compliance obligations, execute compliance activities, and stay current as regulations change.

You'll take full ownership of a Python/Scrapy-based legal/regulatory data ingestion pipeline. This role combines large-scale data engineering, advanced web scraping against hostile sources, complex legal document processing, and cost-efficient cloud infrastructure design. It also includes maintenance of a React frontend application.

REQUIREMENTS

5+ years of experience in data engineering or backend development, with a focus on production pipelines.

Strong experience in Python, including production web scraping (Scrapy or equivalent) and structured-data processing.

Practical understanding of defeating or working around anti-bot measures (rate limiting, Cloudflare, paywalled/gated legal databases) within legal and ethical bounds.

Understanding of relational and non-relational databases; schema design for hierarchical data.

Solid understanding of SQL and data modeling, with the judgment to design schemas that serve both change detection and human-readable display.

Experience with cloud infrastructure (AWS, GCP, or Azure), IaC (Terraform/CloudFormation), and containers (Docker).

Knowledge of Microsoft Azure, including VMs, Blob storage, App Service plans, and container services, with the ability to provision and tear down resources programmatically.

Knowledge of search architectures (full-text, semantic search - Elasticsearch, OpenSearch, or vector DBs).

Familiarity with messaging/notification systems (email APIs, Slack/Teams webhooks).

Understanding of Infrastructure-as-code for repeatable, automated deployments.

Be comfortable owning a system end-to-end with minimal hand-holding, including reading and improving inherited code and documentation.

The ability to take ownership of complex legacy systems without extensive documentation.

Meticulous attention to detail: this work handles legal text where structural precision is critical.

Cost- and efficiency-oriented mindset.

Be comfortable working with ambiguity across heterogeneous, changing data sources.

NICE TO HAVE

Experience with React and Vite for maintaining and extending the display layer.

Prior experience in legal tech, regtech, or compliance software.

Familiarity with anti-detection scraping techniques (proxy rotation, fingerprinting, simulated human behavior).

Familiarity with deterministic hashing and content versioning systems.

Experience designing systems for horizontal scale (clustering, multi-VM, multi-threaded, or containerized workloads).

Experience AWS alongside Azure (the concepts transfer; multi-cloud).

Familiarity with semantic / cross-jurisdictional search techniques.

Exposure to LLM or vision-AI integration - e.g., using AI to parse document structure, generate summaries and action statements, or pre-classify regulatory changes as impacting vs. non-impacting for human review.

RESPONSIBILITIES

Take over and fully understand an existing Python/Scrapy ingestion pipeline, its codebase, and its documented database schemas, then make it your own.

Build and maintain web scrapers across 50+ jurisdictions, each with its own structure, format, and obstacles - XML feeds, HTML pages, DOCX-only sources (e.g., West Virginia), and content behind Westlaw, LexisNexis, Cloudflare, and similar barriers.

Render JavaScript-heavy sources reliably using headless browsers (Playwright/Chromium), including human-like interaction patterns when needed to avoid blocking.

Consolidate tooling where possible: prefer one or two robust, broadly capable scraping tools over a sprawl of point solutions, falling back to specialized tooling only for genuinely esoteric sources.

Normalize ingested content into a structured, plain-text format for deterministic hashing and change detection, while preserving the original, formatted HTML so documents can be displayed exactly as the issuing agency intended.

Faithfully preserve each source's original hierarchy (nested clauses, sub-paragraphs, lettered and numbered subsections, Roman numerals, Unicode markers, appendices, and tables), so stored content remains a true representation of the statute.

Maintain and extend a multi-destination architecture, including the nested-hierarchy schema used by the SaaS compliance platform (white-labeled in some deployments) and a flattened schema supporting full-text, semantic, and cross-jurisdictional search.

Operate and evolve change detection and notification (email, Slack, Teams), with a path toward more granular, paragraph- and clause-level change tracking and side-by-side visual diffs.

Design the infrastructure for cost efficiency: model how many VMs/containers are needed and for how long, automate spin-up and shutdown so nothing runs idle, and right-size compute based on real scraping timings rather than assumptions.

Maintain the React and Vite single-page application that displays ingested regulations, including a searchable table-of-contents navigation pattern.

WE OFFER

US and EU projects based on advanced technologies.

Competitive compensation based on skills and experience.

Annual performance appraisals.

Remote-friendly culture and no micromanagement.

Comprehensive private medical insurance.

Christmas Bonus in the amount of 50% of the monthly payment.

Bonuses for article writing, public talks, other activities.

15 vacation days, 10 holidays, 10 sick leaves.

Personalized learning program tailored to your interests and skill development.

Free tech webinars and meetups organized by Svitla.

Fun corporate online\offline celebrations and activities.

Awesome team, friendly and supportive community!

ABOUT SVITLA

Svitla Systems is a global digital solutions company headquartered in the U.S. and operating across the Americas, Europe, Asia, and APAC. Since 2003, we have served a wide range of clients - from innovative start-ups to Fortune 500 companies.

Our success is built on partnership. By integrating seamlessly with clients’ teams, we create lasting collaborations that drive real results.

We are strong advocates of workplace flexibility, remote culture, individual approach to professional and personal growth.

Our global mission is to build a business that contributes to wellbeing of our partners, personnel, and their families, improves our communities, and makes a lasting difference in the world.

Together, we are coding a brighter tomorrow - and living it.

Join us!

LET'S MEET IN PERSON

Vladyslava Holovina

RECRUITER

Email:

v.holovina@svitla.com