Senior Data Engineer

PharmaLink, LLC is building an enterprise data, analytics, and AI platform powered by the modern Databricks ecosystem. In this onsite role in Largo, FL, the Senior Data Engineer helps design, evolve, and operate scalable data solutions that enable trusted data products across analytics, applications, and AI use cases.

What you’ll do

• Design, build, and operate enterprise data solutions on Databricks.
• Develop scalable ETL/ELT pipelines using Python, PySpark, Spark SQL, and Delta Lake.
• Create curated data models and reusable data products for analytics, applications, and AI.
• Implement Unity Catalog capabilities, including security, lineage, access controls, and data governance.
• Build and optimize Databricks SQL Warehouses and AI/BI dashboards.
• Develop data and AI applications using Databricks Apps.
• Design and implement solutions using Databricks Lakebase.
• Enable AI/ML and Generative AI use cases using Databricks capabilities.
• Automate data quality, reconciliation, anomaly detection, and monitoring processes.
• Integrate data from databases, SaaS platforms, APIs, files, and other enterprise systems.
• Optimize Spark workloads, SQL queries, pipelines, performance, and platform cost.
• Establish engineering standards for Git, CI/CD, testing, monitoring, and production deployment.
• Troubleshoot complex production and data issues and perform root-cause analysis.
• Partner with Data Governance, BI, application, and business teams to deliver trusted data products.

Required experience

• 8+ years of experience in data engineering, data platforms, or related engineering roles.
• 4+ years of hands-on Databricks experience in production environments.
• Advanced skills in SQL, Python, PySpark, and Apache Spark.
• Strong experience with Delta Lake and Unity Catalog.
• Strong ETL/ELT experience, data modeling, and enterprise data architecture experience.
• Experience with Databricks SQL and AI/BI Dashboards.
• Familiarity with Databricks capabilities including AI/ML, Generative AI, Databricks Apps, and Lakebase.
• Strong relational database fundamentals and data-engineering fundamentals.
• Experience with Git, CI/CD, automated testing, monitoring, and production support.
• Experience working with cloud-based enterprise data platforms.
• Must maintain a positive attitude in a high-performance team environment.
• Must pass a background check and random drug screenings.
• Must be adaptable and flexible in dealing with a diverse workforce.

Technologies

• Databricks, Python, PySpark, Spark SQL, Delta Lake, Unity Catalog
• Databricks SQL, AI/BI Dashboards, Databricks Apps, Lakebase
• AI/ML, Generative AI, Git, CI/CD, Apache Spark

Preferred qualifications

• Experience with Databricks on Microsoft Azure.
• Azure Data Factory.
• Microsoft SQL Server.
• Fivetran or similar CDC/ELT technologies.
• Power BI.
• Databricks Workflows.
• MLflow and AI/model lifecycle capabilities.
• API and application development experience.
• Azure DevOps or GitHub-based CI/CD.
• Databricks professional certification.
• Experience in healthcare, life sciences, or another regulated industry.

Location and work setup

• Largo, FL (onsite)

Physical requirements

• Requires light physical activity performing non-strenuous daily activities of an administrative nature (for example, sustained phone, headset, and keyboard use) as well as periods of prolonged sitting.