Senior Data Engineer-Foundational Projects

Other Jobs To Apply

About the position

Responsibilities

  • Designs data patterns that support creation of datasets for analytics; implements calculations, cleanses data, ensures standardization of data, maps/links data from more than one source
  • Performs data validation and quality assurance on work of junior peers and write automated tests
  • Maintains/streamlines existing data pipelines end to end
  • Builds/supports complex data warehousing pipelines into cloud platform GCP/AWS, data integrations, data streaming solutions, predictive model implementations with prompt engineering for AI Models, building out data quality checks into data pipelines
  • Identifies more complex data from upstream sources to enable new capabilities
  • Engages in testing of the technical solutions to ensure data integrity and system functionality of own work and broader system, E2E
  • Builds large-scale batch and real-time data pipelines with big data processing frameworks
  • Designs/develops data integrations to support application engineering and system integration
  • Designs/develops/maintains large data pipelines; diagnoses/solves production support issues
  • Creates documentation and training related to technology stacks and standards within assigned team
  • Designs/implements monitoring capabilities based on business SLA and data quality
  • Uses/contributes to refinement of Digital Engineering-related tools, standards, and training
  • Engages/collaborates with external technical teams to ensure timely, high-quality solutions
  • Engages with shared services teams and vendors when necessary
  • Works closely with Product, Data Science, Application, and Analytics teams to develop a clear understanding of data and data infrastructure needs; assists with data-related technical issues; ensures optimal data design and efficiency
  • Performs full SDLC process, including planning, design, development, certification, implementation, and support
  • Interacts with Product, Business, Analyst stakeholders to confirm data quality, discuss requirements, and support data testing
  • Peer reviews other team members code; learns/adapts from peer review of own code
  • Contributes to overall design, architecture, security, scalability, reliability, and performance
  • Mentors/provides support to junior Data Engineers
  • Builds more complex data models to deliver insightful analytics; ensures highest standard in data integrity
  • Knowledge in machine learning concepts

Requirements

  • 5+ years of experience related to data engineering
  • A related degree or comparable formal training, certification, or work experience
  • Experience in Lean Startup or Agile development methodologies
  • Experience working in large scale infrastructure, large data sets, and mission critical SLAs

Nice-to-haves

  • Advanced knowledge of Lean Startup / Agile methods
  • Knowledge of business intelligence, analytics and reporting, and application integration
  • Knowledge of data architectures such as data warehouse, data lake, and data mesh and when to apply
  • Strong working understanding of data architecture and data modelling best practices and guidelines for various data and analytic platforms
  • Strong working understanding of coding standards and design principles/patterns
  • Strong prioritization skills
  • Strong verbal/written communication and data presentation skills
  • Ability to deliver on ambiguous projects with incomplete information
  • Ability/willingness to learn new technologies as they emerge
  • Ability to calmly work under pressure
  • Ability to work a flexible schedule as needed
  • Ability to collaborate across multiple work locations
  • Ability to work within a team, and willingness to take feedback from peers and mentors
Back to blog