Principal Data Engineer

Position Overview Autodesk seeks a strong principal engineer and technical leader within the Data Acquisition team, designated with designing new robust and resilient data pipelines for analytical abstracts, enrichment, and metadata about design content. These datasets form the basis for data selection and understanding for various AI/ML projects at Autodesk.

This role collaborates with Product Managers, Program/Project managers, Analysts, Platform Architecture, and the Engineering Team to ensure data pipelines and delivery uses modern, scalable, reliable methods. Communication skills are critical, and you must be comfortable distilling complex technical information to a non-technical audience as a technical leader.

We work with a group of highly collaborative individuals who are excited to solve challenges, and implement cloud based, resilient, available, reliable solutions. We are comfortable with several languages and paradigms and aim to choose the technology which is most appropriate for the task. We endeavor to automate our tests and deployments end to end. We seek to mitigate existing issues and balance tech debt with the delivery of value to Autodesk. If you can identify with these traits, we are looking forward to working with you.

Responsibilities

Designing, architecting, solutioning, and implementing data pipelines for enterprise consumption

Mapping and automating data flows, from multiple sources

Develop workflows to run Apache Spark applications using Airflow and AWS EMR; may also work on Snowflake applications using Airflow and dbt

Deploy solutions via automated CI/CD processes to execute data pipelines running in batch

Developing, executing, and documenting software solutions to address complex data collection, processing, transformation, and reporting issues

Interpreting and translating business needs into technical requirements

Troubleshooting code level problems quickly and efficiently as needed

Participating in code and document reviews

Minimum Qualifications

Strong experience in several of the following: Pytho / PySpark, Spark / Spark SQL, Apache Iceberg/Airflow (Astronomer is a plus), Amazon EMR, Lambda, S3, Snowflake / Snowflake SQL, dbt, Fivetran, Hive, Hadoop

Extensive experience with a strong focus across layers of data engineering and pipelines

Strong experience in distributed and cloud architecture patterns

Ability to provide detailed architecture artifacts (L1 / L2 diagrams, sequence diagrams, etc.)

Firm understanding of programming paradigms, their usage, as well as data structure

Effective communication and collaboration skills, understanding how to tailor your message to your audience

Ability to fully develop a feature across all data and/or application layers from inception

Experience building and deploying data applications with AWS

Experience implementing CI/CD and familiarity TDD/BDD. Preferably concrete examples of each

Strong experience designing and using data lakes and data warehouses

Powerful sense of ownership with a bias for action when challenges arise

Ability to work independently and collaboratively across an organization.

Desire to create, investigate and follow best practices

Familiarity with AI/Machine Learning workflows and technologies is a plus

Eagerness to learn and share knowledge with a good attitude

About Autodesk Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – it’s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.

When you’re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!

Salary transparency Salary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.

Diversity & Belonging We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging

#J-18808-Ljbffr