The AWS Business Intelligence Engineer will have responsibility across the entire Cloud Data stack. The BIA will use their experience to build and optimize data pipelines (ELT/ETL), data warehousing and dimensional modeling, and curate data sets for Data Scientists and Business Intelligence users. Additional responsibilities include building AWS Data Lake and on-prem to Cloud Data Migration projects.
- Design, build and operate the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, cloud migration tools, and ‘big data’ technologies
- Optimize various RDBMS engines in the cloud and solve customers’ security, performance, and operation problems
- Design, build, and operate large, complex data lakes that meet functional / non-functional business requirements
- Manages large volumes of data, data modeling, and metadata to support ad-hoc and pre-built reporting
- Train and support business users in achieving unique data insights.
- Optimize various data types of ingestion, storage, processing, and retrieval from near real-time events, and IoT, to unstructured data as images, audio, video, and documents, and in between
- Work with customers’ and internal stakeholders including the Executive, Product, Data, Software Development, and Design teams to assist with data-related technical issues and support their data infrastructure and business needs
- Analytical problem-solving skills
- Ability to manage large volumes of data, modeling data, and metadata
- Experience in Lake Formation Orchestration, Maintain and enhance models for SDLC using CI/CD pipelines in AWS
- Experience in implementing data pipelines for both streaming and batch integrations using tools/frameworks like Glue ETL, Lambda, Spark, Spark Streaming, etc.
- Experience in Relational and NoSQL databases, such as MySQL or Postgres and DynamoDB or Cassandra
- Functional and scripting languages: Python, Java, Scala, etc. Advanced SQL
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ stores
- Articulate with great communication and presentation skills
- Team player that can train as well as learn from others
- Bachelor (Graduate preferred) degree in Computer Science, Mathematics, Informatics, Information Systems, or another quantitative field
- 3+ years of experience in a Data Engineer role in a cloud-native eco-system
- 2+ years working experience in AWS Glue ETL, Redshift and Redshift Spectrum, Step Function, Athena
- Experience building and optimizing ‘big data’ pipelines, architectures, and data sets
- Experience supporting and working with external customers in a dynamic environment
- Experience in Athena and Quicksight is definite plus