Job Description
Remote Lead Data Engineer needs 7+ years' experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales)
Remote Lead Data Engineer requires:
Strong background in math, statistics, computer science, data science or related discipline
- Advanced knowledge one of language: Java, Scala, Python, C#
- Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
- Proficient with:
- Data mining/programming tools (e.g. SAS, SQL, R, Python)
- Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
- Data visualization (e.g. Tableau, Looker, MicroStrategy)
- Comfortable learning about and deploying new technologies and tools.
- Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines.
- Good written and oral communication skills and ability to present results to non-technical audiences
- Knowledge of business intelligence and analytical tools, technologies and techniques.
- Familiarity and experience in the following is a plus:
- AWS certification
- Spark Streaming
- Kafka Streaming / Kafka Connect
- ELK Stack
- Cassandra / MongoDB
- CI/CD: Jenkins, GitLab, Jira, Confluence other related tools
Mandatory Skills Advanced knowledge one of language: Java, Scala, Python, C#? Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
Proficient with Data mining/programming tools (e.g.?SAS, SQL, R, Python) Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
Remote Lead Data Engineer duties:
- Create and manage cloud resources in AWS
- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies
- Data processing/transformation using various technologies such as Spark and Cloud Services.
- Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations
Employment Type: Full-Time
Salary: $ 110.00 Per Hour
Job Tags
Hourly pay, Full time,
Similar Jobs
HireIO Inc
...Proactively search for potential candidates using online databases, job boards, and other... ...preferred but not required.Familiarity with staffing/recruitment processes and industry best... ...within a leading staffing/recruitment agency.Engage in hands-on projects encompassing...
Aerotek
...warehouse equipment and proper use of PPE + Good attendance and work ethic **Experience Level** **:**+ Entry Level **About Aerotek:** We know that finding work is hard, and finding meaningful work is harder. Aerotek connects 180,000+ light industrial and skilled...
Ascension
...position, exclusions may apply for some roles (for example: Travel, PRN, Short-Term Option, etc.). Actual compensation offer will vary... ...for additional specifics. Responsibilities Provide direct nursing care in accordance with established policies, procedures and...
Fud
We are a growing start-up company looking for a Freelance Virtual Assistant Expert and Mentor to join our community. As a Freelance VA Expert... ...E-Commerce - including Amazon, Shopify, and your own online store Gig Economy - like DoorDash, Uber, or Lyft Freelancing...
Dave Wright Nissan Subaru
...NECESSARY!VOTED 'BEST DEALERSHIP TO WORK FOR' 11 TIMES!Wright Wash Core Values:* We Solve It* We Are All One Team* We Do... ...We Say* We Grow Or We Die* We Commit And Never QuitCar Wash Manager Responsibilities:* Ensuring all car wash rules, safety,...