The Data Platform team at Compass is responsible for building and operating a unified, secure and scalable data platform, making high-quality data available for business intelligence, financial reporting, and data science use cases.

As a Senior Engineer of the Data Platform team, you will play a crucial role in building and operating the data infrastructure that serves the entire company. Your responsibilities will include delivering high-quality results to ensure the security, compliance, optimization, and automation of the data infrastructure. Additionally, you will take ownership and actively contribute to the architectural innovations while maintaining a high standard for quality.

To thrive in this role, effective collaboration with engineers and stakeholders across the company is essential. A strong sense of ownership for the product you work on is important to you. You have a genuine passion for learning and find joy in sharing your knowledge with others. You try to understand others before seeking to be understood. Every day you wake up excited about the new things you’ll learn and the people you can inspire.

Responsibilities:

• Data architecture: Evaluate data platform architecture. Build and optimize data services to improve compliance, security, reliability, performance, and resource utilization.
• Compliance: Build process and tools to ensure that the data platform is compliant with SOX (Sarbanes-Oxley Act), GLBA (Gramm-Leach-Bliley Act ), and CCPA (California Consumer Privacy Act).
• Security: Identify infrastructure security-relevant actions, tests, and key performance indicators. Enhances the security posture of the Compass data platform, ensuring the protection of data, systems, and assets.
• Infrastructure automation: Increases reliability, enhances employee productivity, reduces security attack surfaces, and eliminates human errors through standardization of process and Infrastructure as Code.
• Data infrastructure monitoring. Monitor the platform health, reliability, cost, and usage.
• Best practices and guardrails: Champion the best practices and guardrails for using the data platform. Identify cost-saving opportunities and optimize the data platform for all stakeholders.
• Operational support: Support the platform users for troubleshooting, configuration, version updates, and feature adoptions.

Qualifications:

• 8 years of development experience with a focus on architecting cloud data platforms, setting up, managing and automating the infrastructure of cloud data platforms, including Airflow, Databricks, Spark, and Kafka
• 5 years of developer experience using Infrastructure as Code (IaC) technologies, such as AWS Cloud Formation and Terraform
• 3 years of experience with data compliance and governance. Demonstration of collaboration with the internal compliance team and external auditors in planning and execution of all phases of SOX compliance including risk assessment & scoping, documentation of process walkthroughs, identifying controls and key reports, testing, and reporting results. Implement customer data deletion requests that are required by CCPA. Standardize and automate the compliance process.
• 5 years of experience maintaining and improving OE of data platforms - service alerting, monitoring,and automation.
• 8 years of experience and proficient programming skills in Python or Java/C#. 3 years of programming experience in SQL.
• B.S., M.S., or PhD. in Computer Science or equivalent

Desirable to Have:

• Subject matter expert (SME) in architecting cloud data platforms using Databricks, Snowflake, AWS Lake Formation or other relevant technologies.
• SME in big data technologies including Spark, Presto, Airflow, Kafka
• Work in a startup-like environment building agile products and services
• Create clear and concise documentation, including diagrams, service descriptions, decisions, and runbooks.
• Deep knowledge of cloud platform such as AWS, GCP, Azure
• Proven records of improving services in your domain throughout their lifecycle. Identify and remove bottlenecks to address inefficiencies in the developer experience.

Check out Compass’s Engineering blog!