Senior Data Engineer

About Paidy Inc.

We're Japan's leading "Buy Now, Pay Later" payments service with over 10 million customers. You can use Paidy to buy an iPhone at Apple on a monthly payment plan, it's a top payment method on Amazon Japan, and it's offered at thousands of other partner merchants to give Japan more choice for how to shop. That of course means we have lots of data about how Japanese consumers shop online as well as how they interact with our services like our App and web interface.

The Data Department is an 11-member team with data engineers and data scientists that has built a big data platform from the ground up on AWS. We own the data pipelines that ingest data from Kafka topics, vendor APIs, Elasticsearch, and more, built on top of an in-house ETL tool, Glue, EMR, and Spark. We model this data using a medallion architecture and see many projects through to their execution, tailoring gold layer data models to ensure they match the needs of a machine learning project or integration back out to Paidy's services.

We are fully remote (with a great office too) but always collaborating, having internal huddles for a project, helping each other out on Slack, and meeting with product 

About Position

As a Senior Data Engineer in the Data Science Foundations team, you'll work both on data pipeline development and improvements to our data platform itself. That could mean projects ranging from working on a new feature of our Spark SQL Runner application to support a data scientist's project requirements to working on performance improvements to an existing pipeline ingesting data from a Kafka topic. As a senior member of the team, from the first few weeks you already can bring in subject matter expertise in data engineering and are able to mentor other members in building skills in those areas.

The team’s goal is to ensure the company can rely on the team's data platform to easily extract the information they need from a Looker dashboard about last week's business performance and collaborate with the Data Department to build solutions (e.g. with machine learning) that make an impact. Your role involves keeping up with the needs of some areas of the business (usually alongside a Data Scientist) and proposing and delivering the data engineering development work needed to support those business needs. If you want a hands-on fast-paced environment in a small team where you own something big and your data engineering solutions actually make a difference, consider joining us!

Skills and Requirements

● Solid grasp of data engineering and software engineering concepts from real-world experience
● Experienced building and maintaining data ingestion and transformation pipelines
● Worked with Scala and Python in a production environment
● Used Spark and Prefect (or similar tools/frameworks) to develop data engineering systems
● Self-organized, driven to maintain high standards

What We Offer You

● Diversified team with 238+ colleagues from 42+ countries
● Cross-functional collaboration
● Flexible work-from-home arrangement
● Competitive salary and benefits

Paidy Values

• Be a winner / 勝ちにこだわる
  • Beat expectations / 常に期待値を超える
  • Display surprising speed / 人をスピードで驚かす
  • Embrace risk / リスクを恐れない
• Own it and deliver / 結果を出す
  • Commit to what, when and how to deliver/ 目的・やり方・期限にコミットする。
  • Own the actions to deliver / 結果のためのアクションにこだわる
  • Embrace conflict when needed to deliver results / 必要なら対立・衝突も恐れない
• Play an integral role / 大切なピースになる
  • Make an irreplaceable contribution to our business / 替えの効かない貢献をする
  • Embrace and bridge differences in language and culture / 皆が言語と文化の架け橋になる
  • Raise the bar / スタンダードを上げ続ける