SESSION Details

Migrate traditional DW transformations to Spark/Scala

Nowadays, Spark is becoming more and more popular framework for building transformations either big data or traditional DW world. The difference is DW transformations is much more complex than big data specific. Also, moving from SQL-based world to Scala needs an additional efforts from developers. In this session, I will show you how to reuse existed DW experience and programming technics to implement a transformation application in Spark using Scala language. This topic will include: DW objects translation to Spark/Scala, code structuring, piping transformation, templating, unit testing

Feedback link: https://sqlb.it/?7357

Starts: 17:10 10th Mar 2022
Ends: 18:00 10th Mar 2022

Q&As

Short Description
- Join me to learn how to migrate a traditional DW transformation process to Spark/Scala based platform and reuse existed experience
Other technologies
- Scala, IntelliJ

Slidedeck Link

Migrate traditional DW transformations to Spark.pptx

Speakers

Sergiy Lunyakin

Tags

Platform AgnosticManaging Big DataDevelopingTestingBig Data AnalyticsData LakeData Transformation and IntegrationModern AnalyticsSparkAdvancedHow

The SQL Bits Story

SQLBits was formed in 2007 by a group of volunteers who were passionate about the SQL Server product suite and wanted to provide much-needed community-driven education to the data community.

As one of the largest data platform conferences in the world, we offer more opportunities to a wider audience.

15 Years

We’ve grown and expanded a lot since 2007.

2500 Participants

SQLBits is the best place to meet fellow data professionals.

82 Countries

We welcome data professionals from all over the globe.

1140 recorded sessions

All the live sessions are recorded and offered for free, year round.

Experience the SQLBits Conference

Previous
Next

Want to be part of the SQLBits community?

Attend the London conference in-person or virtually on 

March 8-12, 2022 at ExCel London, UK.