A Data Engineer's Guide to Azure Synapse
There has been an explosion of interest in Azure Synapse Analytics as everyone races to get to grips with the all-in-one data analytics platform. But when opening up the box, you find it's a lot more complex that it's made out to be, with several different powerful compute engines, each with their own idiosyncrasies! Why do we have different flavours of each engine? When should you use Spark pools over SQL? What's the most cost effective approach for different scenarios? What types of users should be using each service? The answers to these questions aren't always met with clarity!
This training day breaks down the Synapse workspace into it's component parts and provides a foundation of knowledge for each piece. During the day, we will cover:
- Fundamentals of building a Lake-based analytical platform - how you structure a lake, what file format to choose, what kinds of data work it's suited for
- How the SQL Pools work, patterns for optimising performance and cost and how we can use our SQL endpoints to integrate with other services
- The Synapse Spark engine, demonstrating how you can write dynamic workflows in Python or bring your existing SQL logic to spark directly.
- Data Explorer pools and how you can use it for deep exploration of logs, time series and other fast-moving unstructured data sources
- Synapse Integrations, how you can take your workspace and integrate directly with tools such as Azure Purview, CosmosDB and the wider Dataverse
There is a huge amount to cover, but you'll be guided by Data Platform MVP Simon Whiteley & veteran analytics consultant Zach Stagers, both of whom have deep knowledge across the whole of this wide and sprawling tech stack.
Feedback link https://sqlb.it/?7227
Starts: 09:00 8th Mar 2022
Ends: 17:00 8th Mar 2022
- Azure Synapse Analytics has been steadily maturing and growing since it's release, but it's still hard to know where to start! This introductory training session will give you a grounding in all of the Synapse engines so you'll know your Spark Pool from your SQL Pools and be ready to hit the ground running!
Should delegates bring a laptop with them?
If so what Spec?
- Spec not too important, all workloads will be carried out on the cloud
What software and licences do they need?
- Each delegate should have access to an Azure subscription with at least contributor access to a sandbox resource group and minimum £100 azure credit.
What is the structure of the day?
- The day follows five core modules, each of which consists of theory (slides), practical demos and hands-on labs where appropriate. Agenda: - Introduction to Data Lakes - Lake Zones & Security Design - File Formats, Partitioning & Performance - User Personas & - SQL Pools - Understanding the SQL Pool options & offerings - Loading & Querying Data - Designing for cost & performance - Integration with Power BI - The Synapse Spark Pool - Understanding Spark - Basic Spark Workflows - Engineering Patterns in Pyspark & Spark SQL - Synapse Integrations - Log Exploration with KQL & Data Explorer Pools - Governance & Data Discovery with Azure Purview - Unleashing CosmosDB with Synapse Link
What internet requirements do the delegates have?
- Full wifi throughout the day - speed shouldn't matter too much
What will the delegates get out of the day?
- By the end of the session, delegates will have a complete understanding of when and where to use each component of Synapse Analytics, hands-on practice with the core components and their architecture questions answered.
The SQL Bits Story
SQLBits was formed in 2007 by a group of volunteers who were passionate about the SQL Server product suite and wanted to provide much-needed community-driven education to the data community.
As one of the largest data platform conferences in the world, we offer more opportunities to a wider audience.
We’ve grown and expanded a lot since 2007.
SQLBits is the best place to meet fellow data professionals.
We welcome data professionals from all over the globe.
1140 recorded sessions
All the live sessions are recorded and offered for free, year round.
Experience the SQLBits Conference
Want to be part of the SQLBits community?
Attend the London conference in-person or virtually on