The hardest aspect of Microsoft Ignite was keeping up with all the announcements, especially this year with what seems a focus on the overall state of the enterprise and self-service analytics. This post provides a round-up to the various announcements and sessions in the Data & AI Platform, Power BI, Azure Data Factory, Azure Purview, and associated technologies.
There are also some additional links to resources that I found interesting, such as adding webinar features to Microsoft Teams and a couple of announcements around performance improvements in Excel for the Web. This post is in a series I have dealing with Microsoft Azure learning resources.
This year’s conference had so many Data & AI-related sessions I separated them into the sections below. Most of the session’s descriptions have come directly from the site; however, I have modified some with more detail or commentary.
This section summarizes the links to the various announcements made in the Data & AI Platform.
Azure Book of News
Microsoft does provide a guide that provides as a full summary called Microsoft’s Book of News, which covers the announcements about Microsoft Azure, Security, Microsoft 365, Power Platform and more. Full detail can be found here; Microsoft’s Book of News
Power BI Announcements
Announcing VNet Connectivity for PBI Datasets (Public Preview) – We are excited to announce the public preview of VNet connectivity for Power BI datasets. Protecting your data on the Microsoft cloud and ensuring secure and authorized access to this data has been our key focus. While Azure virtual networks (VNets) offer network isolation and security for your resources on the Microsoft cloud, this VNet connectivity feature enables secure outbound connectivity from Power BI to data sources within an Azure VNet. Please read Data insights without limit, security without compromise for an overview of other new Power BI security capabilities.
Announcing Power BI Premium Per User GA and Premium Gen2 with Autoscale – Power BI empowers every business user and business analyst to get amazing insights with AI-infused experiences. With Power BI Premium, we enable business analysts to analyze and visualize their data and build an end-to-end data platform through drag and drop experiences. Everything from ingesting and transforming data at scale to building automated machine learning models and analyzing massive volumes of data is now possible for our millions of business analysts. Check out all the Power BI Premium and the new capabilities.
New security capabilities in Power BI – With Power BI, customers bring their critical business data to the cloud to empower every individual and every team in their organization to make better decisions with data. Securing and protecting their data is extremely important, especially in a world of remote work. Original announcement here
New Microsoft Power BI security whitepaper – Summary: Power BI is an online software service (SaaS, or Software as a Service) offering from Microsoft that lets you easily and quickly create self-service Business Intelligence dashboards, reports, datasets, and visualizations. With Power BI, you can connect to many different data sources, combine and shape data from those connections, then create reports and dashboards that can be shared with others.
Power BI Embedded Analytics Playground – Learn and explore how you can use embedded analytics in your applications. Site includes; Developer Sandbox, Exploration of APIs, Browse Showcases and real-world examples, Setting up your analytics environment and multiple e-guides and videos. Announcement blog post here.
Power Platform Summary: Announcing new Power Platform capabilities at Microsoft Ignite – The Power Platform team has provided a listing of announcements that includes, not only Power BI, but the full platform.
Learn more about Power BI Premium today and check out all these capabilities – This is the main landing page for Power BI Premium.
Visit the preview announcement blog post: Announcing Preview Availability of Improved Analytics with Next-Generation Power BI Premium and the YouTube Announcement Video – These posts cover a listing of announcements made at Ignite.
Webinars in Teams
Before we get too deep, I enjoyed watching this session, as it reviewed new webinar features released in Microsoft Teams. Being a trainer and creator of content, it is exciting to be able to expand the use of Teams to webinars. Check it out!!
Host webinars in Microsoft Teams | Deep dive on the new presenter and attendee experiences – Host webinars directly from the Microsoft Teams environment and offer unique attendee experiences that help tailor the event to participants’ needs. Teams is now one single and familiar tool for all your meeting types. Online meetings can be extended to webinars and scale-out to a significantly larger audience. As a Teams user, there is no additional expertise required to set up a Team’s webinar. It’s a rich, interactive, and engaging experience for both presenters and attendees, combining voice, chat, video, and of course, live reactions. Troy Batterberry, Microsoft Teams Vice-President, joins host Jeremy Chapman to show you the setup experience before the event, the presenter and attendee experience during the event, and the organizer experience of the event.
Azure Purview Announcements
This section covers Microsoft Purview, which is a new Data Governance tool now in Public Preview. Look to many features and growth as the product moves into general availability. Come back to the Azure Purview Blog for continuing announcements.
Public preview: Register Azure subscriptions and Resource Groups with Azure Purview – Manage your data with greater ease by using Azure Purview to register multiple sources at the same time. Documentation: Manage data sources in Azure Purview (Preview)
Public preview: Azure Purview Data Catalog now supports hierarchical glossary – With this feature, Azure Purview Data Catalog, you can now define the parent-child relationship between terms within the business glossary. This gives you the ability to define the same term name in different parents. E.g., the Customer term can be present in parent Finance as Finance_Customer and in Sales as Sales_Customer with different term templates and custom attributes. Documentation: Understand business glossary features in Azure Purview
Public preview: Scan and classify Amazon AWS S3 data using Azure Purview – Extend your Azure Purview Data Map to include multi-cloud data sources – Azure and Amazon AWS with this feature, now in public preview. Azure Purview customers can now scan and classify data residing in Amazon AWS S3 with the help of automated scanning, AI-powered built-in and custom classifiers, and Microsoft Information Protection sensitivity labels. To get started, visit the blog. Documentation: How To Guide: Azure Purview connector for Amazon S3
Scan and view lineage of data stored in SAP ERP and Oracle DB using Azure Purview – Azure Purview now provides automated scanning and lineage for SAP ECC, SAP S4/HANA and Oracle DB in public preview. Azure Purview now enables customers to know what data exists within their on-premises SAP ERP solutions and Oracle DB with the help of automated scanning in public preview. Customers can also view lineage between assets for these sources.
Public preview: Azure Purview powered search in Azure Synapse Analytics workspaces – Enable effortless discovery of trustworthy, valuable data with an Azure Purview powered search within Synapse Studio. Data scientists and data engineers using Azure Synapse Analytics can now find data they need effortlessly with the help of a native Azure Purview powered search within Synapse Studio. This integration is in public preview. Get started today by connecting your Azure Purview account to Azure Synapse Analytics.
QuickStart: Connect an Azure Purview Account to a Synapse workspace: – In this quickstart, you will register an Azure Purview Account to a Synapse workspace. That connection allows you to discover Azure Purview assets and interact with them through Synapse capabilities.
Register and scan Azure Synapse Analytics – This article discusses how to register and scan an instance of Azure Synapse Analytics (formerly SQL DW) in Purview. Supported capabilities include full and incremental scans to capture the metadata and schema. Scans also classify the data automatically based on system and custom classification rules.
Discover, connect, and explore data in Synapse using Azure Purview – In this document, you will learn the type of interactions that you can perform when registering an Azure Purview Account into Synapse.
Azure Purview available in public preview in South Central US and Canada Central – Azure Purview is now available in the South Central US and Canada Central Azure regions. You can now provision Azure Purview accounts in these regions as a public preview offering.
Azure Data Factory Announcements
A group of Azure Data Factory announcements have been released and available on the following Tech Community Blog post.. The Tech Community is a great resource that provides access to and updated directly from those involved in the product.
Azure Purview and Azure Data Factory Integration – With the integration of Azure Data Factory and Azure Purview, data engineers can quickly & easily find relevant data using a search experience in Data Factory portal, know the data and understand its business value, and easily use them in Data Factory as linked services or datasets. Learn more on how you can use this to understand the lineage for data integration deeply.
Reserved Instance pricing for Data Flows – You can now purchase 1-year or 3-year reservations of Data Flows from the Azure Portal and receive up to 30% off the pay-as-you-go option for General Purpose and Memory Optimized compute options. Learn more about reserved Instance pricing for data flows in Azure Data Factory and Azure Synapse Analytics.
Comman Data Format Format Support with Data Flow – With data flows, you can build powerful ETL processes using CDM formats and then also generate updated manifest files that point to your new, transformed data using CDM as a sink. ADF can use your CDM entity definitions to build ETL projections for transformation and mapping. Learn more about using the Common Data Model format using Azure Data Factory.
Azure Policy Support for Azure Data Factory – Azure Data Factory now includes. Built-in policies that you can assign to your data factory to ensure resource consistency, regulatory compliance, and security. Learn more on how you can use Azure Policy for Azure Data Factory.
Automating Azure Data Factory Deployment – You can now use the validate all and export Azure Resource Manager (ARM) template functionality as part of a CI/CD pipeline. Learn more about Automated publish improvement in ADF’s CI/CD flow
Dual standby Azure SSIS IR pair for high availability – For business continuity and disaster recovery (BCDR), you can now configure a dual standby Azure-SSIS IR pair that works in sync with Azure SQL Database/ Managed Instance failover group.
Azure Database for PostgreSQL in Data Flow – You can now build powerful ETL pipelines that connects to Azure Database for PostgreSQL You can read data directly from Azure Database for PostgreSQL and apply data transformations or join with other data, or post-transformation you can perform inserts, updates, deletes, and upserts to publish the analytics result set into Azure Database for PostgreSQL. Learn more about Azure Database for PostgreSQL connector.
REST Connector as Sink – Generic REST connector is one of the most commonly used connectors in ADF to integrate with various SaaS applications or your own applications. It helps you easily extend and reach a significant number of data sources than the built-in ones. Now using copy activity, not only can you use REST connector to retrieve data, ADF newly enables REST connector as sink so you can also write data via REST APIs. Learn more about REST connector.
Using Worksheet Index for Excel – When parsing Excel files using copy activity or data flow, you can specify the worksheet index. Learn more about Excel format
Delta Lake Format support – Delta Lake is a Spark-based data lake format that makes working with data in your lake for analytics solution super-easy. With ADF data flows, you can read from Delta Lake folders, transform data, and even update, upsert, insert, delete, and generate new Delta Lake folders using the Delta Lake sink format. Learn more about using the Delta format using Azure Data Factory.
Let’s not forget Excel !!
Even with PowerBI, I find myself recommending and using Excel and Office on the Web. Great to see the enhancements announced this week.
What’s New in Excel for the web – The team announced several additional features to help you better navigate and manipulate your Excel files in a browser, including:
- Easier worksheet navigation
- Multiple range selection
- Zoom in/out
- Excel keyboard shortcuts by default
- Version history
- Regional settings
Performance Improvements in Excel for the Web – With an uncompromising goal to provide our customers with the ideal, fast, and smooth user experience, we’re excited to announce the latest significant performance improvements in Excel for the web. We have addressed and improved several scenarios, including opening workbooks, navigating within a workbook, and other interactions.
Links and summary of Ignite Sessions Published to Youtube
Many of these sessions can be found on YouTube under the Microsoft Ignite YouTube Channel. When on The Ignite site, these sessions also provide access to the slide decks, transcripts, links to previous years and other downloadable resources. These are not all the sessions, but those Data & AI of interest.
Azure SYnapse Analytics: Experience a new class of data analytics | LRN242 – Azure Synapse Analytics is a limitless analytics service that brings together data integration, enterprise data warehousing and big data analytics. Azure Synapse brings these worlds together with a unified experience to ingest, explore, prepare, manage and serve data for immediate BI and machine-learning needs. In this session, you’ll get an introduction to when, why and how to get into this area of expertise.
Career Guide:How to be a Data Scientist on Azure | LRN236 – Ever wonder what it means to be a data scientist? By attending this session, you will how data scientists explore and analyze data, and use it to create predictive machine learning models. You’ll also learn how Azure Machine Learning enables data scientists to manage experiments and models through the whole lifecycle from data preparation to production.
Power BI:Using Power BI reports from an end-user perspective (2021) – Not an Ignite session, but an outstanding follow-up. Just an end-user of Power BI? Consuming reports and not authoring at all? Adam walks through how to navigate and use Power BI reports. Jump in and get familiar with reports in Power BI!
Practice Building: Drive a data culture with Power BI and Microsoft Teams | OD351 – The way we work is changing dramatically. It’s more connected, collaborative, and often done remotely. Organizations need tools to help everyone infuse data into every decision. Learn how Power BI and Microsoft teams make it easier to discover insights and collaboratively make decisions based on data. You’ll learn how to easily access and visualize data without leaving Teams, collaborate with colleagues to find and share insights and foster a culture of using data to make decisions.
Power BI:Build systems of intelligence with Power BI and Azure Data | OD352 – Bringing business intelligence to thousands of users in the enterprise requires prepping data and navigating the most complex datasets in the organization at scale. It requires building and distributing insights to those who need it the most without compromising governance, performance and security. Join us in this session to learn how Power BI integrates with Azure Synapse Analytics, Azure Purview and other Azure services to help organizations build and manage a data culture at scale.
Power BI:Embed Power BI analytics in web portals and apps for your organization and customers | OD353 – Power BI embedded empowers application developers within organizations to discover and share valuable insights by embedding visual analytics in their products to gain valuable insights. Join this session to learn how in few easy steps, you can embed Power BI visuals in web portals and apps to adopt decades of analytics expertise in minutes.
Practice Building:Ask the Experts: Drive a data culture with Power BI: Vision Strategy and roadmap | ATE-FS183-R2 – Power BI can help you drive a data culture. Learn why 97% of the Fortune 500 use Power BI today and customers large and small are standardizing on Power BI. We will cover our vision and roadmap. You will see how AI automatically finds patterns in your data. You will learn about how Power BI is built into Teams and Excel to enable incredible collaboration experiences and how Power BI with Synapse meets your most demanding needs. We also have a few exciting new announcements to share!
RoadMap:Ask the Experts: What’s new in Azure Data | ATE-FS174 – Right after Azure Data’s CVP, Rohan Kumar, announces the latest innovations, join members of the Azure Data product team to get your questions answered! Whether it is about what’s new, what’s coming, or something specific, experts will be live to answer your questions in real-time.
Azure Purview:Enable unified data governance with Azure Purview | OD337 – Azure Purview is a unified data governance service that helps you manage and govern your on-premises, multi-cloud, and software-as-a-service (SaaS) data. Get the latest product news and view demos. See how Azure Purview can help you easily create a holistic, up-to-date map of your data landscape with automated data discovery, sensitive data classification, and end-to-end data lineage that will empower data consumers to find valuable trustworthy data.
Data Architecture:A Data Architecture for Cloud-Native Computing | OD413 – Developers need to use the services that best meet their needs, which may result in a “best-of-breed” approach across cloud vendors. This then leads to many complex, unreliable and difficult to debug pipelines and stovepipes to ensure the right data is in the right place. This talk will cover an architectural blueprint for the solution to deliver cloud-native data solutions.
Road Map Q&A:Ask the Experts: What’s new and what’s next for the Microsoft Power Platform | ATE-FS184 – Join this Ask the Experts session to discuss and go deeper into the new features and announcements covered in Charles Lamanna and Julie Strauss’s Featured Session “What’s New and What’s Next for the Microsoft Power Platform.”
Azure Databricks:How Azure Databricks and Delta Lake enabled city-wide data improvements and cost | OD420 – Learn how Azure Databricks and Delta Lake enable a modern data architecture for data engineering, data science, analytics and machine learning. See how the City of Spokane reduced their ELT/ETL total cost of ownership (TCO) by 50% while improving data quality.
Data Engineering:AXA XL Improves Information Availability and Reliability Through Data Virtualization | OD423 – How do you ensure the delivery of consistent and reliable information to the right audience in a timely manner, especially when your data exists in heterogeneous platforms? Moreover, how do you scale systems to provide the appropriate level of performance and availability? AXA XL will share with you their vision of how the reusable components from a data virtualization layer and the flexibility of cloud technologies combine to achieve this elusive goal efficiently and effectively. There is also a case study available, AXA XL Cloud Migration Case Study.
RealTime AI:When Real-Time Insights are Critical Bring AI Capabilities to Your Network Edge | OD432 – Join Cellnex Telecom and Lenovo to explore how intelligence at the edge complements the intelligence in your data center to enable real-time decision making. Lenovo’s PC, mobile, and data center solutions combine to deliver our edge-computing portfolio with a ‘fit anywhere’ form factor, tamper-proof security, built-in AI, and zero-touch manageability.
Data Engineering:Steps for a Successful Database Migration | OD451 – The global pandemic accelerated plans for cloud computing and digital transformation. But database migration—the process of moving a database from one place to another—is no walk in the park. In this session, we’ll review how to explore your data, gather the right performance metrics, and track costs going forward. We’ll also show you tools to accelerate your migration and provide tips to help ensure a smooth process.
AI:AI at Scale: Behind the Scenes | OD382 – What if you could eliminate the barriers to accelerating innovation? The imagination only limits the possibilities for your organization. In this virtual “whiteboard” session David Carmona, GM, AI & Innovation Marketing gives you a behind the scenes look at AI at scale; what it looks like today and where it’s going so you can prepare your organization to be ready as access to state-of-the-art compute resources and multi-purpose models become reality.
Azure Purview:Azure Purview: Take charge of data governance across your digital landscape | OD358 – With the pandemic fast-tracking remote work and continued proliferation of data, companies need to adapt data governance strategies. In this session, you will learn how to use Azure Purview and Microsoft Information Protection across your digital estate.
Career Building:5 Takeaways to Ignite your Creative Edge | OD401 – Would you like your work to be more creative, productive AND have more fun? Discover 5 tips to Ignite your Creative Edge from experts across global organizations. Examples include the excitement of Springboard, leveraging technologies like Azure Arc, how McDonald’s kept their edge in Europe, railway companies setting up a “hackatrain” and a corporate giant becoming more agile. There’ll be plenty of food for thought and five ways to boost your creativity and transform your working life.
Power Platform:What’s new and what’s next for the Microsoft Power Platform | FS184 – The Microsoft Power Platform is an industry-leading low-code development platform rich with capabilities that enable greater development speed and effectiveness for business users, IT Pros and coders. In this session, we’ll demo various capabilities available, cover customer success stories, new features, and the roadmap for later this year. Also, learn how Power Apps, Power Automate and Power Virtual Agent – rapidly enable digitization and automation with a modern set of capabilities.
SQL DevelopmentSQL & Azure Functions: Recipe for Modern Applications: .NET Azure SQL Functions Geospatial JSON | OD328 – Join us to learn the recipe for modern applications in Azure. Through a relatable and real-world scenario, you’ll learn about Azure SQL Database’s built-in capabilities related to JSON, Geospatial, scale, and CI/CD, and how it can help you build modern applications with .NET and Azure App Service. We’ll also show you how to take advantage of other Azure Services, like Azure Functions.
Azure Synapse Analytics:Building end-to-end analytics solutions with a unified | OD333 – Azure Synapse Analytics is a limitless analytics service that brings together data integration, enterprise data warehousing, and big data analytics. It gives you the freedom to query data on your terms, using either serverless or dedicated resources—at scale. Azure Synapse brings these worlds together with a unified experience to ingest, explore, prepare, manage, and serve data for immediate BI and machine learning needs.
Industry:Introducing Microsoft Cloud for Nonprofit | FS187 – I have always had a soft spot for nonprofit organizations. We are lucky to be in a thriving industry and we should all give back. Microsoft Cloud for Nonprofit connects the intelligent and integrated cloud capabilities of the Microsoft stack to the most common nonprofit organizational scenarios, including constituent engagement and fundraising, staff collaboration and volunteer management, and program design and delivery. Bringing together disparate data sources from across the organization enables nonprofits to realize the true value of their data to help accelerate mission outcomes.
RoadMap:Drive a data culture with Power BI: Vision Strategy and roadmap | FS183 – Power BI can help you drive a data culture. Learn why 97% of the Fortune 500 use Power BI today and customers large and small are standardizing on Power BI. We will cover our vision and roadmap. You will see how AI automatically finds patterns in your data. You will learn about how Power BI is built into Teams and Excel to enable incredible collaboration experiences and how Power BI with Synapse meets your most demanding needs. We also have a few exciting new announcements to share!
Architecture:Go Limitless: with Azure Data & AI | FS174 – Azure Data + AI services offer limitless scale, performance and possibilities for what you can achieve. Join Rohan Kumar, Corporate Vice President, Azure Data, as he demonstrates the latest innovations. Come see how Azure is redefining hybrid data, simplifying your migration experience, enabling limitless analytics, democratizing AI, and enabling data governance wherever it lives.
Session on the Ignite Site
These sessions below are a must-watch, however, I could not find the released version on YouTube.
Power BI Roadmap Featured session: Drive a data culture with Power BI: Vision, Strategy, and Roadmap – https://techcommunity.microsoft.com/t5/azure-data-factory/bootstrap-etl-process-by-bringing-azure-purview-assets-into/ba-p/2149194
- Ignite Session -OD351: Drive a data culture with Power BI and Microsoft Teams
- Ignite Session -OD352: Build systems of intelligence with Power BI and Azure Data
- Ignite Session -OD353: Embed Power BI analytics in web portals and apps, for your organization and customers
- Ignite Session -OD354: Turbocharge Dynamics 365 with the Power Platform
Before I wrap up, the following links are to other resources that I came across during the sessions.
MICROSOFT LEARN FOR IGNITE – Continue your learning with Microsoft – Learning doesn’t end just because the show is over. Find your way to deeper content, training options, communities and certification details across all Microsoft cloud solutions right here.
Updated and Expanded Power BI guidance documentation Portal – Power BI guidance documentation provides best practice information from the team that builds Power BI and the folks that work with our enterprise customers. Here you’ll find learnings to improve performance and success with Power BI. We’ll update and add to them as new information is available.
Power BI Course Material – Self Learn or Trainer
Power BI instructor-led training – These workshops and classes are available to our partners and community leads for free to help them train their customers and communities on Power BI. The following sessions are available and include all materials—presenter deck, demo scripts, step-by-step, hands-on lab, dataset, and other content;
- Dashboard in a Day (DIAD)
- Advanced Visualization with Power BI
- Advanced Modeling with Power BI
- Advanced Data Shaping with Power BI
- Administrator in a Day
- Paginated Reports in a Day
- Developer in a Day
- Modern Excel Analyst in a day
- Leap into Power BI
Dax.DO – Code, run, and share DAX online – Write your DAX queries, try them and share them easily with DAX.do. DAX is the query language that powers Microsoft products such as Power BI, Analysis Services, and Power Pivot for Excel. Browse the full language reference on DAX Guide.
Introduction to Azure Purview
Learning Path: Introduction to Azure Purview – With the announcement of Azure Purview, it is a great time to review some of the key features and provide a learning path made up of free online resources to help you get up to speed. This article will take you through an overview of Azure Purview by setting the stage focusing on Data Governance and providing links to the next steps.
OneNote Touch Disappears when using dual monitors
Solved: Microsoft OneNote Touch Not Working When Using Dual Screens – I solved an issue I was having when using Microsoft OneNote on my Surface Laptop with a touch screen when using a secondary, non-touch monitor. I discovered a solution or is a workaround that solved the issue. Microsoft OneNote is one of those productivity tools I cannot live without in my study and work life.
Microsoft Exam Renewal
New Microsoft Certification Renewal Process – One of the main methods technical professionals show their understanding of a subject has been through an intense exam and certification process. Microsoft is changing their certification renewal process to keep up and make renewals easier and cheaper to help you keep up to date. Let’s review some of these changes.
Making Sense of Microsoft Role-Based Certification Process and Resources – The key to certification success is to make sure you have the current requirements and training sites to have the best chance of passing the exam. This article reviews where to learn about the Microsoft exam structure, the program’s role-based nature, and the material provided.
Is your certification expiring soon? Renew it for free today! – Today, we’re proud to announce that 19 certifications are now available to renew on Microsoft Learn, with additional certifications following shortly. Within six months of your role-based or specialty certification expiring, you’re eligible to renew it by passing an online assessment – at no cost and on your own schedule.
Announcement: Free certification renewals
Stay current with in-demand skills through free certification renewals – New Update From Microsoft – March 01, 2021 – We’re excited to announce a couple of upcoming updates to our certification program to help our customers and partners keep pace with technology. We’re providing our certified professionals with a method to renew their Microsoft Certifications, initially earned by passing the rigorous exam(s).