Git Product home page Git Product logo

sqlworkshops-sql2022workshop's Introduction

Workshop: The SQL Server 2022 Workshop

A Microsoft Course from the SQL Server team

About this Workshop

Welcome to this Microsoft solutions workshop on SQL Server 2022 Workshop.

In this course you will learn how to solve modern data challenges with SQL Server 2022 using a hands-on lab approach.

This course is intended to be taken as a self-paced or instructor-led workshop. A supplement slide deck is available for this course in the slides folder.

This course is designed for data professionals who have a basic working knowledge of SQL Server and the T-SQL language.

This README.MD file explains how the workshop is structured, what you will learn, and the technologies you will use in this solution.

Learning Objectives

When you complete this course, you will be able to:

  • Understand how SQL Server 2022 is the most cloud connected version ever.
  • Learn how built-in query intelligence allows your queries to run faster with no code changes
  • Learn how SQL Server 2022 has an industry proven database engine with new innovations in security, scalability, and availability.
  • See new data virtualization capabilities using REST APIs with data sources like s3 compatible providers and see how to backup and restore SQL databases with S3.
  • Learn how to enable new application scenarios using enhancements to the T-SQL language.

Business Applications of this Workshop

  • Enabling hybrid scenarios for disaster recovery, analytics, and security.
  • Ensure your application perform consistently and faster with no code changes.
  • Protect your data integrity with Ledger for SQL Server using blockchain technologies.
  • Save costs for maintenance of tempdb with new scalability improvements.
  • Reduce time for management of high availability with Contained Availability Groups.
  • Access information in data lakes without moving the data.
  • Use new S3 object storage providers for disaster recovery.
  • Maintain your skills for the T-SQL query language with new innovations.

Technologies used in this Workshop

Technology Description
SQL ServerDatabase Platform released and sold by Microsoft
SQL Server 2022The lastest major version of SQL Server
Intelligent Query ProcessingAutomated query processing enhancements in SQL Server 2022
Microsoft AzureMicrosoft's cloud platform for computing, data, and applications.
Query StoreBuilt-in query performance execution statistics stored in a user database
Ledger for SQL ServerBuilt-in tamper evident data integrity using blockchain technologies
PolybaseData Virtualization for data stored outside of SQL Server
SQL Server Management Studio (SSMS)Graphical User Interface Management and Query Tool
Azure Data StudioGraphical User Interface to execute T-SQL queries, notebooks, and manage SQL Server

Before Taking this Workshop

To complete this workshop you will need the following:

  • Before you attend the workshop, download all the scripts and files for hands-on exercises with one of the following methods:
  1. Clone the workshop repo with git clone https://github.com/microsoft/sqlworkshops-sql2022workshop.git. Using this method, the scripts will be under folders by module inside sqlworkshops-sql2022workshop\sql2022workshop. You can download git for windows from https://gitforwindows.org.
  2. Or download a zip file of the scripts from https://github.com/microsoft/sqlworkshops-sql2022workshop/archive/refs/heads/main.zip. You will need to expand the zip file after downloading. Using this method the scripts will be under folders by module inside \sqlworkshops-sql2022workshop-main\sql2022workshop.
  • Setup a machine or VM and install the software and supporting files as listed in the Setup section below

Each module of this workshop can be studied and used independently of each other or taken all as a single set of exercises. The Modules are designed in a sequence but you can use each of them one at a time at your own pace.

Setup

In order to complete the exercises in this workshop you will need the following:

  • A virtual machine or computer running Windows 10, Windows 11, Windows Server 2019, or Windows 2022 with at least 4 CPUs and 8Gb RAM. You will need Administrator rights on the virtual machine or computer. Exercise 3.2 is an advanced exercise in Module 3 that requires 8 CPUs and 24Gb RAM. It may be an optional exercise in some workshop delivery. Check with your instructor..
  • NOTE for Linux or container users: You will be able to go through many of the exercises using Linux as your client and SQL Server on Linux or as a container. However, some exercises require ostress.exe which is a Windows only program. You can also connect with a Windows client to SQL Server on Linux or a SQL Linux container.
  • An Azure subscription is required for Module 2. Check the instructions in each exercise for specific permission requirements. Your instructor will indicate whether you will do Module 2 exercises. Exercises 2.2 and 2.3 in this module cannot be completed using an Azure Virtual Machine (or you set your Azure VM with the following process: https://learn.microsoft.com/azure/azure-arc/servers/plan-evaluate-on-azure-virtual-machine)
  • SQL Server 2022 Evaluation Edition or Developer Edition General Availability from (https://aka.ms/getsqlserver2022) with the Database Engine feature installed. If you use the Developer Edition you must enable the TCP protocol.
    • Exercises in Module 5 require the PolyBase Query Service for External Data Feature (using all defaults). If you have installed SQL Server from an Azure Virtual Machine marketplace image, you will need to add the Polybase feature after deployment. IMPORTANT: If you have an instance from a previous version of SQL Server already on your computer or VM that has the Polybase feature enabled you will have to remove that feature from that instance or uninstall that instance. If this is not possible Module 5 does have T-SQL notebooks for you to view the experience.
    • You must configure SQL Server for mixed mode authentication. Read more how to do this at https://learn.microsoft.com/sql/relational-databases/security/choose-an-authentication-mode.
    • Named instances are supported but you may need to make some edits to some scripts in some modules where the exercise have you connect to SQL Server.
  • Install SQL Server Management Studio (SSMS) latest 18 or 19 build from https://aka.ms/ssms18 or https://aka.ms/ssms19. Several of the modules require features built only into SSMS. You may have a better experience in some modules with new features of SSMS 19.
  • Install Azure Data Studio (ADS) from https://aka.ms/azuredatastudio. T-SQL notebooks are used in this course.
  • Module 3 requires you to restore backups to go through different exercises. Download the backups for these from the following locations. Each exercise has instructions on how and when to restore these backups:
  • Download ostress.exe by downloading RML utilities from https://aka.ms/ostress. Install using the RMLSetup.msi file that is downloaded. Use all defaults. ostress relies on SQLNCLI11. We have seen some situations where this may not be installed. If not, please install SQLNCLI11 from https://www.microsoft.com/en-us/download/details.aspx?id=50402.

Microsoft and any contributors grant you a license to the Microsoft documentation and other content in this repository under the Creative Commons Attribution 4.0 International Public License, see the LICENSE file, and grant you a license to any code in the repository under the MIT License, see the LICENSE-CODE file. All license files are found in the LICENSE directory.

Workshop Details

This workshop uses SQL Server 2022, SQL Server Management Studio, Azure Data Studio, Azure SQL Managed Instance, Azure Synapse Analytics, Azure Active Directory (AAD), and Microsoft Purview you to learn how you can solve new challenges with SQL Server 2022.

Primary Audience:Data professionals looking to understand and use new capabilities of SQL Server 2022
Secondary Audience:Developers, Architects, IT Pros, Data Scientists, and Data Engineers
Level: 300
Type:Self-Paced or Instructor Led
Length: Full Day

Related Workshops

Workshop Modules

This is a modular workshop, and in each section, you'll learn concepts, technologies, and processes to help you complete the solution. This table is provided for you to see the list of modules in the workshop. You can use any module in any order you like but the preferred method is to proceed to Next Steps below to start the workshop.

ModuleTopics
01 - Introduction to SQL Server 2022 Learn how SQL Server 2022 solves challenges for the data professional
02 - Connect SQL Server 2022 to Azure Learn the how SQL Server is a hybrid data platform connecting to Azure services
03 - Accelerate performance with built-in query intelligence Learn how to get faster with no code changes
04 - Power your database with security, scalability, and availabilityLearn the new capabilities of the industry proven SQL Server engine
05 - Access new sources with data virtualization and object storage Learn the new data virtualization and object storage capabilities
06 - Enhance your application with new T-SQL capabilitiesLearn the new T-SQL enhancements for new application scenarios

Next Steps

Next, Continue to Introduction to SQL Server 2022

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

sqlworkshops-sql2022workshop's People

Contributors

microsoft-github-operations[bot] avatar microsoftopensource avatar rgward avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sqlworkshops-sql2022workshop's Issues

ledger table enhancement

Exercise 1: Using an updatable ledger table
Add a step 8.5 : Run the verifyledger.sql to see that the ledger is consistent to this point.
Step 12: Because the ledger hasn't been checkpointed, the same last_verified_block_id is reflected.
Add a new step 13: (same as step 7) Let's generate a new digest by executing the script generatedigest.sql. Save the output value (including the braces) to be used for verifying the ledger.. You will use this in a later step. This provides me a snapshot of the data at a point in time.
Add a new step 14: Run the verifyledger.sql to see that the ledger now reflects the new last_verified_block_id.
Add a new step 15: Remove any character from the hash in verifyledger.sql and re-run the code. Notice the proc states the hash does not match, suggesting tampering.

Exercise 2: Using an append-only ledger table
Script within appchangemaryssalary.sql should be enclosed in a transaction, in case user forgot to create audit table.

Exercise 3: Protecting Ledger tables from DDL changes
Saw a misspelled "syadmin"

Add to Prereqs about Polybase constraints on install

Add to the prereqs that Polybase can be installed only once so that attendees know ahead of time and can make the right decision of how to set up their lab environments.

This addition may save a lot of time and pain for getting the labs done.

Fast Execution on MaxDOP Query

For whatever reason, my queries were running in about 5 seconds and didn't break the 10 second threshold. I'd suggest adding a second XE session so people can quickly identify whether or not their queries are running fast.

delta notebook typo

Step 7 - Query the delta table by using a WHERE clause for a column that is a partiitione column. Not the peformance difference.

Should be "Note" the performance difference

Need a IQP symphony slide

How do all the IQP features work together like PSPO would kick in first and then CE feedback could work against a variant. DOP feedback is just looking at DOP so could use a variant or could still work on a query with CE feedback.

backuprestores3.ipynb - EXIST WHERE clause does not match CREATE for certificate.

Exist check uses IP without port in the WHERE clause, but creates the cert with the port number in the name.

Current:

USE MASTER
GO
IF EXISTS (SELECT * FROM sys.credentials WHERE name = 's3:///backups')
DROP CREDENTIAL [s3://:9000/backups];
GO

Should be:

USE MASTER
GO
IF EXISTS (SELECT * FROM sys.credentials WHERE name = 's3://:9000/backups')
DROP CREDENTIAL [s3://:9000/backups];
GO

Update to Prerequisites

Preqreqs doc says you can use a VM in azure, so I spun up an Azure SQL VM. But it appears that Polybase Query Service is not included in a SQL VM (I did SQL 2022 Developer on Windows Server 2022), and I didn't notice any options to include Polybase when building the VM so I assumed it was just included. Either way it definitely couldn't be enabled.

So if there is in fact no way to enable Polybase when building a SQL VM, I'd note that in the prereqs doc because right now it looks like I should be able to use that.

Consider adding a script for Ledger Module

Consider adding a script or to the script about controlling DDL changes to Ledger Tables. This could show the sorts of "normal" table ALTERs that can be done: certain datatype changes, adding columns, removing columns, etc.

This might help alleviate the concerns by devs and DBAs that Ledger tables are set in stone.

Add Principal to getledgerobjects.sql

In Module 4, Ledger Exercise, Exercise 3: Protecting Ledger tables from DDL changes

Issue: Attendee wanted to know who dropped the table in Exercise 3 (aka which login). The principal login for the user that dropped the table is not in the sys.ledger_table_history DMV.

getledgerobjects.sql queries sys.ledger_table_history and sys.ledger_column_history.

Fix:

add the follwing script which joins sys.ledger_table_history. with sys.database_ledger_transactions in order to get the principal_name column from the sys.database_ledger_transactions.
github screen shot 1

SELECT
t.[principal_name]
, t.[commit_time]
, h.[schema_name] + '.' + h.[table_name] AS [table_name]
, h.[ledger_view_schema_name] + '.' + h.[ledger_view_name] AS [view_name]
, h.[operation_type_desc]
FROM sys.ledger_table_history h
JOIN sys.database_ledger_transactions t
ON h.transaction_id = t.transaction_id

backupdbtos3.sql needs a minor fixup (05_DataVirt/minio/backupdbtos3.sql)

The first backup command will fail:

BACKUP DATABASE WideWorldImporters
TO URL = 's3:/:9000/backups/wwi.bak'
WITH CHECKSUM, INIT, FORMAT;
GO

with this error:

Msg 3078, Level 16, State 1, Line 5
The file name "s3:/10.191.80.74:9000/backups/wwi.bak" is invalid as a backup device name for the specified device type. Reissue the BACKUP statement with a valid file name and device type.
Msg 3013, Level 16, State 1, Line 5
BACKUP DATABASE is terminating abnormally.

We need to add another / before the IP address in the URL specifier.

Exercise PSPOPT - query_store_dispatcher_plan script

The instructions in the PSPOPT README near the end of the script ae as follows:

"Execute the script query_store_dispatcher_plan.sql. If you click on the dispatcher_plan value you will see a graphical plan operator called Multiple Plan."

In the actual script, there is no label "dispatch_plan" - the relevant XML column in the resultset has no label at all.

Also - the resultset for the query returns three columns, but all three values are the same for both plans - I suggest adding the query_variant_query_id column to the query to differentiate the two rows from one another

Thanks for a great session!

Non-Distinct File Names in WWI Backups

Not all of the included wwi backups have unique filenames, preventing all 3 from being restored at the same time. Adjusting the filenames to be unique would make things easier for users to set up.

Typo in Azure Arc exercises

There is a type for Azure Arc exercises in this sentence

est Practices Assessment (BPA) is a service to scan the configuraiton

Issue installing Polybase Feature with 2022 RC1 Eval release

When the SQL installation attempted to install the Polybase feature, I ran across the following error presented in the installer GUI:

"The specified local group does not exist."

Upon further investigation in the Bootstrapper Install logs (detail.txt), the group shown below is apparently a requirement. I had to create this group locally on my workstation for the feature installation to succeed. I am a local admin, the device is domain joined, I have SQL 2017 installed (default instance), but it does not have the polybase feature enabled. Note that the Polybase services failed to install, but to attempt again I had to change the initial installation to remove the feature and attempt reinstallation again. Oddly, although the initial attempts failed, the installer showed that the feature successfully installed.

Detail.txt

(01) 2022-11-13 18:45:52 Slp: SetValue: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL16.RC1\Polybase\Configuration, Name = DMSSvcAcct
(01) 2022-11-13 18:45:52 SQLPolyBase: --RegistrySettings: Wrote registry value DMSSvcAcct = softpro\jsharp...
(01) 2022-11-13 18:45:52 SQLPolyBase: --PolybaseSetupPrivate: Configuring Odbc driver pdwodbc17e -> sqlncli17e.dll
(01) 2022-11-13 18:45:52 SQLPolyBase: --PolybaseSetupPrivate: Creating performance counters: RC1...
(01) 2022-11-13 18:45:58 SQLPolyBase: --PolybaseSetupPrivate: Creating event sources: RC1...
(01) 2022-11-13 18:45:58 SQLPolyBase: --PolybaseSetupPrivate: Creating Access Group
(01) 2022-11-13 18:45:58 Slp: Sco: Attempting to initialize user group PdwComputeNodeAccess
(01) 2022-11-13 18:45:58 Slp: Sco: Attempting to check if user group PdwComputeNodeAccess exists
(01) 2022-11-13 18:45:58 Slp: Sco: Attempting to open root DirectoryEntry object for local computer
(01) 2022-11-13 18:45:58 Slp: Sco: Attempting to check if container 'WinNT://SPRO-L301511,computer' of group exists
(01) 2022-11-13 18:45:58 Slp: Prompting user if they want to retry this action due to the following failure:
(01) 2022-11-13 18:45:58 Slp: ----------------------------------------
(01) 2022-11-13 18:45:58 Slp: The following is an exception stack listing the exceptions in outermost to innermost order
(01) 2022-11-13 18:45:58 Slp: Inner exceptions are being indented
(01) 2022-11-13 18:45:58 Slp:
(01) 2022-11-13 18:45:58 Slp: Exception type: Microsoft.SqlServer.Configuration.Sco.ScoException
(01) 2022-11-13 18:45:58 Slp: Message:
(01) 2022-11-13 18:45:58 Slp: The specified local group does not exist.
(01) 2022-11-13 18:45:58 Slp:
(01) 2022-11-13 18:45:58 Slp: HResult : 0x84bb0001
(01) 2022-11-13 18:45:58 Slp: FacilityCode : 1211 (4bb)
(01) 2022-11-13 18:45:58 Slp: ErrorCode : 1 (0001)
(01) 2022-11-13 18:45:58 Slp: Data:
(01) 2022-11-13 18:45:58 Slp: WatsonData = LOCAL
(01) 2022-11-13 18:45:58 Slp: DisableRetry = true
(01) 2022-11-13 18:45:58 Slp: Inner exception type: System.Runtime.InteropServices.COMException
(01) 2022-11-13 18:45:58 Slp: Message:
(01) 2022-11-13 18:45:58 Slp: The specified local group does not exist.
(01) 2022-11-13 18:45:58 Slp:
(01) 2022-11-13 18:45:58 Slp: HResult : 0x80070560
(01) 2022-11-13 18:45:58 Slp: Stack:
(01) 2022-11-13 18:45:58 Slp: at System.DirectoryServices.DirectoryEntries.Find(String name, String schemaClassName)
(01) 2022-11-13 18:45:58 Slp: at Microsoft.SqlServer.Configuration.Sco.UserGroup.Exists()
(01) 2022-11-13 18:45:58 Slp: ----------------------------------------

Restore Permissions Issues

The 03 Built-in Query Intelligence / pspop lab requires a restore from c:\samples. People are having permissions issues doing the restore. People have successfully changed their sql service account to local system and it worked fine. Other people struggled due to lock-down policies from their IT department.

We could probably prevent this issue by having people copy the backups to the SQL data directory.

dop feedback lab

Step 7, add a note to tell the reader to move on to stop 8 while the workload runs.

Landing Zone Prerequisite

Synapse Link exercise instructions reference a Landing Zone in the Prereqs. However, there do not appear to be any prerequisites for a landing zone (not in the Readme.md on the main page nor in the exercises, nor in the slides).

pspopt demo - need to clear plan cache before re-running demo with perfmon.

current instructions from readme.md for pspopt demo: https://github.com/microsoft/sqlworkshops-sql2022workshop/blob/main/sql2022workshop/03_BuiltinQueryIntelligence/pspopt/readme.md

...

See a PSP problem for a single query execution

  1. Set the actual execution plan option in SSMS. Run query_plan_seek.sql twice in a query window in SSMS. Note the query execution time is fast (< 1 second). Check the timings from SET STATISTICS TIME ON from the second execution. The query is run twice so the 2nd execution will not require a compile. This is the time we want to compare. Note the query plan uses an Index Seek.
  2. In a different query window set the actual execution option in SSMS. Run query_plan_scan.sql in a query windows in SSMS. Note the query plan uses an Clustered Index Scan and parallelism.
  3. Now go back and run query_plan_seek.sql again. Note that even though the query executes quickly (< 1 sec), the timing from SET STATISTICS TIME is significantly longer than the previous execution. Also note the query plan also uses a clustered index scan and parallelism.

See a workload problem for PSP

  1. Setup perfmon to capture % processor time and batch requests/second.
  2. Run workload_index_seek.cmd 10 from the command prompt. This should finish very quickly. The parameter is the number of users. You may want to increase this for machines with 8 CPUs or more. Observe perfmon counters.
  3. Run workload_index_scan.cmd. This should take longer but now locks into cache a plan for a scan.
    ...

Between those two demos, attendees will need to clear the proc cache again so that the second demo does not start by using the parallel scan plan.

CE feedback slide

Add more details on how we identify CE feedback candidates based on estimates vs actual

PSP Optimization minor typo

"Is Parameter Sniffing Gone" slide has a typo on first bullet point.

"Only = operated supported" - guessing you meant "operator"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.