Git Product home page Git Product logo

sqlmonitor's Introduction

SQLMonitor - Baseline SQL Server with PowerShell & Grafana

If you are a developer, or DBA who manages Microsoft SQL Servers, it becomes important to understand current load vs usual load when SQL Server is slow. This repository contains scripts that will help you to setup baseline on individual SQL Server instances, and then visualize the collected data using Grafana through one Inventory server with Linked Server for individual SQL Server instances.

Navigation

Why SQLMonitor?

SQLMonitor is designed as open-source tool to replace expensive enterprise monitoring or to simply fill the gap and monitor all environments such as DEV, TEST, QA/UAT & PROD.

YouTube Tutorial on SQLMonitor

Features

  • Simple & customizable as metric collection happens through SQL Agent jobs.
  • Easy to debug since entire SQLMonitor tools is built of just few tables, stored procedures & sql agent jobs.
  • Grafana based Central & Individual dashboards to analyze metrics
  • Collection jobs using stored procedures with data loading utilizing very small sized perfmon/xevent files puts very minimal performance overhead.
  • Highly optimized grafana dashboard queries using dynamically Parameterized tsql makes the data visualization to scale well even when dashboard users increase.
  • Near to zero manual configuration required. Purging controlled through just one table/job.
  • Depending on version of SQL Server, tables are automatically "Hourly" partitioned & Compressed. So index or other maintenance not even required.
  • Utilizing Memory Optimized tables on central server for core stability metric storage gives it Unlimited scalability.
  • Tools has capability to allow same or different sql instance as Data Target. Thus gives high flexibility & scalability.
  • Works with all supported SQL Servers (with some limitations on 2008R2 like XEvent not available).
  • Utilizing Grafana Unified Alerting gives flexibility to create meaningful alerts.

Live Dashboard - Basic Metrics

You can visit https://ajaydwivedi.ddns.net:3000 for live dashboard for basic real time monitoring.


Live Dashboard - Perfmon Counters - Quest Softwares

Visit https://ajaydwivedi.ddns.net:3000 for live dashboard of all Perfmon counters suggested in SQL Server Perfmon Counters of Interest - Quest Software.


Portal Credentials

Database/Grafana Portal User Name Password
https://ajaydwivedi.ddns.net:3000/ guest ajaydwivedi-guest
Sql Instance -> ajaydwivedi.ddns.net:1433 grafana grafana

How to Setup

SQLMonitor supports both Central & Distributed topology. In preferred distributed topology, each SQL Server instance monitors itself. The required objects like tables, view, functions, procedures, scripts, jobs etc. are created on the monitored instance itself.

SQLMonitor utilizes PowerShell script to collect various metric from operating system including setting up Perfmon data collector, pushing the collected perfmon data to sql tables, collecting os processes running etc.

For collecting metrics available from inside SQL Server, it used standard tsql procedures.

All the objects are created in [DBA] databases. Only few stored procedures that should have capability to be executed from context of any databases are created in [master] database.

For both OS metrics & SQL metric, SQL Agent jobs are used as schedulers. Each job has its own schedule which may differ in frequency of data collection from every one minute to once a week.


Jobs for SQLMonitor

Following are few of the SQLMonitor data collection jobs. Each of these jobs is set to (dba) SQLMonitor job category along with fixed naming convention of (dba) *********.

SQLAgent Jobs Created by SQLMonitor
Job Name Job Category Schedule Job Type Location
(dba) Check-InstanceAvailability (dba) SQLMonitor Every 1 minute PowerShell Inventory Server
(dba) Get-AllServerInfo (dba) SQLMonitor Every 1 minute TSQL Inventory Server
(dba) Get-AllServerCollectedData (dba) SQLMonitor Every 5 minute TSQL Inventory Server
(dba) Update-SqlServerVersions (dba) SQLMonitor Once a week PowerShell Inventory Server
(dba) Collect-PerfmonData (dba) SQLMonitor Every 2 minute PowerShell PowerShell Jobs Server
(dba) Check-SQLAgentJobs (dba) SQLMonitor Every 5 minute TSQL Tsql Jobs Server
(dba) Collect-AgHealthState (dba) SQLMonitor Every 2 minute TSQL Tsql Jobs Server
(dba) Collect-DiskSpace (dba) SQLMonitor Every 30 minutes PowerShell PowerShell Jobs Server
(dba) Collect-FileIOStats (dba) SQLMonitor Every 10 minute TSQL Tsql Jobs Server
(dba) Collect-MemoryClerks (dba) SQLMonitor Every 2 minute TSQL Tsql Jobs Server
(dba) Collect-OSProcesses (dba) SQLMonitor Every 2 minute PowerShell PowerShell Jobs Server
(dba) Collect-PrivilegedInfo (dba) SQLMonitor Every 10 minute TSQL Tsql Jobs Server
(dba) Collect-WaitStats (dba) SQLMonitor Every 10 minutes TSQL Tsql Jobs Server
(dba) Collect-XEvents (dba) SQLMonitor Every minute TSQL Tsql Jobs Server
(dba) Partitions-Maintenance (dba) SQLMonitor Every Day TSQL Tsql Jobs Server
(dba) Purge-Tables (dba) SQLMonitor Every Day TSQL Tsql Jobs Server
(dba) Remove-XEventFiles (dba) SQLMonitor Every 4 hours PowerShell PowerShell Jobs Server
(dba) Run-Blitz (dba) SQLMonitor Once a Week TSQL Tsql Jobs Server
(dba) Run-BlitzIndex (dba) SQLMonitor Every Day TSQL Tsql Jobs Server
(dba) Run-BlitzIndex - Weekly (dba) SQLMonitor Once a Week TSQL Tsql Jobs Server
(dba) Run-LogSaver (dba) SQLMonitor Every 5 minutes TSQL Tsql Jobs Server
(dba) Run-TempDbSaver (dba) SQLMonitor Every 5 minutes TSQL Tsql Jobs Server
(dba) Run-WhoIsActive (dba) SQLMonitor Every 2 minute TSQL Tsql Jobs Server

PowerShell Jobs Server can be same SQL Instance that is being baselined, or some other server in same Cluster network, or some some other server in same network, or even Inventory Server.

Tsql Jobs Server can be same SQL Instance that is being baselined, or some other server in same Cluster network, or some some other server in same network, or even Inventory Server.

Download SQLMonitor

Download SQLMonitor repository on your central server from where you deploy your scripts on all other servers. Say, after closing SQLMonitor, our local repo directory is D:\Ajay-Dwivedi\GitHub-Personal\SQLMonitor.

If the local SQLMonitor repo folder already exists, simply pull the latest from master branch.

Execute Wrapper Script

Create a directory named Private inside SQLMonitor, and copy the scripts of SQLMonitor\Wrapper-Samples\ into SQLMonitor\Private\ folder. Open the script D:\Ajay-Dwivedi\GitHub-Personal\SQLMonitor\Private\Wrapper-InstallSQLMonitor.ps1. Replace the appropriate values for parameters, and execute the script.

Below is sample code present in Wrapper-Samples/Wrapper-InstallSQLMonitor.ps1

Wrapper-Samples/Wrapper-InstallSQLMonitor.ps1

#$DomainCredential = Get-Credential -UserName 'Lab\SQLServices' -Message 'AD Account'
#$saAdmin = Get-Credential -UserName 'sa' -Message 'sa'
#$localAdmin = Get-Credential -UserName 'Administrator' -Message 'Local Admin'

cls
Import-Module dbatools;
$params = @{
    SqlInstanceToBaseline = 'Workstation'
    DbaDatabase = 'DBA'
    #HostName = 'Workstation'
    #RetentionDays = 7
    DbaToolsFolderPath = 'D:\Github\dbatools' # Download using Save-Module command
    #FirstResponderKitZipFile = 'D:\Softwares\SQL-Server-First-Responder-Kit-20231010.zip' # Download from Releases section
    #DarlingDataZipFile = 'D:\Softwares\DarlingData-main.zip' # Download from Code dropdown    
    #OlaHallengrenSolutionZipFile = 'D:\Github\sql-server-maintenance-solution-master.zip' # Download from Code dropdown
    #RemoteSQLMonitorPath = 'C:\SQLMonitor'
    InventoryServer = 'SQLMonitor'
    InventoryDatabase = 'DBA'
    DbaGroupMailId = '[email protected]'
    #SqlCredential = $personal
    #WindowsCredential = $DomainCredential
    <#
    SkipSteps = @( "1__sp_WhoIsActive", "2__AllDatabaseObjects", "3__XEventSession",
                "4__FirstResponderKitObjects", "5__DarlingDataObjects", "6__OlaHallengrenSolutionObjects",
                "7__sp_WhatIsRunning", "8__usp_GetAllServerInfo", "9__CopyDbaToolsModule2Host",
                "10__CopyPerfmonFolder2Host", "11__SetupPerfmonDataCollector", "12__CreateCredentialProxy",
                "13__CreateJobCollectDiskSpace", "14__CreateJobCollectOSProcesses", "15__CreateJobCollectPerfmonData",
                "16__CreateJobCollectWaitStats", "17__CreateJobCollectXEvents", "18__CreateJobCollectFileIOStats",
                "19__CreateJobPartitionsMaintenance", "20__CreateJobPurgeTables", "21__CreateJobRemoveXEventFiles",
                "22__CreateJobRunLogSaver", "23__CreateJobRunTempDbSaver", "24__CreateJobRunWhoIsActive",
                "25__CreateJobRunBlitzIndex", "26__CreateJobRunBlitz", "27__CreateJobRunBlitzIndexWeekly",
                "28__CreateJobCollectMemoryClerks", "29__CreateJobCollectPrivilegedInfo", "30__CreateJobCollectAgHealthState",
                "31__CreateJobCheckSQLAgentJobs", "32__CreateJobUpdateSqlServerVersions", "33__CreateJobCheckInstanceAvailability",
                "34__CreateJobGetAllServerInfo", "35__CreateJobGetAllServerCollectedData", "36__WhoIsActivePartition",
                "37__BlitzIndexPartition", "38__BlitzPartition", "39__EnablePageCompression",
                "40__GrafanaLogin", "41__LinkedServerOnInventory", "42__LinkedServerForDataDestinationInstance",
                "43__AlterViewsForDataDestinationInstance")
    #>
    #OnlySteps = @( "2__AllDatabaseObjects", "29__CreateJobCollectAgHealthState" )
    #StartAtStep = '1__sp_WhoIsActive'
    #StopAtStep = '39__AlterViewsForDataDestinationInstance'
    #DropCreatePowerShellJobs = $true
    #DryRun = $false
    #SkipRDPSessionSteps = $true
    #SkipPowerShellJobs = $true
    #SkipTsqlJobs = $true
    #SkipMailProfileCheck = $true
    #skipCollationCheck = $true
    #SkipWindowsAdminAccessTest = $true
    #SkipDriveCheck = $true
    #SkipPingCheck = $true
    #SkipMultiServerviewsUpgrade = $false
    #ForceSetupOfTaskSchedulerJobs = $true
    #SqlInstanceAsDataDestination = 'Workstation'
    #SqlInstanceForPowershellJobs = 'Workstation'
    #SqlInstanceForTsqlJobs = 'Workstation'
    #ConfirmValidationOfMultiInstance = $true
    #ConfirmSetupOfTaskSchedulerJobs = $true
    #HasCustomizedTsqlJobs = $true
    #HasCustomizedPowerShellJobs = $true
    #OverrideCustomizedTsqlJobs = $false
    #OverrideCustomizedPowerShellJobs = $false
    #UpdateSQLAgentJobsThreshold = $false
    #XEventDirectory = 'D:\MSSQL15.MSSQLSERVER\XEvents\'
    #JobsExecutionWaitTimeoutMinutes = 15
}


#$preSQL = "EXEC dbo.usp_check_sql_agent_jobs @default_mail_recipient = '[email protected]', @drop_recreate = 1"
#$postSQL = Get-Content "D:\GitHub-Personal\SQLMonitor\DDLs\Update-SQLAgentJobsThreshold.sql"
#D:\GitHub\SQLMonitor\SQLMonitor\Install-SQLMonitor.ps1 @Params #-Debug -PreQuery $preSQL -PostQuery $postSQL
D:\GitHub\SQLMonitor\SQLMonitor\Install-SQLMonitor.ps1 @Params

#Get-Help F:\GitHub\SQLMonitor\SQLMonitor\Install-SQLMonitor.ps1 -ShowWindow

<#
$dropWhoIsActive = @"
if object_id('dbo.WhoIsActive_Staging') is not null
	drop table dbo.WhoIsActive_Staging;

if object_id('dbo.WhoIsActive') is not null
	drop table dbo.WhoIsActive;
"@;
F:\GitHub\SQLMonitor\SQLMonitor\Install-SQLMonitor.ps1 @Params -PreQuery $dropWhoIsActive
#>

<#
# **************** Download other github repos/modules/files ***********************

# **__ SQLMonitor __**
Invoke-WebRequest https://github.com/imajaydwivedi/SQLMonitor/archive/refs/heads/dev.zip `
            -OutFile "$($env:USERPROFILE)\Downloads\sqlmonitor.zip"

# **__ dbatools & dbatools.library __**
Save-Module dbatools -Path "$($env:USERPROFILE)\Downloads\"

# **__ PoshRSJob on Inventory __**
Install-Module PoshRSJob -Scope AllUsers -Verbose
Save-Module PoshRSJob -Path "$($env:USERPROFILE)\Downloads\"

# **__ Darling Data __**
Invoke-WebRequest https://github.com/erikdarlingdata/DarlingData/archive/refs/heads/main.zip `
            -OutFile "$($env:USERPROFILE)\Downloads\DarlingData-main.zip"

# **__ Ola Hallengren Maintenance Solution __**
Invoke-WebRequest https://github.com/olahallengren/sql-server-maintenance-solution/archive/refs/heads/master.zip `
            -OutFile "$($env:USERPROFILE)\Downloads\sql-server-maintenance-solution-master.zip"

# **__ First Responder Kit from latest release __**
if ($true) {
    $repo = "BrentOzarULTD/SQL-Server-First-Responder-Kit"
    $tags = "https://api.github.com/repos/$repo/tags"

    $tagName = (Invoke-WebRequest $tags | ConvertFrom-Json)[0].name
    $releaseZip = "https://github.com/$repo/archive/refs/tags/$tagName.zip"

    Invoke-WebRequest $releaseZip `
            -OutFile "$($env:USERPROFILE)\Downloads\SQL-Server-First-Responder-Kit-$tagName.zip"
}

# **__ PoshRSJob - Download from Github __**
if ($true) {
    $repo = "proxb/PoshRSJob"
    $releases = "https://api.github.com/repos/$repo/releases"

    $tagName = (Invoke-WebRequest $releases | ConvertFrom-Json)[0].tag_name
    $releaseZip = "https://github.com/$repo/releases/download/$tagName/PoshRSJob.zip"

    Invoke-WebRequest $releaseZip `
            -OutFile "$($env:USERPROFILE)\Downloads\PoshRSJob.zip"
}

#>

<#
Get-DbaDbMailProfile -SqlInstance '192.168.56.31' -SqlCredential $personalCredential
Copy-DbaDbMail -Source '192.168.56.15' -Destination '192.168.56.31' -SourceSqlCredential $personalCredential -DestinationSqlCredential $personalCredential # Lab
New-DbaCredential -SqlInstance 'xy' -Identity $LabCredential.UserName -SecurePassword $LabCredential.Password -Force # -SqlCredential $SqlCredential -EnableException
New-DbaAgentProxy -SqlInstance 'xy' -Name $LabCredential.UserName -ProxyCredential $LabCredential.UserName -SubSystem PowerShell,CmdExec

Enable-PSRemoting -Force -SkipNetworkProfileCheck # remote machine
Set-Item WSMAN:\Localhost\Client\TrustedHosts -Value SQLMonitor.Lab.com -Concatenate -Force # remote machine
Get-ItemProperty HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System -Name LocalAccountTokenFilterPolicy
Set-ItemProperty HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System -Name LocalAccountTokenFilterPolicy -Value 1

# Incase 
#Set-Item WSMAN:\Localhost\Client\TrustedHosts -Value * -Force # run on local machine
#Set-NetConnectionProfile -NetworkCategory Private # Execute this only if above command fails

Enter-PSSession -ComputerName '192.168.56.31' -Credential $localAdmin -Authentication Negotiate
Test-WSMan '192.168.56.31' -Credential $localAdmin -Authentication Negotiate

Get-ChildItem C:\SQLMonitor -Recurse -File | Unblock-File -Verbose
#>

<#
# Add SQLAgent Service Account to below local windows groups.
    # Computer Management > System Tools > Local Users and Groups > Groups
1) Administrators
2) Performance Log Users
3) Performance Monitor Users
#>


Important

To get a better understand of SQLMonitor installation, I would recommend to watch YouTube Playlist https://ajaydwivedi.com/youtube/sqlmonitor.

Setup Grafana Dashboards

Download Grafana which is open source visualization tool. Install & configure same.

Create a datasource on Grafana that connects to your Inventory Server. Say, we set it with name 'SQLMonitor'. Use grafana as login & password while setting up this data source. The grafana sql login is created on each server being baselined with db_datareader on DBA database.

At next step, import all the dashboard *.json files on path D:\Ajay-Dwivedi\GitHub-Personal\SQLMonitor\Grafana-Dashboards into SQLServer folder on grafana portal. While importing each JSON file, we need to explicitly choose SQLMonitor Data Source & Folder we created in above steps.

Remove SQLMonitor

Similar to Wrapper-InstallSQLMonitor, we have Wrapper-RemoveSQLMonitor that can help us remove SQLMonitor for a particular baselined server. Ensure that all scripts from folder \SQLMonitor\Wrapper-Samples\ are copied into \SQLMonitor\Private\ folder.

Open script D:\Ajay-Dwivedi\GitHub-Personal\SQLMonitor\Private\Wrapper-RemoveSQLMonitor.ps1. Replace the appropriate values for parameters, and execute the script.

Below is sample code present in Wrapper-Samples/Wrapper-RemoveSQLMonitor.ps1

Wrapper-Samples/Wrapper-RemoveSQLMonitor.ps1

#$DomainCredential = Get-Credential -UserName 'Lab\SQLServices' -Message 'AD Account'
#$saAdmin = Get-Credential -UserName 'sa' -Message 'sa'
#$localAdmin = Get-Credential -UserName 'Administrator' -Message 'Local Admin'

cls
Import-Module dbatools;
$params = @{
    SqlInstanceToBaseline = 'Experiment'
    #DbaDatabase = 'DBA'
    #HostName = 'Experiment'
    InventoryServer = 'SQLMonitor'
    InventoryDatabase = 'DBA'
    #RemoteSQLMonitorPath = 'C:\SQLMonitor'
    #SqlCredential = $saAdmin
    #WindowsCredential = $localAdmin
    #SkipRDPSessionSteps = $true
    #SkipSteps = @("43__RemovePerfmonFilesFromDisk")    
    #StartAtStep = '30__DropLogin_Grafana'
    #StopAtStep = '11__RemoveJob_RunBlitzIndex'
    #SqlInstanceForTsqlJobs = 'Experiment\SQL2019'
    #SqlInstanceAsDataDestination = 'Experiment\SQL2019'
    #SqlInstanceForPowershellJobs = 'Experiment\SQL2019'
    SkipDropTable = $true
    #SkipRemoveJob = $true
    #SkipDropProc = $true
    #SkipDropView = $true
    #ConfirmValidationOfMultiInstance = $true
    #ActionType = "Update"
    #OnlySteps = @("16__RemoveJob_RunBlitz","70__DropTable_Blitz")
    #DryRun = $false
}

#$preSQL = "EXEC dbo.usp_check_sql_agent_jobs @default_mail_recipient = '[email protected]', @drop_recreate = 1"
#$postSQL = Get-Content "D:\GitHub-Personal\SQLMonitor\DDLs\Update-SQLAgentJobsThreshold.sql"
#D:\GitHub\SQLMonitor\SQLMonitor\Remove-SQLMonitor.ps1 @Params #-Debug -PreQuery $preSQL -PostQuery $postSQL
D:\GitHub\SQLMonitor\SQLMonitor\Remove-SQLMonitor.ps1 @Params


#Get-DbaDbMailProfile -SqlInstance '192.168.56.31' -SqlCredential $personalCredential
#Copy-DbaDbMail -Source '192.168.56.15' -Destination '192.168.56.31' -SourceSqlCredential $personalCredential -DestinationSqlCredential $personalCredential # Lab
#New-DbaCredential -SqlInstance 'xy' -Identity $LabCredential.UserName -SecurePassword $LabCredential.Password -Force # -SqlCredential $SqlCredential -EnableException
#New-DbaAgentProxy -SqlInstance 'xy' -Name $LabCredential.UserName -ProxyCredential $LabCredential.UserName -SubSystem PowerShell,CmdExec
<#

Enable-PSRemoting -Force # run on remote machine
Set-Item WSMAN:\Localhost\Client\TrustedHosts -Value * -Force # run on local machine
Set-Item WSMAN:\Localhost\Client\TrustedHosts -Value 192.168.56.15 -Force
#Set-NetConnectionProfile -NetworkCategory Private # Execute this only if above command fails

Enter-PSSession -ComputerName '192.168.56.31' -Credential $localAdmin -Authentication Negotiate
Test-WSMan '192.168.56.31' -Credential $localAdmin -Authentication Negotiate

#>


Support

For community support regarding this tool, kindly join #sqlmonitor channel on sqlcommunity.slack.com slack workspace. For paid support, reach out to me directly on #sqlmonitor slack channel.

Related Links


Thanks ๐Ÿ˜ƒ. Subscribe for updates ๐Ÿ‘

sqlmonitor's People

Contributors

imajaydwivedi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

sqlmonitor's Issues

SQL CPU Not Showing in SQLCluster Case

Describe the bug
SQLCluster CPU is not showing on dashboard when the SQLCluster is NOT present on preferred node.

To Reproduce
Assume 2 active-active SQLClusters, say SqlClus01 & SqlClus02 configured on 2 hosts SqlHost01 & SqlHost02. If both the SqlClus01 & SqlClus02 are both moved to SqlHost01, then SqlHost02 does not show SQL CPU on dashboards.

Change Retention of `dbo.all_server_volatile_info_history` to 15 Days

There are scenarios where we need to check historical metrics to figure out max/min/average/pencentile etc for a metrics within x days.

For example, get max memory utilization for last 7 days.

Need to change Retention of dbo.all_server_volatile_info_history to 15 Days in table dbo.purge_table.

Setup Alert Engine Using Python+SQLServer

Design Email & Slack Alerts for following -

  • Instance Unavailability
  • Database Unvailability
  • CPU (High/Critical)
  • Blocking
  • Memory Grant Pending
  • Low SQL Server Ram
  • Jobs Alert
  • ErrorLog based Alerts
  • Login Failures
  • Backups not happening
  • TempDbSaver based alert
  • LogSaver based alert
  • Incorrect tempdb config
  • Alert for long running transaction

Dashboard Exposing Latest Backup History

Is your feature request related to a problem? Please describe.
Sometimes backups are not happening, and we find out very late.

Having a dashboard will help in timely detection of miss backup issues.

  • Added exception for Offline/ReadOnly/Standby databases
  • Ignore database snapshots
  • Added exception table on Inventory server for handling Restore State or any other maintenance activity
  • Implement logic based on Job Category for backup jobs on AG backups. Goal is to figure out which replica is source of backup jobs

We are getting the below error how to fixed this

2024Feb16_1042 ERROR: Seems either [DBA] does not exists, or the data/log files are present in C:\ drive.
Kindly rectify this issue.
F:\GitHub\SQLMonitor\SQLMonitor\Install-SQLMonitor.ps1 : Stop here. Fix above issue.
At F:\Github\SQLMonitor\Private\Wrapper-InstallSQLMonitor.ps1:75 char:1

  • F:\GitHub\SQLMonitor\SQLMonitor\Install-SQLMonitor.ps1 @params

Exception Tables for All Panels on All Server Dashboard

Is your feature request related to a problem? Please describe.

There are times when we can not do anything to one issue appearing on dashboard.
For example, if a database has been set to offline or readonly, then backups won't happen.
So having an exception table would help to skip these records showing on dashboard.

Track Status of SQLAgent Service

After reboot of OS/Box, sometimes, SQLAgent service does not start automatically.

Add feature to track SQLAgent services & display on All Server Dashboard.

We are getting error on two step 3__XEventSession and 9__CopyDbaToolsModule2Host

**2024Feb16_1240 INFO: ***Working on step '3__XEventSession'..
2024Feb16_1240 INFO: $XEventSessionFilePath = 'D:\GitHub\SQLMonitor\DDLs\SCH-Create-XEvents.sql'
2024Feb16_1240 INFO: Computed XEvent files directory -> 'D:\MSSQL15.MSSQLSERVER\XEvents'..
2024Feb16_1240 INFO: Create XEvent session named [xevent_metrics]..

The target, "5B2DA06D-898A-43C8-9309-39BBBE93EBBD.package0.event_file", encountered a configuration error during initialization.
Object cannot be added to the event session. The operating system returned error 5: 'Access is denied.
' while creating the file 'C:\Program Files\Microsoft SQL
Server\MSSQL15.SQL2019\MSSQL\xevents\xevent_metrics_0_133525896392450000.xel'.
At line:97864 char:9

  •     throw $records[0]
    
  •     ~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (SQLMASTER\SQL2019:String) [], Exception
    • FullyQualifiedErrorId : dbatools_Invoke-DbaQuery

2024Feb16_1243 INFO: *****Working on step '9__CopyDbaToolsModule2Host'..
2024Feb16_1243 INFO: $DbaToolsFolderPath = 'D:\Github\dbatools'
2024Feb16_1243 INFO: $dbaToolsLibraryFolderPath = 'D:\Github\dbatools.library'
Import-LocalizedData : Cannot find the Windows PowerShell data file 'dbatools.psd1' in directory
'D:\GitHub\SQLMonitor\SQLMonitor\en-US', or in any parent culture directories.
At D:\GitHub\SQLMonitor\SQLMonitor\Install-SQLMonitor.ps1:1937 char:5

  • Import-LocalizedData -BaseDirectory $dbatoolsConfigFileDirectory  ...
    
  • ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ObjectNotFound: (D:\GitHub\SQLMo...S\dbatools.psd1:String) [Import-LocalizedData], PSInvalidOperationE
      xception
    • FullyQualifiedErrorId : ImportLocalizedData,Microsoft.PowerShell.Commands.ImportLocalizedData

Basic Slack Alerts for Core Metrics

Create following alerts using Python & Slack -

  • CPU
  • Blocking
  • Memory Grants Pending
  • Waits Per Core Per Minute
  • Disk Latency
  • TempDb Space Utilization
  • Log Space Utilization
  • AlwaysOn Latency Health
  • Disk Space Utilization
  • Backup History
  • Offline SQLAgent Service
  • SQLInstance Unavailable
  • [ ]

Capture `sp_Blitz` Result & Create Dashboard

Is your feature request related to a problem? Please describe.
sp_Blitz exposes lots of issues that could be helpful to DBA & Developers.

Describe the solution you'd like
Capture sp_Blitz result at least once a week, and create dashboards exposing various kind of informations from same.

Add Collection Time on All Server Dashboard

Is your feature request related to a problem? Please describe.
It's very important to know when the data collection happened while analyzing the data.

So need to add collection_time on All Server Dashboard panels.

Infra to Support Password Rotation for `grafana` Login

Due to restrict oranizational security policies, login password should change very 90 days.

Due to this, password for grafana login should be rotated every 90 days.

Need to build the infra to support password rotation for grafana login

Control Immediate Removal of Perfmon File in Job `(dba) CollectPerfmonData`

Is your feature request related to a problem? Please describe.
I was to analyze the generated perfmon blg files on my server. So don't want to remove them immediately.

Describe the solution you'd like
A parameter in script SQLMonitor\perfmon-collector-push-to-sqlserver.ps1 to decide if immediately remove a file or not.

Dashboard - Deadlock Analysis

Dashboard - Deadlock Analysis

Below query can be used for this infra -

declare @_endDate datetime;
set @_startDate = dateadd(hour,-24,getdate());
set @_endDate = GETDATE()

exec sp_BlitzLock @StartDate = @_startDate, @EndDate = @_endDate
				,@OutputDatabaseName = 'tempdb'
go

select *
from tempdb..BlitzLock

Add Parameters to Skip Particular Wait Type in `usp_waits_per_core_per_minute`

Is your feature request related to a problem? Please describe.
On some of my servers, there are some WaitTypes that always occurring, and overshadowing other relevant wait types.

Describe the solution you'd like
Below 2 options -

  1. A parameter to ignore comma separate list of wait types
  2. A table that keeps a list of wait types that can be ignored for easy handling. This table should be present on each server.

Self-Sign PowerShell Scripts

In highly secured environment, only remote-signed powershell scripts are allowed.

Self-signing SQLMonitor powershell scripts will help in smooth deployments.

Job `(dba) Collect-DiskSpace` failing

Describe the bug
Job (dba) Collect-DiskSpace failing with below error message after upgrade of dbatools to latest release.

2023Nov22_1832 INFO:      Fetch disk space on $HostName = [21L-LTPABL-1187]..
2023Nov22_1832 INFO:      Push disk info to SqlServer [21L-LTPABL-1187].[DBA].[dbo].[disk_space]..
The certificate chain was issued by an authority that is not trusted
At line:97904 char:9
+         throw $records[0]
+         ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ConnectionError: (21L-LTPABL-1187:String) [], Exception
    + FullyQualifiedErrorId : dbatools_Write-DbaDbTableData

Add additional verification step for Instance-Availability

Is your feature request related to a problem? Please describe.
Sometimes job (dba) Check-InstanceAvailability may fail due to various reasons like file reuse, temporary glitch, etc
Due to this, the job sets a server OFFLINE

Describe the solution you'd like
Add additional verification using tsql method to reset the OFFLINE flag for offline servers.

Setup Alert Engine Using Python+Postgres

Design Email & Slack Alerts for following -

  • Instance Unavailability
  • Database Unvailability
  • CPU (High/Critical)
  • Blocking
  • Memory Grant Pending
  • Low SQL Server Ram
  • Jobs Alert
  • ErrorLog based Alerts
  • Login Failures
  • Backups not happening
  • TempDbSaver based alert
  • LogSaver based alert
  • Incorrect tempdb config
  • Alert for long running transaction

`Remove-SQLMonitor` - Save InstanceDetails are until the Instance is Removed Full

If Remove-SQLMonitor fails after step 49__DropTable_InstanceDetails, then with current code, its not possible to finish the unistallation as the details of instance are lost.

In order to get around this issue, kindly save the instance_details somewhere at start of installation, and remove same after successfull uninstallation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.