This Splunk TA (add-on) is Open Source and not an officially support Microsoft product. Support is best effort. Members of the community (you) are encouraged to participate so that this popular TA can get the support its users richly deserve.
Splunk updated their Python requirement to v3.x a few months ago. This TA still takes a dependency on Python 2.7.
The repo is archived due to lack of resources to keep it up to date.
This add-on is built using Node.js and Python 2.7 and has been tested on Ubuntu 14.04, Windows 10 and RHEL 7.
It consumes Metrics, Resource Diagnostic Logs, and Tenant Diagnostic Logs (AAD Activity Log) and the Azure Activity Log according to the techniques defined by Azure Monitor, which provides highly granular and real-time monitoring data for Azure resources, and passes those selected by the user's configuration along to Splunk.
Here are a few resources if you want to learn more about Azure Monitor:
- Overview of Azure Monitor
- Overview of Azure Diagnostic Logs
- Overview of the Azure Activity Log
- Overview of Azure Active Directory Activity Logs, known in Azure Monitor as Tenant Diagnostic Logs.
- Overview of Metrics in Microsoft Azure
- Stream Azure monitoring data to an event hub for consumption by an external tool
This add-on requires an Azure Event Hub, Key Vault, Azure AD Service Principal and other configurations to properly integrate Splunk with Azure. Creating and configuring the Azure resources can be accomplished using one of the scripts available in the .\scripts folder as shown here:
If you are planning to use MSI authentication instead of Service Principal (SPN), first ensure that you VM is up and running within your Azure subscription.
- Windows users can use the PowerShell script
.\scripts\azure-setup.ps1. Proceed to the section Azure configuration for Windows users. - Linux and Mac users can use the Bash script
.\scripts\azure-setup.sh. Proceed to the section Azure configuration for Linux / Mac users.
- Azure PowerShell, which you can download from here. The script was tested with version 5.7.0 of the Azure PowerShell cmdlets running on Windows Server 2016.
- The following resource providers must be registered in your Azure subscription. You can find out which resouce providers are registered in your subscription using the command
Get-AzureRmResourceProvider | Where-Object { $_.RegistrationState -eq "Registered" } | Select ProviderNamespace.- Microsoft.Authorization
- Microsoft.EventHub
- Microsoft.KeyVault
- Microsoft.Storage
- microsoft.insights
-
Open
.\scripts\azure-setup.ps1. Replace the variables at the top of the script with values from your environment.$subscriptionId: Your Azure subscription Id.$tenantId: The tenant / directory Id for your Azure subscription.$splunkResourceGroupName: The name of the resource group to deploy the cluster into. This can be a new or existing resource group.$splunkResourceGroupLocation: The location you want to deploy the cluster in. For example, eastus, westus, etc.
An example showing the variables populated is shown here:
-
Run the script. The script will prompt you to authenticate to your Azure subscription. The output for the script will look similar to the output shown here:
Proceed to the section Splunk Enterprise configuration.
- Azure command-line interface (CLI), which you can download from here. The script was tested with version 2.0.42 of the Azure CLI running on Ubuntu 18.04 LTS.
- The following resource providers must be registered in your Azure subscription. You can find out which resouce providers are registered in your subscription using the command
az provider list --query '[?registrationState==`Registered`].namespace'.- Microsoft.Authorization
- Microsoft.EventHub
- Microsoft.KeyVault
- Microsoft.Storage
- microsoft.insights
-
Open a terminal window and navigate to the
.\scriptsfolder. The bash script requires four parameters as shown here:usage: azure-setup.sh [options] options: -l <location> : [Required] Location to provision resources in. Ex. westus, eastus, etc. -r <resource group name> : [Required] Resource group to deploy resources into. -s <subscription id> : [Required] Azure subscription Id. -t <tenant id> : [Required] Azure Active Directory / Tenant Id. -m <vm name> : [Optional] VM name of VM that has MSI enabled. This will skip SPN setup and assign RBAC access for the VM.
-
Run the script. The script will prompt you to authenticate to your Azure subscription. The output for the script will look similar to the output shown here:
Proceed to the section Splunk Enterprise configuration.
-
Install the add-on in Splunk Enterprise using the latest package file from the github releases page.
-
In Splunk, open the apps manager page by clicking on the gear icon.
-
Click on the button labeled Install app from file.
-
In the dialog window, click the Browse... button and select the downloaded
.splfile. Next, click the Upload button.
-
After uploading the add-on app, the apps manager page should show the application installed in Splunk. An error message may also appear if the indicating the add-on could not initialize. This is typically because dependencies for Python and/or Node.js are not present.
-
To resolve the error message stating Splunk is "Unable to initialize modular input", install Node.js and Python on your Splunk Enterprise instance as described here. An example for an instance running on Ubuntu is shown below:
-
Run the following commands to install the Python and Node.js dependencies:
# Elevate to root user sudo -i # Download script to setup Python dependencies curl -O https://raw.githubusercontent.com/Microsoft/AzureMonitorAddonForSplunk/master/packages/am_depends_ubuntu.sh # Set the execution attribute on the downloaded script chmod +x ./am_depends_ubuntu.sh # Run the script ./am_depends_ubuntu.sh # Download Node.js and it's dependencies curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - # Install Node.js apt-get install nodejs # Install Nodel modules in the add-on's app folder. cd /opt/splunk/etc/apps/TA-Azure_Monitor/bin/app npm install # Return back to a non-root user exit
-
Go back to Splunk and click the Disable link for the add-on.
-
Click the Enable link to re-enable the add-on. The add-on should be enabled now without any error messages appearing.
-
-
-
Using the output from either the
.\scripts\azure-setup.ps1or.\scripts\azure-setup.shabove, configure the add-on's data inputs.-
In Splunk, click on Settings -> Data Inputs at the top of the page.
-
For each of the add-on's data inputs, add a new configuration by copying and pasting the settings from the script's output into the data input's configuration.
-
If you have encountered difficulties with the add-on, the first thing to do is ensure that all Python and Nodejs dependencies are installed correctly according to the installation instructions in the wiki.
The Activity Log and Diagnostic Log data inputs use AMQP to connect to event hub over TLS using ports 5671 / 5672 as described in the AMQP 1.0 Service Bus and Event Hubs protocol guide. So, if you are having connection/authentication issues, check that these ports are open on your Splunk instance.
The Azure Monitor Metrics data input uses HTTPS to call into the Azure Monitor Metric API. As such, outbound traffic over port 443 needs to be enabled on the server.
If that doesn't help, the next thing to do is switch logging for ExecProcessor to Debug (Settings / Server Settings / Server Logging in Splunk Web) and recycle the add-on (disable/enable). Then search for 'azure_monitor' ERROR and DEBUG messages. There will be a lot of DEBUG messages. If you don't see anything helpful, open an issue in the repo.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
To test changes, you will need to generate a version specific package file that includes your changes, such as .\packages\TA-Azure_Monitor_1_2_7.spl. Follow the steps below to generate the version specific package file.
Note: The Windows script package.cmd requires 7-Zip to build the the file structure and contents. So, make sure you have this installed on your computer and that 7z can be run from a command/shell prompt. On Mac and Linux, package.sh will use tar if 7-zip is not installed.
-
Open
.\default\app.confand bump the version property in the [launcher] section. -
This step generates the version specific package file. If you are running Windows, then you will use the
.\deployment\package.cmdscript. If you are running Mac or Linux, use the.\deployment\package.shscript. Open a command/shell prompt and change to the.\deploymentdirectory. Execute the script, passing in the version specific string as shown below. Notice the use of underscores in the string.Windows
package.cmd 1_2_7Mac or Linux
./package.sh 1_2_7The version specific package file will be in the
.\packagesfolder, as shown below.
-
The version specific package file should be committed to the repository with your code changes. It is tradition to remove the oldest version specific package file when creating a new one.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent