Git Product home page Git Product logo

tabular's Introduction

Tabular

This repo is a collection of useful code for automating processes within tabular modeling. All of these scripts are to be executed in Tabular Editor so make sure to download and install it.

For addtional information on these scripts and more, check out my blog Elegant BI.

Auto-generated aggregations supporting base fact tables in both import mode and direct query. Also check out the Agg Wizard which has additional functionalities and a supporting user interface.

Run this script in Tabular Editor to create an automated data dictionary. This script works for Analysis Services, Azure Analysis Services, and Power BI Premium models (XMLA R/W endpoints enabled).

Run this script in Tabular Editor to create an automated data dictionary where the Data Dictionary table is stored in Excel. This script works for Analysis Services, Azure Analysis Services, and Power BI Premium models (XMLA R/W endpoints enabled).

Run this script in Tabular Editor to list out all of the base measures in a given tabular model. A base measure is defined as a measure which does not depend on another measure.

Run this script in Tabular Editor against a live-connected model to quickly make a list of all relationships that contain a blank row in the 'to-table'. This has now been integrated into the Vertipaq Analyzer scripts (see below) as well as the latest Best Practice Rules.

The elements in this folder enable you to scan all models within a folder using Tabular Editor's Best Practice Analyzer and show the summarized results in a Power BI report. Below are instructions how to do it:

Run the BPAScanFolder.cs script in Tabular Editor 2, setting the folderName parameter to a folder containing model files (.bim , database.json, model.tmd). Download and open the BPAReport.pbix and update the BPAResults table's source to be the BPAResults.txt file generated by the BPAScanFolder.cs script. You will now see a Power BI report showing the Best Practice Analyzer violations in your models.

Screenshot 2023-06-15 124459

Run this script in Tabular Editor against a live-connected model to cancel the data refresh of that model.

Run this script in Tabular Editor against a live-connected model to cancel user queries over a given threshold. Simply enter the threshold (in seconds) in the 'thresholdSec' parameter. This script specifically avoids cancelling data refresh operations and can be customized to not cancel queries run by a list of user names.

Run this script in Tabular Editor against a live-connected model while selecting a single table within the TOM (Object) Explorer. It will return a data preview of the table.

Run this script in Tabular Editor against a live-connected model while selecting one or more columns from a table within the TOM (Object) Explorer. It will return a data preview of the columns (distinct values).

Run the ExportDescriptions.cs script in Tabular Editor to export objects and existing descriptions in your tabular model to an Excel file.

Run the ImportDescriptions.cs script in Tabular Editor to import object descriptions back into your tabular model from the Excel file.

Running this script in Tabular Editor will run the Best Practice Analyzer and output the results. The output can easily be copied into Excel for further analysis.

Run the ExportReportObjects.cs script in Tabular Editor to export the objects used in a Power BI report (or a collection of Power BI reports within a specified folder). The image below shows an example output:

image

  • Bookmarks
    • Report Name, Bookmark Name, Bookmark Id, Page Id, Visual Id, Visual Hidden Flag
  • Connections
    • Report Name, Server Name, Database Name, Report Id, Connection Type
  • Custom Visuals
    • Report Name, Custom Visual Name
  • Page Filters
    • Report Name, Page Id, Page Name, Filter Name, Table Name, Object Name, Object Type, Filter Type
  • Pages
    • Report Name, Page Id, Page Name, Page Number, Page Width, Page Height, Page Hidden Flag, Visual Count, Page Background Image, Page Wallpaper Image, Page Type
  • Report Filters
    • Report Name, Filter Name, Table Name, Object Name, Object Type, Filter Type
  • Unused Objects
    • Report Name, Table Name, Object Name, Object Type
  • Visual Filters
    • Report Name, Page Name, Visual Id, Table Name, Object Name, Object Type, Filter Type
  • Visuals Objects
    • Report Name, Page Name, Visual Id, Visual Type, Custom Visual Flag, Table Name, Object Name, Object Type, Source
  • Visuals
    • Report Name, Page Name, Visual Id, Visual Name, Visual Type, Custom Visual Flag, Visual Hidden Flag, X Coordinate, Y Cooridnate, Z Coordinate, Visual Width, Visual Height, Object Count, Show Items No Data Flag, Slicer Type
  • Visual Interactions
    • Report Name, Page Name, Source Visual ID, Target Visual ID, Type ID, Type

Note: If using Tabular Editor 3, comment out the last line (starting with 'static void') and also comment out the closing bracket in line 2287.

Note: 'Source' within Visual Objects shows as 'Standard' for objects used within rows/columns etc. of visual itself. Objects used for conditional formatting or to set titles, backgrounds etc. will show as such within the 'Source' column.

Note: Unused Objects lists objects (measures, columns etc.) not used in the report and checks the dependencies listed below. This should be used in conjunction with the 'Remove Unnecessary Columns' Best Practice Rule for the greatest efficacy.

  • Measures
  • Relationships (key columns)
  • Sort-by Columns
  • Calculated Columns
  • Hierarchies
  • Calculation Groups
  • Auto-date Tables

Running this script in Tabular Editor will update the DAX for all measures to ensure that the case of referenced tables, columns or measures matches the case of the actual table, column or measure name in your model. This is necessary for DirectQuery and DirectLake models.

Metadata Import - Perspectives

Run this script to automatically update the perspectives in your model (or add new perspectives). This script coordinates with the output text file from the Metadata Export script.

Metadata Import - Translations

Run this script to automatically update the translations in your model (or add new translations). This script coordinates with the output text file from the Metadata Export script.

Running this script opens a program within Tabular Editor that allows you to create or modify perspectives akin to the way it is done in SQL Server Development Tools (SSDT). It also gives you a tree-view of all the objects that are in a perspective relative to all the objects in the model.

Related Tables In Common

Run this script in Tabular Editor to identify the tables which are related to all the tables you have selected in the TOM explorer.

Want to migrate measures created within a Power BI Desktop report to your tabular model? This script does exactly that. Setting the 'createMeasures' parameter to 'true' will create the measures in the model file within Tabular Editor. Setting this paramter to 'false' will dynamically generate C# code which can be copied and executed in order to create the measures in a model.

Save all models on server

Run this script in Tabular Editor (while connected live to an Analysis Services instance) and it will save the metadata files of all the models on the server to the location of your choice. Just need to input 2 parameters (folderPath, saveType).

Run this script against a live-connected model to save Vertipaq Analyzer statistics as annotations on model objects. These annotations may be referenced to create Best Practice Analyzer rules for your model. See the link below for more info on Tabular Editor's Best Practice Analyzer.

Note: If running this script against a Power BI Desktop model (using Tabular Editor as an External Tool), you must select the following setting within Tabular Editor:

File -> Preferences -> Features -> Allow Unsupported Power BI features (experimental)
  • Model: Model Size

  • Tables: Row Count; Table Size; Table Size as a Percentage of the Model Size

  • Partitions: Record Count; Segment Count; Records Per Segment

  • Columns: Cardinality; Column Hierarchy Size; Column Size; Data Size; Dictionary Size; Column Size as a Percentage of the Table Size; Column Size as a Percentage of the Model Size

  • Hierarchies: User Hierarchy Size

  • Relationships: Relationship Size; Max From Cardinality; Max To Cardinality; Referential Integrity Violation Invalid Rows

This script creates the same annotations as the Vertipaq Annotations script. The only difference is that this script loads the Vertipaq Analyzer data from a Vertipaq Analyzer (.vpax) file. The .vpax file can be generated by selecting 'View Metrics' within the 'Advanced' tab in DAX Studio.

tabular's People

Contributors

m-kovalsky avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tabular's Issues

Descriptions enhancement : add support for different code page

Your scripts for exporting/exportings descriptions are very usefull, however it uses the default Excel code page which can be an issue on non English Excel version.

It would be nice to add an origin variable to set the code page (close to other variables definitions) in ExportDescriptions.cs:
object origin = Type.Missing; // 65001 for UTF-8;
and use excelApp.Workbooks.OpenText(textFilePath, origin, 1, ... instead of excelApp.Workbooks.OpenText(textFilePath, Type.Missing, 1, ...

Exporting and importing DisplayFolder could be nice also even if it's not strictely descriptions.

Thanks

Unable to load Tabular Model (Compatibility Level 1200+)

Hello @m-kovalsky ,

While we are integrating the BPA Scan in .NET core project, we are getting the below issue. Can you please help on to fix this issue.
"Unable to load Tabular Model (Compatibility Level 1200+) from .src\model.bim. Error: The type initializer for 'Microsoft.AnalysisServices.Core.Utils' threw an exception".

BPA issue

Is there a way to create parameters/shared expression?

Hi Michael - thanks so much for your scripts. I use several of them and save me a lot of time and efforts.

Is there a way to create parameters in TE using scripts? I was able to create Shared expression but couldn't figure out how to set the kind to M and add Expression via script.

Thanks

ExportReportObjets.cs: VisualObjects missing elements

It looks as if from the non-Standard VisualObjects only the "Label" makes it into the final txt file.
All other elements (starting from row 682) don't make it into the final file.

And: For the vcObjects, the extraction path is not correct. The field "FillRule" doesn't exist any more and can simply be deleted.
However, even after that fix, those objects won't make it into the txt currently.

Doesn't work with PPU

Is a Service Principal required for running this script?

As Power BI Admin I still get this:
Error on line 29 ( Model.Database.TOMDatabase.Server.CancelSession(sID);)
The '[email protected]' user does not have permission to perform a cancel operation.

ExportReportObjects.cs accumulating report pages when looping through multiple .pbixs

Observing cases where the ReportObjects_Visuals.txt from using "Method 2" for multiple .pbix results in all of the pages from the first report also being associated to the second. Then all those pages also being associated to the third report. Then all those pages also being associated to the fourth and so on.

No issue with Method 1 for a single .pbix.

This is an incredibly helpful tool but may create confusion when using across multiple .pbixs.

TE version 2.16.1.

Example: Created 3 .pbixs each having one page. First .pbix shows the one page correctly, but subsequent reports scanned repeat the report + page listed previously.

Example

No update after import

Hi,
When importing with TE2, the description field does not update.
The advanced "unsupported filters" is on.
Is there any thing to know ? (only work on api connection (not local model), needs for a specific compiler ?

ArgumentNullException

I am getting an error on line 50. Could you please help me navigate through it?
image

I don't have one data source but data is pulled in using dataflows and then creating the model in PowerBI
image

ExportReportObjects.cs: conditional formatting, RLS, OLS, and relationships not considered?

When I view ReportObjects_UnusedObjects.txt, I see columns that are part of RLS or OLS or a relationship, as well as measures that are used for conditional formatting. Is this expected?

My model has two tables:
image

In addition to the relationship, there's an RLS role, an OLS role, and a table visual with conditional formatting (background color).

I expect to see only [Unused measure] in ReportObjects_UnusedObjects, yet I see everything but 'Table'[Column for a table visual]: ReportObjects_UnusedObjects.txt

Sample PBIX file: Sample for issue 14.zip

In case it matters, I'm connecting to the model locally. The only parameters I changed were pbiFolderName, pbiFile, addPersp (the latter set to true).

Is there something I'm doing incorrectly?

AutoAggs: Object Reference not set to an instance

When running the script this error comes up at line 25:

grafik

When outputting p.Query it returns Null. p is correctly set to the first partition of the fact table.

The way I understand it, it tries to find the query for a partition, but it's empty? I've already setup incremental refresh on the fact table, whch is why the partitions themselves do not have it's own M query. It's in the "SourceExpression" attribute of the table object. Is this why it fails?

ExportReportObjects.cs Error on line 155 (Connections)

While my folder path and file name are correct, I get the following error when I try to run the script:
image
It's just a simple file I created for test purposes and saved locally. Report elements and the data model are in one PBIX file.

Creating Agg Table with M-Partition instead Legacy

Hello Sir, In this snippet below, it creates a Legacy-Partition for the AggTable by default. I was wondering if it is possible at all to create a M-Partition instead. If so, I would greatly appreciate if you could point me to some documentation to do that. Thanks in advance.

foreach(var p in Model.Tables[tableName].Partitions.ToList()) { var aggPartitionName = p.Name.Replace(tableName,aggTableName); var aggQuery = p.Query.Replace(tableName,aggTableName); var pName = Model.Tables[aggTableName].AddPartition(aggPartitionName); // Update Data Source pName.DataSource = Model.DataSources[dataSource]; // Update Query pName.Query = aggQuery; } // Remove default partition Model.Tables[aggTableName].Partitions[aggTableName].Delete(); }

Updates to Script, but specific to TE3 - help with TE2 for all-in-one solution that combines Visual Object, Model, and Workspace Use detail?

Michael, first off, your visual object layer script has been amazing for my team and our business.

Over the last few months, I've made a lot of additions (additional object details pulled) and also added a process to combine each export into a single excel file with tabs.

I'm wanting to share, but all my changes have been for Tabular Editor 3 and I can't quite figure out the Tabular Editor 2 errors.

Any chance you can reach out via email at [email protected] or message in Teams at [email protected]? Similar to your current script, I think these changes are super helpful to anyone/any organization and would love to get them in your main script !

Chris

Cannot get BPA-script to work

Hi Michael

Thanks for all the BPA work.

I cannot run the BPA-script that collects info on several model. It says the ObjectReference for TabularModelHandler is null?

TabularModelHandler

Data Dictionary without Excel and without DWH

Hi,

I have created (used your script) a script to create a data dictionary without using Excel or a live connection.

But i cant get it to work yet.
What i did i created a SQL statement based on your SQL insert script into the a DWH.
But what i did i create a statement and put that in the Partition.

I also create a table and datasource, but in this case there isnt a data source.
And i think its failing because the datasource doenst exists.

Could you help me ?

The script is in the attachtment.

Kind regards,

Ben

Unable to run script against report

I'm facing an issue when pointing to a OneDrive path. Have you resolved your issue?

I'm getting an error at line 137: File does not exist. Must use a valid .pbix or .pbit file.

Not facing this issue when moving the file to a local folder.

Export report object - filter values

In the Export Report Object you get the columns measures visuals and so on.
But what i don't see are the filter values.

I don't know if this issue should be at report analyzer. That's why I added it here.

Txt path error

Trying to use the code to export descriptions and getting this error :
I made sure to have the files in that path. Any tips on how to solve? Thanks!
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.