In May 2020 a new pipeline appeared. Deployment Pipelines
Having worked with Devops, It looks like it is identical to the Pipeline that allows you run builds, perform tests and release code to the various production environments.
Power BI deployment Pipelines are a new way for BI teams to manage the content lifecycle.
It is only available in Power BI Premium
Another current restriction is that it doesnt work with dataflows
There are some great reasons to use deployment pipelines
- To Improve your productivity
- Faster delivery of your updates
- Reduction of manual work and errors
Lets see how it works with an example
- the Workspaces must be in Premium capacity (You can set up the Dev and test areas on an A SKU to save money)
- the Developer must have a power BI Pro License
- The developer must be an owner of the data sets in the target workspace
Setting up the pipeline Example
In this example, I want a development Test and Production area.
Each of these areas has a separate data source. One each for Dev Test and Prod (But this might not be the case for your own development)
The first instance we will create a dataset that contains the dataflow.
You need a workspace with a diamond against it (Which signifies its in Premium capacity)
You will need to be Admin in the workspace to carry out the task of creating the Pipeline
A test report is created over the development data (Which is a table in the Dev Azure Database)
You can set up power BI to move to different environments using the following guide Moving Power BI to different Environments (Dev Test Production etc)
I have parameters set for each environment
At this point, make sure you change the current value of the parameter to check they are all working. This report has been published up to the Premium workspace
You don’t have to do this. You can use Rules within the Pipeline if you wish.
Start a Power BI Deployment pipeline
The Look and feel of the Deployment Pipeline is great
Create a Pipeline
Now we are into the actual main Pipeline area. You can only assign one workspace to the pipeline. When we move through the pipeline it automatically creates the other workspaces for you.
Our Pipeline testing workspace is going to be set up as the development area
Assign a Workspace
You dont need to start with development. You can start with test or Production but in this case we are going straight to the Dev area. Also you don’t need to have all three environments. This now gets assigned to the Development pipeline
At this point you can see what will be part of the Dev Pipeline. Show more shows you the content within. At this point you can visibly select items that you want to deploy up to the next stage, but in this example all three of them is required.
Rename Pipeline Testing to Pipeline testing (Dev). Click on … and go to Workspace settings
Your report consumers and testers will want to use your reports and dashboards as apps. You can create Apps at every stage. To do this click on … and Publish app
Deploy to test
Clicking Deploy to test creates a copy in your Test area
It will copy your Reports, Datasets and dashboards into the test area. If you have dataflows you should note that these currently don’t get included
Rename to Pipeline testing (Test) if required
At this point we may want the test area to be connected to a different data source than the development environment. Because we set up Parameters in the pbix file to change to different databases, we can use parameter rules. If if you dont have parameters set up you can create a data source rule.
At this point. go back to your New Test Premium Workspace.
Against the data set click … and Settings
I have changed the Parameter to the correct one
Now refresh the credentials
And repeat when you get to Production App Workspace
Deployment Settings (Back in the Pipeline)
Get to Deployment Settings when clicking on the lighting bolt
Parameters have been set in the pbix files so these should be used in this example.
You can use rules(Below) if you don’t have parameters but remember to check your Data set settings first.
Because the source is a database, the pipeline knows to ask for a server and a database. Make sure your Database is set up correctly first within Service.
Deploy to Production
Clean up and set up the same rules (Ensuring after deployment you check your Data set Settings before setting up the rules).
Updates to Development Area
You can also just deploy specified items that you have worked on
For this test, go back to the desktop and add a couple of visuals. Then Publish into the development workspace.
Look to compare the changes
The comparison shows that two items have changed. Deploying into test will copy the new information across
You will also see icons for new and deleted.
Now we can see that production is still using the previous datasets, reports and dashboards. we wont copy across until we are happy the changes are correct.
These are now three individual Workspaces with individual data sets and reports. You will need to set up scheduled refresh for each area.
You can also publish downstream if required by clicking … If the item is available
- Not for Dataflows or Excel items
- Premium Only
The Production workspace must be in premium. You can use A SKU or Power BI Embedded to save money. (A Sku’s can be set up within Azure and are Test environments. they can be paused)
It doesn’t currently plug into Azure Devops. Hopefully this will be coming soon.
Workspace using Dataflows
I’m moving across to another workspace now. lets have a look at the Lineage
There is an Adventureworks Dataflow which connects to a dataset and a report.
go to Pipelines. Create a Pipeline and then…..
In this instance, The Dataset and report that sits over the dataflow is specifically selected.
Power BI is really not happy about this
The two items are copied across.
If we want to set up rules for this workspace…..
No actions are available. Your dataflow is sat in the Development area. You cannot change it
Lets do the same for Production
If you go and look at your workspaces
There are now 3 workspaces. Lets have a look at Lineage to see how the dataflow is shown for test.
Your data cannot be viewed because your dataflow is not supported.
Considering we have spent a lot of time supporting people to move to dataflows, this is a real problem
Comparing Dev to Test
Still looking at the reports that use the dataflow. Lets see if it can compare. The pbix files is opened and amended. then published to the dev Workspace.
At least it tells you which items have changed
With the push to move to dataflows to separate transformation to the actual creation of DAX analytics, it seems like an urgent requirement.
Hopefully this will be resolved soon.
When this is all up and running it is recommended to separate the datasets from reports and dashboards. to do this use the selective deploy
Plan your Permissions model
- Who should have access to the pipeline?
- Which operations should users with pipeline access be able to perform in each stage?
- Who’s reviewing content in the test stage and should they have access to the pipeline?
- Who will oversee deployment to the production stage?
- Which workspace are you assigning and where in the process are you assigning it to?
- Do you need to make changes to the permissions of the workspace you’re assigning?