The news is out, the cat is out of the bag, the secret has been revealed!
If you haven´t noticed Microsoft revealed their new lake-centric software-as-a-service solution for data analytics. It´s a one stop shop for data integration, data engineering, data warehousing, data science, real-time analytics, applied observability, and business intelligence. It´s built on a data lake using the open delta file format.
You might be thinking, why invent the wheel when they have a bunch of good solutions already? The answer to that is that they are not. They are taking the good things from Synapse Analytics, Azure Data Factory and Power BI and adding a new monitoring solution. But it´s not just taking those solution as-is. The foundation of the platform is OneLake a data lake based on the delta file format. It´s called OneLake as you should only have one copy of your data and all workloads (spark, sql, real time analytics or data science) as well as Power BI will all work on top of the same copy of the data. Even for Power BI you don´t have to import the data anymore (if your data is in the OneLake).
The Synapse tools will now all work directly on the lake, even the Data Warehouse part.
So, what does it mean for you and how do you get started?
I have been so fortunate to be able to test out these solutions for the past few months. It´s very promising. I´ve not tested production workloads but what I have seen I like. It´s only preview, and some of the stuff is pretty revolutionary, so there have been some issues but overall, I´ve been impressed. I´m not saying you should drop your current solution and move everything to Microsoft Fabric, but you should start looking into what it is and where it might benefit you. I see huge potential in the platform for all sizes of organizations. I think companies of any size, even SME´s are going to see some cost savings and I think it´s going to save a lot of time because you don´t need to move as much data around. You are still going to be doing the same architecture (that never really changes) which is great.
What I really like about it is that you can have mixed workspaces with fx. a spark notebooks, data factory pipelines and Power BI reports. So if you work across you can stay in one workspace. It might also make it easier to work as a team in one workspace. I still think that for many IT teams you will have different workspaces for different workloads/personas/tools but there will be times when it makes sense to have some or all of it in one workspace.
My advice to you is to take it easy. Find a good scenario where you can test out the capabilities over the next few months and be ready to take advantage of the platform when it becomes generally available. I´m hoping for an easy “migration” path from Synapse Analytics and Azure Data Factory. I´m assuming that if you have a data lake already it should be fairly straightforward to move that into the OneLake but I´m not sure at this point in time as I have only tested copy/pasting.
How do you get started?
Before you can get started you need to turn Microsoft Fabric on in the Power BI tenant settings. You can turn it on for the whole organization (I don´t recommended that at this point in time) or for a subset of the organization through specific security group(s). Not that it´s off by default but if you don´t touch it Microsoft are going to turn it on for everyone sometime in June.
When the setting is turned on, there are two ways to get started with Microsoft Fabric.
If you have existing Power BI capacity it will be converted to Microsoft Fabric capacity, and you can use it for all the other workloads (during the free preview you won´t actually use your capacity but you can monitor how the workloads would affect your capacity). If you don´t have Power BI you can start a free 60 day Microsoft Fabric trial and get free capacity during that. You need to go to https://app.fabric.microsoft.com/. There you can click on your user image in the top right corner. There will be a start trial button there you can press to start your trial.
After you “turn fabric on” you can work with the different components on https://app.fabric.microsoft.com/ or on https://app.powerbi.com/ via the icon at the very bottom on the left. Clicking that will allow you to choose a tool to work with.
I will be posting more as I test out more production like scenarios in the coming months, so stay tuned.
The third of the five pillars of Power BI Governance is the Training and support pillar. The order of the pillars is not important, so all the pillars are equally important.
Thebest way to ensure compliance is to make sure your users know what they are doing. If they know that, the likelihood of an accidental breach is far reduced. This is what the Training and support pillar revolves around.
As said above a well-trained user base (IT and business) is the best way to maximize compliance. In the ideal situation you would train every user with the best possible classroom training and test them afterwards to gage their knowledge. But in reality, there is not budget or time to do that. So how do you plan for a more pragmatic training and how do you support your users?
I suggest you start by grouping people after how they use Power BI. This could be something like (not an exhaustive list):
Report consumer
Report developer
Dataset developer
Administrator
Supporter
You then decide what type of training would give, what the impact of the training is and what cost is associated with each type. For an example you might argue that report consumers would get value out of short videos, online self-paced training, and instructor-led training. Maybe the cost is least with videos, then online self-paced training with instructor-led training the most expensive.
The next phase is to order the groups into how much impact a breach would have. The bigger the impact the more you should be willing to spend on training to minimize the chance of it happening.
Of course there are constraints such as peoples location, finite training budget and other things that will impact the decision. In the end you will find a solution that will be pragmatic I hope.
When I help customers design training plans I normally start with a rule of thumb which is like this:
Report consumer – short videos
Report developer – online self-paced training
Dataset developer – instructor-led training
Administrator – mentoring
Supporter – internal training
Then we start with the exercise above to figure out what is best for each group.
The bottom line is, that if you want to have a successful Power BI implementation training is very important. You want to train everyone who touches Power BI but in a different way depending on their role. You want to make sure you get to everyone and deliver the right training based on their needs. It´s not only governance training that is important. Training users in properly using Power BI and using best practices will deliver value faster and will make report and dataset developers more compliant.
One of the things we have been doing is to automate the training offer to users by using Microsoft Power Automate in combination with Office 365 (who has license) and the Power BI activity log (what are they doing). When a user gets a license or when they publish their first report or dataset they receive an email with the training being offered in the organization as well as relevant document and processes needed for their role. There are many variations on how you can go about this but the goal is to minimize the effort from the governance responsible to figure out who needs to be trained.
When a user it trained it´s good idea to continue to give support. Power BI changes every month, new features get added frequently and governance requirements might change over time. A good training will have taught the user how to be compliant as Power BI was the time the training was given and while it´s rare that new features that introduce compliance issues are added, it does happen. When it does you should have a support plan in place to either update the users training or at least make them aware of the risk. Do sessions in “what´s new in Power BI” or send out a video explaining a new feature if you decide it introduces potential compliance risk. However you do it make sure you get to everyone who needs the information. The Power BI audit log can help you identify active users and if they are just consuming or developing. The main thing is to understand that you are not done when the training is done.
This concluded part 4, Training and support pillar. Part 5 will cover pillar 4 Monitoring
The second of the five pillars of Power BI Governance is the Process and frameworks pillar. The order of the pillars is not important, so all the pillars are equally important.
Processes and frameworks are the backbone of a good governance strategy. They are used to guide administrators and users how to use Power BI in a compliant manner and using best practices.
In my opinion, when it comes to governance, a framework is just a collection of processes. Therefore, I will only talk about processes in this blog.
It´s important to say at this point that we are just talking about documents to help people use Power BI correctly. If you prefer to call them something else than processes, there is no reason not to. Many of my customers prefer to call these documents best practices documents, while others don´t mind the name process. The important thing to remember is that you make it clear to the users which parts of the documents are required, and which are optional best practices. In my experience people tend to take processes more seriously than best practices but also dread them more. You need to find the best way in your organization to keep people interested and make sure they take the documents seriously.
There are many different processes you could create in an organization. It all depends on your governance strategy and how much you want to split up topics. For instance you might create a development process that covers everything from Power BI Desktop development to publish and share or you might have one development process and another process for publish etc.
When I work with customers on Power BI governance, I will suggest that we cover at a minimum:
Power BI Development guide
Power BI publish / deployment guide
Power BI sharing guide
Power BI Administration guide
Power BI Tenant settings documentation
There are others that could be done separately such as a security process or a naming standard but that depends on the customer and the users (their skill, tolerance to documentation and usage scenarios)
Normally I will create the documents with my customer by first running workshops to better understand what they want and need. Normally, I will do one for each document or audience depending on how the customer wants to proceed. The important thing here is to listen to the stakeholders.
For the admin docs, you need to talk to current Admins on how they are administrating the system and balance that on best practices and the governance strategy (what do we allow of settings, separation of duties etc.). I will often use the tenant setting documentation to make sure everyone understands each setting and set it appropriately. In my opinion it´s very important to understand them well and challenge each decision to turn off settings that might make life easier to the user. Preferably you will write in the tenant settings documentation why the setting is sat in the documented manner.
For the development docs you will need to talk to representation of developers, both IT and business to understand how they are developing and want to develop and balance that against the governance strategy. You might end up with one process for IT and another for business users. For example, you might decide that it shouldn’t be a problem requiring IT to use source control, but it might be too much for business users. They might be comfortable with OneDrive version control instead.
For the publish and sharing docs the audience for the workshop depends on how it works in your organization. Do you have a deployment process in place or does everyone publish from Power BI Desktop? Do you allow users access to all workspace, or do you have separation of duties on some (production) workspaces? Does the same person do both publish and share? How you proceed with the publish and sharing docs will always depend on your environment.
If you figure out that the governance requirements would fundamentally change how the users work with Power BI or turn off settings that are crucial to the way, they work you have a huge communication task ahead of you and need strong management backing.
This concluded part 4, Process and framework pillar. Part 5 will cover pillar 3 Training and support
First of the five pillars of Power BI Governance is the People pillar. The order of the pillars is not important, so all the pillars are equally important.
The people pillar is about having the right roles in place and actually recognize that people who have those roles need time to perform them. There are no fixed roles in Power BI but there are some roles that I see many customers have in common. The roles are not always organizational/technical roles but sometimes a set of tasks that the same person performs that are important to the governance of the platform. The Power BI related roles I typically see in organizations are:
Power BI Administrator
Power BI Gateway Administrator
Power BI Auditor
Power BI Supporter(s)
In many cases multiple roles will be covered by one person but there might also be multiple people for a single role. As there is no one way this is done from organization to organization I´m not going to dig deep into the roles.
The main point I always try to make when it comes to roles in the Power BI governance effort is that organization acknowledge that these roles exist even though they are not described in the persons job description. I want them to also acknowledge that the roles require time from the person performing them. So, it´s about placing the hat (role) and either allocate time or understand this role is being performed at the cost of other roles the person has. All to often I see that these roles are unofficial/invisible, and people are expected to perform them besides their “normal” day job which is most often developing in Power BI. If you work in Power BI or are responsible for Power BI in your organization, I encourage you to figure out which tasks your Power BI people are doing besides developing and try to figure out if there is a need for a role description and time allocation for the role to be performed to the standard your organization wants. You should also consider if they need training to perform these tasks effectively and to a high standard.
This concluded part 3, People pillar. Part 4 will cover pillar 2 Processes and Framework.
This is part 2 of my Power BI Governance series. You can read part 1, Introduction to Power BI Governance here
Governance strategy
In Power BI, as with so many other things, the main governance issue is people. You are trying to influence people’s behavior with either guidance or technical restrictions. Although technology does help in many cases, more often than not, governance is about influencing people’s behavior with training and best practice documents. Keep that in mind when you design your governance strategy. Don´t focus too heavily on technology. Having well trained users that know how to use Power BI in the right way is the best way to stay compliant.
Having a good governance strategy and implementing it properly is a huge step in securing compliance. In this part 2 of my Power BI governance series, we will explore what you should keep in mind when creating your governance strategy and what the key things to implementing it successfully are. A governance strategy is most often a separate document describing the purpose and goals of your governance effort. It, most likely, won´t go into details of the controls themselves.
When you start creating your governance strategy there are few things, I think. you should keep in mind:
Consider current IT Governance strategy
In your organization is Power BI:
Enterprise BI tool
Self-service BI tool
Managed self-service
All the above
Other considerations
How sensitive is your data?
How do developers and users work with Power BI?
How experienced are your developers?
What kind of security requirements and/or industry standards do you have to adhere to?
How much audit trail do you need?
The answer to the questions above should get you one step closer to figuring out what your governance strategy should contain. You can then use the five pillars of Power BI Governance described in part 1 of this blog series to help you understand what topics to cover in your strategy.
When it comes to implementing a Power BI governance strategy there are few things that can help you to be successful with it.
The key to success is, in my mind, is:
That you secure management buy-in. Without management buy-in you will have hard time implementing your strategy. Governance is often about restricting people and making them use tools in a certain way which might be different to what people want to do. Convincing people to follow your strategy without management backing will be an uphill battle in most cases.
Find a way to document your control measures. It might sound very simple but deciding before you start how you are going to document the strategy and the controls you will implement can be extremely beneficial. You need to make sure that the documents are easy for users to find, read and understand. What the right level of documentation, language and storage are for your organization will depend a lot on what your users are used to. If you are in a highly regulated business your users will be used to reading and understanding heavy texts and will know where governance documents are stored. If on the other hand you are operating in a business where users are not used to that, you might need to keep the documents on a lighter level so that you don´t risk users dismissing them or not reading them properly. There are several techniques that you can use to help your less experienced users to understand governance documents such as having short summaries at the top with key takeaways or breaking them into smaller documents that don´t require as much reading.
Figure out how you want to enforce the controls you put in place. If you put in place controls that you expect people to follow you need to be able to enforce them. When you set up your controls you need to ask yourself two questions. How do I understand if the control is being followed or not and how do I react if they are not? If you don´t know if your controls are being followed or not, they are not very useful. Yes, they might help people use Power BI correctly but it´s very important to understand if they are or not. Likewise, you need to know how you will enforce the control if people are not following them as if you don´t do anything or if you react in an unpredictable way it´s hard for people to take the control seriously. This is where management backing is very important as they usually have bigger say in how people behave.
This concludes part 2, Power BI Governance Strategy. Part 3 will cover the first pillar People
This is the first part in a 7 part series on Power BI governance. I will add links to the next parts as I publish them.
Part 1. Introduction to Power BI governance
Governance can mean many things and often different things to different people. In this article series I want talk about my view on Power BI governance and what I think you should be doing when it comes to governing your Power BI environment.
Governance is about making sure the right people do the right thing within the defined boundaries of the organization. We need to make sure the BI system (Power BI) does not expose data to the wrong people and that the artifacts are stored, shared, and maintained in the right way. Furthermore, we need to make sure that the users, creators, and administrators know how to use, manage, and secure the artifacts
As Power BI is partly self-service, it is vital that the governance is implemented early and in such a way that it does not impede creators and users unless necessary. Being restrictive in the wrong place can lead to implementation failure and un-governed solution frequently known as Shadow IT. It´s important to tread carefully to avoid that situation but at the same time make sure your organization is compliant and secure
Governance strategy
In my opinion Power BI governance strategy has 5 pillars, People, Processes and framework, Training and support, monitoring and Settings and external tools.
Most of these pillars are non-technical. Only Monitoring and Settings and external tools are technical. This often distracts organizations as many like to think that problems should be solvable with technology. The reality is that technology can only partly help. As with so many other things the main governance issue is people. Having well trained users that know how to use Power BI in the right way is the best way to stay compliant.
Having a good governance strategy and implementing it properly is therefore a huge step in securing compliance.
The 5 pillars cover all of what your governance strategy implementation should cover (in my mind).
The people pillar is about having the right roles in place and actually recognize that people what have those roles need time to perform them. All too often I see that people have unofficial Power BI roles with no time allocation. For example, I see with few of my clients that the Power BI Administrator is the best Power BI person in the company who is expected to do the administration besides their Power BI development. It might work and often does but it should still be recognized that it takes time and it comes with responsibility which requires it to be done properly.
The processes and framework pillar is about having the proper documents in place so users can use Power BI correctly and be compliant. Processes or best practices are document that describe how to use or administer Power BI. Frameworks often describe the method on which you base the process/best practice documents on.
The training and support pillar is about making sure everyone that uses Power BI has gotten the required training. Here you will describe your training plan, decide what type of training each user type should get and how to make sure you reach everyone with your training. It´s also here you might describe how you support your users going forward with things such as internal user groups or subscription to external training library.
The monitoring pillar is about setting up monitoring of Power BI. Usually, it involves extracting data from the Power BI activity log as well as the Power BI REST APIs for information about existing artifacts in your Power BI tenant. Sometimes you might extract data from other parts of Microsoft 365 such as employee data to supplement the activity and inventory data. This part of the governance effort is both about describing your monitoring (documentation) as well as implementing it.
The settings and external tools pillar is about making sure Power BI settings are correctly sat as well as how to use other approved tools to support Power BI. Here you will describe all the settings and their correct value in a document. You will also describe how other tools such as Microsoft 365 sensitivity labels or Tabular Editor should be used with Power BI.
This concludes part 1, introduction to Power BI Governance. Part 2 will cover Power BI Governance strategy
This summer has been very eventful for me and my family and not just because of the world pandemic. Some of you might have noticed that I haven´t been very active in blogging or on social media for couple of months. This is due to some happy personal situations.
To recount what has happened we need to start by going back to June. On June 24th we had our third child. She´s fantastic and perfect like her older sisters. It´s been 11 years since our middle one was born so it took some time getting back into the baby parenting role again but I´m thoroughly enjoying it.
On August 1st we moved from Iceland to Denmark. We had been planning the move for a while as we wanted to be closer to my wife’s family as I travel quite a bit (or I used to at least). The plan was to move at the end of May when our older daughters were done with exams in their school but before our youngest was born. Unfortunately, the COVID-19 pandemic prevented us from taking to Denmark to look at houses. I managed to do that on June 15th, less than a week before the expected birthdate. My wife and I made a pact. I would find us the perfect house and she would keep the baby in until I came back. Both things worked out as planned ?. It´s a big thing moving between countries separated by an ocean. You need to pack your stuff into a shipping container two weeks before you want to receive it in the new house. This meant we risked being homeless for 14 days with a newborn. Luckily my sister could loan us an apartment, so it worked out well.
These two big events took all my energy in the last 2-3 months as you might understand.
Now I´m back to full work and ready to start to contribute more to the community and participate more.
While I was in my online hiatus some great professional things happened as well.
My MVP award got renewed for the third time
I got selected to speak at PASS Summit
I got selected to speak at SQL Saturday Gothenburg on September 5th (really looking forward to that one)
I´m still speaking at SQL Bits although now virtual. I have a training day that I´m adapting to online delivery. That is going to be awesome.
#DataWeekender #TheSQL has opened call for speaker.
Me and Michael Johnson finished the first draft of the book we are writing
Expect to see some posts about the coming events from me and also some posts where I continue my series on Power BI monitoring and governance.
If you like to connect or get in touch you can subscribe to my newsletter in the box to the right or find me on social media. If you like to see what we have to offer you can navigate to https://northinsights.com
Power BI is essentially a self-service BI tool where users traditionally have a lot of freedom to create the reports and dashboards that they need and organize it in a way that suits them.
If you are a Power BI admin or if you´re concerned with governance or security, you often want to know what´s going on in your Power BI environment. Since Power BI is first and foremost a self-service BI tool, Microsoft has not (yet) developed good, out of the box, monitoring tools. This means that you need to develop your own way of monitoring Power BI.
This series of blogs describe what you should be monitoring in Power BI and what method works best for each.
The blogs are:
Power BI Admin Portal Settings
Power BI Artifact Inventory
Power BI Activities
Power BI Capacities
We will start this blog series with looking at how you should monitor your Power BI Admin Portal settings.
Part 1. Power BI Admin Portal
The Power BI Admin Portal is the place where the Power BI Admin can change settings and monitor certain things.
One of the main points of interest is the Tenant settings. Some of the settings that you can change in the Tenant settings part of the portal are who can publish to web, who can share externally, who can create workspaces and where the internal help portal is. There are in all, at the time of this writing, 31 settings you can change. Some of them are fine in the default settings while others like Publish to web should be changed as soon as possible.
Besides the Tenant Settings some of the other things you can change are Capacity Settings, Dataflow Settings, look at all workspaces in the tenant, turn on audit logs, brand the Power BI portal, manage Protection metrics and add Featured content. What ever you decide to change the purpose of this blog is to encourage you to document and monitor the settings
Figure 2: Power BI Admin Portal
Record and monitor Tenant settings
It´s very important that the Tenant settings are documented and monitored regularly. Unfortunately, you cannot monitor these settings automatically, so someone needs to login to the portal and manually check the settings. We recommend that you write down all the settings and have the admin check them once a month. This is especially important if you have more than one administrator. The main reason for that is that any change made in the portal is not logged anywhere you can access. If you have not written down how you want the settings to be, it´s very difficult for an admin to know if the settings are correct as they cannot see if they have been changed unless they remember the previous setting.
Figure 3: Example of Power BI Admin Portal Settings documentation
Besides the Tenant settings we recommend that you turn on Audit logs which are needed for activity monitoring and review Embed codes to make sure there is no sensitive data being embedded outside of an approved system. If you have Power BI Premium you can also use the Capacity settings to control your capacities.
Conclusion
Go through all the settings in the Power BI Admin Portal. Change the settings as needed and then documents every setting. Manually monitor that the settings have not been changed at least once a month as a part of your governance process. Turn on Audit logs and make sure there are no reports being embedded outside of approved systems.
Come back for the next blog on Power BI Artifact collection and monitoring
If you want to discuss Power BI monitoring or governance or get help with implementing it in your organization please contact Ásgeir Gunnarsson on asgeir@northinsights.com or go to https://northinsights.com and find out what we offer and how to get in touch. We offer consulting and advisory as well as training on the whole Business Intelligence lifecycle including Power BI.
Power BI, like many other self-service BI tools, suffers for its dual purpose of being self-service but also used as an enterprise BI tool. Power BI started out as a pure self-service tool but has increasingly been moving to be more of an enterprise tool and can rightly be called a hybrid BI tool. No matter if you use Power BI as a self-service tool, as an enterprise BI tool or both, it’s important to include governance into your implementation. Far too many organizations start using Power BI without thinking about governance and then have the problem of trying to get their users to stop doing things as they are used to, and to start using process they are not used to and often feel will hinder their progress.
No matter if your organization is starting its Power BI journey or has already ventured in the Power BI “Wild West”, governance is an important and necessary part of any Power BI implementation.
This article will focus on the 4 pillars of a good governance strategy:
Figure 1: The Four Pillars of Power BI Governance Strategy
Process
At the heart of a governance plan are processes. There can be many smaller processes or few bigger ones but without them there is not much governance.
It´s important to have a formal governance process in place. This process is often broken down into smaller processes and usually contains processes for Development, Publishing, Sharing, Security, Naming standards, Support and Tenant Settings.
Most often these processes describe how to work with Power BI and sometimes they describe how to support Power BI. It´s vital that the processes are easily discoverable and are setup as a part of a whole so that users will know how each process ties into the whole governance strategy. One way is to have one master process document with links to all the process documents. Another way is to store all the process documents in the same library and categorize them so it´s easy to navigate between them and they are logically grouped.
We will look at the seven most common (in the our opinion) processes and see examples on what they might contain.
Development process
A development process most often describes how a report, datasets or both are developed. They describe where you develop the Power BI content and how you store and version your files.
Publishing Process
The publishing process usually describes how to set up multiple environments and how to promote Power BI content between them.
Sharing Process
The Sharing process describes how to share reports, dashboards and datasets and links to the security process for more details.
Figure 2: Example of a sharing process image
Security Process
A security process describes how to secure Power BI content. This is usually split into two categories: Object level security and data security.
Naming standard process
One of the most undervalued process is the Naming Standard process. Having this process early in the Power BI implementation will greatly improve the usability of the Power BI environment. Finding workspaces, reports, dashboards and datasets can be very tricky when you have hundreds of workspaces with no clear naming convention.
Support process
Many organizations neglect to create a proper support organization when implementing Power BI (see Roles section). Having a good support process will enable your current support organization or dedicated Power BI support people to more easily assist users when needed. A support process will help non-Power BI supporters to know when to dig in and try to solve a problem and when to refer the problem to the report owner or Power BI Support people.
Tenant Settings process
There are several settings in the Power BI admin portal that are important when it comes to governance. Publish to web, Sharing outside of organization, Export data, Internal support page to name few are all very important for different reasons. As there is no way to monitor the tenant settings automatically, it´s very important that the organization has a process in place defining how each setting in the Power BI Admin portal should be set, who the setting should apply to and describe why it´s important.
Training
If you want to have a successful Power BI implementation training is very important. You want to train everyone who touches Power BI but in a different way depending on their role. You want to make sure you get to everyone and deliver the right training based on their needs. It´s not only governance training that is important. Training users in properly using Power BI and using best practices will deliver value faster and will make report and dataset developers more compliant.
One of the things we have been exploring is to automate the training offer to users by using Microsoft Flow in combination with Office 365 (who has license) and the Power BI activity log (what are they doing). When a user gets a license or when they publish their first report or dataset they receive an email with the training being offered in the organization as well as relevant document and processes needed for their role.
Training categories
The most common training categories are Consumer, Report Developer and Report and Dataset Developer. For each category there is a definition of who belongs to it as well as what training content is appropriate and how it should be delivered. Most often the Consumer training is delivered either as videos or training manuals. Developer training is most often either classroom training or online training course.
Monitoring
One of the cornerstones of governance is monitoring. Monitoring what users are doing and monitoring what users are creating. From a governance perspective monitoring creation, access, usage, changes, deletion and data exports are the most important. Besides that, monitoring the settings of Power BI Admin Portal is very important.
The Power BI Rest API can tell you what artefacts exists and who has access to what. Besides that, the Rest API has powerful administration endpoints that allow you to get information about various administration objects as well as allow you to perform admin tasks. To access the Power BI Rest API, you can either create your own web application and call the API or you can use PowerShell to call it. Microsoft has put some effort into wrapping many of the endpoint in the API into PowerShell cmdlets and they have also created a cmdlet to wrap the call to the Rest API. You can read more about the PowerShell cmdlets at https://docs.microsoft.com/en-us/powershell/power-bi/overview?view=powerbi-ps and you can read more about the Power BI Rest API at https://docs.microsoft.com/en-us/rest/api/power-bi/. The user will have access to some of the endpoints through normal Power BI workspace access, but a lot of the endpoints require the user to be a Power BI administrator, at least if they want tenant level information.
Monitoring usage
The Power BI Activity log can tell you who accessed what and who changed or deleted what. The Power BI activity log is turned off by default but can be turned on in the Power BI Admin Portal. The activity log can be accessed in two places:
Power BI Rest API
The last 30 days of the activity log is now replicated from the Office 365 Security and Compliance to the Power BI admin portal and you can get to that with the Power BI Rest API if you are a Power BI admin.
In the Office 365 Security and Compliance
The activity log is part of the Office 365 Security and Compliance Centre. To get access to the activity log in the Office 365 Security and Compliance Centre you need to have the View-Only Audit Logs or Audit Logs role in Exchange Online or be an Office 365 admin. It is possible to fetch data from the Office 365 acitvity log in two ways. One is to log into the Office 365 Security and Compliance Centre, run the log query and either view the results on the screen or download the results as a CSV file. Another way is to use the Office 365 Rest API which is the preferred way if you want the automate the collection of the log information. Note that the log is only stored in the Office 365 Security and Compliance Centre for 90 days so if you want to keep it for a longer time you will need to collect it and store it in a different place such as data warehouse. More information about the acitivty log and how to collect the data can be found here: https://docs.microsoft.com/en-us/power-bi/service-admin-auditing.
We recommend that both the activity log and artefact inventory is collected and stored in a database. Partly because of governance issues as described before but partly because there is valuable information in there about adoption, development over time and user behavior which could be beneficial for the organization at a later time.
Figure 3: Monitoring Power BI strategy
Monitoring Power BI Admin Portal
There are several settings in the Power BI Admin Portal that are very important when it comes to governance. One of the main points of interest is the Tenant settings. Some of the settings that you can change in the Tenant settings part of the portal are who can publish to web, who can share externally, who can create workspaces and where the internal help portal is. There are in all 31 settings you can change. Some of them are fine in the default settings while others like Publish to web should be changed as soon as possible.
Besides the Tenant Settings some of the other things you can change are Capacity Settings, Dataflow Settings, look at all workspaces in the tenant, turn on audit logs and brand the Power BI portal.
Figure 4: Power BI Admin Portal
It´s very important that the Tenant settings are documented and monitored regularly. Unfortunately, you cannot monitor these settings automatically, so someone needs to login to the portal and manually check the settings. We recommend that you write down all the settings and have the admin check them once a month. This is especially important if you have more than one administrator. The main reason for that is that any change made in the portal is not logged anywhere you can access. If you have not written down how you want the settings to be it´s very difficult for an admin to know if the settings are correct as they cannot see if they have been changed unless they remember the previous setting.
Besides the Tenant settings we recommend that you turn on Audit logs which are needed for activity monitoring and review Embed codes to make sure there is no sensitive data being embedded outside of an approved system. If you have Power BI Premium you can also use the Capacity settings to control your capacities.
At the time of this writing there are 91 events that are monitored in the Power BI audit log. If your organization does not want to store all that data you should consider taking all events that are about viewing, editing (including deleting) and exporting. When you have started the collection of the data you might want to join it to further information from the artefact inventory discussed in the previous section as well as information about the organization employees and the organization structure.
Monitoring the Power BI On-Premise Gateway
The Power BI On-Premise Gateway is a Windows service running on an on-premise server. The gateway needs to be monitored as other Windows services. The main things to monitor are the service uptime and server performance. Normally monitoring is in the hands of an infrastructure team (if one exists).
Roles
To be successful with a Power BI implantation in the long run it´s important to have well defined roles. This is most likely different from organization to organization and in some cases the same person might have more than one role. The most common roles are Power BI Administrator, Power BI Gateway Administrator, Data steward, Power BI Auditor and Power BI Supporter(s).
Automation
For the training and processes you can automate the discoverability by sending content to users as soon as they get a license or as soon as they create content. That way you are sure that all your users are aware of the processes and training and you can control what they receive depending on where they are in their Power BI journey.
Summary
This article suggest that a good Power BI governance strategy has 4 pillars, Processes, Training, Monitoring and Roles. Organizations need to define processes so that their users do Power BI right, train them to follow the processes as well as best practices when it comes to Power BI, Monitor the Power BI environment and have defined roles and responsibilities. Each pillar has equal importance and for a successful Power BI implantation you want to make sure you think about them all.
Governance is a necessary part of a Power BI implementation and the earlier you can start the easier it will be.
If you want to discuss Power BI governance or get help with implementing it in your organization please contact Ásgeir Gunnarson on asgeir@northinsights.com or go to http://northinsights.com and fill out the contact form. We offer consulting, training and advisory on the whole Business Intelligence lifecycle including Power BI.
We have been considering many options when it comes to how we can incorporate glossary information into our Power BI reports. We have started the work of creating master data around our glossary and are considering how best to get it into our cubes.
When Microsoft announced the tooltip pages for Power BI I saw that this could be the way we display the glossary in our reports. So, while we solve the problem of getting the glossary into the cubes I decided to try to add the glossary manually to a tooltip page to see how it looks.
Traditionally we have been showing support measures in the tooltip. An example of this is when Net Sales Growth is in a graph the tooltip would show Net Sales and Net Sales Growth %. I really like how you can for most visuals put in multiple tooltip measures, but my problem has been that everything in the visual is displaying in the tooltip. For an example we sometimes include a measure only meant to colour the graph. This can be a Net Sales vs. EBTIDA measure or something like that. This measure is not meant for displaying in the graph but because you don‘t have any control over the tooltip it is displayed.
With the tooltip page you have full control over what you display and what you don‘t and you get more space to work with. In the images below, you see that when looking at Gross Profit Growth in a graph we were displaying the Gross Profit and Gross Profit Growth and are using Gross Profit Growth % vs Net Sales Growth % to colour the graph. This is nice enough but the extra colouring measure is annoying and you are, in my mind, missing the definition of Gross Profit.
Below you see an image of how we are now doing using the new tooltip pages. Here we are showing prior year Gross Profit, current period Gross Profit, the Gross Profit Growth % and the definition of Gross Profit. Now many of you might feel the definition is not important but when you work in a multinational company the definitions often become important. While the definition of Gross Profit is simple the definition of measures such as Cost of goods sold (COGS) can be complicated and so it‘s is important that everybody knows exactly what definition is used.
This is of course just the start and I see plenty of opportunity to relay more messages to the users via the tooltip page. This can be definitions of calculations where appropriate or dimension definitions.
