We are going to add some new information into the data set
In Power BI Service, Click Edit … against the #Taskmaster dataset
I’m actually going to add 7 more fields
- Description, Sentiment and Name are Text
- CreatedAt is a date
- Count, Favorited and followersCount are Numbers. Click Done
Return to Microsoft Flow
Edit the flow and go to add rows to a dataset
- Count is an Expression
- It is simply set as 1
You can Test and Save
Any new data that comes through will contain these data items
Creating sentiment is slightly more involved but lets go ahead and create sentiment
After Detect Sentiment update your flow with the following logic
Create Sentiment which is Neutral. If Score is over 0.7 then Positive. If under 0.3 then negative
We can add this to our data set
We now have lots of new information to use in our streaming reports
Back in Power BI Service
I’ve attempted to add a new Custom Streaming Tile to the dashboard based on a Line chart to look at the count of records
Unfortunately this Streaming visual doesn’t seem to work and immediately I can see a fatal flaw with using streaming data set visuals for this kind of data
These visuals are for data that pulls through almost constant stream of data. They are not for data that has a feed that at some points doesn’t have much data coming through. You need to have lots of data in the time frame of the streaming dataset (For example 60 minutes).
I have the #Taskmaster Stream set up for Historical Data Analysis
when Historic data analysis is enabled, the dataset created becomes both a streaming dataset and a push dataset
A streaming dataset, has no underlying database
A Push data set
For a push dataset has a few limitations on how much data can be pushed in:
- 75 max columns
- 75 max tables
- 200,000 max rows stored per table in FIFO dataset
- 5,000,000 max rows stored per table in ‘none retention policy’ dataset
The data is then stored in Power BI and we can actually access it from Desktop.
This means I can create some really nice reports against my taskmaster data set
You cant create Hierarchies of calculated columns over the Push data set. Only Measures. You are very limited on what you can do. The Key Influencers visual is also unable to work with push data which is a shame because this is the perfect visual for analysing positive and negative tweets
I should have brought date across as a date in the first instance because Month is now just a number and I cant change this in Power BI. We have date but only against the very latest data
Time was already in the data set but this consists of date and Time which you cant reset to create a date hierarchy
I cant add day names to the day number(Taskmaster is on on Wednesday so I expect the levels to go up then)
So the push data set is fairly simple to set up but its incredibly limiting to what you can do
Our initial page by Month. I needed to Add Month Name into the Flow
Next I drill through on the 9th of October. the name of the day would be great because taskmaster is on a Wednesday
Finally we drill through to the hourly tweets.
As you can see, We have a few negatives in the data set which, on reading aren’t actually negative. The Cognitive API doesn’t recognise sarcasm for instance.
There you go, We started out with a streaming data set and ended up with a push data set.
The push data set doesn’t need a refresh which is them main reason to go for them.
One last quick thing about this push dataset. When I attempted to republish the report I got the following error
To get past this error I had to delete my report in Power BI Service before trying again. I have never seen this kind of conflict before and I’m assuming its an issue with the Push data set