A current project being worked on starts with a folder in Sharepoint with multiple files that need to be imported together (All with the same Spec)
There are lots of groups of files to import into Power BI
- We need the File Name – it will be used to create the file snapshot date
- There is a header over the columns – this needs removing and then the headers promoting
Get Data > Folder creates helper folders that clutter up the back end and aren’t the best solution.
This is the code that gets created
let
Source = Folder.Files("C:\Users\Name\Company\Project\Data"),
#"Filtered Rows" = Table.SelectRows(Source, each Text.Contains([Name], "FileName")),
#"Filtered Hidden Files1" = Table.SelectRows(#"Filtered Rows", each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File (8)", each #"Transform File (8)"([Content])),
#"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File (8)"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File (8)", Table.ColumnNames(#"Transform File (8)"(#"Sample File (8)"))),
#"Promoted Headers" = Table.PromoteHeaders(#"Expanded Table Column1", [PromoteAllScalars=true]),
You don’t want lots of these files and folders cluttering up the beck end of Power BI slowing things down. Try the following.
We are going to run through this again but change what we do
Get Data / File
Let’s collect all FileA files
Select Binary and Name and Right Click – Remove Other Columns
Now go to Add Column – Custom Column
Lets build this function up
Returns the contents of the Excel workbook. Content was added here. Normally when you click the arrow button on Content, this is where Power BI creates the Helper queries for you which we don’t want.
If you click the new custom column (Just next to Table) you can see the content
Now we can amend this custom column even more
= Table.AddColumn(#"Removed Other Columns", "Custom", each Table.Skip(Excel.Workbook([Content]){0}[Data]))
Table.Skip allows us to skip over the header which start at {0}
And we can even add promote headers to this one bit of code Table.Promoteheaders
= Table.AddColumn(#"Removed Other Columns", "Custom", each Table.PromoteHeaders(Table.Skip(Excel.Workbook([Content]){0}[Data])))
Now we can click the Arrow on the Custom column and bring through all the columns in the table.
No Helper Queries are created and we have done numerous tasks within that one line of code
If you have already created everything you can still do this and grab the code to replace the old code with at the start of each query. then remove the helper queries.