Parse Code from Markdown Files¶
Introduction¶
Are you testing your documentation? If you write PowerShell scripts or modules, you are hopefully using Pester to test your code. And if you use PlatyPS to generate markdown documentation like I do, then you have a bunch of example PowerShell code sitting in .md files. But what happens if you rename a command, a parameter, or make a breaking change?
Your documentation is the face of your product. It's the source of truth for the people who use it, whether it's a PowerShell module or something else entirely unrelated. When your examples have errors in them, it won't be obvious to everyone. Some people may copy and paste your examples, see an error, and move on. Maybe they see the use of aliases and other coding patterns that are generally not recommended to use in source code or documentation and pick up those habits, or they become unsure about the overall quality of the product behind the documentation?
The MilestonePSTools PowerShell module I work on has 413 markdown files under the docs folder, and 394 of those files were generated by PlatyPS for commands in the module (in English and Spanish). I have a bunch of tests for the module itself, but until today I was not testing any examples or other PowerShell code blocks found in the documentation.
Oh aliases...¶
My PowerShell journey started in 2019 when I began building a module. I was learning PowerShell at the same time I was building what would become a commercially used module, and learning best practices and common patterns from the community. One important best practice I failed to learn early on was use a prefix for the nouns in command names to prevent collisions with commands from other modules. So after I while, I started to add a "Vms" prefix to the commands in the module, and I started renaming commands and adding an alias to the new command matching the old one to help prevent breaking changes.
The Pester test screenshot at the top of this post shows that there are some old pre-prefix commands still in use. At the time the documentation was written, these weren't aliases at all. But they are now, and people reading this documentation might be confused about the command names, or they may just naturally start using the alias version of those commands because it's in the documentation so it must be right!
Demonstration¶
Let's take a look at an excerpt of the docs from another command, this time in markdown format. In the first
example for Update-Bookmark, I used the "%" alias in place of ForEach-Object. To be fair, I wanted to keep the example
line from being too long. But I know there are better strategies to achieve that.
The Get-MdCodeBlock command uses regular expressions to determine whether a line represents the beginning, or end of
a code fence, and whether inline code is present in that line. If a language shortcode is used, that information is
grabbed and returned with each code block. For the markdown example above, that looks like...
Get-MdCodeBlock -Path .\Update-Bookmark.md | Select-Object Source, LineNumber, Position, Inline, Language | Format-Table
# Source LineNumber Position Inline Language
# ------ ---------- -------- ------ --------
# Update-Bookmark.md 9 0 False
# Update-Bookmark.md 15 4 True
# Update-Bookmark.md 30 0 False powershell
For brevity I didn't include the Content property in the example output above, but you can probably see the value in
checking all of the example code you wrote years ago and never looked at again, despite the code base seeing dramatic
changes and growth over time.
Sample Pester Test¶
Here's a basic Pester test which uses Get-MdCodeBlock to extract the powershell example and pass the content to Invoke-ScriptAnalyzer.
Describe 'Markdown Tests' {
Context 'PowerShell Code Blocks are Valid' {
BeforeDiscovery {
. $PSScriptRoot\Get-MdCodeBlock.ps1
$script:codeBlocks = Get-ChildItem '*.md' | Get-MDCodeBlock -Language powershell
}
It 'Analyze codeblock at <_>' -ForEach $script:codeBlocks {
$analysis = Invoke-ScriptAnalyzer -ScriptDefinition $_.Content -Settings PSGallery
$analysis | Where-Object Severity -ge 'Warning' | Out-String | Should -BeNullOrEmpty
}
}
}
I absolutely love having this improved visibility into the health of the documentation. The tests call out the file, line number, and give me the formatted output from PSScriptAnalyzer. And you can get even more creative by using the PowerShell language parser to extract an abstract syntax tree and inspect all code hiding in markdown files for just about anything.
Code¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 | |