Using that math you are figuring if there was a spawn 5 out of 5 times which would be the 10%*10%*10%*10%*10=.00100%.
Yes if the chance it 10% to spawn a boss and there are 5 mobs, then the chance to get 5 bosses from 5 mobs is exactly 0.001 %
The chance to spawn EXACTLY 1 boss is 0.1 * 0.9 * 0.9 * 0.9 * 0.9 * 5 (ways this can happen) = 0.32805 = 32.805 %
The chance to spawn EXACTLY 2 bosses is 0.1 * 0.1 * 0.9 * 0.9 * 0.9 * 10 (ways this can happen) = 0.0729 = 7.29 %
The chance to spawn EXACTLY 3 bosses is 0.1 * 0.1 * 0.1 * 0.9 * 0.9 * 10 (ways this can happen) = 0.0081 = 0.81 %
The chance to spawn EXACTLY 4 bosses is 0.1 * 0.1 * 0.1 * 0.1 * 0.9 * 5 (ways this can happen) = .00045 = .045%
And as you said the chance to get all 5 is 0.001 %
Since all of these outcomes don't stomp on each other we can add them to =
40.951% of 1 OR MORE bosses spawning from 5 spawns
Checked by my original method of NO bosses spawning 0.9 * 0.9 * 0.9 * 0.9 * 0.9 = .59049 = 59.049%
Means the chance of 1 or more spawning = 100% - 59.049% =
40.951%