Let's say there are two attacks: one rolls a 10d10 and the other rolls a 5d20. Assuming the modifier was the same, which rolls would have better chances of rolling average, and which would have better chances of rolling the minimum or maximum?

To specify I am aware that more dice would have a higher avg and min roll but both of those would be by a slim margin and I'm not sure how those would affect your "odds" of hitting each number.

From what I can gather what you're asking, you want to know the probabilistic difference between rolling 10d10 and 5d20. You've rightly pointed out that each roll has the same maximum and that each has a better chance at rolling their given averages. They obviously have different minimums (10 vs 5), and so you want to know precisely how different the rolls are.

Using AnyDice.com we can calculate the probability very simply with the commands output 10d10 and output 5d20. And really that's all there is to it. The black line below represents 10d10, and the yellow line represents 5d20.

Generally speaking, when you have a greater number of smaller dice, your rolls are less "swingy". Meaning, there are better odds at rolling the "average". But, you have worse odds at rolling higher numbers.

Put another way: Look at this graph, it represents the odds that you will roll at least a given number. You can see in general it's better to roll 10d10 because you have greater odds at hitting a certain number until about 60, then 5d20 gives you better odds at hitting those values, but only slightly.

So with 5d20, you have higher odds at hitting a greater range of values, meaning that if you roll 5d20 often, you'll see more "swingy" results. But with 10d10, the odds are more stacked in the middle, meaning it should feel like you're hitting the "average" or the "middle" results more often.

But let's simplify. Lets look at output 2d10 vs output 1d20. Same idea as 10d10 vs 5d20. Here we can see that each value of 1d20 has an equal chance to be rolled. But with 2d10, the odds change because there are a greater number of rolls that represent the middle values (11). there's 10-1, 9-2, 8-3, 7-4, 6-5, 5-6, 4-7, 3-8, 2-9, and 1-10 representing 11. 10% of all the combinations are 11. But for higher values (20), there is only 10-10 representing that, which is only 1% of all possibilities.

Similarly, if you wanted to compare 1d100 to 5d20 and 10d10, you would see a flat probability: a 1% chance for each value between 1 and 100.

More dice lead to more average results

There are a number of dice calculators around the internet to illustrate the probabilities of whatever combinations you'd like to see, but a general statement is that rolling a combination of dice and summing them for a result will increase the probability of average results and reduce the probability of extreme results as compared with rolling fewer dice.

For a simple example consider 2d4-1 and 1d7 (this is for you @SevenSidedDie) both produce a number between 1 and 7. Your chance of a 1 with the 7-sider is 1 in 7 or about 14%. Your chance with the 2d4 is 1 in 16 (6.25%), because there are 16 different results possible, but only one of them is the 2 ones which give you a 1. On the other hand there are 3 ways to get 3: 1 & 3, 2 & 2, and 3 & 1, so 3 chances in 16 (18.75%), but still only 1 in 7 with the 7-sider.

I'll just give a decent example. Rolling 1d6 is comepletely random. Every part has only one way to be rolled, and thus there are an equivalent amount of ways to roll each number. Given that the die is not weighted, then each roll is random. Rolling 2d6 means you suddenly can roll 6 in multiple different ways. The greatest example is that you can only roll "2" one way via 2d6, two "1s". However, 3+3 = 6, 2+4 = 6, 1+5 = 6, 4+2 = 6, and finally 5+1 = 6, so that means there are five ways to roll a six.

This is actually something relavent in the game Mekton. There is an optional rule people use to use 2d6 instead of 1d10. It was introduced in the liscensed spin off, (not created by tallorn games) Gundam Senki. I've been in a campaign where limited Senki rules were used instead of Metkon rules.

Also, it means a critical success, and thus exploding dice, was limited to a 1/36 vs a 1/10 chance. This is actually desirable. I'm playing in a campaign that's ironically Gundam themed, and we have quite the instance rate of critting and then disabling various famous mobile suits. Half the Gundams were captured by Zeon by killing the pilot, including the full armor 7th gundam, Netix, and a ton of interesting stuff. With a 2d12 system, the chance of that happening drops drastically, even if you still use the 1d10 table to roll for critical damage, to 1/180 vs 1/50 (there are two ways knock out enemy mobile suits without killing). It's slightly less than that, as there's more ways to disable a mekton without called shots, but there's my comparison. Still, we've had not quite good luck to die to a ball randomly, like had happened to Dozle, who took one right in the face.

Die rolls have mean equal to the average of the largest and smallest number so for a die with f faces (a "df"), the average is (1+f)/2 and the variance is equal to the mean times (f-1)/6; i.e. (f+1)(f-1)/12. The mean and variance of a sum of dice is the sum of the means and the sum of the variances respectively. If the dice are all the same (ndf), then the mean is n(f+1)/2 and the variance is n(f+1)(f-1)/12.

The chance of rolling the minimum (or the maximum) is (1/f)^n - if you halve the faces but double the number of dice, the second case has a larger minimum but the same maximum, but the maximum will be more likely with the larger die.

If you're adding more than a few dice, the probability of the nearest-to-average roll will be roughly 1.38/[f√n]; if you halve the number of faces and double the number of dice you move the average up by 1/2 for each die you started with, and the probability of rolling the nearest-to-average roll will increase by roughly 42%. (These numbers are a bit more accurate with more faces and more dice and not so accurate with few faces and few dice.)