Not so random loot (again)

from Elemental Forums

Hi all,

Just been doing a bunch of playtesting, have got a few mods installed... and I notice I seem to quite often get repeat drops of loot from goodie huts.  Eg. item A drops twice in a row, then something else.... then item B drops twice in a row etc.

I've found like 10 items so far, and 4 of them have been duplicates.  Same thing happened previous game.  Based on pure probability that should (almost) never happen.

Found other old threads on this here:

https://forums.elementalgame.com/450227/page/2/#replies

https://forums.elementalgame.com/445758/page/1/#3374010

 

Just wondering if anyone else still gets this strange behavior from goodie hut loot ?

 

Edit -> There is definitely something funky with the loot randomizer.

 

Thanks.

21,616 views 21 replies
Reply #1 from Elemental Forums Top

Yes, I do.  It would be nice if the loot was more random in LH.

PS: You might want to change the forum to Legendary Heroes (a subforum of Fallen Enchantress).

Reply #2 from Elemental Forums Top

Quoting StevenAus, reply 1

PS: You might want to change the forum to Legendary Heroes (a subforum of Fallen Enchantress).
End of StevenAus's quote

Oops mistake, fixed it now thanks.

Galactic Civilizations III
Reply #3 from Elemental Forums Top

Moved to LH Support.

@Stardock - can this be looked at for the next patch (whenever that is).  There is definitely something strange about the randomization of loot drops... it is *very* common that you get 2 or 3 of the same item dropped from a goodie hut in the same game, within the first few drops even.

I just got 2 silence scrolls as the first 2 items items dropped from goodie huts.  I haven't even seen this item before.... then all of a sudden I get 2 of them in the first 2 drops.

o_O

Reply #4 from Elemental Forums Top

And just got 2 Guardian Blades in a row from consecutive Treasure Chests. 

Edit-> Just scored another one.

 

And i've got 4 Ice Spears now, from about 25 items.

Reply #6 from Elemental Forums Top

O.k., I'm alive...

 

Creating additional loot will NOT improve randomness

Yes, that's counter-intuitive, but simply put: no matter the size of the map and the number of treasure chests you loot, you will not (randomly) get all the common, uncommon, or rare items available in the xml. The maximum number of randomly available items per class of items (Common, Uncommon, Rare) seems to be 54. The list of items seems to be created at the same time as the map generation. This means duplicates (triplicates, etc.) will be occurring much more than expected. Now, there might be other reasons why this is happening, but I assume this is main reason. This also means that many interesting items may not make it into your current game while rusty bracers might. Worth mentioning, although this has been implied in my initial assertion, non-random loot can always be added on top of the 54 limit.

 

How to fix this

Assuming this isn't something that can be tracked and fixed by Stardock, the simple, but tedious solution, is to create additional class of items for every set of 54 items: Common1, Common2, Common3, Uncommon1, Uncommon2, etc. Then, you have to fix all the Treasure Chests / Creatures / etc. so that they can drop all related class of items equally (Common1-2-3-...). Doing this will 1. prevent LH from removing items from a game (e.g. Instead of, say, 54/162 Common items making it into your current game, you will get 54/54 Common1, 54/54 Common2, 54/54 Common3), 2. will increase randomness (e.g. 33% probability for each pool of items) and, 3. reduce duplicates, triplicates, etc. Note: I've tried it and it works.

 

_________________________

LH Mods by Primal

XtraDeconstruct

XtraDeconstruct Canons

Reply #7 from Elemental Forums Top

I can verify this from a while ago when I created some additional tiers for loot.  I set all the loot for each tier to the same values and ended up with a greater loot diversity.

Reply #8 from Elemental Forums Top

Hmm this is good info thanks guys.

Reply #9 from Elemental Forums Top

To sum up

In the XML

- change "RarityDisplay" from common -> common1/2/3 

- change dropsites to handle the new rarities.(does this have to be done in core files?)

Primal_Savage can you show a XML example where and how to do the second step?

 

Does anyone consider adapting this in a mod?(Black Market Bazaar could be a candidate)

 

As to the flaw in game, it does sound like they have a hardcoded limit of 54 in the game(string or record length), somewhere after the randomization algorithm has run. That should be an easy fix once they find it.

Reply #11 from Elemental Forums Top

mmmm soul-destroying

I already love the sound of that, this is going to be a party! it is just a question of selecting some music that matches the assignment and get to work

 

i suggest Sunn O)))

https://www.youtube.com/watch?v=v90ldK_8f_U

Reply #12 from Elemental Forums Top

 

I have walked through the code and there isn't any max of 54 items for each rarity.

This is the game logic:

1. Start a count at 0.

2. Go through all of the items in the game (this includes all dlc, mods, etc). If that item matches the given rarity (ie: Common) then add that items liklihood to the count.

ex: Spear, rarity Common, liklihood 20, count = 20 (former count + 20)
     Hate Stone, rarity Common, liklihood 20, count = 40 (former count + 20)
     Amulet of Haste, rarity Uncommon, skipped

3. Once it has cycled through all the games items it has a count which is equal to the sum of all the objects of that rarity. It then generates a random number between 0 and that total count.

4. The game goes through all the items again, if the item matches the supplied rarity it subtracts its liklihood form the random number it generated.

5. Once the random number get to 0 or below it stops and the item it is on is the one that is given.

 

Now this isn't quite the function I would have written for this (we can certainly come up with examples that do the same in one pass), but it works. The problem prior to 1.7 was that the data type it uses to store the total liklihood wasnt large enough for all the liklihoods added up. Once that number hit its max, no toehr items were counter. This wasnt a problem with the base game, but you could run into it with mods and/or DLC. To fix it we reduced the liklihoods to a tenth of what they were (base liklihood on common went from 400 to 40) so we can now cycle through 10 times as many items.

Mods would also have to be updated to use the same scale or else items from them will drop a lot more frequently than base game items or they run the risk of running out of the count cycle (if they have as many items as the base game).

 

Reply #13 from Elemental Forums Top

Hi Derek,

Thanks for the response, always great to hear from you.

 To fix it we reduced the liklihoods to a tenth of what they were (base liklihood on common went from 400 to 40) so we can now cycle through 10 times as many items.

End of quote

Ahhh that's interesting, ok I had totally missed that in the 1.7 update.

:-"


My mod overwrites a lot of default weapons etc and I've still go the old likelihood values (200 for Uncommon) against lots of gear... so that would most likely explain a lot of the problems i'm having.  I'm not 100% convinced it would explain why I get so many duplicates, but it might.

I'll sort out my mod files, do some tests with and without mods to see how it goes and post an update here.

Thanks

 

Reply #14 from Elemental Forums Top

400 (average likelihood) * 100(or so items) = 40000. Type was not enough? Really some 16 bit type was used?

Was the reason behind this rand() which can't actually generate more than 32K? No surprise random things are not very random.

 

Reply #15 from Elemental Forums Top

The problem prior to 1.7 was that the data type it uses to store the total liklihood wasnt large enough for all the liklihoods added up. Once that number hit its max, no toehr items were counter.
End of quote

You bastard, I knew it. This is exactly what I said back in march 2012, including the solution, but you guys refused to believe me. 

<X3


Edit: post quoted

Confirmed... the likelihood calculation is a range that is capped at something close to 32767.

Imagine five items with likelihood 200. The game then sets up a random roll between 0 and 1000, where a 0-200 means the first item gets picked, 201-400 the second, 401-600 the third and so on. But the range is also capped at 32767 or something close by. And because the core armor, weapons and items have a likelihood that adds up to 32900, it is not possible to add new items without changing all the core items.

So what I did was, I reduced all the core items' likelihood by half, so they were occupying the range of 0-16450. Then, I added 50 new items, that all had a likelihood of 2000. If I was correct, this would mean that I would only ever see my own items drop from the range of 16450-32767.

Then I proceeded to test the drops in-game, with these new variables in mind. There were two facts that confirmed my initial hypothesis.

First, out of the 50 items with likelihood 2000 that I added, I only ever got the first 9 items. This corresponds to the "free range", ie16 317 (32767-16450), or in other words, 16317/2000 = 8.1585.

Second, out of a 100 random items I looted, 49 were from my own items with 2000 likelihood. This would make sense, as the 0-16450 range is of core items, and the remaining 16451-32767 range is of modded items - in other words, half the likelihood were my own items, and half were core items.

The solution is to reduce all likelihood of core items by half (or more), to open up the range for our modded items to drop.

End of quote


Reply #16 from Elemental Forums Top

Likelihood was reduced in 1.6 following this report, though I suppose it could have been fixed much earlier based on the previous post. The current problem is not related to likelihood. Here is a description of some of the tests I ran:

 

I created a list of 63 items. Ended up looting 53 different ones. (53/63)

I then did the same test with a list of 87 items. Ended up looting 53 different ones. (53/87)

I repeated said test and got 54 different items. (54/87)

This is where the 54 comes from. As I said it "seems to be 54". Would more tests reveal that it is 55? 56? I think the point is you will not be getting all the items in a single category if that category has more than X items, X being around 54.

 

Also, the last items (xml wise) in a category seems very very unlikely to drop. I ran 5 tests with a list of 33 items. I never looted more than 31 different ones. I received item #32 in only 1 test. I never received item #33.

 

 

_________________________

LH Mods by Primal

XtraDeconstruct

XtraDeconstruct Canons

Reply #17 from Elemental Forums Top

 

I'm not sure what would cause that Primal_Savage. If you want to send me your xml I'll hook it up here and step through the debugger to see what it is doing. I'm sending you a PM with my email address.

I was going to post my email here, but I already get enough penis enlargement emails and I just know you guys would start signing me up for all sorts of mailing lists.

Reply #18 from Elemental Forums Top

Quoting Derek, reply 17

...I already get enough penis enlargement emails and I just know you guys would start signing me up for all sorts of mailing lists...
End of Derek's quote

Ha!

Reply #20 from Elemental Forums Top

Just wanted to say that since I fixed up the likliehood values in my mod it seems to have sorted out the issues I was having, so thanks guys/Kael for reminding me about the change.... looking much better now.

 

Reply #21 from Elemental Forums Top

@ Kael

Quoting Derek, reply 12
I have walked through the code and there isn't any max of 54 items for each rarity.
End of Derek's quote

 

I'm not sure if this is a coincidence or not, but maximum number I came with for loot was 54. However, assuming it is 53 (Two tests did result in 53, let's assume a mistake from me for the result 54; who said I could count...),

Quoting Primal_Savage, reply 16
I created a list of 63 items. Ended up looting 53 different ones. (53/63)

I then did the same test with a list of 87 items. Ended up looting 53 different ones. (53/87)

I repeated said test and got 54 different items. (54/87)

End of Primal_Savage's quote

then that number (53) is exactly the same number found in another thread on this forum, the ResourceType issue. When you hit 53 new ResourceType in the \Mods\ folder, the map generation starts having some serious issues.

Sooo, something to do with the \Mods\ folder and 53, maybe? (Hopefully referring to \Mods\ didn't kill any support this issue might have gotten)

 

 

 

_________________________

LH Mods by Primal

XtraDeconstruct

XtraDeconstruct Canons