We're playing on LAN.
I doubt it's a computer issue since we run so different computers, for example - I crashed (C2D 2.0 Ghz, 8600 GT, 3 gb ram) while another guy with a very bad PC just continues playing as usual.
Also, we thought about the ram issue and we all rebooted our computers, same thing kept happening.
We've had crashes with a medium sized map with 6 players and 10 players.
So in short:
We're playing on LAN
We're usually more than 6 people
Who it is that crashes seems rather random
Map-size doesn't seem to matter
Various configurations, both in-game settings and hardware
I've never experience this kind of crash in singleplayer.
to "the Monk" - We have a, pretty much, game of the week/month

Seems to be Audiosurf right now (awesome game btw, you should check it out, you load your own music into the game and it automatically builds a racing track from it. Think Wipeout mixed with Guitar Hero ^^).
Usually I'm pretty good at troubleshooting stuff like this myself but with this.. i simply can't figure out what could be wrong. First I thought it had something to do with Vista (since I was the host) but it kept happening with Windows XP hosts.. the list goes on :< We sent in our mini-dumps and savefiles but didn't recieve a response.
EDIT:
Also, since the game can get stuck crashing at a certain point, I don't think it's peoples computers.
Thanks for the replies, appreciate it!
//Wihl