Is there hope for multi-core processing

Is this in SP or MP??? If it is MP, it could be someone else slowing you down as the game synchronises as it goes....

Given that the latest Entrenchment patch eliminated nearly all stuttering and late-game slowdowns, and was tied to memory useage, I'm going to submit that the slowdowns in your case aren't because of the CPU bottleneck, but insufficient RAM.

Also, more units = more stuff to process.

Multi-core support is unlikely. It's a massive change.

And far more difficult to implement properly than people realize.

One solution you could try, it's a bit of a pain in the ass, but depending on how much stuff you have running in the back-ground it can make a huge difference, change the core affinity for every process to one core, and the affinity for Sins to the other, I've mostly only used this for converting video files, but it helps a shit load

if this is the case, then i will wait around for Entrenchment to show up since I dont use the beta.

as for RAM, i have 2gb already and its not being fully utilized by SINS or the OS.

this happens on SP and MP.  but I know that the second computer in the MP enviroment happens to be exactly the same as mine.  the only difference being that i put two branding stickers on my computer, and 1 sticker on my wifes. 

i do realize the complexity of tackling multicore support.

I've done this in addition to giving Sins a High Priority but I'm never sure if it makes a big difference or not.

generally thats a bad idea, if you are referring to going into processes, right clicking the sins exe and upping it priority.  i've seen that essentially lock machines.  i will just wait for Entrenchment.

I like to play without entrenchment though. Multicore shouldn't be tied to only the expansion.

I've simply set my one of my largest processes to one affinity (the music player) and the
sins process to another affinity. It helps a little bit but not very noticeable.

How about the devs set one core for ai and troop movement to one core and network communication to another core until multicore can be properly instituted?

we arent saying the multi core is tied to the expansion, only certain fixes for memory handling in long games.

OK, short of getting a dev in here I know no one is going to believe this...  and given the current status of my education, I don't have the credentials to say this with the kind of absolute certainty needed, but...

This game was not designed around multiple cores from the beggining; as such, it will be very difficult to change it to multiple core design.  In all likelyhood, we're talking about it being difficult enough to change as to be in effect re-writing the code from scratch.  It is not as easy as simply going "OK, lets use more than one core!"

For years, computers have been based around a linear processing path -- do A, then B, then C.  With multiple cores available, suddenly parallell paths become more efficient.  But other than the "infinitely parrallizable" (SP) graphics (and to a degree, physics) we (as a 'society' ) just don't have the experiance to write things in parrallell, rather than serial.

This isn't someone being lazy; doing multiple cores will require a lot of work, and very likely be outside of the normal programming standards the devs normally work too.

(NOTE:  The opinions expressed herein are the ramblings of a computer science major undergrad, who has done a fair bit of 'homework' on this subject but has no real experiance or understanding of the field.  I'm better than 99.9% of the people out there, but any real professional will probably laugh at my poor useage of proper vocabulary, and how I've mangled the concepts contained herein).

In other words, it's not exactly likely to appear in any version of Sins. Entrenchment or later expansions included.

I have not seen an objectionable slow down, although on truly massive maps (200+ planets with 10 players) I can imagine how it could become that way.  I have seen observable slowdowns in SP and MP on 3v3 and 4v4 maps where the number of planets are up in the 40-50 range. The slow down doesn’t become noticeable until somewhere around mid-game (1+ hours). Again this is noticeable although not objectionable on my system.

Typically I have friends join as part of my team (not hosted at ICO) and we play against a computer team or in SP I still frequently do 4v4 with 50+ planets. It is somewhat normal in our games for my partners and I to each have 750-1000 supply points each by game’s end (2-3 hours). In this case that means typically 150-200 frigates with 10-30 carriers and 1+ capital ships for each player. The one observable item that I see is that in late game situations things seem to be moving slower in both SP and MP mode and the network status confirms cpu usage climbing as the game progresses. If the game is saved and then loaded as soon as the game loads ships move much faster for a while (haven’t timed how long the speed up lasts) and the network status screen shows low cpu numbers. This was true up to and including v1.12, I haven’t made an attempt to see how 1.13 works in regard to this. While I can’t tonight, I will by the end of this weekend play a 4v4 ~50 planet map SP game using v1.13 and log the utilization for the memory and all four cores along with the graphs of the ship counts and post the results (last time I did this the max on a single core was ~80% and bounced from core to core).

Here are some things that I believe contribute significantly to this issue in MP mode. ICO has done a great job in improving their support; however it’s still an issue. When using ICO the network status screen for me never shows pings under 60 and seems to randomly bounce up into the 200+ range for all players. When I host on my system all players seem to average in the 60-90 range with minimums of 25-35 and maximums of around 120; seems to be related mostly to time of day. The larger ICO latency numbers would naturally contribute to sync issues and slow downs. Additionally any use of a wireless non-N class router and NIC card will make those numbers worse by a factor of 2-3. When I was using a G router and NIC card it was common to see maximum pings in the 500+ range with averages in the 170 range.

IMO having mutli-core support would be most noticeable in SP games or MP games where the host is managing a live player and all of the AI players. I understand some of the difficulties, since I write multi-threaded applications for a living. While on the surface it may seem odd that when this game came out even the hyper-threaded cores and dual-core chips were common with quads being less common, remember that the business decision as seen in the minimum spec requirements states a single-core chip of 1.8GHz. Regardless the devs have done a great job with the game and I appreciate the extras that they have put in like the ability to save and load a MP game.

System Specs for systems used by myself and friends

Phenom 9850 BE, 4GB DDR2, 8800GTS(SLi), Vista64

AMD 6000+, 2GB DDR2, 8800GTX, XP

Q6600, 4GB DDR2, 8800 Ultra, Vista64

I actually set it to realtime when I do that, but that's why I mess with the affinity first, you only overburden one core and the other limps along well enough to open taskman and kill the process if it tries to take over

Post!  Ron is right.  Parallelization is not something you do as an afterthought.

<pedant>

Ron is partly right; while parallelization is not something you do as an afterthought, heavy-duty multiprocessing is actually a very old and very common art.   All those 'big iron' servers with multiple CPUs and all that engineering work into designing good systems for sharing memory between 'em -- that's been around for many, many years.  All those distributed database systems?  They're running many queries simultaneously, in different locations, while still attempting to maintain consistency.   And engineers who need to write performant code have long been aware of multiprocessing to hide the latency of slower actions like most I/O, means multithreading.  And so forth.

It's not something to add as an afterthought, because you have to worry about everything from atomicity of transactions and serializing them without spending too much time spinning (and depending on the task, this may be unavoidable to the point of making it not worth it), to making sure that you're not being given different results by the whims of the operating system's task scheduler.

</pedant>

Also, it is only something you can do if the tasks can be done at the same time independently.

If you can do task A and B at the same time, then you can do parrallel stuff. Having network code on a different thread won't help if there is nothing to generate/processs the data for/from the network code.

My understanding of Sins works like this

1) Process a round of simluation

2) Send the data to all other users

3) Host checks the cheksum data for all parties and if they are valid, tells everyone to process the data and do the next round of simulation. If an error occurs, throw a Mini Dump and log what happened so the Devs can look into it...

4) Goto 1

So having multi-cores isn't going to help as the networking code does not get invoked until the simulation code occurs.

Now, some people will ask about splitting the simulation up. However, if you refer to the Dev's diaries on the sync bugs, you will note that they use a deterministic random number generator so each machine ends up generating the same random number. What they have to guarantee is that the number called for Ship1 on machine A is the same as on machine B.

What happens if there are two ships?

Machine A (one core) asks for a number for Ship 1 and then one for Ship 2.

Machine B (2 cores) asks for a number for Ship 3 and then for one for Ship 2 due to how the OS is scheduling threads etc...

This would then put the game out of sync and you have a mini-dump.

Now, like Ron I may be way off base here, but unless someone who understands the Sins engine fully can point out where I am wrong, I believe this is a reasonable argument as to the dangers to multi-core for Sins. This does not mean these issues CANNOT be fixed - only that the issues are an order of complexity greater then for a single core environment...

I have the Core 2 Duo, I have no problems running sins, though, even if every thing is at max

I don’t think that the newer multi-core chips will experience the problem…where I have seen it was in an MP game and one of the players had an older P4 with and AGP card. We were 1+ hours into the game and three fleets jumped into the pirate system resulting in a large multi-fleet action; at which point stuttering started to occur. It didn’t halt the game and was certainly still playable. I believe the situation I just describe will be worse if that kind of system is used to play 4v4, large maps in SP mode.

As I mentioned in a post above this weekend I’ll be running a test to see how heavily loaded the cpu really is. To date I’ve not seen a slow down occur on a 9850BE @ 2.7. I’ll be using a 4v4 map with50+ planets in SP mode and logging cpu utilization by core every 30 seconds.

Let me restrict my comment further -- gaming programmers only have single-thread experiance to draw on; database and similar operations are a very different set of problems that don't really apply.  I think.  (Things that are designed to be distributed across multiple computers aren't similar to things designed to run on *one* computer)

I went back to the top and reread all of the posts. Based on the OP I don’t see any information telling me that enough homework has been done. Since the OP references two identical machines having the same issue I would do some research first. Bring up the task manager and set the update speed to low. Play the game for an hour and alt-tab out to the desk top. If the one of the CPUs is consistently pegged at 100% then you do probably have an overloaded core. If the heaviest loaded core is sitting at 80% or less then look at these candidates.

1.       How are your machines communicating? Through a wired switch or are they using G class NIC cards. I guarantee that if you’re using G wireless cards you’re very likely to see random lags in the 500ms range. G class routers are intended for internet surfing not internet gaming, for that you need an N class router and N class NIC cards. A good way to see if the LAN is the issue host a local Sins game between your two machines and bring up the Network option and watch the ping values for few minutes, they update every few seconds. If you see values approaching 500 then you’re likely to experience stutter waiting for the lagging player to catch up.

2.       How powerful are the video cards? Try downloading FRAPS and see what your frame rate is. If video is the problem you might in some later game large fleet actions see a single digit frame rate. If you do and the cpu utilization is not 100% then you might have a video problem (HW or driver).

I would be surprised if an all human MP game were to experience slowdowns; other than those caused by lag issues or poor video card issues. If all of the players are human then the host is more of a recording agent and I would expect a minimal cpu load as compared to SP play with 7-9 AI players. In this situation there is all of the normal live MP load plus the AI load of determining both strategic and tactical issues.

Greetz,

CPU load never pegs.  The OS and RAM are doing their jobs in that regard keeping thing as tidy as they can.

1.    1gb ethernet NIC switched at 1gb over LAN.
2.    My cards were decent for their time.  256mb memory nvidia 7300gs.  ram and gpu speeds were some of the highest in the class compared to other 7xxxGS models.  in small sp games i can boost the fx to highest all the way around and all speeds are great all the way though the game.  more planets, more crap happening, slower the game.  if it IS the video cards, then oh well, new cards are on the upgrade list, but not for several more months.

so far we have only play 2human v 2ai on medium maps.  peformance isnt terrible.  the game is still very playable.  but i can image when we play 8ai and dozens of planets we will run into some pain.

i think a dead horse is getting beaten here.  i know there wont be a patch for Multicore functionality.  but maybe it will be handled in the next full version or maybe in mini-expansion 3 down the road.

not really true...

By example, the simplest means of parallelizing loops (data decomposition) is to let the compiler do all the work for you. The Intel C++ Compiler, version 7.0 and later, has support to analyze loops and parallelize those that it considers good candidates. The -Qparallel switch enables the feature... But since the code was not created for parallelization, the speed win is only around the 28%...

While automatic parallelization is great, there are times when the developer can, or would like to direct the compiler what to thread. After all, he know much more about the overlying algorithm than the compiler could ever attempt to understand. The OpenMP standard allows the developer the kind of flexibility he desires, while limiting the time investment necessary to thread code. OpenMP works on an explicit fork/join method, so the programmer must specify the start and end of a parallel region.

The Intel C++ Compiler version 7.0 supports OpenMP, but not all C++ compilers at the time of this writing do so. Use the -Qopenmp switch to enable processing of the OpenMP directives or pragmas to generate threaded code.

The cool thing about OpenMP is that it allows easy parallelization of loops or regions of your code without large-scale modifications. In fact, the original serial code is left largely intact.

Both method was used long time ago in the game industry when HT ( Hyper Threading ) have appear on Desktop computer... it is nothing new...

For synchronization a simple spin-wait loop may suffice but spin-waits can quickly become a hindrance on performance. A spin-wait locks up processor resources because it runs so efficiently, therefore stalling both logical processors. Inserting an assembly-level pause instruction can alleviate the detrimental effects of spin-waits...

So, yes, it is possible... but not really interesting for dual core... 4 or 8 core ( or more ) are better because these reduce the latency ( spin-wait )...Sins already use a second core ( if it exist ) for load texture... so, Sins is already using the multi-core architecture...and don't forget that the real goal of Stardock was to make run Sins on the more computer... not only the high end one...

Maybe a article related to a old Stardock game, called Galactic Civilization II will show that multi-threaded game is nothing new but ...

http://www.gamasutra.com/features/20060405/wardell_03.shtml

read the part 2 and point 3 of the lesson learned...

multi-threaded II

To follow up on what Ron said... Databases can be written to run in parallel as multiple users request something at the same time, so it is doing X jobs for Y people. A game is one job for 1 person unless you are talking about the server part (if implemented)

Wrong... a game have a lot of "job"... AI, 3D render, sound, human interface ( mouse, joystick, keyboard ), physx, etc.... once you have multitasking, you can make multithread software...

About the original post :

Game speed on giant maps still progressively gets slower for me.  Doesn't really seem like a video issue...maybe Im wrong.

Not really a problem from video or multicore... more a problem of ram... if you lower your GFX, thing speed up a little simply because some ram become free... this problem can be resolved with 64 bits support but it is not a option for Stardock now... the sync problem will become huge problem between people who are 32 bits and 64 bits, it will cut the online community in two group who are incompatible for online game... for single player, it will not be  problem...

I don't think that Ironclad will make a new engine for Sins... but due to the very big success of Sins, i think that it is not impossible that a Sins 2 appear in some years, with multicore and/or 64 bits support... it is usually the way it work in the gaming system... money return from a first game feed the creation from a new one... but a new engine cannot be created in one day... these type of work can ask years...