Stardock.com ‚Äč

Beta 1.2 vs The Sins Optimization Project: Results?

By on February 15, 2011 7:40:49 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums External Link

boshimi336

Join Date 03/2008
+65

For those that do not know what the Sins Optimization Project is, please click the link below and some descriptions from their original post.

http://forums.sinsofasolarempire.com/378149

 

The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To look like nothing has been changed at all.

I studied the Particle files, and textures. Many of the particle textures IMO were also way to high resolution. A reduction by 1/2 showed no visible difference in game in most cases.  There were some exceptions where it did effect the appearence in game, and those textures were left alone. Remember the goal was not to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses far less memory, and is much more efficient. The particles also utilize the mip mapping features of DDS.

The developers mentioned something about a Memory Leak. I heard that a leak can be caused by a file calling for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins has MANY particle, and mesh files that do that. I removed all of the redundant entries that pointed to non existent textures in the mesh, and particle files. I also removed the entries that have lines like "C;\gs\main\datasource\textures\effects\aura4.dds", and just put "aura4.dds". This will default the search to the mods texture folder, and speed things up. Also various typo's were corrected in some mesh, and particle files. Some ships have mesh nulls that were miss-spelled. For example the Akkan had typos on its engine nulls, and flairs. The engine trail, and flair didnt show up in game. the Marza was missing its bomb nulls. All of this has been corrected as well. If there is a ship mesh file in the mod then it had something wrong that was fixed. Regardless if this caused a memory leak, or not these errors had to be fixed.

After the Meshes, Particles, Textures, and the User Interface were optimized, and some very obvious bugs were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of version 0.07 no game i ran ever went above 1.5 gigs on Diplomacy with maximum settings. Including Bloom enabled. Tested on Huge random multi star maps with 9 AI's. The 2 gig crash problem is solved. Our primary goal with this project is achieved!

 

 

What I would like to know is how much of this has been transferred over into the 1.2 Beta patch? The community has, from a Mod standpoint, done a lot of down in the dirt work to make larger games playable reliably online. I would like to hope that this effort has been rewarded by being included in some part in the official patch. Already I have noted the 1.5gig 'observed max usage' comments from the 1.2 Beta and between the TSOP Mod being comparable. Has anyone looked in detail at a comparison between these two excluding game balance changes? Thoughts?

81 Replies
Search this post
Subscription Options


Reason for Karma (Optional)
Successfully updated karma reason!
February 15, 2011 7:52:25 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I haven't looked in detail, but I'm doubt the devs did do a lot of transferring since they have access to the engine.

 

Reason for Karma (Optional)
Successfully updated karma reason!
February 15, 2011 8:31:11 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

From the sounds of it most of the optimization in the Beta patch came from optimizing the engine. This seems not only to have been more effective, but it means that the optimization mod could still be used in future Sins versions to increase performance further if necessary.

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 10:04:43 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

well, I'd hope they dont downgrade the textures. Heck, I'd like them to upgrade them a bit. How bout some tesselation on the roids while I'm dreaming? The other optimizations im okay with (though I'd question whether some g-cards might not like the new texture format. Then again, it might be fine, I dont know). I myself have a really beefy machine and once I did the Large Address Aware thing that let the game address more than 2 gigs, all was good (before, starbases were bright blue!). I havent seen anything weird with the beta, but does anyone think that a patch would cause me to have to redo the LAA? I suppose I could look for myself when I get home.

Which makes me wonder if they did anything along the lines of LAA?

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 1:37:54 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting SithLordAJ,
well, I'd hope they dont downgrade the textures. Heck, I'd like them to upgrade them a bit. How bout some tesselation on the roids while I'm dreaming?

There's very little in which you'd see a noticeable improvement. As for tessellation... yeah, that's dreaming of flapping your arms and flying to the moon.

 

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 3:31:49 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I understand some people dont have machines as awesome as mine (I'm pretty poud of it, if you can't tell), and I'm all for those people running on downgraded textures. Sins, like most PC games, has different levels of textures for different configurations. I am okay with making the lower resolution textures even lower so that more people have access to the game. I am okay with adding more sets of lower resolution textures so that more things are affected by the low resolution settings in the same regard.

I am not okay with high resolution textures being downgraded so someone on an old laptop doesnt have to go get a decent gaming machine. Either you want to play games or you dont.

Btw, I often zoom way the hell in. That's one of the main attractions of this game for me, to be zoomed in so that a tiny little constructor or fighter takes up nearly the whole screen and then zoom all the way out so that the whole sector of stars are blurs on top of one another.

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 3:36:51 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

It doesn't matter what settings you run it on, it doesn't make a visible difference.

Hence this line:

There were some exceptions where it did effect the appearence in game, and those textures were left alone.

Besides, Sins requires very little in terms of computer specs to run max graphics. Bought my computer more than three years ago and it was mid-end then, but it still runs max Sins graphics.

 

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:01:34 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

The textures that are loaded are the same for high video settings or low. Its how your graphics card puts them in-game. In TSOP, none of the ship or structure textures were touched as they were already converted to .dds which is a form of compression with very little loss in detail. Major Stress just took this one step further and converted the remaining textures to .dds which can cut the memory usage by 50-75% on each texture sheet.

We have stated over and over that no changes will be made that effect the quality of the game. Only the major waste of memory.

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:05:31 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I still stand by my statement. do not downgrade the graphics. you can 'optimize' by numerous tricks including texture compression and other things, but i see no reason to downgrade anything. If other people have problems, the lower resolution settings should be the answer. If they aren't, then that needs to be fixed instead of changing the texture.

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:12:16 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Hey fella... you don't need to install the Sins Optimization Project if you don't want to. If you have no probs with lag then why even bother with it yourself? So you have nothing to worry about.

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:23:29 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

@AJ - If you read the TSOP post, you would understand the 2 gig limit on the game and not very many people can or will use that LAA fix. All the awesome textures you talk about zooming in on ARE ALREADY OPTIMIZED by Ironclad. Do you take screenshots and zoom in on your research icons? or race pictures? or menu backdrops? Do you open the texture sheets and zoom in on the 50% empty space? How about opening the textures that are loaded in-game that are not even used? Look at the lower part of this post and thats just 2 of many. 

@rowanlad - I think he is trying to make the point of Ironclad using TSOP

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:26:25 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Oh, I know. The point of this thread, however, is a suggestion that it should be integrated into the native code; that is what I am opposed to. I am fine with there being a mod for an even smaller footprint. I am fine with tweaks to the engine. I am fine with more options for lower-end machines (like texture compression enable/disable). I am fine with additional lower resolution textures for the same reason.

I am not okay with any sort of downgrading of textures. If people still want to argue me about this, I can keep it civil. Tell me why, I, with my super-awesome-calculates-what-you're-thinking-while-simultaeously-rendering-10-feature-length-films-in-real-time computer need this?

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:33:02 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting myfist0,
@AJ - If you read the TSOP post, you would understand the 2 gig limit on the game and not very many people can or will use that LAA fix. All the awesome textures you talk about zooming in on ARE ALREADY OPTIMIZED by Ironclad. Do you take screenshots and zoom in on your research icons? or race pictures? or menu backdrops? Do you open the texture sheets and zoom in on the 50% empty space? How about opening the textures that are loaded in-game that are not even used? Look at the lower part of this post and thats just 2 of many. 

So this is an argument for Ironclad to implement the LAA fix? I'm okay with that. And as for the empty space in the texture sheets, if that's what we're arguing about, then I think I haven't made myself clear: optimizations that dont sacrifice the detail in anyway are okay by me. And I have said alot of what is mentioned in the OP about TSOP is okay with me. Just not the texture downgrading. If by dropping resolution you mean decreasing the size of the texture sheets, but each texture on the sheet is still the same size.... I'm okay with that.

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:44:13 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Major Stress tried to lower the size of the planet and skybox textures but it was abandoned as was stated. No other textures were made smaller in TSOP, only converted to DDS which as you stated is fine when you zoom in on.

I do have 1 question about your LAA. Does it still allow you to play online or does it change the native checksum?

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 5:47:22 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Its ok, im pretty sure Ironclad won't endeavor to decrease detail in order to increase performance. Sins is already pretty old and so a lot of the market will have upgraded their machine in the intervening years anyway. If anything it would be as Myfist has stated and just polishing up the code and removing unused textures. 

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 9:00:25 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

What about all the typo's, coding referencing things in an inefficient manner.

 

Has anyone compared how the 1.19 beta is setup to the TSOP mod? I'd like to know if they just redid the engine and left the rest of the game still highly inefficient or if they actually went in (since most of the work has already been done by the TSOP mod) and used that as a check list to fix many of the problems with Sins.

 

On Max graphics I haven't been able to tell the difference between an original huge texture and an 'optimized' texture. I don't see what the problem is. If there is NO VISUAL DIFFERENCE why the complaint? That's like asking someone to model the head inside each spaceship in the Sins engine so they're there when we're flying around in space, even though we can't see them whether they existed or not. If there is no noticeable difference ... why care?

 

Again, my point is this. What is the difference between TSOP and Beta 1.19?

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 9:54:52 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

It will be left to us modders again to do all the grunt work and compare files to see all the differences and new errors but I will not beta test anything but my own or my teams work unless they are paying me for the time so I will wait for the final release.

Reason for Karma (Optional)
Successfully updated karma reason!
February 16, 2011 10:37:34 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I'm generally not an online player, however because you were asking about it, I launched the game and went into the multiplayer lobby. Unfortunately, no one was in the lobby and there were no games that I could join, but 4 players were listed as being online and 2 games going(I have the beta patch, so that maybe why). So its kinda unclear as to whether LAA prevented anything as far as the multiplayer is concerned.

If you have not used it, LAA is pretty simple. you simply add the executable to a list and it tells you whether the file is currently LAA (True or False), and whether the original value was True or False. It seems to me like a simple file property, and in fact, when I first ran it I didn't believe I had done it right and spent another 20 minutes trying to figure out how to use the damn thing. When I eventually realized I had it right all along, I thought it was a scam or something that wasnt going to work. Well, it worked fine. Haven't messed with the thing since. (Actually, I did just have to mess with it. It seems the patch has a new .exe, which means I had to set it again.)

[edit]I was curious about more info on LAA. I guess its just a flag thrown in the image header of an executable that allows it to use the larger memory footprint. Generally, it should be set when the application is linked. I guess the LAA fix simply re-links the .exe with the flag on. I don't see why Ironclad doesn't simply turn on this option when they compile the new .exe, as it doesnt seem like it would hurt, but could potentially help

Reason for Karma (Optional)
Successfully updated karma reason!
March 2, 2011 3:27:29 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Sith,

 

Sorry I am trying to track the LLA conversation. This is the first mention of it that I have seen in the forum. Is this a function of Windows or is it a third party program that creates the list of executables you mentioned in your post?

Cheers

Reason for Karma (Optional)
Successfully updated karma reason!
March 2, 2011 3:39:42 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting cepheidsoul,
Sith,

 

Sorry I am trying to track the LLA conversation. This is the first mention of it that I have seen in the forum. Is this a function of Windows or is it a third party program that creates the list of executables you mentioned in your post?

Cheers

It's a function of Windows. Flagging your executable as LAA (Large Addressing Aware) basically tells Windows that it can give it about an extra 1gb of available memory space. This only works on 64-bit operating systems. It essentially lets windows give the app around 3gb of memory, up from 2. Turning on LAA tends to help on some games that are very memory heavy (helped a lot in SupCom, Empire: Total War, etc). Typically, anything with big maps and lots of units has a large memory footprint and benefits from LAA.

There are third party programs written for various games that will turn the header on, and they all work on any executable because it is a generic process that's the same for any executable file. You can also do it through the command line if you really wanted to.

Reason for Karma (Optional)
Successfully updated karma reason!
March 2, 2011 8:00:26 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

No no no no.  LAA does NOT let you use 3GB of RAM.  The 32bit executable is hardcoded at 2GB or RAM, period.  What it does is allow the executable the option of addressing RAM in that above 2GB range.  It allows the program access to space that other programs aren't using.  In as simplest terms as I can explain this...

Example 32bit system with no LAA enabled programs.  Grey is the OS, the other colors are random programs.  Purple is your game.
No LAA

Only the system OS is using RAM above the OS implemented 2GB wall, and no program is using more then 2GB.  They are all sharing the space.  The system is fully capable of using addresses up to the 4GB limit, however, limits were put on this long before there was even an issue.  10 years ago, long after the development of the 32bit OS and 5 years after mass deployment of it, a system with 500Megs of RAM was close to top of the line and everyone was happy because those limits seemed to be unreachable any time soon.  They weren't completely wrong.

 

Example 32bit system with 1 LAA enabled program (Mind you that you have to throw some settings in the OS to allow this as well, otherwise nothing happens)
LAA Yes

The purple program is now allowed to address space above 2GB, however, it doe NOT use more then 2GB of RAM.  You get better performace out of the program in the sense that it is no longer directly competing for the lower more crowded addresses and getting shuffled around.  A 64 bit system plays things out in this manner as well without having to change all of the system settings to allow it.

Don't get me wrong, it's actually more complicated then this, but this is as simplistic as it can be made to explain what is actually happening.

Remember people, use common sense, it's called LAA (Large address AWARE) for a reason, not UMR (use more ram).  The biggest problem with this is that right now, only gamers are affected by this issue and let's be honest, gamers aren't the smartest of the computer using bunch.  They're not educating themselves on what is really happening and only repeating part of the solution and confusing everyone else they talk to.  LAA has mutated into a monster that it was never intended to be.  It was intended to be used for servers with high volume traffic, not the casual PC user, that's why it's not an easy thing to do and has restrictions on it's use.

Reason for Karma (Optional)
Successfully updated karma reason!
March 2, 2011 11:25:26 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Heres the breakdown:

  • 32-bit system maximum ram= 4gb (without certain tricks), 64gb (with said tricks, but motherboards dont usually allow this)
  • 32-bit windows default maximum alotted to programs= 2gb
  • 32-bit windows maximum alotted to programs via \3gb switch= allows virtual addressing up to 3 gb without messing with the OS's ability to address.
  • 64-bit system maximum theoretical ram= 16eb (exabyte. its just plain ridiculous. It will be reached eventually, but probably not for a long long time)
  • 64-bit system current maximum ram= 256tb (still ridiculous, but at least we are using recognizable numbers now)
  • Windows hardcoded limits= 128 for vista/xp, 192 for windows 7
  • Large Address Aware is a linking option. If your not sure what that is, its basically an option set when the program is compiled. This option tells it that the application can handle memory addresses larger than 2 gb
  • both 32-bit and 64-bit applications are not allowed to address more than 2gb of virtual ram in any version of windows unless the program is LAA
  • If a 32-bit program, runs on a 64-bit windows, with LAA, it can use 4gb of virtual address space.
  • If a 64-bit program, runs on a 64-bit windows, with LAA, it can use 8tb of virtual address space.
  • If a 32-bit program, runs on a 32-bit windows, with LAA, it can use 3gb of virtual address space. (might have conflicts if \3gb switch is not enabled)
  • sins is obviously a 32-bit program

Stant123, I am not sure where you are getting your information about fixed address sizes. Can you give me a link to your info? I am very interested in all these types of things. The LAA info I have I got from microsoft.

Reason for Karma (Optional)
Successfully updated karma reason!
March 3, 2011 6:04:01 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting SithLordAJ,
(exabyte. its just plain ridiculous. It will be reached eventually, but probably not for a long long time)


Don't be so sure

It keeps happening faster than people think. 

Reason for Karma (Optional)
Successfully updated karma reason!
March 3, 2011 11:26:17 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting rowanlad,

Quoting SithLordAJ,
reply 21
(exabyte. its just plain ridiculous. It will be reached eventually, but probably not for a long long time)

Don't be so sure

It keeps happening faster than people think. 

well, I believe that some servers might have terrabytes of RAM (not sure as I dont work with servers much), but theres a whole other scale between terrabyte and exabyte: petabyte. So.... at least 3 weeks. thats a long long time to me.

Reason for Karma (Optional)
Successfully updated karma reason!
March 3, 2011 3:11:28 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

OK guys.

The 32-bit memory limit is a result of the pointer length. a pointer on a 32 bit OS is a 32 bit number representing a memory address. Windows 32bit (NT) was designed to reserve 2GB of a max of 4GB (32 bit can represent a maximum of 4.294.967.296 adresses, 2^32) for Kernel space (to help with performace)

Using LAA on 64Bit OS, 32bit applications can target the entire 32bit (4GB) for themselves (using the 3GB switch on 32bit windows only 3GB, 1GB is stays reserved for kernel)

A 64 bit executeable on a 64 bit windows can target as much ram as is available (up to 192TB on win7)

The behavior of games using the LAA flag depends on the programming. If the program just tries to get another adress for more ram (using malloc, calloc, alloc or whatever) without checking the return and just crashing (sins?) than LAA will help, as long as you got the ram to spare. LAA using the 3GB switch on win32bit can create performance problems (Less space reserverd for kernel modules) but will help allivate the crashes.

I'm not sure if it will change the checksum though, but I've heard it does not.

 

 

As for the optimisation project, I wish the would include all the corrections (inkluding wasted texture space as myfist0 pointed out) and bugfixes (not the optimisations) into the patch.

Reason for Karma (Optional)
Successfully updated karma reason!
March 4, 2011 12:26:17 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

As for the optimisation project, I wish the would include all the corrections (inkluding wasted texture space as myfist0 pointed out) and bugfixes (not the optimisations) into the patch.

 

Agreed!

Reason for Karma (Optional)
Successfully updated karma reason!
Stardock Forums v1.0.0.0    #108432  walnut2   Server Load Time: 00:00:00.0000656   Page Render Time:
Facebook Twitter YouTube Google+