[X:CE 0.34.2] Loading an in-progress GC save freeze

[YaK]

Personally i have had this bug also with the old Grimple's Ambient Sound, so what Policenaut said makes totally sense.

Anyway, i am talking oc CE+X-Division.

I also noticed the problems with "suppression", the game often freezes when the bullet "touch" the enemy and his sound should be played. This happens a lot in overcrowded maps (Terror Mission) and, don't know if it's just a coincidence, but always happened with Sebs units.

 

[Charon]

12 minutes ago, YaK said:

I also noticed the problems with "suppression", the game often freezes when the bullet "touch" the enemy and his sound should be played. This happens a lot in overcrowded maps (Terror Mission) and, don't know if it's just a coincidence, but always happened with Sebs units.

In the whole time i was testing, playing and developing X-Division i only experienced this bug once, when i made a new melee unit for Solver which had only the minimum things coded to work - aiprops and ufocontents. I havent been able to reprouduce this in over 400 hours of gameplay ? Why is that so ?
And i´ve played with dozents of different mods, changing files on the fly and activating and deactivating other mods, including grimples ambient sound and the Ultimate Ambient Sound mod. Its weird that for other people this seems to be a regular occurance (?).

[YaK]

Don't know.

I suspect that this kind of problems show themselves more often with low specs configuration (like mine).

This started to happen not at the beginning of 0.99, but a few patch/hotfix later, "probably" when i installed Grimple's mod. It rendered the game quite unplayable the game for me as soon as i hit the 0.99.3 branch and my campaign was in phase 3.

I put my playing in statis till the updating of CE and i've done a clean installation/patching, not using any Sound mods, but haven't got time to start a new campaign, yet.

I also experienced a "variant" of the GC savegame bug.

I NEVER used quicksaves nor usually reload from autosave, but in my case the game didn't "freeze", it just get stuck on the loading screen with just a tiny bit of che loading bar from completion and GC music playing.



 

Edited November 21, 2016 by YaK

[Policenaut]

Some crash testing ahead:
Windows 7 Laptop - cannot reproduce crashing when loading GC saves
Windows XP PC - can reproduce crashing when loading GC saves
XP PC has better graphics card and processor. XP uses Intel processor and W7 laptop uses AMD processor. Both are 32bit systems.

Test on Windows XP PC - each test was repeated multiple times.

• #1. No ambient sound mod enabled - no apparent problems
• #2. Ambient sounds without fix - freeze/crash during load of GC save from main menu and when switching to Geoscape, random crashing afterwards on Geoscape more details later below
• #3. Ambient sounds with fix (group 1 instead of 2 + modmerge improvements) - worked fine when loading from main menu, had one crash when changing to Geoscape, random crashing afterwards on Geoscape more details later below
• #4. Ambient sounds with alternate fix 1 (#3 + looping ambient sounds use group 2 with no repeattimer) - worked fine when loading from main menu, ambient sounds no longer looped and hard CTD which created a .dmp file when changing to Geoscape - .dmp file here: XenonautsCrashDump(Ambient sounds with alternate fix 1).dmp
• #5. Ambient sounds with alternate fix 2 (#3 + looping ambient sounds use repeattime="0" instead of repeattime="0-0") - worked fine when loading from main menu and doesn't seem to crash most of the time when loading Geoscape, random crashing afterwards on Geoscape more details later below
• #6. Ambient sounds with alternate fix 3 (#3 + looping ambient sounds use repeattime="0.01-0.02" instead of repeattime="0-0" or repeattime="0") - worked fine when loading from main menu and doesn't seem to crash most of the time when loading Geoscape, random crashing afterwards on Geoscape but much less than with repeattime="0-0" or repeattime="0" more details later below
• #7. Ambient sounds with alternate fix 4 (#3 + no looping ambient sounds) - worked fine when loading from main menu and doesn't seem to crash most of the time when loading Geoscape, random crashing afterwards on Geoscape but much less than with looping ambient sounds more details later below

During all tests except #1 after completing a GC mission via loading a GC save from the main menu and returning to the Geoscape, and then trying to load another GC save would cause the game to crash every time during load. Test #6 and test #7 was able to load saves sometimes but never more than once before crashing and the loaded save would be more crash prone.
During all tests except #1 after completing a GC mission from a GC save from the main menu and then returning to the Geoscape, the game would randomly crash after fast forwarding time for a bit on the Geoscape. Test #6 and test #7 had much less crashing but it could still happen.
During all tests except #1 the switch to Geoscape could sometimes crash, test #3-7 experienced it much less than test #2. Test #4 always crashed.
Loading a Geoscape save from the main menu and then never loading another save seems to never trigger any of these crashes.
There was no apparent difference between quicksaves, autosaves and manual saves during my testing.
The logfile.txt never reported anything of note.

Summary:

It seems that the prime cause of the crashing was the looping ambient sounds using repeattime="0-0" and group="2". Changing looping ambient sounds to use repeattime="0" seems to improve it somewhat but the problem still remains. Changing looping ambient sounds to be above 0, i.e repeattime="0.01-0.02", seems to improve it even more but there's still a chance of crashing. Removing the looping ambient sounds completely was similar to changing them to use a repeattime above 0. Not using any ambient sounds at all seems to work flawless. I could load saves any number of times without any issues.

There seems to be some inherent problem with repeattime that the ambient sounds use. I'm not a programmer but my guess is that the timer data is not properly saved in the GC save file which causes the crashing. I have no idea why the Windows 7 laptop, which is less powerful than my Windows XP PC, never experienced any crashing. Is it an Intel/AMD issue or is it related to the operating system?

However, I will later upload a version of the tropical/swamp tileset to steam workshop/nexusmods without ambient sounds and update the Ultimate Ambient Sounds to include those ambient sounds. I will also change it to use the data in test #6. It should hopefully reduce the crashing when loading GC saves for people who experience it but it won't fully eliminate it. There's nothing I can do to fix the bugs related to the ambient sound system repeattime and the sound system in general.

Thanks to everyone who helped investigate this issue and hopefully this gets fully fixed in a future X:CE update.

Edited November 21, 2016 by Policenaut

[Policenaut]

5 minutes ago, YaK said:

Don't know, i suspect that this problems show themselves more often with low specs configuration (like mine).

This started to happen not at the beginning of 0.99, but a few patch/hotfix later, "probably" with Grimple's mod and rendered quit unplayable the game for me a the 0.99.3 branch.

I put my playing in statis till the updated of CE and done a clean installation/patching, not using Sound mods, but haven't got yet time to start a new campaign.

I also experienced a "variant" of the GC savegame bug.

I NEVER used quicksaves nor usually reload from autosave, but in my case the game didn't "freeze", it just get sutck on the loading screen with just a tiny bit of che loading bar from completion and GC music playing.



 

That's the variant I've experienced the most. Overall it seems to either freeze/crash when almost finishing the loading and when clicking OK after finishing a GC mission and trying to return to the Geoscape. It can also freeze before even loading when trying to load a GC save when already in the Geoscape. These are all caused by the ambient sounds according to the tests I did above.

Edited November 21, 2016 by Policenaut

[Charon]

14 minutes ago, Policenaut said:

Loading a Geoscape save from the main menu and then never loading another save seems to never trigger any of these crashes.

This is what i do 99.99% of the time.

15 minutes ago, Policenaut said:

I have no idea why the Windows 7 laptop, which is less powerful than my Windows XP PC, never experienced any crashing. Is it an Intel/AMD issue or is it related to the operating system?

I got windows 7 too, so maybe there are less crashes for that ?

16 minutes ago, Policenaut said:

The logfile.txt never reported anything of note.

Jup, we are far beyond anything the logfile will report. The deeper you dig, the less error reports you get and the more bugs happen. Its like we are being blidnfolded the further we go.
Looks like we are on our own here ... on virgin lands.

19 minutes ago, Policenaut said:

There's nothing I can do to fix the bugs related to the ambient sound system repeattime and the sound system in general.

Jup, its a wonder its even working. And i planned for 400 more sounds to add to the system, hurray .

21 minutes ago, Policenaut said:

Thanks to everyone who helped investigate this issue and hopefully this gets fully fixed in a future X:CE update.

Well done digging so deep, at least we are capable of reducing the crashes a bit.

26 minutes ago, YaK said:

I NEVER used quicksaves nor usually reload from autosave, but in my case the game didn't "freeze", it just get stuck on the loading screen with just a tiny bit of che loading bar from completion and GC music playing.

I rarely experienced this as well. It seems all connected to loading autosaves from inside of the game instead of the main menu, and it can happen from the main menu as well.

 

So, all in all i think we can agree that:

starting up the game > Going to straight New Game/Load Save Game > never reload anything

is the safest way to play. Anything else beyond that randomely produces crashes at random places, mostly connected to sounds. At least thats what it looks like.

 

I mean the loading mechanism ingame seems to be something you should be able to use, but at this state you shouldnt really use it all, i guess. So its basically a broken system, which barely works correctly and only under the right circumstances.

[Policenaut]

Both the Tropical/Swamp tileset and Ultimate Ambient Sounds have now been updated on both Steam Workshop and Nexusmods.

[Solver]

I have never experienced a crash-on-load while playing by myself. I do have some saves that crash on loading, but there my attempts to disable all sound when ending GC didn't improve anything, as reported by you guys. So I am still unsure as to whether there's a specific improvement to be made.

[Charon]

40 minutes ago, Solver said:

I have never experienced a crash-on-load while playing by myself.

Its easy to miss, especially for the more experienced players and developers. because most of them start a game, load a save, and dont really load around except for testing purposes.

It can be reproduced very easily by loading a GC game and then quitting, just to reload another GC game from the main menu without leaving the game. This has proven to be the most realiable method of reproducing crashes on load.

Here are some facts:

• GC loading has in incredible higher rate of crashing in comparasion to Geoscape loading.
  This is true for modded and unmodded games. The relation between modded and unmodded games is roughly 1:10 for the average modded game ( excluding X-Division )
• There is an almost 100% certainty that the first load after the startup of the game loads correctly, no matter the save type
• Sounds have been proven to be especially complicated.
  Example 1: I´ve just added 164 sounds to the aircombat and experienced my first "on aircomabt load" crash in over 800 hours of gameplay. "Maybe" it was just an coincidence, but it never happened before
  Example 2: UAS mod has been proven to make GC unstable. This concerns the loading, tabbing, and ingame behaviour. My own X-Division Developers Diary playthrough with the updated UAS and other recommended mods is so buggy that
  
  1. alt tabbing out of the game crashes it
  2. Loading a save from a GC produces funny, very unregular behaviour all around. The best bet you have is to never load a GC mission. Beyond that, there cant be any promises that the game fucntions as expected.
   
• Unmodded games, like XCE, have proven to not have this many bugs to a point where you can say its a rare occurence. Modded games however pump it up for small mods by around 10%, and for heavily modded games to around 60% of the game where buggy behaviour happens all the time. Every "sound" addition has proven to be especially responsible for those, although nobody can really say how or why. ( Shit game engine we suppose )

 

So under all of this light the best way to play modded games is to

1. Only load a save ONCE, on startup, and never reload anything
2. DONT load GC saves. Always play it through in one go.
3. Use the least amount of sound mods possible

[Solver]

1 minute ago, Charon said:

It can be reproduced very easily by loading a GC game and then quitting, just to reload another GC game from the main menu without leaving the game. This has proven to be the most realiable method of reproducing crashes on load.

Two or three weeks ago when investigating this, I probably tried this 15 times

I'm sure there's something wrong with the ambient sounds though. Perhaps instead of cutting music, I should forcibly terminate all ambient sounds as combat ends?

[Charon]

Just now, Solver said:

Two or three weeks ago when investigating this, I probably tried this 15 times

Like i stated, you cant reproduce this without heavily modding your game. XCE functions 99.99% of the time correctly. Once you start modding the fireworks start to happen.

1 minute ago, Solver said:

I'm sure there's something wrong with the ambient sounds though. Perhaps instead of cutting music, I should forcibly terminate all ambient sounds as combat ends?

We dont know, but take into account that things can also crash on the loading for something. For instance aircombat.
It looks like the game trips itself up when too many sound possibilities are present. This is why the 5 radio chatter sounds "rarely" cause this, in comparasion to my added 164 sounds. The same is true for sound mods, the less you have the more stable the game becomes.

[Solver]

Yes, that seems to be the general pattern. I've noticed the game has some difficulty playing sounds, of course, but this is engine territory - with it being as bad as it is, large amounts of sounds is just one of the many things that it cannot properly handle.

[Charon]

We are currently investigating the game "freeze on hit" bug, and it could be sound related too.

 

I have a save game: Special 7.sav , and the bug is reproducable. It always happens in the same way on the last bullethit. It appears it can happen for single bullets as well.

I might be totally wrong here, but it seems like the "hit" sound fades just before the last bullets hits and the last bullet triggers another reaction and the game starts to loop.

The bug is only reprdoucable from this position and from this angle, if you move the soldier and shoot from another angle it cant be reproduced.

This bug seems to be unit specific, as some units have to be proven more reliable to trigger this bug. The things we counted are sebillian medic, harridan soldier, and a policeofficier at the beginning of the game.

 

Ill try to deactivate the UAS and see if that brings any results, although im unsure how this can be related. Until that im stuck with the most bugged save ever.

[Policenaut]

1 hour ago, Solver said:

Two or three weeks ago when investigating this, I probably tried this 15 times

I'm sure there's something wrong with the ambient sounds though. Perhaps instead of cutting music, I should forcibly terminate all ambient sounds as combat ends?

This might be worth doing since I made the ambient sounds use the group="1" sound group used for general sounds instead of the group="2" sound group used for music. Could you see if there's some issue with repeattimer that ambient sounds use as well? It might be related. 

Edited November 26, 2016 by Policenaut

[Solver]

There's not an issue in the actual ambient sound code as far as I can tell, but the few related problems I have seen do point to the engine being stuck somewhere in its sound system. And that's where I get stuck as well since I don't have the source code. 

[Charon]

9 minutes ago, Solver said:

There's not an issue in the actual ambient sound code as far as I can tell,

+1 as @Policenaut and I have been going over the code extensively.

9 minutes ago, Solver said:

And that's where I get stuck as well since I don't have the source code. 

Wut ? I thought all XCE coders have soruce code access. Is it only half of the package ?

[Policenaut]

Alright, that's a shame about this seemingly being an engine issue. Stopping the entire group="1" + ambient sounds system when the GC result screen pops up might still help with GC to Geoscape crashing though. Thanks for the effort though.

@Charon

I think Goldhawk never got access to the engine source code and the company behind it has abandoned all support for it.

[Charon]

Just now, Policenaut said:

@Charon

I think Goldhawk never got access to the engine source code and the company behind it has abandoned all support for it.

This is very frustrating.

[Solver]

That's right, neither the X:CE developers nor Goldhawk have the engine source code. Geez guys, why do you think X:CE is the way it is?  Half of the things that suck in this game could be fixed by coders with engine access. And a lot of the code - original and X:CE - consists of very creative ways to work around the engine's limitations.

The choice of engine was the biggest mistake in the development of Xenonauts. The engine is intended for games much simpler than Xenonauts (and is quite suited to that), it's also a free engine with no licensing costs or such so it could be used on a low budget, but that means there's no support for the company. And the company abandoned the engine and all support for it some 2 years before Xenonauts was even released.

If we had the engine source code, we'd be able to fix issues like this with sound, further improve loading speed and performance, and fix things such as UI scaling or those stupid bugs with upper UFO floors.

[Solver]

Quoting Chris on the subject:

Quote

Our choice of engine was the single worst decision we ever made.

And:

Quote

(We also could have dumped the engine at any point any started over in an engine like Unity. In retrospect, this could have been a good idea. But an indie team with no proven track record starting over in a new engine halfway through development? That would not have gone down well and the negative publicity could have killed the project. I decided to stick with the devil we knew.)

 

[Charon]

19 minutes ago, Solver said:

Quoting Chris on the subject:

And:

 

I know, i read it . My 900 - 1000 sounds mods will come with the instability issues. Its really frustrating, but i know now you cant do anything.

 

Looks like the best choice is to simply wait for X2 and start over, or something along those lines.

[Charon]

@YaK Can see if never loading a GC save changes things in your next playthrough ?

[Solver]

@Charon, I will make a note to try something with the ambient sounds when GC ends (can't hurt at least) but I think your attempts to add hundreds of sounds will just be a stability disaster no matter what.

[Policenaut]

I've seen old threads of people trying to get the company behind the engine to give the source code and even found some of Chris' posts asking about it when searching around about it. If the source code still even exists at this point I wouldn't mind trying to revive the efforts of aquiring it again. I really like Xenonauts and I wish its flaws could be fully fixed but without full source code access that seems not quite feasible.

Nevertheless thanks for still supporting Xenonauts even though Xenonauts 2 is closing in. Expect to see some mods from me for Xenonauts in the future, including bugfixing of the building destruction, more tilesets and an unofficial expansion/rebalance mod. 

[Solver]

1 minute ago, Policenaut said:

If the source code still even exists at this point I wouldn't mind trying to revive the efforts of aquiring it again.

What specifically do you think could be done?

The company (PlayFirst) still exists and still makes casual browser/mobile games but I'm sure they don't care about the engine anymore. I've been involved in one successful effort to get a source code release before (over 10 years ago), and that was quite challenging even though the source was available. With the Xenonauts engine, I wouldn't be surprised if it's not even available anymore.

One more reason I have to add about why having the source would be nice is, it would be an improvement just to recompile the engine with a different compiler. Currently Xenonauts only compiles with MSVC2008, and I know even llunak wasn't able to make it compile with another compiler - even VC2010 - due to some issues with linking, and he's more competent at this than I am.