[SECRET PROJECT] Alien Missions Overhaul

[kabill]

One of the secret projects I'm working on is a reworking of alien mission selection to allow a greater range of options for modders. I'm going to collect and document information in this thread and use it as a place for discussing further expansion. If you have any questions or suggestions, feel free to post them here.

Mission Arrays:

The first project has been to permit changes to the relative chance of alien missions spawning as the game progresses. This uses a new .xml spreadsheet file introduced in XCE v0.33 (when it releases). Each row contains a 'mission array' which specifies the alien ticker value at which it comes into effect and relative spawn chance for each of the nine alien mission. These values replace the values found in the individual AM_* mission files, which are no longer used at all. As such, at any given ticker point you are now able to change the distribution of alien missions, for example, increasing the chance of Terror and Base Attack missions as the game progresses relative to other kinds.

In addition, multiple mission arrays can be set to the same ticker point with a relative chance of being chosen. When a wave spawns, the game will randomly choose which of the eligible arrays is used for that wave (the total chance for all arrays in a group must equal 100). Thus, for example, you could have two mission arrays at a particular ticker point. The first, with an 80% chance of being chosen, would have a distribution of alien missions similar to the current vanilla game. The other, with a 20% chance of being chosen, would be a special array with a very high chance of Terror and Air Superiority missions being spawned and a low chance of others.

Note that all this feature does is allow more control over the alien mission spawn chances. It does not affect things like the oncePerWave variable or the mission cooldown times in gameconfig.xml so you may need to adjust those too if you want to take full advantage of this feature in your mod. Furthermore, the UFO type values in the AM_* files are still used there.

Progressive Crew Lists:

The second project has been to allow alien crews to progress independently of UFO type. This has been achieved using the races.xml file found in ufocontents. In this file you can specify the name of the race type, the ticker point at which it becomes active, and the chance of being selected relative to other race types in that band. The race type here must match the name of a crew list file, e.g. adding a race "caesan_advanced" would require accompanying crew list files ending ".caesan_advanced" for all UFO types you want to use that race type.

Using this system, you could do something like the following:

1) In races.xml, set up six additional race types in addition to the original 3: andron_advanced; andron_elite; caesan_advanced; caesan_elite; sebillian_advanced; and sebillian_elite. The 'advanced' types are set to ticker-point 300 and the 'elite' types set to ticker-point 600. They are all given an even chance.

2) For all UFOs, new crew lists are made for the new race types. E.g. there are now six additional corvette crew lists: aircraft.alien.corvette.andron_advanced; aircraft.alien.corvette.andron_elite; aircraft.alien.corvette.caesan_advanced; and so on.

3) The crew lists are rewritten such that the original files contain only low-ranking aliens, the 'advanced' files contain moderately ranked aliens and the 'elite' files contain the highest ranked aliens.

The result of this is that the strength of alien units now progresses independently of the UFO type. Thus, if you made a mod where all UFO types are active from the start, a battleship could have a large but low-ranking crew early on while late in the same a light scout might have a small but elite crew roster. And that's just one example of how you might use this system.

Plans for the Future:

- More control over certain mission types (UFO speed, landing chance, variable escort chance, aggressive/passive classification, aggressiveness of pursuit of objectives, etc.), possibly allowing these to vary in relation to the alien ticker like with the mission chances above.

- Resource and AI system so that the game actively works against the player rather than simply rolling random missions (a long term project).

Edited March 9, 2015 by kabill

[kabill]

I'll reserve this on the off chance I need the space.

[kabill]

Updated OP with information about the progressive crew lists feature I've just completed (after spending 5 hours hunting for a bug which required entering a single word of code to fix).

[drages]

Great idea.. with this you can even use bigger ships at beginning too. Independence Day like beginning would be cool

[llunak]

It's not clear to me how much data you want to have in this new mission arrays file, but are you sure using XML spreadsheet is a good format? XML spreadsheet is not very flexible (I already find it limited for AM_* files), both with regards to structure (it has to be a table thing, which already AM_* files aren't really) to mod merging (adding new missions types wouldn't be trivial). If mission changes is the only data that is supposed to be there, I guess that's good enough, but in case you want to include more (such as dumping AM_* altogether), then maybe normal XML would be better.

Progressive Crew Lists: Technical: Rather than having fake races, IMO it'd be better to have sub-races. Should be clearer and easier to do some things, such as falling back to generic race values if there's no data for a specific sub-race. It'd also keep stock X:CE simpler, as you'll possibly need to keep this to mods like your dynamic UFOs. The point of keeping easy UFOs even in mid- and late-game is to allow easier missions for e.g. training up new rookies, so it's a question if stock X:CE should ever have a scout with elites on it. I'm not saying the answer is necessarily no, but it's something to at least think about (and, thinking of it, I can't help thinking that it might be "interesting" to have only an occassional smaller ship have such a soldier, just to keep the tension fresh ).

[kabill]

It's not clear to me how much data you want to have in this new mission arrays file, but are you sure using XML spreadsheet is a good format? XML spreadsheet is not very flexible (I already find it limited for AM_* files), both with regards to structure (it has to be a table thing, which already AM_* files aren't really) to mod merging (adding new missions types wouldn't be trivial). If mission changes is the only data that is supposed to be there, I guess that's good enough, but in case you want to include more (such as dumping AM_* altogether), then maybe normal XML would be better.

I can post up a copy of the .xml spreadhseet I've put together later by way of demonstration but currently a spreadsheet format feels most appropriate. The AM_* files aren't replaced other than their "chance" variable, so they're still needed (especially for the UFO rosters).

I can see a case for replacing with regular .xml files if the AM_* files were to be replaced though. I don't know if you've seen but I would actually like to try and rewrite the UFO loading code at some point because it works in a way that seems strange to me (the game reads directly from the AM_* files every time it loads a UFO rather than uploading the AM_* file data on game load, which I assume is why there's a lag-spike whenever a new wave is generated). This might end up being beyond me, but in any case it's something to keep in mind for if/when this happens.

Progressive Crew Lists: Technical: Rather than having fake races, IMO it'd be better to have sub-races. Should be clearer and easier to do some things, such as falling back to generic race values if there's no data for a specific sub-race. It'd also keep stock X:CE simpler, as you'll possibly need to keep this to mods like your dynamic UFOs. The point of keeping easy UFOs even in mid- and late-game is to allow easier missions for e.g. training up new rookies, so it's a question if stock X:CE should ever have a scout with elites on it. I'm not saying the answer is necessarily no, but it's something to at least think about (and, thinking of it, I can't help thinking that it might be "interesting" to have only an occassional smaller ship have such a soldier, just to keep the tension fresh ).

Ha, I should probably make it clear (this is a general point) that any non-trivial change I suggest is intended as a feature for modders rather than default changes. Thus, I don't expect progressive crew lists to be used in base XCE and the way I have implemented it is such that the base game shouldn't be any more complex than it already is, at least in terms of setup - the ufocontents files remain exactly the same and all that's changed externally is that the races.xml file has a couple more variables per element (but for 'vanilla' XCE can use the standard Caesan, Sebillian and Andron racial groups as normal). In other words, I'm making an effort to keep disruption to a minimum.

[kabill]

So, here's a copy of the base file for controlling UFO mission chances:

https://www.dropbox.com/s/p27oyj9hrhhipdt/ai_missionchances.xml?dl=0

And here's the one for controlling race types:

https://www.dropbox.com/s/ij0uipk8cl9959z/races.xml?dl=0

Both these seem a reasonable format to me (but then I would say that since I made them!)

[Sentelin]

I read description just now and took brief look at XML sheets.Both of projects are something that I wished we had year ago when we were developing XNT (they would have allowed unique events and boss fights possible,alas we had to make compromise that didn't work well in the end) when in general mod community was more active.XML sheet for controlling UFO mission is pretty clean and easy to read even if you are forced to use text editor.For controlling race types is also good formatted I don't believe anyone is going to get confused .

I don't have what else really to say for now(I wanted to ask similar of what Illunak posted -why would you go with fake races instead of sub-races - but he beat me to it),at least until you more progress is shown

Edited March 11, 2015 by Sentelin

[kabill]

I went with 'fake' races because it required less work to implement. The game already supports the addition of extra racial groups fine so honestly it didn't occur to me to do anything differently. Indeed, I'm not really sure what the advantage of using sub-races would be: it would require extra work and make the internal system more complex to produce exactly the same outcome.

(It's possibly not apparent without looking at the code, but the race names in races.xml are only used as a means of identifying valid ufocontents files for loading. Beyond that they don't do anything at all, which is the case even without my changes. Hence, there's functionally there's no difference between using "Caesan_Advanced" as a 'race' and implementing a system which reads "Caesan" as race followed by "Advanced" as a subrace. Except, of course, for the fact that the former system is already coded.)

[Skitso]

I don't know if you've seen but I would actually like to try and rewrite the UFO loading code at some point because it works in a way that seems strange to me (the game reads directly from the AM_* files every time it loads a UFO rather than uploading the AM_* file data on game load, which I assume is why there's a lag-spike whenever a new wave is generated). This might end up being beyond me, but in any case it's something to keep in mind for if/when this happens.

Yes please! That lag spike really bugs me. Also geoscape scrolling lag when a continent in selected would be nice to have fixed.

[llunak]

I can post up a copy of the .xml spreadhseet I've put together later by way of demonstration but currently a spreadsheet format feels most appropriate. The AM_* files aren't replaced other than their "chance" variable, so they're still needed (especially for the UFO rosters).

I know the format, saw it in the commit. My point is that even though the format may be good enough now, it may not be that way for long. Soon there might be e.g. the idea that there could be more than just 1 type of terror wave, with different craft composition (or some other settings), and these would be properties of the wave rather than mission type, so would go naturally into this file. So there's a good chance this new format would very soon become obsolete again.

[kabill]

Yeah, that's fair. I spent a little bit of time last week making a mock-up of a regular .xml file replacement for the current AM_* files and it convinced me that there's a lot that could be done with that to expand existing capabilities. I suspect the change I've already done could be included in the same file, such that all the alien mission properties are kept in the same place (possibly even transporting the variables that are currently set in gameconfig to allow some flexibility with those as well).

If I have time, then, I might try working on this over the current week to see if I can get it working, even if just a replacement for the function of the existing AM_* files to try and get rid of the lag caused by wave-spawning. Extra functionality should be pretty easy to add on top of that, I think.