Jump to content


Photo

Xcom + Python + Dosbox


  • Please log in to reply
2 replies to this topic

#1 Pi Masta

Pi Masta

    Sergeant

  • Xenocide Recruit
  • 68 posts

Posted 15 January 2007 - 04:34 PM

Ok, I'll start off telling you what has inspired me for this. I downloaded Scott Jones's XcomUtil. It really looked like it has a good potential as a platform for other mods. Unfortunately it uses batch files, which are good scripting files for common stuff, but utterly horrible for modifing a game. I begin wondering if I could modify the game easily.

I recently learned Python and so I thought, hey why not use it to modify XCOM. Python is freaking great as it's fast, compact, and best of all interpreted (aka, no seperation from code and execution). However there is one problem: Python was made well after the days of DOS. I found this PythonD site, but it is very choosy in the DOS it can run, seems their target is to run in NT Shells and all. Suffice to say I can't get it to run in DOSBox, namely because I can't get Long File Name support on it. Then I remembered a little comment on the windows python installer thanking a person saying without him Python for Windows would still be Python for DOS. So looked at the archives and tried some older distributions in DOSBox and got one to work!

I'm just tickled about this fact as parsing and making adjustments in Python is a sinch. I'm not sure about the version of Python I got. It was in the 1.4 downloads, but when it runs it claims its 1.5.2, however some stuff that is in the 1.5.2 documentation doesn't work in it, namely the True and False constants aren't defined, nor operators like +=.

So the past day or two I've been using the Ufopedia website and hatfarm's website to make a sort of 'back-end' on all the formats of the files. I'm kinda surprised at how far I have gotten and how smooth it has been going. I'm hoping when I get most of the files into nice Python classes, making a front end shouldn't be too hard. I'm actually thinking of making 2 front-ends, one for windows with a nicer GUI and a command line for DOS (though this can be achieved by: 'python some.py arg1 arg2').

I've posted the RAR file of the scripts I have so far. Basically the name of the python file is the same as the dat file (with .py instead of .dat of course), however there is one datread.py file that holds a class for parsing the files. Essentially I only need to specify the record length and the fields in the other files and this file will do the rest. Also note this code is poorly documented and messy in some areas, I have done as best as I can to make it compatible with the old Python version, but it was mostly developed under the current 2.5.

Just thought I'd share and see if there are any thoughts on this.

EDIT: My absent mind forgot to actually attach the RAR. LOL

Edited by Pi Masta, 16 January 2007 - 10:31 PM.


#2 zaimoni

zaimoni

    Sergeant

  • Forum Members
  • PipPipPip
  • 70 posts

Posted 16 January 2007 - 06:16 PM

....I found this PythonD site, but it is very choosy in the DOS it can run, seems their target is to run in NT Shells and all. Suffice to say I can't get it to run in DOSBox, namely because I can't get Long File Name support on it.

The HX DOS-extender linked from http://www.japheth.de/ reportedly lets Windows console applications run in true DOS. I'm not sure about how it interacts with DLLs, though.

I've posted the RAR file of the scripts I have so far. Basically the name of the python file is the same as the dat file (with .py instead of .dat of course), however there is one datread.py file that holds a class for parsing the files.

I'll have to take a look. I've done some preliminary work along these lines (although my intention is to implement advanced turn swapping and "proper" soldier sorting; I'm no longer damaging the score files, but the hardcoding of melee attacks is very limiting.)

#3 Pi Masta

Pi Masta

    Sergeant

  • Xenocide Recruit
  • 68 posts

Posted 05 February 2007 - 12:29 AM

Ok, I'm going to upload a newer better version. I've been spending too much time on this little project, but I guess I'm enjoying and learning at the same time.

My main concern is how easy this could be integrated with platforms. While yeah python is extremely portable, it was kind of made after the days of DOS. Getting one that works takes some digging/trial and error especially with DOSBox. I'd like for it to work with DOSbox, mainly because that's what I'm using but there's some limitations such as no long file names, and I don't think it handles environment variables correctly either (one python installation kept looking for /usr/bin, even though I set the appropriate environment variables).

I also have a kinda crazy solution to this. Making a sort of 'monitoring' program with python and have it run on the host environment (instead of trying to make it fit into DOS with XCOM). Then when the switching is made from geoscape to tactical a batch file or simple program can wait until a certain file exists (which the monitoring system would make once it's done). Then the batch program would delete the file and continue on to tactical or geoscape. I am mainly thinking to do this to give python a bit more speed. The python that I can get to run on DOS box takes about 2-3 seconds to load (because of the 32-bit DOS extender). Once it is up, it's being emulated so the speed isn't exactly great either.

As a side note I tried to get python to compile to a 16-bit DOS program (I downloaded 1.5.2 source and hacked away). It has been very frustrating just getting to compile, then to maybe run. I did get an interpreter prompt a few times but I couldn't import any modules (including site) because it was out of memory. I started getting the impression I was trying to get a square peg into a round hole.

Alright, well if anyone's keeping track there's a pretty big change I did. I found out about the struct module in python that loads C-structs from files, much much faster than my parsing code. There's only a few file structures that aren't implemented but most of those are files we don't understand yet. Well on second thought I don't really have any 'maps' decoded, mainly because there are great tools for them already and using this structure would be tedious if not silly.

There's a module/folder datfile that contains all the data specific loaders that will be the 'back-end' for the project. They are basically there to provide a python-ic interface each of the files, and I'm actually kinda proud how much those can do.

I guess I'll go into detail a bit on how it's set up. Each file is split into records, many have lots of records like soldiers (250), while others only have 1 like saveinfo. To load a file simply import the datfile module with the same name as the file (or of the same format, ie obpos can be used for obpos2). Inside each of the files is a class of the same name but with 'dat' appended to the end. They simply need file to load (either when contructed or explicitly through load). Alright enough blabberin, I'll give an example

from datfile.soldier import Soldierdat

x = Soldierdat('missdat\\soldier.dat')  #Don't forget to escape the back slashes in the path
print x[0].Name
x[0].iniFiring = 60					 #Set the initial firing accuracy to 60
x[0].incFiring = 10					 #set the increase in firing to 10, thus he has 70 accuracy
print x[0].getField("Rank")			 #Returns the formatted Rank of the soldier, x.Rank is the actual
										#number stored in the file

x[0].Rank = 15						  #This is an invalid value and will raise an exception unless...
x[0].unlock()						   #Unlocks the data checks to allow values that could not be obtained
x[0].enforce(True)					  #enables (or disables if argument is false) the lock
x[0].getEnforce()					   #returns the state of locking/enforcement
x[0]['Rank']							#this will also get the rank value (just the number)

x[0].pprint()						   #does a pretty print, basically giving long descriptions and formatting
										#of the values ie Booleans will be 'Yes/No' Ranks, armor, etc will 
										#map to string values

x[0][1]								 #numbers can be used as well, they are in the order of file and also
										#include 'Unknown' values that are left as raw strings. These values
										#can be modified and can be reached by a name like _UNKNOWN_##

for s in x[0]:
	print s							 #This will print all the raw values for the soldier

x.parseAll()							#This will parse all the records, 250 in this case of Soldier.dat
										#Normally the records are parsed on demand. Loading a file just places
										#the entire file (as a string) in the variable. The record classes
										#are not made until they are requested or explicitly parsed. Also
										#parsing a record will remove any unsaved changes

for s in x:
	print s.Name, s.Kills, s.Missions   #Prints every soldier's (including dead and unused slots) name, kills and
										#missions

Ok, I also have a psudo-front end for the soldiers called soldInfo.py (I'm sticking to 8x3 filenames) That has a slightly better interface for soldiers, and pprint() will print more like a grid (along with ASCII-ified stat bars). For example if s is an instance then s[0].firing would return the total firing for the first soldier, and s[0].bravery will return the ingame result for bravery (stored differently in the file). Of course these values can't be changed mainly to save myself headaches. You can do s[0].data to get the same data interface as above so s[0].data.incFiring = 10 will work.

Ok, back to coding, just keep in mind I haven't really tested these much.