Open Source Disassembly / Reimplementation of F1GP/WC

[Alejandro Caceres]

Hello to everyone! I hope you all are staying safe.

First, an introduction...

My name is Alejandro, I'm from Argentina and I've been a long time MicroProse fan: started playing back as a kid in 93 w/ F1GP in my A500 and F1GP-ed, then moved to PC GP2/3/4... Before that, tried Stunt Car Racer on the C64 (sadly missed Revs at the time!). MicroProse got me hooked on simracers (mostly AC, rF2 and AMS), and browsing related content I recently found Ted Meat's great F1GP season stream. So I decided to try F1GP with wheel and pedals and was very pleasantly surprised.

Looking around some more landed me on this site, and that got me thinking. From the last time I checked, TSRs, addons and editors for F1GP continued evolving a lot and nowadays they let you do things with F1GP that were hard to imagine a couple of years ago (I'm glad to see that names from years ago such as Rene Smit are still active!). I was also lurking around this group and found a copy of the F1GP EXE IDA database, with several funcions/names/strings/datatypes already decyphered/defined/established.

I'm also a Free Software/Open Source advocate, and right now the open source sim racing landscape is quite barren, with projects such as TORCS and its forks currently not delivering the potato-PC-safe, back-to-basics, fun but accurate racing sim experience a hypotetical "OpenGP" could. My thinking is, with F1GP/WC it may be possible to jump from where the Transport Tycoon Deluxe community was several years ago (TTDpatch, patching the executable to add new functionality) to a new, opensource reimplementation (OpenTTD) which, in time, became an excellent game on its own.

So, questions:

1. Is there any reverse-engineering/dissasembly/reimplementation effort of F1GP/WC currently on the works? If so, in what status is it?

2. Is the disassembly in a complete state somewhere? Can an assembly be successfully regenerated and a new, compatible EXE compiled from it?

3. What are the biggest obstacles to overcome? I assume that most game logic, rendering pipeline, AI and physics are still uncharted territory. Is that so?

4. Are there any legal hurdles to accomplish this? From other open source reimplementations, I understand that it is fair game to reimplement commercial software through reverse engineering: CannonBall (OutRun), OpenRA, OpenXcom, OpenTTD, CorsixTH (Theme Hospital) are some examples (IANAL).

5. How can I help?

These are some of my possibly useful skills:

1. Intermediate/Advanced C/JS/Python/PHP/sh/regex coding skills (been a Linux sysadmin for almost 20 years now)

2. Development workflow, toolchain, version control and debugging skills (ditto)

3. Multiple OS experience (Windows, Linux, Mac OS...)

4. Polygon and NURBS 3d modeling (currently studying Industrial Design, so I'm fluent in most CAD and 3d modeling environments)

5. Industrial and graphic design (ditto)

6. Documentation and translation to Spanish (my native language)

In this context, I find myself with more free time than I expected so I may be able to dedicate a few hours per week to this project. Plus, it should help me properly learn x86 assembly to boot!

Anyway, let me know what you think.

Take care,

Alejandro.

[René Smit]

Hello Alejandro,

Good to see a new old fan :)

There has not been an attempt to reassemble the game from the current disassembly. I've thought about it a few times, but I was sure it wouldn't work so I didn't even try. You're free to try it of course :)

The uncharted territory in the disassembly is mostly in the low level graphics, and AI and physics. The game logic is what I mostly concentrated on. To be honest I don't think it's feasible to recreate the game from scratch. For me it has always been much more effective to hook into the game and add new features from there.

Only recently I got involved in enchancing the TSRs made by Trevor Kellaway back in 1995, after he made it open source. Before that I've been working on tools for GP2/GP3/GP4. The big difference is that F1GP runs in 16-bit DOS. Though part of the assembly code is still 16-bit even in GP4. Geoff just didn't migrate all of it to 32-bit (it was handwritten assembly, not compiled from C for example).

Cheers,

Rene Smit

Op maandag 1 juni 2020 17:31:21 UTC+2 schreef Alejandro Caceres:

[Alejandro Caceres]

Hello René,

Thanks for the welcome!

I've been scanning through the disassembly database and it looks to be almost a third of the way, in terms of quantities. Many data structures are there, and many functions commented/figured out. It looks that it is a daunting task just to get to that point.

I could help with ongoing projects to get to grips with assembly. Or maybe dive into the disassembly and try to document what I can. I think I could try to get to the modesetting/framebuffer code first, it should be pretty straightforward to get there since F1GP is VGA-only. I think this may be useful to patching and TSR projects too. Then again, I may fail spectacularly.

So right now I'm catching up, reading the Development site and linked repos. Is there a place where I can download a more recent IDB file? Should I email to f1gp.archive for access? What is the recommended workflow/toolchain for patching binaries?

Thanks again,

Alejandro

[René Smit]

Hello Alejandro,

Where did you get the .idb file from? What is the development site you are referring to? The latest version is in the "F1GP Ongoing Development" folder on Google Drive, but at least for write access you'll need to be given access to it first by Hrvoje. 

It would be great if you could help out. Besides checking out the latest .idb file you can look at my fork of Trevor's F1GP utilities repository: https://github.com/sdidit/f1gp-utils

Drawing in 2D on the screen is already possible and done by e.g. GPPERF. In my GPTEST utility I also print lots of data on the screen.

The Google Drive folder also contains the tools needed to compile most of the TSRs and other utilities in DOSBox. It also has a debug version of DOSBox with which you can debug executables and set breakpoints etc.

The workflow for patching the executable depends a bit on the application. I prefer not to touch it and always patch in memory using a TSR, after it has been unpacked. Others are patching either the packed or unpacked binary, but it's not guaranteed that you can find the right offset in the packed version. There's also the support for multiple versions of the executable. My approach is to calculate the EU 1.05 offset and calculate the other offsets from that. I've made a Python utility to calculate them, but I also implemented this in GPTEST in assembly.

Rene Smit

Op dinsdag 2 juni 2020 20:19:06 UTC+2 schreef Alejandro Caceres:

[F1GP Archive]

Hello Alejandro and welcome.

I will send you the invitation to the development folder that René mentioned by email.

This way you will be able to access the latest version of the IDA database and other development-related files.

Cheers,

Hrvoje

[Alejandro Caceres]

Hello Hrvoje,

Thank you!

Alejandro

[Alejandro Caceres]

Hello Rene,

I found the IDB file as an attachment on this very forum, in an older thread. Will check the latest version, read only access is more than enough. Got a lot of catching up to do!

Thanks for the tips,

Alejandro

[Klaus Six]

Hello Alejandro,

welcome to the group.
Having F1GP reimplemented as an open source project sounds interesting. I'm afraid it is a very challenging task.
Anyway, having more people look into the IDB database and figuring out what is going on there will not hurt.

I'm sure I have a bunch of information currently not included in the IDB database as it is only in my own copy.
@Hrvoje: could I please get access to the files Rene mentioned as well?
I would love to contribute what I know.

My main focus currently is to figure out the meaning of the data in the track files. Some of it is still unknown.
I have some informations about internal object structures, mainly deducted from the existing GP2 documentation, which is at least not included in ArgDocs currently. From that it should be possible to display the internal objects. It sounds like this could be a task for you Alejandro if you have experience with 3d modeling.

Greetings,
Klaus

[F1GP Archive]

Hey Klaus,

You've been added to the F1GP Development folder.

I've sent the invite to your email.

Some of the files you supplied are already there, so feel free to rearrange them.

There's also the free version of IDA, that should be used to open the database.

Cheers,

Hrvoje

[Alejandro Caceres]

Thanks Klaus!

I figured out the same thing - having a full disassembly will be of great help, and it is the first step for a reimplementation anyway. I think that it may be daunting but definitely doable. Been looking at the database and already found a couple of small functions I could document so looking forward to the updated IDB file.

I'll gladly take a look at the 3d track objects data representation. Since the trackside objects are not exactly high polygon count, starting from a known object some assumptions could be made about the underlying data.

Best regards,

Alejandro

[Fredrik Meyer]

Hello,

Any information about the 3D shapes would be very welcome, not least from an ArgTrack perspective, and hopefully converting GP2 tracks (including track side objects) to F1GP.

I know a little more about the 3D shapes than what is currently documented at the ArgDocs site, but not much. But I think I maybe figured out the split between the groups of data (lines, vertices, etc) at least. This can be seen (to a very limited effect) in ArgTrack, if you choose double-click on an Object Shape in the tree on the left. There are some buttons there for editing "Data at Offset 1", "Data at Offset 2" etc. This is very likely to be quite incorrect, and I must admit I haven't played around with it at all. I'm pretty sure that with some disassembly and an IDB database you will get much further than that.

Cheers,

/ Fredrik

[Alejandro Caceres]

Hello Fredrik,

Will take a look at the newer IDB and try to decypher the correct data structure for trackside objects. It is my understanding that this was advanced quite a bit already so I'll try to help.

Regards,

Alejandro