Cool thing, wanted to add some more Info

[KnackiKitsune]

Only just now saw this and got interested, made an account because I saw some stuff on the wiki section is currently unclear or unknown, and I happened to have looked into various inti creates ds games (including the zx games):

-the "dm" files are short for "demo", and are all cutscene images
-the area data files (a01, b02, etc) are all tilesets, while you did mention in the rom hacking notes, it is not on the rom map wiki section
-the stage layout data are all in overlay files, for example: area a4s layout is in overlay 0047 (in the JP version at least)

also there seems to be no mention about the headers for the bin files, and/or the offsests at the start of the file, and what they mean in the rom map or rom hacking wiki pages, which are very crucial for understand these, I assume you already know all that, but for people not familiar, it would be appreciated if they get some more insight
very roughly worded (and not pretty looking) example:

"area" files: 0x0 length 4 = how many things are in this file,
starting at 0x4 length 4* offset of the first thing (*first 3 bytes are the offset, while the 4th byte is a compression indicator IF that part is compressed, represented as 0x80 mainly it seems),
same as 0x4 until last thing reached,
4 bytes before the first thing = file length/EoF

hope I didn't come off as rude, I do find this so promising looking, and really hope this can progress well, not just for zx hacking, but some of it could be applied to other inti creates (ds) games as well, so any information more public and accessible is good

(one additional note, the rom map may be a bit confusing for some, I would either just say describe what is in these files, or make it more clear that they do belong together and are not separate)

anyways, if my comment is confusing in any way, let me know, and of course hope this project will continue to go well :)

[AlaryVanEeckhout]

Thanks a lot for the info and feedback!

Interesting, so that's where the stage layout is stored. Do you happen to know how to determine if an overlay has that data or how it is structured? And maybe where the data for sprite/enemy placement is?

For the header structures, I did put comments documenting the structure of the files in the code, but I will definitely put it in the ROM hacking notes section, since it is probably a more appropriate place to put that information as you pointed out.

For the ROM map, I kind of want to keep a detailed list of each entry in the files. What do you suggest I do to make it look better? Should I have a column exclusively for file name and leave it empty for entries in the same file?

[KnackiKitsune]

enemy placements I am not sure of for now, one thing I did notice that in the overlay file, there will often be a file name that likely is there to tell the game what tileset to load (so 0047, there will a04.bin):

if you look at a (decompressed) overlay file and notice a lot of repeating numbers like, 20 0x01 in a row, that is also a good indicator of that; those should be the tiles loaded in, and there should be 2 or 3 of these parts (don't have a decompressed overlay rn to showcase it well)
the game does stages almost the same as the mmz games from what I have seen, maybe the z3 decomp project could be of any help?

the rom map thing is probably a bit trickier, what you suggested sounds good
-adresss -file -description
-000 -file.bin -this
-000 ___________ -more

but not sure if that format work in a way that is also pleasing to look at, if it doesn't, oh well
like for the o01.bin part, the backgrounds and explosions are obviously part of the file, don't know if that, or like a - or ^ could help to make it more clear or not, hard to say as while I get it, not sure how others would
this was more "maybe" suggestion tbh, what works, works in the end, so no pressure
otherwise a few of the section just don't have their file names (the talk files for example, and how for example the ones ending with 1 are for vents story, and the ones ending with 2 are for ailes story), should probably include it or add it as part of the description

anyways thanks for the reply! and appreciate the kind words :)

edit: found an image I took when I was messing around with overlay 0047, where a change I made changed whater transparency, so stuff like that can be changed too:

[KnackiKitsune]

update: I just did some more testing, mainly by changing the repeated same byte section I mentioned to random numbers, and the part I edited caused the background to scramble up (and at one part the game just turned into a white screen, hooray results!), so if it is like the mmz games like I assume it is, then 2 parts of these should be for background related things, while one part is for the layout and tiles you walk on, some closer debugging and also taking a look at that these overlay in a disassembler, may get us closer to fully understanding it
(also good job on the updates you made)

[AlaryVanEeckhout]

Thanks for the detailed responses!
I'll look into the ARM9 overlays at some point, not sure when, but thanks to the info you gave me, it should be a lot easier to figure out.
I'm sure there's a lot of cool things we'll be able to do with that data. I saw that in area X-2, the moving clouds in the background are actually individual screens, so the concerned overlay is probably making them scroll horizontally in the background in a certain order/sequence.

[KnackiKitsune]

no problem!
the clouds thing is probably a (h)dma type thing, the ds only has 4 bakground layers (if I remember correctly), unless they did dedicate a whole layer for it.
there is this video I saw once partially explaning it, while this is for new super mario bros ds, all ds games have the 4 background limit, so you can apply similar things here, but there are multiple possibilities of course
so maybe it is handled by the stages' respective overlay, or it could also be the arm9, (or a whole overlay soley dedicated for that, which I honestly doubt)
and agreed, the potential is there, just a matter of fully figuring it out

edit: seeing that you are doing the jp character set now, I have some of the kanji and one of the chracaters you are missing, the jp versions are the ones I usually take a look at, tho I have not finished this particular table, am just gonna share what I have:

3F=ー it doesn't show here, but the line goes slightly up on the left

F001=-
F008=?
F00D=「
F00E=」
F00F=･･･
F010=運
F011=屋
F012=部
F013=隊
F014=合
F015=流
F016=本
F017=社
F018=何
F019=者
F01A=出
F01B=現
F01C=原
F01D=因
F01E=調
F01F=査
F020=子
F021=生
F022=命
F023=体
F024=戦
F025=争
F026=発
F027=地
F028=見
F029=会
F02A=国
F02B=助
F02C=年
F02D=新
F02E=今
F02F=回
F030=変
F031=身
F032=少
F033=女
F034=転
F035=送
F036=青
F037=赤
F038=上
F039=後
F03A=世
F03B=界
F03C=進
F03E=王
F03F=人
F040=間
F041=数
F042=百
F043=支
F044=配
F045=男
F046=勇
F047=気
F048=前
F049=闘
F04A=用
F04B=時
F04D=平
F04E=和
F050=名
F051=・
F053=下
F054=左
F05D=中
F05E=開
F061=長
F062=大
F065=同
F066=動
F067=続
F068=悪
F06B=空
F06E=使
F074=一 same deal with 0x3f, but the line goes up a bit on the right

F077=向
F07B=各
F07F=止
F080=在
F083=仲
F089=目
F08A=々
F08D=伝
F08E=説
F08F=入
F092=守
F093=手
F09B=.

if you find that any of them are wrong (some kanji look very similar to each other) let me know
also if you can't find anything immeditaely in the font file , try to modify some of the actual text and see the results in game there (I personally only do it that way, even if it is a often more tedious and looking at the font file is often just good enough)

[KnackiKitsune]

Guess I can also mention this even if it is not useful at all, and just moreso interesting, one of inti creates' other ds games (kabu trader shun) has a dummy file that actually turned out to be an overlay file from (very likely to be assumed jp version) zx.
Not only that, it also turned to NOT match the one found in the final release (which would be overlay9_0047)!
However, everything related to the actual stage stuff matches 1:1 with final, hence why there really is not much use for it, the main difference are seemingly: some pointers being different, and a whole section that has names for various things, like the doors, items, and enemies. which is why I assume this came from either a late proto build, or some debug rom, but have no evidence for either, because its the only fine we have for it.
obviously the file is compressed, so it looks like this at first:

after being decompressed, you get something fully readable:

most of the strings found there are things that appear in the actual area too!
there are also pointers to the strings below them, which actually order them quite nicely, so it easier to know what is supposed to be part of which "category"

again not useful much, but something to look at and compare if you are interested