fix: use raw bytes for r8 map type order rules

[AbhiTheModder]

more info: #497 (comment)

• closes [DETECTION] Several detection issues for com.nextcloud.client_330010090.apk #497

[CalebFenton]

This seems extremely bad. Why can't we use the dex module? If we hardcode these values, do we know that they'll never break? We're hardcoding the same value for different things, we're doing some strange casting.

There are no tests, there's no explanation for any of this.

[CalebFenton]

Instead of making the code much worse, why not just fix the dex module?

[AbhiTheModder]

This seems extremely bad. Why can't we use the dex module?

1. Yara C has been moved to maintainance mode i.e, they no longer will provide updates (https://virustotal.github.io/yara-x/blog/yara-x-is-stable/), yara C does supports dex 39 which would for now fix this issue (and that too both yara-python or yara-python-dex maintainers need to do on their side by updating their yara submodule) but still future dex versions will not be supported in near future.
2. To use dex module we could switch to yara-x but that plan was held off for now @enovella

There are no tests, there's no explanation for any of this.

I mentioned it in the description above as:

more info: #497 (comment)

Once again, kindly read: #497 (comment)

[AbhiTheModder]

If we hardcode these values, do we know that they'll never break? We're hardcoding the same value for different things, we're doing some strange casting.

these are tied to the foundational DEX file format specification, which hasn't changed its core header layout or map item sizing in years afaik (and doing so would break backward compatibility with the Android runtime). Because the current YARA DEX module fails to parse DEX versions 39+, we have to read the raw bytes manually to support newer files.

Just to clarify, here's what uint16(uint32(52) + 4 + N * 12) is doing according to the official Android DEX format docs:

• uint32(52): Reads the 32-bit integer at offset 52 (0x34). In the DEX header, this is map_off (the offset to the map_list).
• + 4: Skips the 4-byte size field at the very beginning of the map_list structure so we land on the first map_item.
• + N * 12: A map_item struct is exactly 12 bytes long (type: 2 bytes, unused: 2 bytes, size: 4 bytes, offset: 4 bytes). Multiplying the index by 12 jumps to the correct item in the array.
• uint16(...): Reads the first 2 bytes at that final calculated offset, which gives us the type field of the map_item.