GLDv2 test performance reproduction

[pavelsuma]

Hi,
Thank you for your great work and sharing your code.

I was able to reproduce mAP scores for ROxford and RParis datasets, also with +1M distractors, both with and without re-ranking. Unfortunately, I cannot reproduce mAP@100 scores for GLDv2 retrieval test set. I am around 0.8 below your scores even without re-ranking. The same applies when I try to reproduce original CVNet - I can reproduce ROP but not GLDv2.

I cannot figure out what I am doing wrong. Is there any special pre-processing of the GLDv2 images which is different to ROP images? For example some image resizing? I noticed that if I keep GLDv2 images in their original size, some of them have their smaller side too small and the extraction with your code crashes. This does not happen with ROP as they are generally larger images with less extreme aspect ratios.

Thank you for your assistance.

[ShihaoShao-GH]

Hi, thank you for your interest in our work! The GLDv2 evaluation pipeline is with my Google collaborators. I have already informed them to provide some support. Stay tuned.

[pyeun08-alt]

Hello @ShihaoShao-GH ,
Thank you again for sharing your excellent work and code.

I would like to follow up on the previous issue regarding the GLDv2 retrieval test set, as I am currently facing the same problem described above. I was able to reproduce the reported performance on the Revisited Oxford and Paris datasets, including the +1M distractor setting, and both the global retrieval and re-ranking results are consistent with the reported numbers. However, I am still unable to match the reported mAP@100 on GLDv2, even without re-ranking.

RN101-SuperGlobal	GLDv2-private
No reranking (paper)	33.4
No reranking (reproduce)	32.68

Because a very similar question was already raised before, I wanted to ask whether there has been any update or clarification since then. The fact that the Revisited Oxford/Paris results are reproducible, while the GLDv2 results remain consistently lower, makes me suspect that there may be some dataset-specific preprocessing or evaluation detail involved.

RN101-SuperGlobal	ROxf-M	+1M	RPar-M	+1M	ROxf-H	+1M	RPar-H	+1M
No reranking (paper)	85.3	78.8	92.1	83.9	72.1	61.9	83.5	69.1
No reranking (reproduce)	85.36	78.68	92.24	83.8	72.8	65.5	83.79	69.02
Rerank top-400 (paper)	90.9	84.4	93.3	84.9	80.2	71.1	86.7	71.4
Rerank top-400 (reproduce)	91.28	84.04	93.37	85.76	80.83	70.86	86.84	72.82

In particular, I would like to confirm whether there is any special preprocessing for GLDv2 images that differs from the ROxford/RParis pipeline, such as image resizing, filtering, or handling of images with very small shorter sides or extreme aspect ratios. I also noticed that some GLDv2 images can cause issues during feature extraction if kept at their original sizes.

If there are any details about the exact GLDv2 evaluation pipeline, preprocessing settings, or implementation notes that were not included in the public code, I would greatly appreciate it if you could share them.

Thank you very much for your time and assistance.