Skip to content

Updates in FreesoundExtractor #1510

Open
ffont wants to merge 22 commits into
masterfrom
ffont-patch-1
Open

Updates in FreesoundExtractor #1510
ffont wants to merge 22 commits into
masterfrom
ffont-patch-1

Conversation

@ffont
Copy link
Copy Markdown
Member

@ffont ffont commented May 4, 2026

This PR includes some updates in the FreesoundExtractor:

  • Removed check for completely silent audio in FreesoundExtractor. There are files in Freesound which will be complete silence or almost silence, so it is fine to provide results for them as well.
  • Add computation of the Freesound similarity space vectors, based on a normalization of the low level descriptors and PCA transformation. Coefficients for such operations are hard-coded in a separate (huge) header file.
  • Add some post-processing of computed descriptors that will output midi note numbers, names, and some other minor stuff (in a separate fs.x namespace). This is used in Freesound when indexing the descriptors. We used to do that post-processing in Freesound code, but doing it here it is useful as people that uses the FreesoundExtractor to analyze sounds locally, gets descriptors which are directly compatible with those indexed in Freesound.

ffont and others added 19 commits May 4, 2026 14:08
Removed check for completely silent audio in FreesoundExtractor. There are files in Freesound which will be complete silence or almost silence, so it is fine to provide results for them as well. Also, this happens before descriptors have been computed, so it should not make things fail?

I have to test that, for now making this PR and see how tests behave...
Before it was hardcoded to use python3.6, which is no longer installed
in our manylinux images.
We don't need to use FFmpeg 2.8 as since #1494 we support FFmpeg 5+
Eigen 5.x (current Homebrew formula) requires C++14 features
(std::enable_if_t, std::integer_sequence, etc.), causing build
failures with the previous C++11 default.
Update cibuildwheel-tensorflow.toml arm64 override:
MACOSX_DEPLOYMENT_TARGET 15.2 → 15.4 (matching the current libtensorflow
2.20.0 bottle's minimum)
Let's use the same pipeline in both workflows so that we don't have to
fix deps individually everytime we change something.
This avoids running checkout@v4, which depends on Node 20 on Centos7 as
it is not supported.
Build doc inside docker to mimic the cibuildwheel behaviour.
- 15.0 for the normal wheels as required by libcrypto.3.dylib and
libssl.3.dylib
- 15.2 for the tensorflow wheels as required by libtensorflow
Let's take the opportunity to bump our tooling:
- ubuntu22.04 --> ubuntu24.04
- checkout v4 --> v6
- cibuildwheel v2.23 -> v3.2
- upload-artifact v4 -> v5
- cp38: project does not target Python 3.8.
- *t-*: free-threaded (no-GIL) builds. The Essentia C bindings rely on
  legacy PyTypeObject definitions that fail to instantiate under
  cp313t/cp314t ("Type does not define the tp_name field"). Re-enable
  once the bindings are audited for free-threading compatibility.
- *-musllinux*, *i686: not supported by our manylinux2014 build images.
Since Node.js 20 actions are deprecated
Co-authored-by: Copilot <copilot@github.com>
@ffont ffont changed the title Remove silence check in FreesoundExtractor Remove silence check in FreesoundExtractor and add sim space vector computation May 6, 2026
@ffont ffont changed the title Remove silence check in FreesoundExtractor and add sim space vector computation Updates in FreesoundExtractor May 8, 2026
@ffont ffont marked this pull request as ready for review May 8, 2026 15:07
@ffont
Copy link
Copy Markdown
Member Author

ffont commented May 8, 2026

I think the implementation is probably fine, but I'm a bit concerned about the size of the hard-coded literals in src/essentia/utils/extractor_freesound/FreesoundSimSpaceDescriptorsData.h.

Also note that this includes Pablo's changes to make builds work. Maybe better to merge these first to master so the diff here makes more sense.

@ffont ffont requested review from dbogdanov and palonso May 8, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants