Time scale modifications algorithms are used to speed up or slow down the reproduction velocity of an audio. When you change the sample rate of an audio, the velocity changes but it also changes the pitch (when the audio is speed up it sounds highier pitch). There are different algorithms that change the velocity of the audio but mantain the pitch.

The main reference for this study is the following article, where all the different algorithms are describe in detail.

A Review of Time-Scale Modification of Music Signals.
— Jonathan Driedger and Meinard Müller¹

Algorithm comparison

There are two main algorithms, the Overlap-and-add (OLA) and the Phase Vocoder (PV). Both achieve good results under different signals and conditions. For this, a final implementation using Harmonic Percussion Separation (HPS) combines both algorithms and achiving the best results.

OLA

This method works on time domain and overlap sections of the audio (windows) and rearange it to achive a certain desire change on speed. This method works well for percussive signals, but it introduces artifacts when used with harmonic or tonal signals.

PV

This method works on frequency domain, and it combines chunks of audio in the frequency domain to achieve the desire change in time. This uses the phase vocoder principle to propagate the phase between the windows, this grantice the continuity of when applied to harmonic signals. On the contrary, it does not work for percussive signals becouse the phase propagation process eliminate the transients in the signals.

I created visualizations using Manim to enhance my class presentation. The first video demonstrates how the PV algorithm aligns windows to ensure smooth transitions in the generated signal over time. To achieve this, a Gaussian window is applied, which maintains continuity and smoothness, even at the start and end of the sequence.

The second video showcases the effects of applying the PV algorithm to a signal containing transients.

As predicted by theory, the transients vanish because the PV algorithm disrupts the vertical phase alignment. While these examples utilize idealized signals, they effectively demonstrate the key strengths and limitations of the algorithm.

HPS

To use both methods with their ideal signals, the HPS algorithms is used. This algorithm separete the signal into the harmonics and the percussive parts. It works by comparing the continuty of the signal in the STFT representation and using a filter to compare vertical versus horizontal presence on the spectrogram. With a threshold, a binary mask can be define over the spectrogram to separete the percussive parts from the harmoincs sections.

Results

We succesfully implement all the algorithms and compare them, verifying the theortical contents presented on the referenca article. In the process, we develope the toolkit to use this algorithms with the python programming language. All the code is avaliable on this repo

Academic Presentation

The study was preseented with my classmates on the JAAS (Jornadas de Acustica, Audio y Sonido). The main ideas and conclusions were preseneted on the conference. In the following repoert there is all the details and anlysis done for this proyect (in spanish).

ALGORITMOS DE MODIFICACIÓN DE ESCALA TEMPORAL.
— Matías Di Bernardo; Matías Vereertbruhggen; Sebastían Carro ²