Personal Project on Matias Di Bernardo

Z-Plane Visualizer

Sat, 12 Aug 2023 00:00:00 +0000

On the Procesamiento Digital de Señales (Digital Signal Processing) course at UNTREF, we explore the Z-Plane for filtering design with discrete variables. During the class, the professor introduced a tool built in MATLAB that visualizes the magnitude and phase plots in relation to the positions of zeros and poles on the Z-Plane. Inspired by this, I decided to recreate this tool in Python.

Leveraging my experience with the PyGame library, I was able to build a real-time application that allows for the movement of poles and zeros, enabling users to see how the transfer function changes in real time.

How It Works

First, I map the pixel positions on the Z-Plane to coordinates according to the unit circle representation. Then, I construct the transfer function, where each zero $z_{n}$ is a term in the numerator, and each pole $z_{i}$ is a term in the denominator. With the transfer function $H(z)$, I can plot both the magnitude and phase plots, which are also mappings from the normalized response into a space within the app.

$$ H(z) = \frac{\sum_{n=0}^{N} (z - z_{n})}{\sum_{i=0}^{N} (z - z_{i})} $$

Each frame recalculates the transfer function. The program performs efficiently because I store the zeros and poles in arrays, enabling faster computations with the help of numpy, which is already highly optimized. This allows real-time visualizations of the changes and provides a more intuitive understanding of the behavior of the Z-Plane. The following code snippet shows how the transfer function is computed using complex exponentials:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


def e(w, root):
 return (np.exp(1j*w) - root)

def transfer_function(zeros, poles):
 w = np.linspace(-np.pi, np.pi, RES)

 for zero in zeros:
 num = e(w, zero)

 for pole in poles:
 den = e(w, pole)

 H_z = num/den

 mag = np.abs(H_z)
 ang = np.angle(H_z)

 return mag, ang

This project is designed as educational material, providing students with a practical tool to better understand the interactions between poles and zeros in the Z-Plane. It is not intended as a professional filter design tool.

Functionality

The application has the following functionality:

Display and move poles/zeros: The user can select and choose the position of zero or pole in the Z plane. With the symmetry option active, all the poles and zeros selected or moved are attached to their symmetric par with respect to the imaginary axes.
Order: The order of the poles/zeros can be modified with the mouse wheel up to increse or down to decrease. It supports up to order 4. The color of the zero/pole change with the order.
Information: Holding the cursor over a pole or zero displays information about the position, symmetry and order.
Zoom: With the plus and minus symbols, the user can zoom in and out of the Z plane.
Delete: The trash bin symbol clears all the poles and zeros from the plane, and the user can also delete specific poles or zeros by pressing right click on them.
Magnitude Graph: The magnitude spectrum displayed in the app is normalized. This decision helps to keep the focus on the shape of the magnitude and it makes sure that the graph is visually meaningful for the user. The downside is that the difference in peak values is not captured.
Phase Graph: The phase is displayed unwrapped between $-\pi$ and $\pi$.

Demo

Here is a quick demo of the app in action:

The source code for this project can be found on this repo.

AI learns to play 2048 game

Fri, 17 Feb 2023 00:00:00 +0000

I used this project as an introduction to Reinforcement Learning. Having mostly worked with supervised and unsupervised learning, I wanted to start with a small, simple project to quickly grasp the main ideas and create something fun. I followed this YouTube video as a reference, which explains how to use Reinforcement Learning (RL) to train a model capable of playing the snake game. To make it more challenging, I applied the same network to the 2048 game.

Code

The code has three main components:

The Game: Implements the game logic and the graphical user interface (GUI).
Agent: Controls the gameplay.
AI Model: A neural network that learns how to play and guides the agent.

Game State

I modeled the game state as a 4x4 matrix representing the board. The actions in the game are represented by a vector, with the following possibilities:

Left: [1, 0, 0, 0]
Right: [0, 1, 0, 0]
Up: [0, 0, 1, 0]
Down: [0, 0, 0, 1]

Reward

The RL model works with rewards to quantify when the agent performs well or poorly. Initially, I defined the following rewards:

+10 points: When the agent doubles the points. This is crucial because, in 2048, points increase exponentially.
-10 points: When the game is lost. This serves as a straightforward negative reward.

Initially, the model showed slow improvement. To address this, I added an additional reward:

+2 points: When points are increased. This reward incentivizes actions that maximize board clearance and progress.

How the Model Works

The model is a simple linear neural network. It takes the game state as input and predicts the best next action to maximize the reward.

Rewards are managed using a technique called Q-Learning. A Q Value represents the quality of a decision based on the loss function. The loss function is derived from the Bellman Equation:

$$ NewQ(s, a) = Q(s, a) + \alpha [R(s, a) + \lambda \, \text{max}Q'(s', a') - Q(s, a)] $$

Where:

$Q(s, a)$: The Q Value for a specific state and action.
$\alpha$: Learning rate.
$R(s, a)$: Reward for a specific state and action.
$\lambda$: Discount rate.
$\text{max}Q’(s’, a’)$: Maximum expected future reward.

Experiments and Results

Random Test as a Baseline

To establish a baseline, I tested the average score achievable by taking random actions. After 1,000 iterations, the average score was 170, far below the 2048 points needed to win the game.

Initial Results

My initial attempts were discouraging. The model performed worse than random movements. Here are some early results:

Game 1,000 | Score: 208 | Record: 416 | Mean Score: 200 | Reward: 640
Game 1,000 | Score: 116 | Record: 346 | Mean Score: 161 | Reward: 310
Game 1,000 | Score: 112 | Record: 348 | Mean Score: 146 | Reward: 320

In these attempts, the agent developed a suboptimal strategy of filling the board before increasing points.

Improvements

After experimenting with the reward parameters, I focused on the exploration phase. Initially, the exploration games parameter, which randomly picks moves, was set to 25 games. Increasing this parameter allowed the agent to explore more strategies, leading to better results:

Game 1,000 | Score: 478 | Record: 478 | Mean Score: 230
Game 1,000 | Score: 514 | Record: 964 | Mean Score: 382

As the model improved, I extended the training to more games:

Game 3,796 | Score: 770 | Record: 1,366 | Mean Score: 469
Game 4,852 | Score: 631 | Record: 1,462 | Mean Score: 483

Finally, with 200 exploration games and 5,000 training games, the results were as follows:

Game 5,000 | Score: 840 | Record: 1,678 | Mean Score: 512

Although the model didn’t beat the game, I was satisfied with the progress. I believe that with longer training (the final run lasted 4 hours) and additional network layers, it would be possible to win the game using this architecture.

Demo

Here is a demo of the application, showcasing the agent’s learning process:

I implemented keybindings to control the game’s speed, providing three options:

Fast: For rapid training.
Slow: To observe and analyze the agent’s progress and mistakes.
Medium: Rarely used.

The complete code for this project is available in this repository.

Evaluation of different models for Speaker Diarization task

Fri, 04 Nov 2022 00:00:00 +0000

This project started as the final assignment for a seminar class at UNTREF called Seminario en Aplicaciones de Redes Neuronales en la recuperación de información musical. The objective was to use a Siamese Neural Network (SNN) in a different context than the one explored in class (music similarity detection).

For the final project of this course, we developed an SNN model from scratch using the Keras framework and the SincNet architecture to reduce audio dimensionality, achieving good results. Later, to expand this project, I tried another approach by using Wav2Vec for dimensionality reduction and re-implementing the entire project in the PyTorch framework. This attempt, however, yielded suboptimal results, indicating that the dimensionality reduction using Wav2Vec lost critical information required for the speaker diarization task.

Speaker Diarization Task

The goal of a speaker diarization model is to identify different speakers in an audio stream containing multiple speakers. For example, in a podcast with two people (A and B), the model must determine the time steps where speaker A is talking and the time steps where speaker B is speaking (and implicitly identify the periods of silence). These models are incredibly useful for audio editing and analyzing long audio sequences with multiple speakers.

Why Siamese Neural Networks?

In class, we explored the Siamese architecture to compare similarities between pieces of music, developing a tool capable of identifying covers of famous songs.

A Siamese Neural Network consists of two or more identical subnetworks sharing the same weights and parameters. It is designed to compare input pairs and measure their similarity, typically using a distance metric like Euclidean distance. Each subnetwork processes one input, and the outputs are combined to compute a similarity score.

With this similarity comparison in mind, we wanted to apply these networks to the speaker diarization task. The idea was to generate speaker embeddings from audio using a pre-trained model and compare the outputs of different audio segments. Based on the similarity score, we aimed to identify the segments where different speakers are talking.

Experiments

I tested two different methods for audio feature extraction to serve as speaker embeddings.

Keras Implementation with SincNet

In this approach, we used the SincNet architecture to extract speaker-specific features from audio. SincNet applies learnable sinc functions as its filters, which are particularly well-suited for audio processing as they mimic traditional bandpass filters. These features were then fed into the Siamese Neural Network, which compared pairs of audio segments to calculate their similarity scores. The model was trained on labeled audio datasets, and we observed strong performance in clustering audio segments by speaker, achieving clear boundaries between different speakers.

A report with the results can be found in the following Jupyter notebook (in Spanish).

PyTorch Implementation with Wav2Vec

For this method, I utilized Wav2Vec, a powerful pre-trained model for extracting deep audio embeddings. Unlike SincNet, Wav2Vec embeddings are derived from self-supervised learning, capturing high-level representations of audio. These embeddings were used in the Siamese Neural Network for similarity comparisons. However, the results were suboptimal for the diarization task. It appears that Wav2Vec, while excellent for speech recognition tasks, lost some speaker-specific details necessary for distinguishing between speakers in our setup.

Results

The experiments demonstrated that the choice of feature extraction method is crucial for speaker diarization. The Keras implementation with SincNet outperformed the PyTorch implementation with Wav2Vec, showing higher accuracy in identifying speaker transitions. This suggests that task-specific feature extraction, like SincNet, is more effective than general-purpose embeddings like Wav2Vec for speaker diarization.

The code for this project is available in this repository.

Conclusions

This project was one of my first experiences with deep learning models, where I applied my knowledge to a problem without following a specific paper or using a pre-trained model. I explored different solutions and concluded on the importance of feature extraction and model selection.

It also helped me become familiar with the syntax of the most popular deep learning frameworks and solidified my understanding in the process.