End-To-End Raw Audio Based Instrument Resynthesis

End-To-End Raw Audio Resynthesis System for Piano -> E-Piano

This project focused on research and implementation of an end-to-end raw audio instrument re-synthesis system on the PyTorch platform. The aim of the project is to accurately model mappings from one set of instrument’s timbral characteristics to another. In this case the two instruments are a sampled Steinway piano and an electronic instrument that imitates the sound of the Yamaha DX7 synthesizer (an electronic piano sound). Audio files for both instruments were automatically synthesised from MIDI data and subsequently used to train two separate models. A convolutional autoencoder is used to find the mappings, which are cleared of noise in the second step, using a modified U-Net structure.

Report (PDF) • Code