momonala / aposynthese Goto Github PK

this is a big ticket. I would like to to not create the video with the Decomposer class. Instead it should only decompose a song into its spectrogram, chromogram, and a 3rd matrix of size (88, 128, 25*t), where 88=number of keyboard keys, 128 is the amplitude range, and t is the number of seconds (so 25fps). This matrix can be json serialized and sent as a HTTP response to a front end software for visualization.

The idea is that the intermediate matrix of the keyboard visualization is the largest memory in object (several GB), and the computation of filling this matrix plus the build movie step is quite time expensive. This can all be pushed to a separate process on the client side to speed up the entire request.

Solve: WARNING:root:IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (536, 960) to (544, 960) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to None (risking incompatibility). You may also see a FFMPEG warning concerning speedloss due to data not being aligned.

Issue cause because of approximation below, where self.width_full no longer divisuble by 16, which is needed for compatibility with most video codecs.

 # sizes and variables for genreating full frame visualization
        num_time_steps_in_1_sec = int(self.t_final / (self.stop_time or self.duration))
        stretch_vec_factor = int((self.width_full - self.keyboard_width) / num_time_steps_in_1_sec)
        self.piano_roll_width = num_time_steps_in_1_sec * stretch_vec_factor
        self.width_full = self.keyboard_width+self.piano_roll_width  # update full frame size to approximated

clean it up

The following can be precomputed and stored to disk

    freqs['Helmholtzname'] = freqs.Helmholtzname.apply(lambda x: x.replace('′', ''))
    freqs['Helmholtzname'] = freqs.Helmholtzname.apply(lambda x: x.replace('͵', ''))
    freqs['Helmholtzname'] = freqs.Helmholtzname.apply(lambda x: x.split(' ')[0])
    freqs['Helmholtzname'] = freqs.Helmholtzname.apply(lambda x: x.upper())
    freqs['Frequency (Hz)'] = freqs['Frequency (Hz)'].astype(np.float16)
    freqs['Keynumber'] = freqs['Keynumber'].astype(np.uint8)
    freqs = freqs[freqs.Keynumber <= 88]
    freqs = freqs.sort_values('Keynumber', ascending=False)
    freqs.index = range(1, 89)