Everyone loves music visualization, but not all apps support it in a sensible way. Maybe you want to shuffle a random assortment of video and audio files in a player that doesn’t handle that well (VLC!), or not at all (mplayer!). Or maybe you want to upload something to youtube, with gorgeous HD visualizations instead of that lame static cover art image?
The few google results on the topic that weren’t spam suggested screencapping software. Yeah, that’s great… until you have more than two files.
Once again, everyone’s favourite multimedia swiss army knife – GStreamer – steps up to the plate.
Here’s an example of encoding an MP3 to an H.264 .mkv file using the gorgeous goom visualizer (requires the mp3 and x264 plugins for gstreamer):
gst-launch filesrc location=input.mp3 ! queue ! tee name=stream ! queue ! mp3parse ! matroskamux name=mux ! filesink location="output.mkv" stream. ! queue ! mp3parse ! mad ! audioconvert ! queue ! goom ! ffmpegcolorspace ! video/x-raw-yuv,width=1280,height=720 ! x264enc ! mux.
It’s beautiful – and the video is pretty sweet as well.
It’s worth noting that this approach does not re-encode the MP3, like some less awesome approaches would do (causing loss of quality). It simply muxes it together with the visualizer’s video stream. x264 even seems to distribute itself well across cores.
No, wait, what? MP3 and H.264? Of course, I meant Vorbis and Theora! Let me rephrase:
gst-launch filesrc location=input.ogg ! queue ! tee name=stream ! queue ! oggdemux ! vorbisparse ! oggmux name=mux ! filesink location="output.ogg" stream. ! queue ! oggdemux ! vorbisdec ! audioconvert ! queue ! goom ! ffmpegcolorspace ! video/x-raw-yuv,width=1920,height=1080 ! theoraenc ! mux.
The same goodness applies, except for the parallelism. If you have a multicore CPU, there’s massive speedup to be had through simple shell script based multithreading. (Why full HD this time? VLC on Windows crashes on 720p Theora!)
And there you have it. A simple, hack-free, modular and flexible way of encoding visualization videos for MP3 and Ogg Vorbis files. Thanks, GStreamer!
If you’re creating your own pipeline and GStreamer gets stuck during preload, try adding more queue elements.
If you get errors like “/GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data flow error”, read further down in the error output. If it says “reason not-negotiated (-4)”, you’re probably missing some suitable converters like audioconvert or ffmpegcolorspace.
updated the mp3 version for 1.0
gst-launch-1.0 filesrc location=input.mp3 ! queue ! tee name=stream ! queue ! mpegaudioparse ! matroskamux name=mux ! filesink location=\output.mkv\ stream. ! queue ! mpegaudioparse ! mad ! audioconvert ! queue ! goom ! videoconvert ! video/x-raw,forma=yuv,width=1280,height=720 ! x264enc ! mux.
so much for my editing skills corrected below:
gst-launch-1.0 filesrc location=input.mp3 ! queue ! tee name=stream ! queue ! mpegaudioparse ! matroskamux name=mux ! filesink location=\output.mkv\ stream. ! queue ! mpegaudioparse ! mad ! audioconvert ! queue ! goom ! videoconvert ! video/x-raw,width=1280,height=720 ! x264enc ! mux.
Wow! Works great. I have been trying to figure this out for ages! I have also been playing with some of the other visualisations(libvisual) but they seem to do weird things. I encoded a song that was 4:00 minutes long but the end result reported it was 6:07 with the last two minutes causing most media players to pause. odd.
@frente69
Odd indeed. Could it be a sync or frame rate issue? If nothing else, I guess there’s ffmpeg to crop it :/
First code(s) creating 0 byte file on Ubuntu 14.04 x64.
Code from comment gives the following error:
WARNING: erroneous pipeline: could not link filesink0 to queue2
Hi !
Great work ! I’m new in GStreamer. Is it possible to get a Text Overlay in this :
gst-launch filesrc location=input.mp3 ! queue ! tee name=stream ! queue ! mp3parse ! matroskamux name=mux ! filesink location=”output.mkv” stream. ! queue ! mp3parse ! mad ! audioconvert ! queue ! goom ! ffmpegcolorspace ! video/x-raw-yuv,width=1280,height=720 ! x264enc ! mux.
I tried by myself but it doesnt work. Where do i have to put the code in ?
Sup peeps. If you are on 18.04 running this with gstreamer1 you may get hella mad when getting the error about not having an element called “mad”
sudo gst-launch-1.0 filesrc location=Downloads/1/audio.mp3 ! queue ! tee name=stream ! queue ! mpegaudioparse ! matroskamux name=mux ! filesink location=output.mkv stream. ! queue ! mpegaudioparse ! mpg123audiodec ! audioconvert ! queue ! goom ! videoconvert ! video/x-raw,width=1280,height=720 ! x264enc ! mux.
^in case some nubs need clarification, I simply replaced mad with mpg123audiodec