The sessions are held over Google Hangouts on Air, and the videos are posted to YouTube. The material is based on content that was used for a Frontend Masters course.
Dmitri Voronianski sent in a very nicely designed React library for SoundCloud (GitHub: soundblogs/react-soundplayer, License: MIT, npm: react-soundplayer). It has a high-level component (SoundPlayerContainer), and there are lots of other components relating to audio playback. This includes buttons like PlayButton and NextButton, but also play position (Progress, Timer), and album art (Cover).
My screenshot is from the example near the bottom of the page, which is a full-blown audio player that uses most of the features of the library. It looks very nice, and is all done through HTML5, React, and the SoundCloud API.
The AnalyserNode interface represents a node able to provide real-time frequency and time-domain analysis information. It is an AudioNode that passes the audio stream unchanged from the input to the output, but allows you to take the generated data, process it, and create audio visualizations.
Because the AnalyserNode API does the more complicated FFT stuff, the source is actually fairly easy to understand. Take a look at Amelie.jsx to see how the colours are generated and then plotted using D3.js.
Enjoying Syro? I'll admit I was a little bit too obsessed by the cover art, which contains a list of Aphex Twin's expenses and a list of the audio hardware used on the album. So I thought it was pretty cool to see Spectroface by Daniel Rapp. This project uses the Web Audio API to recreate Aphex Twin's spectrogram face that was hidden in the Windowlicker b-side.
Daniel's website explains how spectrograms work, and the source code is heavily commented, so with a little bit of effort you should be able to follow it.
A spectrogram is a visual representation of the frequencies which make up a sound. Say you whistle a pure "middle C", then a spectrogram would light up right at 261.6 Hz, which is the corresponding frequency for that tone. Likewise, the "A" note makes the spectrogram turn bright white at 440 Hz.
If you hover over "middle C" and "A" on the original page (http://danielrapp.github.io/spectroface) it'll actually play the notes, which is a nice touch.
I tried out Daniel's examples and found they work best in Chrome with the webcam and mic active. You should play the sounds from the speaker rather than headphones to see the image encoding effect in the spectrogram visualisations.
The source is concise, and amazingly doesn't require too much hacking to get the audio values translated into pixels. For example, the code that determines the shade of grey to use for position x, y in the image looks like this:
The last example treats the video as an instrument, so you can wave things in front of the camera to produce different sounds. Although it sounds extremely loud and strange, it's very interesting and comes from a surprisingly small amount of code.
It starts acting like an instrument! Notice that if you place your finger over the webcam, the sound is muted. You can produce low-pass or high-pass filters by covering only part of the webcam.
Party Mode (GitHub: preziotte / party-mode, License: MIT) by Mathew Preziotte is a music visualiser with a slick UI and lots options. If you press m it'll show a menu for each visual effect, and there's also a keyboard icon near the bottom of the screen that documents each shortcut.
Using the web audio api, I can get an array of numbers which corresponds to the waveform of the sound an html5 audio element is producing. There's a good tutorial on how to do this. Then, using requestAnimationFrame (with a little frame limiting for performance reasons) I'm updating that array as the music changes. I then normalize the data a bit (or transform it slightly depending on the visualization) and redraw the screen based on the updated array. I'm using d3.js to draw and redraw SVG based on this normalized data. Each visualization uses the data a bit differently -- it was mostly trial and error to get some stuff I liked looking at.
Jack Frigaard sent in his Bézier Clock, which got lots of attention on Hacker News this week. It's made with Processing.js, which is loads of fun to play with, and you can click it to visualise the curve splines and poitns.
There are keyboard options as well:
space: Toggle continual animation
s: Show intermediate figures and the standard ones
a: Cycle through linear, quadratic, cubic and sinusoidal easing
Cristiano Belloni has relaunched KievII Host as Hya.io. This is a web-based plugin-based audio application that supports hardware MIDI though Web MIDI, audio synthesis, sequencers, and lots of plugins.
The project has a blog with announcements about updates, plugins, and some technical background. The web interface is closed source, but the plugins Cristiano has written are open source. For example, hy-osc is an oscillator, and kmx is a mixer.
Hya supports AMD and ES6 modules. They use a simple JSON object that describes the plugin, and a constructor function. You can bind to the Hya GUI using various events, like handleMouseOver and handleMouseDown. The post Developing a canvas plugin for Hya.io explains how to create plugins in detail.
The documentation has details on MIDI interaction and the initPlugin constructor. It's amazing to think that you could connect a MIDI control surface or keyboard to a web application.