Post pobrano z: Introduction to Web Audio API

Web Audio API let’s us make sound right in the browser. It makes your sites, apps, and games more fun and engaging. You can even build music-specific applications like drum machines and synthesizers. In this article, we’ll learn about working with the Web Audio API by building some fun and simple projects.

Getting Started

Let’s do some terminology. All audio operations in Web Audio API are handled inside an audio context. Each basic audio operation is performed with audio nodes that are chained together, forming an audio routing graph. Before playing any sound, you’ll need to create this audio context. It is very similar to how we would create a context to draw inside with the <canvas> element. Here’s how we create an audio context:

var context = new (window.AudioContext || window.webkitAudioContext)();

Safari requires a webkit prefix to support AudioContext, so you should use that line instead of new AudioContext();

Normally the Web Audio API workflow looks like this:

create source -> connect filter nodes -> connect to destination” />

There are three types of sources:

1. Oscillator – mathematically computed sounds
2. Audio Samples – from audio/video files
3. Audio Stream – audio from webcam or microphone

Let’s start with the oscillator

An oscillator is a repeating waveform. It has a frequency and peak amplitude. One of the most important features of the oscillator, aside from its frequency and amplitude, is the shape of its waveform. The four most commonly used oscillator waveforms are sine, triangle, square, and sawtooth.

It is also possible to create custom shapes. Different shapes are suitable for different synthesis techniques and they produce different sounds, from smooth to harsh.

The Web Audio API uses OscillatorNode to represent the repeating waveform. We can use all of the above shown waveform shapes. To do so, we have to assign the value property like so:

OscillatorNode.type = 'sine'|'square'|'triangle'|'sawtooth';

You can create a custom waveform as well. You use the setPeriodicWave() method to create the shape for the wave, that will automatically set the type to custom. Let’s listen how different waveforms produce different sounds:

See the Pen.

Custom waveforms are created using Fourier Transforms. If you want to learn more about custom waveform shapes (like how to make a police siren, for example) you can learn it from this good resource.

Running the oscillator

Let’s try to make some noise. Here’s what we need for that:

1. We have to create a Web Audio API context
2. Create the oscillator node inside that context
3. Choose waveform type
4. Set frequency
5. Connect oscillator to the destination
6. Start the oscillator

Let’s convert those steps into code.

var context = new (window.AudioContext || window.webkitAudioContext)();

var oscillator = context.createOscillator();

oscillator.type = 'sine';
oscillator.frequency.value = 440;
oscillator.connect(context.destination);
oscillator.start();

Note how we define the audio context. Safari requires the webkit prefix, so we make it cross-browser compatible.

Then we create the oscillator and set the type of the waveform. The default value for type is sine, so you can skip this line, I just like to add it to make it more clear and easy to update. We set the frequency value to 440, which is the A4 note (which is also the default value). The frequencies of musical notes C0 to B8 are in the range of 16.35 to 7902.13Hz. We will check out an example where we play a lot of different notes later in this article.

Now when we know all of that, let’s make the volume adjustable as well. For that we need to create the gain node inside of the context, connect it to the chain, and connect gain to the destination.

var gain = context.createGain();
oscillator.connect(gain);
gain.connect(context.destination);

var now = context.currentTime;
gain.gain.setValueAtTime(1, now);
gain.gain.exponentialRampToValueAtTime(0.001, now + 0.5);
oscillator.start(now);
oscillator.stop(now + 0.5);

Now you have some knowledge of working with the oscillator, here’s a good exercise. This Pen has the oscillator code setup. Try to make a simple app that changes the volume when you move the cursor up and down your screen, and changes the frequency when you move the cursor left and right.

Timing of Web Audio API

One of the most important things in building audio software is managing time. For the precision needed here, using the JavaScript clock is not the best practice, because it’s simply not precise enough. However the Web Audio API comes with the currentTime property, which is an increasing double hardware timestamp, which can be used for scheduling audio playback. It starts at 0 when the audio context is declared. Try running console.log(context.currentTime) to see the timestamp.

For example, if you want the Oscillator to play immediately you should run oscillator.start(0) (you can omit the 0, because it’s the default value). However you may want it to start in one second from now, play for two seconds, then stop. Here’s how to do that:

var now = context.currentTime;
oscillator.play(now + 1);
oscillator.stop(now + 3);

There are two methods to touch on here.

The AudioParam.setValueAtTime(value, startTime) method schedules change of the value at the precise time. For example, you want to change frequency value of the oscillator in one second:

oscillator.frequency.setValueAtTime(261.6, context.currentTime + 1);

However, you also use it when you want to instantly update the value, like .setValueAtTime(value, context.currentTime). You can set the value by modifying the value property of the AudioParam, but any updates to the value are ignored without throwing an exception if they happen at the same moment as the automation events (events scheduled using AudioParam methods).

The AudioParam.exponentialRampToValueAtTime(value, endTime) method schedules gradual change of the value. This code will exponentially decrease the volume of the oscillator in one second, which is a good way to stop sound smoothly:

gain.gain.exponentialRampToValueAtTime(0.001, context.currentTime + 1);

We can’t use 0 as the value because the value needs to be positive, so we use a very small value instead.

Creating the Sound class

Once you stop an oscillator, you cannot start it again. You didn’t do anything wrong, it’s the feature of the Web Audio API that optimizes the performance. What we can do is to create a sound class that will be responsible from creating oscillator nodes, and play and stop sounds. That way we’ll be able to call the sound multiple times. I’m going to use ES6 syntax for this one:

class Sound {

  constructor(context) {
    this.context = context;
  }

  init() {
    this.oscillator = this.context.createOscillator();
    this.gainNode = this.context.createGain();

    this.oscillator.connect(this.gainNode);
    this.gainNode.connect(this.context.destination);
    this.oscillator.type = 'sine';
  }

  play(value, time) {
    this.init();

    this.oscillator.frequency.value = value;
    this.gainNode.gain.setValueAtTime(1, this.context.currentTime);
            
    this.oscillator.start(time);
    this.stop(time);

  }

  stop(time) {
    this.gainNode.gain.exponentialRampToValueAtTime(0.001, time + 1);
    this.oscillator.stop(time + 1);
  }

}

We pass the context to the constructor, so we can create all of the instances of the Sound class within same context. Then we have the init method, that creates the oscillator and all of the necessary filter nodes, connects them, etc. The Play method accepts the value (the frequency in hertz of the note it’s going to play) and the time when it shall be played. But first, it creates the oscillator, and that happens every time we call the play method. The stop method exponentially decreases the volume in one second until it stops the oscillator completely. So whenever we need to play the sound again, we create a new instance of the sound class and call the play method. Now we can play some notes:

let context = new (window.AudioContext || window.webkitAudioContext)();
let note = new Sound(context);
let now = context.currentTime;
note.play(261.63, now);
note.play(293.66, now + 0.5);
note.play(329.63, now + 1);
note.play(349.23, now + 1.5);
note.play(392.00, now + 2);
note.play(440.00, now + 2.5);
note.play(493.88, now + 3);
note.play(523.25, now + 3.5);

That will play C D E F G A B C, all within the same context. If you want to know the frequencies of notes in hertz, you can find them here.

Knowing all of this makes us able to build something like a xylophone! It creates a new instance of Sound and plays it on mouseenter. You can check the example and try make one by yourself as an exercise.

See the Pen Play the Xylophone (Web Audio API) by Greg Hovanesyan (@gregh) on CodePen.

I’ve created a playground, containing all the required HTML and CSS, and the Sound class we’ve created. Use the data-frequency attribute to obtain the note values. Try here.

Working with a recorded sound

Now that you’ve built something with an oscillator, let’s now see how to work with a recorded sound. Some sounds are very hard to reproduce using the oscillator. In order to use realistic sounds in many cases, you’ll have to use recorded sounds. This can be `.mp3`, `.ogg`, `.wav`, etc. See the full list for more info. I like to use `.mp3` as it’s lightweight, widely supported, and has pretty good sound quality.

You can’t simply get sound by a URL like you do with images. We have to run an XMLHttpRequest to get the files, decode the data, and put into the buffer.

class Buffer {

  constructor(context, urls) {  
    this.context = context;
    this.urls = urls;
    this.buffer = [];
  }

  loadSound(url, index) {
    let request = new XMLHttpRequest();
    request.open('get', url, true);
    request.responseType = 'arraybuffer';
    let thisBuffer = this;
    request.onload = function() {
      thisBuffer.context.decodeAudioData(request.response, function(buffer) {
        thisBuffer.buffer[index] = buffer;
        updateProgress(thisBuffer.urls.length);
        if(index == thisBuffer.urls.length-1) {
          thisBuffer.loaded();
        }       
      });
    };
    request.send();
  };

  loadAll() {
    this.urls.forEach((url, index) => {
      this.loadSound(url, index);
    })
  }

  loaded() {
    // what happens when all the files are loaded
  }

  getSoundByIndex(index) {
    return this.buffer[index];
  }

}

Let’s take a look at the constructor. We receive our context there as we did in the Sound class, receive the list of URLa that will be loaded, and an empty array for the buffer.

The we have two methods: loadSound and loadAll. loadAll loops through the list of URLs and calls the loadSound method. It’s important to pass the index, so that we put the buffered sound into the correct element of the array, regardless of which request loads first. This also let’s us see which request is the last, which means that on its completion the buffer is loaded.

Then you can call the loaded() method, which can do something like hiding the loading indicator. And finally the getSoundByIndex(index) method gets the sound from the buffer by index for playback.

The decodeAudioData method has a newer Promise-based syntax, but it doesn’t work in Safari yet:

context.decodeAudioData(audioData).then(function(decodedData) {
  // use the decoded data here
});

Then we have to create the class for the sound. Now we have our complete class to work with the recorded sound:

class Sound() {

  constructor(context, buffer) {
    this.context = context;
    this.buffer = buffer;
  }

  init() {
    this.gainNode = this.context.createGain();
    this.source = this.context.createBufferSource();
    this.source.buffer = this.buffer;
    this.source.connect(this.gainNode);
    this.gainNode.connect(this.context.destination);
  }

  play() {
    this.setup();
    this.source.start(this.context.currentTime);
  }  

  stop() {
    this.gainNode.gain.exponentialRampToValueAtTime(0.001, this.context.currentTime + 0.5);
    this.source.stop(this.context.currentTime + 0.5);
  }

}

The constructor accepts the context and the buffer. We create by calling createBufferSource() method, instead of createOscillator as we did before. The buffer is the note (element from the buffer array) that we get using the getSoundByIndex() method. Now instead of the oscillator we create a buffer source, set the buffer, and then connect it to the destination (or gain and other filters).

let buffer = new Buffer(context, sounds);
buffer.loadAll();

sound = new Sound(context, buffer.getSoundByIndex(id));
sound.play();

Now we have to create an instance of buffer and call the loadAll method, to load all of the sounds into the buffer. We also have the getSoundById method to grab the exact sound we need, so we pass the sound to the Sound and call play(). The id can be stored as a data attribute on the button that you click to play the sound.

Here’s a project that uses all of that: the buffer, the recorded notes, etc:

See the Pen The Bluesman – You Can Play The Blues (Web Audio API) by Greg Hovanesyan (@gregh) on CodePen.

You can use that example for for reference, but for your own exercise, here’s a playground I’ve created. It has all the necessary HTML and CSS and the URLs to the notes that I have recorded on a real electric guitar. Try writing your own code!

Intro to Filters

The Web Audio API lets you add different filter nodes between your sound source and destination. BiquadFilterNode is a simple low-order filter which gives you control over what parts of the frequency parts shall be emphasized and which parts shall be attenuated. This lets you build equalizer apps and other effects. There are 8 types of biquad filters: highpass, lowpass, bandpass, lowshelf, highshelf, peaking, notch, and allpass.

Highpass is a filter that passes higher frequencies well, but attenuates lower frequency components of signals. Lowpass passes lower frequencies, but attenuates higher frequencies. They are also called „low cut” and „high cut” filters, because that explains what what happens to the signal.

Highshelf and Lowshelf are filters are used to control the bass and treble of the sound. They are used to emphasize or reduce signals above or below the given frequency.

You will find a Q property BiquadFilterNode interface, which is a double representing the Q Factor. Quality Factor or Q Factor control the bandwidth, the number of frequencies that are affected. The lower the Q factor, the wider the bandwidth, meaning the more frequencies will be affected. The higher the Q factor, that narrower the bandwidth.

You can find more info about filters here, but we can already build a parametric equalizer. It’s an equalizer that gives full control for adjusting the frequency, bandwidth and gain.

Let’s build a parametric equalizer.

See the Pen.

Let’s take a look on how we can apply distortion to the sound. If you wonder what makes an electric guitar sound like one, it is the distortion effect. We use the WaveShaperNode interface to represent a non-linear distorter. What we need to do is to create a curve that will shape the signal, distorting and producing the characteristic sound. We don’t have to spend a lot of time to create the curve, as it’s already done for us. We can adjust the amount of distortion as well:

See the Pen.

Afterword

Now that you’ve seen how to work with the Web Audio API, I recommend playing with it on your own and making your own projects!

Here are some libraries for working with web audio:

• Pizzicato.js – Pizzicato aims to simplify the way you create and manipulate sounds via the Web Audio API
• webaudiox.js – webaudiox.js is a bunch of helpers that will make working with the WebAudio API easier
• howler.js – Javascript audio library for the modern web
• WAD – Use the HTML5 Web Audio API for dynamic sound synthesis. It’s like jQuery for your ears
• Tone.js – A Web Audio framework for making interactive music in the browser

Introduction to Web Audio API is a post from CSS-Tricks

Post pobrano z: Weapons of Math Destruction

I think you’d do well to read Cathy O’Neils Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. I saw her read at the Miami Book Fair several months ago and immediately bought a copy. I even got her to sign it which is kinda cool 😉

Cathy’s big idea is that we’re absolutely surrounded by algorithms that inform big decision making. There are lots of good algorithms that help us. Sadly, there are lots of insidiously, dangerous, bad algorithms that do serious damage, and they are lurking all about disguised as good algorithms.

One aspect of a good algorithm is some kind of feedback and correctional system. Early on Cathy points to some advertising algorithms as an example of a healthy algorithm. For example, if an algorithm is in place to recommend a product you should buy, and it does a terrible job at that, it will be tweaked until fixed, thereby correcting what is has set out to do. Moneyball-style algorithms are the same. The data is open. Baseball team managers use algorithms to help recruit for their team and manage how they play. If it isn’t working, it will be tweaked until it does.

A bad algorithm might lack a feedback loop. One of her strongest examples is in the algorithms that rate teachers. There is plenty of evidence that these algorithms are often wrong, ousting teachers that definitely should not have been. And not in a „they tested badly, but have a heart of gold” way, in a „the algorithm was actually just wrong” way. What makes something like this a „weapon of math destruction” (WMD) then, is the fact that it affects a lot of people, screws up, and there is no correction mechanism. There are lots of interesting criteria, though. I’ll let you read more about it.

There is an awful lot of considerations and nuance here, and I think Cathy delivers pretty gracefully on all that. She has an impressive pedigree academically, professionally, and journalistically. There is some pitchfork raising here, but the prongs are made of research, data, and morals.

Weapons of Math Destruction is a post from CSS-Tricks

Post pobrano z: Just Another HTTPS Nudge

I was strongly reminded about the scariness of non-secure websites the other day.

I’m using Xfinity as an internet service provider, and they give you a device that is both a cable modem and a router.

Here’s a tiny bit of backstory. I use a VPN, and I discovered that in using their modem directly, the VPN wouldn’t work. I’m not sure why. I didn’t dig into it very far, because I have a modem of my own I’d prefer to use. So I plugged that in, which worked… but not particularly well. The connection was spotty and slow, even right in my own house.

I think (maybe?) it was competing WiFi signals from the two routers sitting right next to each other. Don’t quote me on that. The reason I think that is because, fortunately, I was able to turn off the router on the Xfinity device, and that solved the problem. Thde speed and connectivity was back. To their credit, it was really fast. The Xfinity device has a featured called „Bridge Mode” that is specifically for turning off the router so that you can use your own. I was able to enable that, use my own router, get the speed back, and connect to the VPN.

Win! That lasted for a few months. Then recently there was some weird big internet outage in our area. Xfinity notified us about it. They had to push some updates or something to our device, and that broke everything again. I struggled with it for days, but what ultimately worked was turning off Bridge Mode, and turning it back on again (isn’t it always?).

In those in-between days, the only thing I could figure out to get online was to connect to the SSID „xfinitywifi” that this router seemed to be emitting. This „xfinity” network is unusual because it behaves kinda like a coffee shop or university hotspot in that it pops up that weird browser modal and you have to log in with your (Xfinity) credentials. It’s a value-add kinda thing for their service. Their routers are dotted all over the place, so if you’re a customer of theirs, you get internet („for free”) a lot of places. My fiance was at the doctor the other day, and she was using it there.

If that’s the network you’re connected to, Xfinity performs man-in-the-middle attacks on websites to send you messages. Here’s an example of me just looking at a (non-secure) website:

Man-in-the-middle, meaning, this website had no such popup in its code. Xfinity intercepted the request, saw it was a website, and forcefully injected its own code into the site. In this case, to advertise an app and to tell you about security. Ooozing with irony, that.

If they can do that, imagine what else they can do. (Highly recommended listening: ShopTalk #250) They could get even more forceful with advertising. Swap out existing advertising with their own. Install a keylogger. Report back information about what you’re doing and where you are. You might not even know if anything is happening at all.

This might seem a little tin foil hatish, but realize: they’ve already been incentivized to do this. All the incentive is there to keep milking value out of this superpower they have.

Some good news: Individual websites can stop this with HTTPS. That’s a massively good step. With HTTPS, the traffic packets are encrypted and Xfinity can’t read or manipulate them effectively. Through metadata, they might be able to guess what they are (e.g. know you’re streaming a video and throttle speed), but there isn’t much else they can do.

It’s not just this one indiscretion, Xfinity also uses this tactic to send you other messages.

@chriscoyier @XFINITY also how they warn you about bandwidth or billing issues. not fun.

— David Bisset (@dimensionmedia) February 24, 2017

@chriscoyier @XFINITY I have seen an ISP adding ads to bing home page. 😕

— AKT (@itsakt) February 25, 2017

It’s this double whammy of scary:

• Seriously?! You require me to have a box in my house that broadcasts a public WiFi hotspot that I can’t turn off? You’re automatically opted into it, but you can turn it off.
• Seriously?! You use that hotspot to perform man-in-the-middle attacks on anybody using it?

I’m sure it’s not just Xfinity, it’s just that’s what I’m using now and have now seen it with my own eyes. To be clear, I’m sure I signed something that allows them to do everything they are doing and I don’t think anything they are doing is technically illegal (again, don’t quote me on that).

Being upset at them, and telling them about it, is a good step. Fighting back is another. Internet access is vital, so you have to use something, but if you have an option, is there an ISP that doesn’t do this available to you? Use them. Money talks.

Again, HTTPS solves this on a per-website basis. Jeff Atwood sums this up pretty well:

1. You have an unalienable right to privacy, both in the real world and online. And without HTTPS you have zero online privacy – from anyone else on your WiFi, from your network provider, from website operators, from large companies, from the government.

2. The performance penalty of HTTPS is gone, in fact, HTTPS arguably performs better than HTTP on modern devices.

3. Using HTTPS means nobody can tamper with the content in your web browser. This was a bit of an abstract concern five years ago, but these days, there are more and more instances of upstream providers actively mucking with the data that passes through their pipes. For example, if Comcast detects you have a copyright strike, they’ll insert banners into your web content … all your web content! And that’s what the good guy scenario looks like – or at least a corporation trying to follow the rules. Imagine what it looks like when someone, or some large company, decides the rules don’t apply to them?

The move to HTTPS is non-trivial, and introduces somewhat complicated dependencies. It’s easy to forget to renew your certificate and break your entire website just like that. I’m not arguing against HTTPS (exactly the opposite), but you should know that it requires some upfront work and some diligent maintenance.

If you’re on WordPress like me, I wrote up how I moved to all-HTTPS going on two years ago. It involved a little database work even, getting URL’s pointing to the right places.

SSL certificates (the main prerequisite for HTTPS) also have traditionally cost money. No more! Let’s Encrypt is here:

Lets Encrypt is a free, automated, and open Certificate Authority.

There is an in-progress WordPress plugin for it. Let’s hope that gets off the ground. Just a few days ago I used the Let’s Encrypt Plesk extention to put HTTPS on ShopTalk’s website and it took me like 5 minutes. I’ll have to write that up soon.

Also check out the really excellent Moving To HTTPS Guide:

A community site to help site owners migrate to HTTPS with a simple tested process. Allowing you to filter the plan based on multiple platforms (WordPress, Magento, and more), hosting environments (cPanel, Apache, and more) along with the level of control / access you have over the site.

Just Another HTTPS Nudge is a post from CSS-Tricks

Post pobrano z: Majorette Events

Post pobrano z: Veintidos Grados

Post pobrano z: Ditalia

Post pobrano z: Das Bevo