Jonathan Snook on the complexity of animating the <progress> element. If you’re unfamiliar, that’s the element that spits out a bar chart-like visual that indicates a position between two values:
This example has custom styles, but you get the point.
Jonathan’s post shows off a method for animating a change in progress value using CSS and a touch of JavaScript while making sure that it animates properly in every modern browser. The demo he made looks pretty neat. I’m sure this is going to be one of those posts I come back to again and again.
Or the priority navigation pattern, or progressively collapsing navigation menu. We can name it in at least three ways.
There are multiple UX solutions for tabs and menus and each of them have their own advantages over another, you just need to pick the best for the case you are trying to solve. At design and development agency Kollegorna we were debating on the most appropriate UX technique for tabs for our client’s website…
We agreed it should be a one-liner because the amount of tab items is unknown and narrowed our options down to two: horizontal scroll and adaptive with “more” button. Firstly, the problem with the former one is that horizontal scroll as a feature is not always visually obvious for users (especially for narrow elements like tabs) whereas what else can be more obvious than a button (“more”), right? Secondly, scrolling horizontally using a mouse-controlled device isn’t a very comfortable thing to do, so we might need to make our UI more complex with additional arrow buttons. All considered, we ended up choosing the later option:
Planning
The main intrigue here is if it’s possible to achieve that without JavaScript?Partly yes, however the limitations that it comes with probably make it only good for a concept museum rather than real life scenarios (anyway, Kenan did a really nice job). Still, the dependency on JS doesn’t mean we can’t make it usable if for some reason the technology is not available. Progressive enhancement and graceful degradation for the win!
Since the amount of tab items is uncertain or volatile, we will make use of Flexbox which ensures the items are nicely spread in the container element without setting the widths.
Initial Prototype
There are two lists both visually and technically: one is for items that fit in the container, and one for items that don’t. Since we’ll depend on JavaScript, it’s totally fine to have our initial markup with a single list only (we will duplicate it with JS):
With a tiny touch of flex-based CSS things are starting to get serious here. I’ll skip the decorative CSS properties in my examples here and place here what really matters:
Now before enhancing it progressively with JavaScript, let’s make sure it degrades gracefully if there is no JS available. There multiple of reasons for JS absence: it’s still loading, it has fatal errors, it failed to be transferred over the network.
And once JavaScript is here, the --jsfied class name is added to the container element which neutralizes the CSS above:
container.classList.add('--jsfied')
Turns out the horizontal scroll strategy that I mentioned before might make a fine use here! When there’s not enough room for menu items the overflowing content gets clipped inside the container and scrollbars are displayed. That’s way better than empty space or something that’s broken, isn’t it?
Missing Parts
First off, let’s insert the missing DOM parts:
Secondary (dropdown) list which is a copy of the main list;
Here we are nesting secondary list into primary and using some aria-* properties. We want our navigation menu to be accessible, right?
There’s also an event handler attached to the „more” button that toggles the --show-secondary class name on the container element. We’ll use it to show and hide the secondary list. Now let’s style the new parts. You may want to visually accent „more” button.
Obviously, we need some code that hides and shows the tabs…
Hiding and Showing Tabs in the Lists
Because of Flexbox, the tab items will never break into multiple lines and will shrink to their minimum possible widths. This means we can walk through the each item one by one, add up their widths, compare it to the width of .tabs element and toggle the visibility of particular tabs accordingly. For that we will create a function called doAdapt and wrap in the code below in this section.
To begin width, we should visually reveal all the items:
On a side note, .--hidden works the way you’ve probably expected:
.tabs .--hidden {
display: none;
}
Math time! I’ll have to disappoint you if you expected some advanced mathematics. So, as described previously, we walk through the each primary tab by adding up their widths under stopWidth variable. We also perform a check if the item fits in the container, hide the item if not and save its index for later use.
Hereafter, we need to hide the equivalent items from the secondary list that remained visible in the primary one. As well as hide “more” button if no tabs were hidden.
I could probably end my article here, but there is an extra mile we can walk to make it better and some things to note…
Enhancements
It has been implemented in the demo above, but we haven’t overviewed a small detail that improves the UX of our tabs widget. It’s hiding the dropdown list automatically if user clicks anywhere outside the list. For that we can bind a global click listener and check if the clicked element or any of its parents is the secondarylist or „more” button. If not, the dropdown list gets dismissed.
document.addEventListener('click', (e) => {
let el = e.target
while(el) {
if(el === secondary || el === moreBtn) {
return;
}
el = el.parentNode
}
container.classList.remove('--show-secondary')
moreBtn.setAttribute('aria-expanded', false)
})
Edge Cases
Long Tab Titles
You might have been wondering how the widget behaves with long tab titles. Well, you have at least two options here…
Let titles wrap to the next line which is how they behave by default (you can also enable word wrapping with word-wrap: break-word):
Or you can disable all kinds of wrapping in the primary list with white-space: nowrap. The script is flexible enough to put the too-long items to the dropdown (where the titles are free to wrap) by stepping aside the shorter siblings:
Many Tabs
Even though the secondary list is position: absolute it doesn’t matter how long your document’s height is. As long as the container element or its parents are not position: fixed, the document will adapt and the bottom items will be reachable by scrolling down the page.
A Thing to be Aware Of
Things may become tricky if the tabs are buttons rather than anchors semantically, which means their response to clicks are decided by JavaScript, e.g.: dynamic tabs. The problem here is that tab button event handlers aren’t duplicated along with the markup. I see at least two approaches to solve that:
Place dynamic event handler attachments right after the adaptive tab code;
Use an event delegation method instead (think of jQuery’s live()).
Unfortunately, events occur in quantity: most likely your tabs will have a selected state that visually indicates the current tab so it’s also important to manage the states simultaneously. Otherwise, flip the tablet and you’re lost.
Browser Compatibility
Even though I used ES6 syntax in the examples and demo, it should be converted to ES5 by a compiler such as Babel to significantly widen the browser support (down to IE9 including).
You can also expand the Flexbox implementation with an older version and syntax (all the way down to IE10). If you need to also support non-Flexbox browsers you can always do feature detection with CSS @supports, apply the technique progressively, and rely on horizontal scroll for older browsers.
Roel Nieskens released a tool that lets you upload a font file and see what’s inside, from how many characters it contains to the number of languages it supports. Here’s what you see once you upload a font, in this case Covik Sans Mono Black:
Why is this data useful? Well, I used this tool just the other day when I found a font file in a random Dropbox folder. What OpenType features does this font have? Are there any extra glyphs besides the Roman alphabet inside? Wakamai Fondue answered those questions for me in a jiffy.
CSS is getting increasingly powerful, and with features like CSS grid and custom properties (also known as CSS variables), we’re seeing some really creative solutions emerging. Some of those solutions focus around not only making the web prettier, but making it more accessible, and making styling accessible experiences better. I’m definitely here for it!
A common UI pattern that we see on the web are dropdown menus. They’re used to display related information in pieces, without overwhelming the user with buttons, text, and options. Somewhere that we see these a lot is inside of headers or navigation areas on websites.
A Google search for „dropdown menu” yields many examples
Let’s see if we can make one of these menus with CSS alone. We’ll create a list of links within a nav component like so:
We now have our two-tiered navigation system. In order to have the content hidden and displayed when we want it to be visible, we’ll need to apply some CSS. All style properties have been removed from the following example for clarity on interaction:
li {
display: block;
transition-duration: 0.5s;
}
li:hover {
cursor: pointer;
}
ul li ul {
visibility: hidden;
opacity: 0;
position: absolute;
transition: all 0.5s ease;
margin-top: 1rem;
left: 0;
display: none;
}
ul li:hover > ul,
ul li ul:hover {
visibility: visible;
opacity: 1;
display: block;
}
ul li ul li {
clear: both;
width: 100%;
}
Now, the submenu dropdown is hidden, but will be exposed and become visible when we hover over its correlating parent in the navigation bar. By styling ul li ul, we have access to that submenu, and by styling ul li ul li, we have access to the individual list items within it.
The Problem
This is starting to look like what we want, but we’re still far from finished at this point. Web accessibility is a core part of your product’s development, and right now would be the perfect opportunity to bring this up. Adding role="navigation" is a good start, but in order for a navigation bar to be accessible, one should be able to tab through it (and focus on the proper item in a sensible order), and also have a screen reader accurately read out loud what is being focused on.
You can hover over any of the list items and clearly see what is being hovered over, but this isn’t true for tab navigation. Go ahead and try to tab through the example above. You lose track of where the focus is visually As you tab to Two in the main menu, you’ll see a focus indicator ring, but when you tab to the next item (one of its submenu items), that focus disappears.
Now, it’s important to note that theoretically you are focused on this other item, and that a screen reader would be able to parse that, reading Sub-One, but keyboard users will not be able to see what’s going on and will lose track.
The reason this happens is because, while we’re styling the hover of the parent element, as soon as we transition focus from the parent to one of the list items within that parent, we lose that styling. This makes sense from a CSS standpoint, but it’s not what we want.
Luckily, there is a new CSS pseudo class that will give us exactly what we want in this case, and it’s called :focus-within.
The Solution: `:focus-within`
The :focus-within pseudo selector is a part of the CSS Selectors Level 4 Spec and tells the browser to apply a style to a parent when any of its children are in focus. So in our case, this means that we can tab to Sub-One and apply a :focus-within style along with the :hover style of the parent and see exactly where we are in the navigation dropdown. In our case it would be ul li:focus-within > ul:
ul li:hover > ul,
ul li:focus-within > ul,
ul li ul:hover {
visibility: visible;
opacity: 1;
display: block;
}
Now, when we tab to the second item, our submenu pops up, and as we tab through the submenu, the visibility remains! Now, we can append our code to include :focus states alongside :hover to give keyboard users the same experience as our mouse users.
In most cases, such as on direct links, we usually can just write something like:
a:hover,
a:focus {
...
}
But in this case, since we’re applying hover styles based on the parent li, we can again utilize :focus-within to get the same look at feel when tabbing through. This is because we can’t actually focus on the li (unless we add a tabindex="0"). We’re actually focusing on the link (a) within it. :focus-within allows us to still apply styles to the parent li when focusing on the link (pretty darn cool!):
li:hover,
li:focus-within {
...
}
At this point, since we are applying a focus style, we can do something that’s typically not recommended (remove the styling of that blue outline focus ring). We can do this by:
li:focus-within a {
outline: none;
}
The above code specifies that when we focus within list items via the link (a), do not apply an outline to the link item (a). It’s pretty safe to write it this way, because we’re exclusively styling the hover state, and with browsers that do not support :focus-within, the link will still get a focus ring. Now our menu looks like this:
Final menu using :focus-within, :hover states, and customizing the focus ring to disappear
If you’re familiar with accessibility, you may have heard of ARIA labels and states. You can use these to your advantage to also create these types of dropdowns with built-in accessibility at the same time! You can find an excellent example here by Heydon Pickering. When including ARIA markup, your code would look a little more like this:
You’re adding aria-haspopup="true" to the parent of the dropdown menu to indicate an alternative state, and including aria-label="submenu" on the actual dropdown menu itself (in this case our list with class="dropdown".
These properties themselves will give you the functionality you need to show the dropdown menu, but the downside is that they only work with JavaScript enabled.
Browser Support Caveat
Speaking of caveats, let’s talk about browser support. While :focus-within does have pretty good browser support, it’s important to note that Internet Explorer and Edge are not supported, so your users on those platforms will not be able to see the navigation.
This browser support data is from Caniuse, which has more detail. A number indicates that browser supports the feature at that version and up.
Desktop
Chrome
Opera
Firefox
IE
Edge
Safari
60
47
52
No
No
10.1
Mobile / Tablet
iOS Safari
Opera Mobile
Opera Mini
Android
Android Chrome
Android Firefox
10.3
No
No
No
64
57
The ultimate solution here would be to use both ARIA markup and CSS :focus-within to ensure a solid dropdown experience for your users.
If you want to be able to use this feature in the future, please upvote it on Edge User Voice! And upvote :focus-ring while you’re at it, so that we’ll be able to style that focus ring and create a beautiful interactive web experience for all 😀
More on `:focus-within` and A11Y
Scott O’Hara wrote about :focus-within, highlighting demos like highlighted <table> rows and dropdown menus
React components can, and often do, have state. State can be anything, but think of things like whether a user is logged in or not and displaying the correct username based on which account is active. Or an array of blog posts. Or if a modal is open or not and which tab within it is active.
React components with state render UI based on that state. When the state of components changes, so does the component UI.
That makes understanding when and how to change the state of your component important. At the end of this tutorial, you should know how setState works, and be able to avoid common pitfalls that many of us hit when when learning React.
Workings of `setState()`
setState() is the only legitimate way to update state after the initial state setup. Let’s say we have a search component and want to display the search term a user submits.
Here’s the setup:
import React, { Component } from 'react'
class Search extends Component {
constructor(props) {
super(props)
state = {
searchTerm: ''
}
}
}
We’re passing an empty string as a value and, to update the state of searchTerm, we have to call setState().
setState({ searchTerm: event.target.value })
Here, we’re passing an object to setState(). The object contains the part of the state we want to update which, in this case, is the value of searchTerm. React takes this value and merges it into the object that needs it. It’s sort of like the Search component asks what it should use for the value of searchTerm and setState() responds with an answer.
This is basically kicking off a process that React calls reconciliation. The reconciliation process is the way React updates the DOM, by making changes to the component based on the change in state. When the request to setState() is triggered, React creates a new tree containing the reactive elements in the component (along with the updated state). This tree is used to figure out how the Search component’s UI should change in response to the state change by comparing it with the elements of the previous tree. React knows which changes to implement and will only update the parts of the DOM where necessary. This is why React is fast.
That sounds like a lot, but to sum up the flow:
We have a search component that displays a search term
That search term is currently empty
The user submits a search term
That term is captured and stored by setState as a value
Reconciliation takes place and React notices the change in value
React instructs the search component to update the value and the search term is merged in
The reconciliation process does not necessarily change the entire tree, except in a situation where the root of the tree is changed like this:
// old
<div>
<Search />
</div>
// new
<span>
<Search />
</span>
All <div> tags become <span> tags and the whole component tree will be updated as a result.
The rule of thumb is to never mutate state directly. Always use setState() to change state. Modifying state directly, like the snippet below will not cause the component to re-render.
// do not do this
this.state = {
searchTerm: event.target.value
}
Passing a Function to `setState()`
To demonstrate this idea further, let’s create a simple counter that increments and decrements on click.
At this point, the counter simply increments or decrements the count by 1 on each click.
But what if we wanted to increment or decrement by 3 instead? We could try to call setState() three times in the handleDecrement and handleIncrement functions like this:
Object.assign() is used to copy data from a source object to a target object. If the data being copied from the source to the target all have same keys, like in our example, the last object wins. Here’s a simpler version of how Object.assign() works;
So instead of the call happening three times, it happens just once. This can be fixed by passing a function to setState(). Just as you pass objects to setState(), you can also pass functions, and that is the way out of the situation above.
If we edit the handleIncrement function to look like this:
…we can now increment count three times with one click.
In this case, instead of merging, React queues the function calls in the order they are made and updates the entire state ones it is done. This updates the state of count to 3 instead of 1.
Access Previous State Using Updater
When building React applications, there are times when you’ll want to calculate state based the component’s previous state. You cannot always trust this.state to hold the correct state immediately after calling setState(), as it is always equal to the state rendered on the screen.
Let’s go back to our counter example to see how this works. Let’s say we have a function that decrements our count by 1. This function looks like this:
Each time the button to decrement is clicked, the count will decrement by 1 instead of 3. This is because the this.state.count does not get updated until the component has been re-rendered. The solution is to use an updater. An updater allows you access the current state and put it to use immediately to update other items. So the changeCount() function will look like this.
Now we are not depending on the result of this.state. The states of count are built on each other so we are able to access the correct state which changes with each call to changeCount().
setState() should be treated asynchronously — in other words, do not always expect that the state has changed after calling setState().
Wrapping Up
When working with setState(), these are the major things you should know:
Update to a component state should be done using setState()
You can pass an object or a function to setState()
Pass a function when you can to update state multiple times
Do not depend on this.state immediately after calling setState() and make use of the updater function instead.
Una Kravets shows how to make layouts in CSS Grid with flexbox fallbacks for browsers that don’t support those grid properties just yet. Una writes:
CSS grid is AMAZING! However, if you need to support users of IE11 and below, or Edge 15 and below, grid won’t really work as you expect…This site is a solution for you so you can start to progressively enhance without fear!
The site is a provides examples using common layouts and component patterns, including code snippets. For example:
JAMstack sites are often seen as being static. A more accurate mental model for them would be that they are sites which have the ability to be hosted statically. The difference might seem semantic, but thanks to the rise of many tools and services which simplify running a build and deploying to static hosting infrastructure, such sites can feel much fresher and dynamic than you might imagine, while still capable of being served from static hosting infrastructure, with all the benefits that brings.
A feature often used as an example of why a site cannot be hosted statically is comments. A comments engine needs to handle submissions, allow for moderation, and is by its very nature, „dynamic”.
Comment systems are generally thought of as quite dynamic content
Thanks to the growing ecosystem of tools available for JAMstack sites, there are solutions to this. Let’s look at an example which you could use on your own site, which:
Sends notifications when new comments need moderating
Bakes the comments into your site, so that they load quickly and appear in searches
This example makes use of some of the features of Netlify, a platform for automating, deploying and hosting web projects, but many of the principles could be used with other platforms.
We’ll create 2 forms to receive all of our comments at the different stages of their journey from commenter to content. When Netlify sees a <form>, it creates a unique data store for the data the form collects. We’ll make great use of that.
Form 1) A queue to hold all of the new comment submissions. In other words, a store to hold all comments awaiting moderation.
Form 2) Contains all approved comments.
The act of moderation will be somebody looking at each new submission and deciding, „yep!” or „nope!”. Those that get nope-d will be deleted from the queue. Those that are approved will be posted over to the approved comments form.
All of the comments in the approved comments form are used by our static site generator in subsequent site builds thanks to the API access Netlify gives to the submissions in our forms.
The comment form
Each page includes an HTML <form>. By adding the boolean attribute of netlify to any HTML form element in your site, Netlify will automatically generate an API for your form, and gathers all of the submissions to it for you. You’ll also be able to access the submissions via that API later. Handy!
The comments <form> on each page will look a lot like this (some classes and extra copy omitted for clarity):
You’ll may notice that the form also includes a type="hidden" field to let us know which page on the site this comment is for. Our static site generator populates that for us when the site is generated, and well use it later when deciding which comments should be shown on which page.
Submissions and notifications
When a new comment is posted via the comment form, Netlify not only stashes that for us, but can also send a notification. That could be:
an email
a Slack notification
a webhook of our choosing.
These give us the chance to automate things a little.
New submissions result in a notification being posted to Slack. We’ll get to see what was submitted and to which page right there in our Slack client.
To make things extra slick, we can massage that notification a little to include some action buttons. One button to delete the comment, one to approve it. Approving a new comment from a Slack notification on your phone while riding the bus feels good.
We can’t make those buttons work without running a little bit of logic which, we can do in a Lambda function. Netlify recently added support for Lambda functions too, making the process of writing and deploying Lambdas part of the same deployment process. You’ll not need to go rummaging around in Amazon’s AWS configuration settings.
With a freshly approved comment being posted to our approved comments form, we are back to using the submission event triggers that Netlify provides. Every time something is posted to the approved comments form, we’ll want to include it in the site, so we have Netlify automatically rebuild and deploy our site.
Each time we run our site build, whether in our local development environment or within Netlify through their automated builds, the first step is to pull all of our external data into local data files which our SSG can then use. We do that with a Gulp task.
Armed with a `comments.json` file which we have populated from a call to Netlify’s form submission API which grabbed all of our approved comments, our SSG can now build the site as normal and bake the correct comments right into the HTML of our pages.
Some benefits
Well that was fun, but why bother?
Yes, you could use something like Disqus to add comments to a statically hosted site via a few lines of JavaScript. But that adds an external JavaScript dependency and results in the comments about your content living far away from the content itself. And it’s only available once the JavaScript has loaded, pulled in the data and then injected it into your site.
Whereas this method results in comments that are baked right into the content of your site. They’ll show up in searches on Google and they will load as part of your site without the need for any client-side JavaScript.
Even better, they’ll be available for you to commit right into your code repository with the rest of your content, giving you more peace of mind and portability in the future.
The example site and all of its code are available to explore. You can try submitting comments if you like (although poor old Phil will need to moderate any comments on this example site before they appear, but that will just make him feel loved).
Better still, you can clone this example and deploy your own version to Netlify with just a few clicks. The example site explains how.
Just show me behind the scenes right now!
If you’d want to take a look at how things behave for the moderator of a site using this system without grabbing a copy of your own, this short video will walk through a comment being made, moderated and incorporated into the site.
This is an extract from chapter 11 of Ashley Davis’s book Data Wrangling with JavaScript now available on the Manning Early Access Program. I absolutely love this idea as there is so much data visualization stuff on the web that relies on fully functioning client side JavaScript and potentially more API calls. It’s not nearly as robust, accessible, or syndicatable as it could be. If you bring that data visualization back to the server, you can bring progressive enhancement to the party. All example code and data can be found on GitHub.
When doing exploratory coding or data analysis in Node.js it is very useful to be able to render a visualization from our data. If we were working in browser-based JavaScript we could choose any one of the many charting, graphics, and visualization libraries. Unfortunately, under Node.js, we don’t have any viable options, so how otherwise can we achieve this?
We could try something like faking the DOM under Node.js, but I found a better way. We can make our browser-based visualization libraries work for us under Node.js using a headless browser. This is a browser that has no user interface. You can think of it as a browser that is invisible.
I use Nightmare under Node.js to capture visualizations to PNG and PDF files and it works really well!
The headless browser
When we think of a web-browser we usually think of the graphical software that we interact with on a day to day basis when browsing the web. Normally we interact with such a browser directly, viewing it with our eyes and controlling it with our mouse and keyboard as shown in Figure 1.
Figure 1: The normal state of affairs: our visualization renders in a browser and the user interacts directly with the browser
A headless browser on the other hand is a web-browser that has no graphical user interface and no direct means for us to control it. You might ask what is the use of a browser that we can’t directly see or interact with.
Well, as developers we would typically use a headless browser for automating and testing web sites. Let’s say that you have created a web page and you want to run a suite of automated tests against it to prove that it works as expected. The test suite is automated, which means it is controlled from code and this means that we need to drive the browser from code.
We use a headless browser for automated testing because we don’t need to directly see or interact with the web page that is being tested. Viewing such an automated test in progress is unnecessary, all we need to know is if the test passed or failed — and if it failed we would like to know why. Indeed, having a GUI for the browser under test would actually be a hindrance for a continuous-integration or continuous-deployment server, where many such tests can run in parallel.
So headless browsers are often used for automated testing of our web pages, but they are also incredibly useful for capturing browser-based visualizations and outputting them to PNG images or PDF files. To make this work we need a web server and a visualization, we must then write code to instance a headless browser and point it at our web server. Our code then instructs the headless browser to take a screenshot of the web page and save it to our file system as a PNG or PDF file.
Figure 2: We can use a headless browser under Node.js to capture our visualization to a static image file
Nightmare is my headless browser of choice. It is a Node.js library (installed via npm) that is built on Electron. Electron is a framework normally used for building cross-platform desktop apps that are based on web-technologies.
Why Nightmare?
It’s called Nightmare, but it’s definitely not a Nightmare to use. In fact, it’s the simplest and most convenient headless browser that I’ve used. It automatically includes Electron, so to get started we simply install Nightmare into our Node.js project as follows:
npm install --save nightmare
That’s all we need to install Nightmare and we can start using it immediately from JavaScript!
Nightmare comes with almost everything we need: A scripting library with an embedded headless browser. It also includes the communication mechanism to control the headless browser from Node.js. For the most part it’s seamless and well-integrated to Node.js.
Electron is built on Node.js and Chromium and maintained by GitHub and is the basis for a number of popular desktop applications.
Here are the reasons that I choose to use Nightmare over any other headless browser:
Electron is very stable.
Electron has good performance.
The API is simple and easy to learn.
There is no complicated configuration (just start using it).
It is very well integrated with Node.js.
Nightmare and Electron
When you install Nightmare via npm it automatically comes with an embedded version of Electron. So, we can say that Nightmare is not just a library for controlling a headless browser, it effectively is the headless browser. This is another reason I like Nightmare. With some of the other headless browsers, the control library is separate, or it’s worse than that and they don’t have a Node.js control library at all. In the worst case, you have to roll your own communication mechanism to control the headless browser.
Nightmare creates an instance of the Electron process using the Node.js child_process module. It then uses inter-process communication and a custom protocol to control the Electron instance. The relationship is shown in Figure 3.
Figure 3: Nightmare allows us to control Electron running as a headless browser
Our process: Capturing visualizations with Nightmare
So what is the process of capturing a visualization to an image file? This is what we are aiming at:
Acquire data.
Start a local web server to host our visualization
Inject our data into the web server
Instance a headless browser and point it at our local web server
Wait for the visualization to be displayed
Capture a screenshot of the visualization to an image file
Shutdown the headless browser
Shutdown the local web server
Prepare a visualization to render
The first thing we need is to have a visualization. Figure 4 shows the chart we’ll work with. This a chart of New York City yearly average temperature for the past 200 years.
Figure 4: Average yearly temperature in New York City for the past 200 years
To run this code you need Node.js installed. For this first example we’ll also use live-server (any web server will do) to test the visualization (because we haven’t created our Node.js web server yet), install live server as follows:
npm install -g live-server
Then you can clone the example code repo for this blog post:
Now go into the repo, install dependencies and run the example using live-server
cd nodejs-visualization-example/basic-visualization
bower install
live-server
When you run live-server your browser should automatically open and you should see the chart from Figure 4.
It’s a good idea to check that your visualization works directly in a browser before you try and capture it in a headless browser; there could easily be something wrong with it and problems are much easier to troubleshoot in a real browser than in the headless browser. live-server has live reload built-in, so now you have a nice little setup here when you can edit and improve the chart interactively before you try to capture it under Node.js.
This simple line chart was constructed with C3. Please take a look over the example code and maybe look at some of the examples in the C3 gallery to learn more about C3.
Starting the web server
To host our visualization, we need a web server. It’s not quite enough that we have a web server, we also need to be able to dynamically start and stop it. Listing 1 shows the code for our web server.
Listing 1 – Code for a simple web server that can be started and stopped
const express = require('express');
const path = require('path');
module.exports = {
start: () => { // Export a start function so we can start the web server on demand.
return new Promise((resolve, reject) => {
const app = express();
const staticFilesPath = path.join(__dirname, "public"); // Make our 'public' sub-directory accessible via HTTP.
const staticFilesMiddleWare = express.static(staticFilesPath);
app.use('/', staticFilesMiddleWare);
const server = app.listen(3000, err => { // Start the web server!
if (err) {
reject(err); // Error occurred while starting web server.
}
else {
resolve(server); // Web server started ok.
}
});
});
}
}
The code module in listing 1 exports a start function that we can call to kickstart our web server. This technique, being able to start and stop our web server, is also very useful for doing automated integration testing on a web site. Imagine that you want to start your web server, run some tests against it and then stop it at the end.
So now we have our browser-based visualization and we have a web server that can be started and stopped on demand. These are the raw ingredients we need for capturing server-side visualizations. Let’s mix it up with Nightmare!
Rendering the web page to an image
Now let’s flesh out the code to capture a screenshot of the visualization with Nightmare. Listing 2 shows the code that instances Nightmare, points it at our web server and then takes the screenshot.
Listing 2 – Capturing our chart to an image file using Nightmare
const webServer = require('./web-server.js');
const Nightmare = require('nightmare');
webServer.start() // Start the web server.
.then(server => {
const outputImagePath = "./output/nyc-temperatures.png";
const nightmare = new Nightmare(); // Create the Nightmare instance.
return nightmare.goto("http://localhost:3000") // Point the browser at the web server we just started.
.wait("svg") // Wait until the chart appears on screen.
.screenshot(outputImagePath) // Capture a screenshot to an image file.
.end() // End the Nightmare session. Any queued operations are completed and the headless browser is terminated.
.then(() => server.close()); // Stop the web server when we are done.
})
.then(() => {
console.log("All done :)");
})
.catch(err => {
console.error("Something went wrong :(");
console.error(err);
});
Note the use of the goto function, this is what actually directs the browser to load our visualization.
Web pages usually take some time to load. That’s probably not going to be very long, especially as we are running a local web server, but still we face the danger of taking a screenshot of the headless browser before or during its initial paint. That’s why we must call the wait function to wait until the chart’s <svg> element appears in the browser’s DOM before we call the screenshot function.
Eventually, the end function is called. Up until now we have effectively built a list of commands to send to the headless browser. The end function actually sends the commands to the browser, which takes the screenshot and outputs the file nyc-temperatures.png. After the image file has been captured we finish up by shutting down the web server.
You can find the completed code under the capture-visualization sub-directory in the repo. Go into the sub-directory and install dependencies:
cd nodejs-visualization-example/capture-visualization
cd public
bower install
cd ..
npm install
live-server
Now you can try the code for yourself:
node index.js
This has been an extract from chapter 11 of Data Wrangling with JavaScript now available on the Manning Early Access Program. Please use this discount code fccdavis3 for a 37% discount. Please check The Data Wrangler for new updates on the book.
Let’s say I told you we can get the results below with just one HTML element and five CSS properties for each. No SVG, no images (save for the background on the root that’s there just to make clear that our one HTML element has some transparent parts), no JavaScript. What would you think that involves?
The desired results.
Well, this article is going to explain just how to do this and then also show how to make things fun by adding in some animation.
CSS-ing the Gradient Rays
The HTML is just one <div>.
<div class='rays'></div>
In the CSS, we need to set the dimensions of this element and we need to give it a background so that we can see it. We also make it circular using border-radius:
Here we keep the gaps in between rays equal to the rays (so that’s .5*$p for either a ray or a space), but we can make either of them wider or narrower. We want an abrupt change after the ending stop position of the opaque part (the ray), so the starting stop position for the transparent part (the gap) should be equal to or smaller than it. So if the ending stop position for the ray is .5*$p, then the starting stop position for the gap can’t be bigger. However, it can be smaller and that helps us keep things simple because it means we can simply zero it.
$nr: 20; // number of rays
$p: 100%/$nr; // percent of circle allocated to a ray and gap after
.rays {
/* same as before */
mask: repeating-conic-gradient(#000 0% .5*$p, transparent 0% $p);
}
Note that, unlike for linear and radial gradients, stop positions for conic gradients cannot be unitless. They need to be either percentages or angular values. This means using something like transparent 0 $p doesn’t work, we need transparent 0% $p (or 0deg instead of 0%, it doesn’t matter which we pick, it just can’t be unitless).
There are a few things to note here when it comes to support:
Edge doesn’t support masking on HTML elements at this point, though this is listed as In Development and a flag for it (that doesn’t do anything for now) has already shown up in about:flags. The Enable CSS Masking flag in Edge.
conic-gradient() is only supported natively by Blink browsers behind the Experimental Web Platform features flag (which can be enabled from chrome://flags or opera://flags). Support is coming to Safari as well, but, until that happens, Safari still relies on the polyfill, just like Firefox. The Experimental Web Platform features flag enabled in Chrome.
WebKit browsers still need the -webkit- prefix for mask properties on HTML elements. You’d think that’s no problem since we’re using the polyfill which relies on -prefix-free anyway, so, if we use the polyfill, we need to include -prefix-free before that anyway. Sadly, it’s a bit more complicated than that. That’s because -prefix-free works via feature detection, which fails in this case because all browsers do support mask unprefixed… on SVG elements! But we’re using mask on an HTML element here, so we’re in the situation where WebKit browsers need the -webkit- prefix, but -prefix-free won’t add it. So I guess that means we need to add it manually:
$nr: 20; // number of rays
$p: 100%/$nr; // percent of circle allocated to a ray and gap after
$m: repeating-conic-gradient(#000 0% .5*$p, transparent 0% $p); // mask
.rays {
/* same as before */
-webkit-mask: $m;
mask: $m;
}
I guess we could also use Autoprefixer, even if we need to include -prefix-free anyway, but using both just for this feels a bit like using a shotgun to kill a fly.
Adding in Animation
One cool thing about conic-gradient() being supported natively in Blink browsers is that we can use CSS variables inside them (we cannot do that when using the polyfill). And CSS variables can now also be animated in Blink browsers with a bit of Houdini magic (we need the Experimental Web Platform features flag to be enabled for that, but we also need it enabled for native conic-gradient() support, so that shouldn’t be a problem).
In order to prepare our code for the animation, we change our masking gradient so that it uses variable alpha values:
And we animate it from the CSS using a keyframe animation:
.rays {
/* same as before */
animation: p .5s linear infinite alternate
}
@keyframes p { to { --p: #{$p} } }
The result is more interesting in this case:
Alternating ray size animation (live demo, only works in Blink browsers with the Experimental Web Platform features flag enabled).
But we can still spice it up a bit more by flipping the whole thing horizontally in between every iteration, so that it’s always flipped for the reverse ones. This means not flipped when --p goes from 0% to $p and flipped when --p goes back from $p to 0%.
The way we flip an element horizontally is by applying a transform: scalex(-1) to it. Since we want this flip to be applied at the end of the first iteration and then removed at the end of the second (reverse) one, we apply it in a keyframe animation as well—in one with a steps() timing function and double the animation-duration.
$t: .5s;
.rays {
/* same as before */
animation: p $t linear infinite alternate,
s 2*$t steps(1) infinite;
}
@keyframes p { to { --p: #{$p} } }
@keyframes s { 50% { transform: scalex(-1); } }
Now we finally have a result that actually looks pretty cool:
Alternating ray size animation with horizontal flip in between iterations (live demo, only works in Blink browsers with the Experimental Web Platform features flag enabled).
CSS-ing Gradient Rays and Ripples
To get the rays and ripples result, we need to add a second gradient to the mask, this time a repeating-radial-gradient().
$nr: 20;
$p: 100%/$nr;
$stop-list: #000 0% .5*$p, transparent 0% $p;
$m: repeating-conic-gradient($stop-list),
repeating-radial-gradient(closest-side, $stop-list);
.rays-ripples {
/* same as before */
mask: $m;
}
Sadly, using multiple stop positions only works in Blink browsers with the same Experimental Web Platform features flag enabled. And while the conic-gradient() polyfill covers this for the repeating-conic-gradient() part in browsers supporting CSS masking on HTML elements, but not supporting conic gradients natively (Firefox, Safari, Blink browsers without the flag enabled), nothing fixes the problem for the repeating-radial-gradient() part in these browsers.
This means we’re forced to have some repetition in our code:
$nr: 20;
$p: 100%/$nr;
$stop-list: #000, #000 .5*$p, transparent 0%, transparent $p;
$m: repeating-conic-gradient($stop-list),
repeating-radial-gradient(closest-side, $stop-list);
.rays-ripples {
/* same as before */
mask: $m;
}
We’re obviously getting closer, but we’re not quite there yet:
Intermediary result with the two mask layers (live demo, no Edge support).
To get the result we want, we need to use the mask-composite property and set it to exclude:
Note that mask-composite is only supported in Firefox 53+ for now, though Edge should join in when it finally supports CSS masking on HTML elements.
XOR rays and ripples (live demo, Firefox 53+ only).
If you think it looks like the rays and the gaps between the rays are not equal, you’re right. This is due to a polyfill issue.
Adding in Animation
Since mask-composite only works in Firefox for now and Firefox doesn’t yet support conic-gradient() natively, we cannot put CSS variables inside the repeating-conic-gradient() (because Firefox still falls back on the polyfill for it and the polyfill doesn’t support CSS variable usage). But we can put them inside the repeating-radial-gradient() and even if we cannot animate them with CSS keyframe animations, we can do so with JavaScript!
Because we’re now putting CSS variables inside the repeating-radial-gradient(), but not inside the repeating-conic-gradient() (as the XOR effect only works via mask-composite, which is only supported in Firefox for now and Firefox doesn’t support conic gradients natively, so it falls back on the polyfill, which doesn’t support CSS variable usage), we cannot use the same $stop-list for both gradient layers of our mask anymore.
But if we have to rewrite our mask without a common $stop-list anyway, we can take this opportunity to use different stop positions for the two gradients:
// for conic gradient
$nc: 20;
$pc: 100%/$nc;
// for radial gradient
$nr: 10;
$pr: 100%/$nr;
The CSS variable we animate is an alpha --a one, just like for the first animation in the rays case. We also introduce the --c0 and --c1 variables because here we cannot have multiple positions per stop and we want to avoid repetition as much as possible:
$m: repeating-conic-gradient(#000 .5*$pc, transparent 0% $pc) exclude,
repeating-radial-gradient(closest-side,
var(--c0), var(--c0) .5*$pr,
var(--c1) 0, var(--c1) $pr);
body {
--a: 0;
/* layout, backgrounds and other irrelevant stuff */
}
.xor {
/* same as before */
--c0: #{rgba(#000, var(--a))};
--c1: #{rgba(#000, calc(1 - var(--a)))};
mask: $m;
}
The alpha variable --a is the one we animate back and forth (from 0 to 1 and then back to 0 again) with a little bit of vanilla JavaScript. We start by setting a total number of frames NF the animation happens over, a current frame index f and a current animation direction dir:
const NF = 50;
let f = 0, dir = 1;
Within an update() function, we update the current frame index f and then we set the current progress value (f/NF) to the current alpha --a. If f has reached either 0 of NF, we change the direction. Then the update() function gets called again on the next refresh.
(function update() {
f += dir;
document.body.style.setProperty('--a', (f/NF).toFixed(2));
if(!(f%NF)) dir *= -1;
requestAnimationFrame(update)
})();
And that’s all for the JavaScript! We now have an animated result:
Ripple alpha animation, linear (live demo, only works in Firefox 53+).
This is a linear animation, the alpha value --a being set to the progress f/NF. But we can change the timing function to something else, as explained in an earlier article I wrote on emulating CSS timing functions with JavaScript.
For example, if we want an ease-in kind of timing function, we set the alpha value to easeIn(f/NF) instead of just f/NF, where we have that easeIn() is:
function easeIn(k, e = 1.675) {
return Math.pow(k, e)
}
The result when using an ease-in timing function can be seen in this Pen (working only in Firefox 53+). If you’re interested in how we got this function, it’s all explained in the previously linked article on timing functions.
The exact same approach works for easeOut() or easeInOut():
function easeOut(k, e = 1.675) {
return 1 - Math.pow(1 - k, e)
};
function easeInOut(k) {
return .5*(Math.sin((k - .5)*Math.PI) + 1)
}
Since we’re using JavaScript anyway, we can make the whole thing interactive, so that the animation only happens on click/tap, for example.
In order to do so, we add a request ID variable (rID), which is initially null, but then takes the value returned by requestAnimationFrame() in the update() function. This enables us to stop the animation with a stopAni() function whenever we want to:
/* same as before */
let rID = null;
function stopAni() {
cancelAnimationFrame(rID);
rID = null
};
function update() {
/* same as before */
if(!(f%NF)) {
stopAni();
return
}
rID = requestAnimationFrame(update)
};
On click, we stop any animation that may be running, reverse the animation direction dir and call the update() function:
addEventListener('click', e => {
if(rID) stopAni();
dir *= -1;
update()
}, false);
Since we start with the current frame index f being 0, we want to go in the positive direction, towards NF on the first click. And since we’re reversing the direction on every click, it results that the initial value for the direction must be -1 now so that it gets reversed to +1 on the first click.
The result of all the above can be seen in this interactive Pen (working only in Firefox 53+).
We could also use a different alpha variable for each stop, just like we did in the case of the rays:
In the update() function, the only difference from the first animated demo is that we don’t change the value of just one CSS variable—we now have four to take care of: --a0, --a1, --a2, --a3. We do this within a loop, using the ease-in function for the ones at even indices and the ease-out function for the others. For the first two, the progress is given by f/NF, while for the last two, the progress is given by 1 - f/NF. Putting all of this into one formula, we have:
(function update() {
f += dir;
for(var i = 0; i < 4; i++) {
let j = ~~(i/2);
document.body.style.setProperty(
`--a${i}`,
TFN[i%2 ? 'ease-out' : 'ease-in'](j + Math.pow(-1, j)*f/NF).toFixed(2)
)
}
if(!(f%NF)) dir *= -1;
requestAnimationFrame(update)
})();
The result can be seen below:
Multiple ripple alpha animations (live demo, only works in Firefox 53+).
Just like for conic gradients, we can also animate the stop position between the opaque and the transparent part of the masking radial gradient. To do so, we use a CSS variable --p for the progress of this stop position:
The JavaScript is almost identical to that for the first alpha animation, except we don’t update an alpha --a variable, but a stop progress --p variable and we use an ease-in-out kind of function:
/* same as before */
function easeInOut(k) {
return .5*(Math.sin((k - .5)*Math.PI) + 1)
};
(function update() {
f += dir;
document.body.style.setProperty('--p', easeInOut(f/NF).toFixed(2));
/* same as before */
})();
Alternating ripple size animation (live demo, only works in Firefox 53+).
We can make the effect more interesting if we add a transparent strip before the opaque one and we also animate the progress of the stop position --p0 where we go from this transparent strip to the opaque one:
In the JavaScript, we now need to animate two CSS variables: --p0 and --p1. We use an ease-in timing function for the first and an ease-out for the second one. We also don’t reverse the animation direction anymore:
const NF = 120,
TFN = {
'ease-in': function(k, e = 1.675) {
return Math.pow(k, e)
},
'ease-out': function(k, e = 1.675) {
return 1 - Math.pow(1 - k, e)
}
};
let f = 0;
(function update() {
f = (f + 1)%NF;
for(var i = 0; i < 2; i++)
document.body.style.setProperty(`--p${i}`, TFN[i ? 'ease-out' : 'ease-in'](f/NF);
requestAnimationFrame(update)
})();
This gives us a pretty interesting result:
Double ripple size animation (live demo, only works in Firefox 53+).
The team at Kapwing has collected a lot of images from the Internet Archive’s Wayback Machine and presented a history of how the homepage of popular websites like Google and the New York Times have changed over time. It’s super interesting.
I particularly love how Amazon has evolved from a super high information dense webpage that sort of looks like a blog to basically a giant carousel that takes over the whole screen.
