Exploring Data with Serverless and Vue: Filtering and Using the Data

In News by admin

In this second article of this tutorial, we’ll take the data we got from our serverless function and use Vue and Vuex to disseminate the data, update our table, and modify the data to use in our WebGL globe. This article assumes some base knowledge of Vue. By far the coolest/most useful thing we’ll address in this article is the use of the computed properties in Vue.js to create the performant filtering of the table. Read on!

Article Series:

  1. Automatically Update GitHub Files With Serverless Functions
  2. Filtering and Using the Data (you are here!)
showing how the demo works

You can check out the live demo here, or explore the code on GitHub.

First, we’ll spin up an entire Vue app with server-side rendering, routing, and code-splitting with a tool called Nuxt. (This is similar to Zeit’s Next.js for React). If you don’t already have the Vue CLI tool installed, run

npm install -g vue-cli
# or
yarn global add vue-cli

This installs the Vue CLI globally so that we can use it whenever we wish. Then we’ll run:

vue init nuxt/starter my-project
cd my-project
yarn

That creates this application in particular. Now we can kick off our local dev server with:

npm run dev

If you’re not already familiar with Vuex, it’s similar to React’s Redux. There’s more in depth information on what it is and does in this article here.

import Vuex from 'vuex';
import speakerData from './../assets/cda-data.json';

const createStore = () => {
 return new Vuex.Store({
   state: {
     speakingColumns: ['Name', 'Conference', 'From', 'To', 'Location'],
     speakerData
   }
 });
};

export default createStore;

Here, we’re pulling the speaker data from our `cda.json` file that has now been updated with latitude and longitude from our Serverless function. As we import it, we’re going to store it in our state so that we have application-wide access to it. You may also notice that now that we’ve updated the JSON with our Serverless function, the columns no longer correspond to what we’re want to use in our table. That’s fine! We’ll store only the columns we need as well to use to create the table.

Now in the pages directory of our app, we’ll have an `Index.vue` file. If we wanted more pages, we would merely need to add them to this directory. We’re going to use this index page for now and use a couple of components in our template.

We’re going to bring all of our data in from the Vuex store, and we’ll use a computed property for this. We’ll also create a way to filter that data in a computed property here as well. We’ll end up passing that filtered property to both the speaking table and the speaking globe.

  computed: {
    speakerData() {
      return this.$store.state.speakerData;
    },
    columns() {
      return this.$store.state.speakingColumns;
    },
    filteredData() {
      const x = this.selectedFilter,
        filter = new RegExp(this.filteredText, 'i')
      return this.speakerData.filter(el => {
        if (el[x] !== undefined) { return el[x].match(filter) }
        else return true;
      })
    }
  }
}

You’ll note that we’re using the names of the computed properties, even in other computed properties, the same way that we use data- i.e. speakerData() becomes this.speakerData in the filter. It would also be available to us as {{ speakerData }} in our template and so forth. This is how they are used. Quickly sorting and filtering a lot of data in a table based on user input, is definitely a job for computed properties. In this filter, we’ll also check and make sure we’re not throwing things out for case-sensitivity, or trying to match up a row that’s undefined as our data sometimes has holes in it.

Here’s an important part to understand, because computed properties in Vue are incredibly useful. They are calculations that will be cached based on their dependencies and will only update when needed. This means they’re extremely performant when used well. Computed properties aren’t used like methods, though at first, they might look similar. We may register them in the same way, typically with some accompanying logic, they’re actually used more like data. You can consider them another view into your data.

Computed values are very valuable for manipulating data that already exists. Anytime you’re building something where you need to sort through a large group of data, and you don’t want to rerun those calculations on every keystroke, think about using a computed value. Another good candidate would be when you’re getting information from your Vuex store. You’d be able to gather that data and cache it.

Creating the inputs

Now, we want to allow the user to pick which type of data they are going to filter. In order to use that computed property to filter based on user input, we can create a value as an empty string in our data, and use v-model to establish a relationship between what is typed in this search box with the data we want filtered in that filteredData function from earlier. We’d also like them to be able to pick a category to narrow down their search. In our case, we already have access to these categories, they are the same as the columns we used for the table. So we can create a select with a corresponding label:


 

We’ll also wrap that extra filter input in a v-if directive, because it should only be available to the user if they have already selected a column:


  
  

Creating the table

Now, we’ll pass the filtered data down to the speaking table and speaking globe:

Which makes it available for us to update our table very quickly. We can also make good use of directives to keep our table small, declarative, and legible.

{{ key }}
{{ post[entry] }}

Since we’re using that computed property we passed down that’s being updated from the input, it will take this other view of the data and use that instead, and will only update if the data is somehow changed, which will be pretty rare.

And now we have a performant way to scan through a lot of data on a table with Vue. The directives and computed properties are the heroes here, making it very easy to write this declaratively.

filtering the data in the table

I love how fast it filters the information with very little effort on our part. Computed properties leverage Vue’s ability to cache wonderfully.

Creating the Globe Visualization

As mentioned previously, I’m using a library from Google dataarts for the globe, found in this repo.

The globe is beautiful out of the box but we need two things in order to work with it: we need to modify our data to create the JSON that the globe expects, and we need to know enough about three.js to update its appearance and make it work in Vue.

It’s an older repo, so it’s not available to install as an npm module, which is actually just fine in our case, because we’re going to manipulate the way it looks a bit because I’m a control freak ahem I mean, we’d like to play with it to make it our own.

Dumping all of this repo’s contents into a method isn’t that clean though, so I’m going to make use of a mixin. The mixin allows us to do two things: it keeps our code modular so that we’re not scanning through a giant file, and it allows us to reuse this globe if we ever wanted to put it on another page in our app.

I register the globe like this:

import * as THREE from 'three';
import { createGlobe } from './../mixins/createGlobe';

export default {
 mixins: [createGlobe],
  …
}

and create a separate file in a directory called mixins (in case I’d like to make more mixins) named `createGlobe.js`. For more information on mixins and how they work and what they do, check out this other article I wrote on how to work with them.

Modifying the data

If you recall from the first article, in order to create the globe, we need feed it values that look like this:

var data = [
    [
    'seriesA', [ latitude, longitude, magnitude, latitude, longitude, magnitude, ... ]
    ],
    [
    'seriesB', [ latitude, longitude, magnitude, latitude, longitude, magnitude, ... ]
    ]
];

So far, the filteredData computed value we’re returning from our store will give us our latitude and longitude for each entry, because we got that information from our computed property. For now we just want one view of that dataset, just my team’s data, but in the future we might want to collect information from other teams as well so we should build it out to add new values fairly easily.

Let’s make another computed value that returns the data the way that we need it. We’re going to make it as an object first because that will be more efficient while we’re building it, and then we’ll create an array.

teamArr() {
  //create it as an object first because that's more efficient than an array
  var endUnit = {};
  //our logic to build the data will go here

  //we'll turn it into an array here
  let x = Object.entries(endUnit);
  let area = [],
    places,
    all;

  for (let i = 0; i < x.length; i++) {
    [all, places] = x[i];
    area.push([all, [].concat(...Object.values(places))]);
  }
  return area;
}

In the object we just created, we'll see if our values exist already, and if not, we'll create a new one. We'll also have to create a key from the latitude and longitude put together so that we can check for repeat instances. This is particularly helpful because I don't know if my teammates will put the location in as just the city or the city and the state. Google maps API is pretty forgiving in this way- they'll be able to find one consistent location for either string.

We'll also decide what the smallest and incremental value of the magnification will be. Our decision for the magnification will mainly be from trial and error of adjusting this value and seeing what fits in a way that makes sense for the viewer. My first try here was long stringy wobbly poles and looked like a balding broken porcupine, it took a minute or so to find a value that worked.

this.speakerData.forEach(function(index) {
   let lat = index.Latitude,
      long = index.Longitude,
      key = lat + ", " + long,
      magBase = 0.1,
      val = 'Microsoft CDAs';

   //if we either the latitude or longitude are missing, skip it
   if (lat === undefined || long === undefined) return;

   //because the pins are grouped together by magnitude, as we build out the data, we need to check if one exists or increment the value
   if (val in endUnit) {

     //if we already have this location (stored together as key) let's increment it
     if (key in endUnit[val]) {
       //we'll increase the maginifation here
     }
   } else {
     //we'll create the new values here
   }

 })

Now, we'll check if the location already exists, and if it does, we'll increment it. If not, we'll create new values for them.

this.speakerData.forEach(function(index) {
...

  if (val in endUnit) {
    //if we already have this location (stored together as key) let's increment it
    if (key in endUnit[val]) {
      endUnit[val][key][2] += magBase;
    } else {
      endUnit[val][key] = [lat, long, magBase];
    }
  } else {
    let y = {};
    y[key] = [lat, long, magBase];
    endUnit[val] = y;
  }

})

Make it look interesting

I mentioned earlier that part of the reason we'd want to store the base dataarts JavaScript in a mixin is that we'd want to make some modifications to its appearance. Let's talk about that for a minute as well because it's an aspect of any interesting data visualization.

If you don't know very much about working with three.js, it's a library that's pretty well documented and has a lot of examples to work off of. The real breakthrough in my understanding of what it was and how to work with it didn't really come from either of these sources, though. I got a lot out of Rachel Smith's series on codepen and Chris Gammon's (not to be confused with Chris Gannon) excellent YouTube series. If you don't know much about three.js and would like to use it for 3D data visualization, my suggestion is to start there.

The first thing we'll do is adjust the colors of the pins on the globe. The ones out of the box are beautiful, but they don't fit the style of our page, or the magnification we need for this data. The code to update is on line 11 of our mixin:

const colorFn = opts.colorFn || function(x) {
  let c = new THREE.Color();
  c.setHSL(0.1 - x * 0.19, 1.0, 0.6);
  return c;
};

If you're not familiar with it, HSL is a wonderfully human-readable color format, which makes it easy to update the colors of our pins on a range:

  • H stands for hue, which is given to us as a circle. This is great for generative projects like this because unlike a lot of other color formats, it will never fail. 20 degrees will give us the same value as 380 degrees, and so on. The x that we pass in here have a relationship with our magnification, so we'll want to figure out where that range begins, and what it will increase by.
  • The second value will be Saturation, which we'll pump up to full blast here so that it will stand out- on a range from 0 to 1, 1.0 is the highest.
  • The third value is Lightness. Like Saturation, we'll get a value from 0 to 1, and we'll use this halfway at 0.5.

You can see if I just made a slight modification, to that one line of code to c.setHSL(0.6 - x * 0.7, 1.0, 0.4); it would change the color range dramatically.

two different color patterns for the pins on the globe

We'll also make some other fine-tuned adjustments: the globe will be a circle, but it will use an image for the texture. If we wanted to change that shape to a a icosahedron or even a torus knot, we could do so, we'd need only to change one line of code here:

//from
const geometry = new THREE.SphereGeometry(200, 40, 30);
//to 
const geometry = new THREE.IcosahedronGeometry(200, 0);

and we'd get something like this, you can see that the texture will still even map to this new shape:

showing how we can change the shape of the globe to an icosahedron

Strange and cool, and maybe not useful in this instance, but it's really nice that creating a three-dimensional shape is so easy to update with three.js. Custom shapes get a bit more complex, though.

We load that texture differently in Vue than the way the library would- we'll need to get it as the component is mounted and load it in, passing it in as a parameter when we also instantiate the globe. You'll notice that we don't have to create a relative path to the assets folder because Nuxt and Webpack will do that for us behind the scenes. We can easily use static image files this way.

mounted() {
  let earthmap = THREE.ImageUtils.loadTexture('https://cdn.css-tricks.com/world4.jpg');
  this.initGlobe(earthmap);
}

We'll then apply that texture we passed in here, when we create the material:

uniforms = THREE.UniformsUtils.clone(shader.uniforms);
uniforms['texture'].value = imageLoad;

material = new THREE.ShaderMaterial({
  uniforms: uniforms,
  vertexShader: shader.vertexShader,
  fragmentShader: shader.fragmentShader
});

There are so many ways we could work with this data and change the way it outputs- we could adjust the white bands around the globe, we could change the shape of the globe with one line of code, we could surround it in particles. The sky's the limit!


And there we have it! We're using a serverless function to interact with the Google Maps API, we're using Nuxt to create the application with Server Side Rendering, we're using computed values in Vue to make that table slick, declarative and performant. Working with all of these technologies can yield really fun exploratory ways to look at data.

Article Series:

  1. Automatically Update GitHub Files With Serverless Functions
  2. Filtering and Using the Data (you are here!)

Exploring Data with Serverless and Vue: Filtering and Using the Data is a post from CSS-Tricks

Powered by WPeMatico