Chris Essig

Walkthroughs, tips and tricks from a data journalist in eastern Iowa

Archive for the ‘coding’ Category

Leaflet formula: Counting markers within a radius (version 2)

with 2 comments

A couple of years ago, I open-sourced a project that allows you to count how many markers are within a circle using Leaflet. As part of the template, a user needs to enter an address to get a marker to show up on the map.

Two weeks ago, Jefferson Guerrón reached out to me because he was creating something similar that didn’t require a user to type in their address. He was using the plugin Leaflet.draw to get the marker on the map. But he was having a hard time getting a circle to show up around the marker and count the other markers within that circle.

After a few back-and-forth emails, we were able to come up with a solution. Here’s a look of the final project. And the code is available on Github.

If you have questions, don’t hesitate to e-mail or comment.

Advertisements

Written by csessig

December 14, 2016 at 5:15 pm

D3 formula: Splitting elements into columns

leave a comment »

icons_columnsD3 can be a tricky — but powerful — beast. A month ago, I put together my most complex D3 project to date, which helped explain Iowa’s new Medicaid system

One of the first places I started on this project was building a graph that will take an icon and divide it into buckets. I didn’t see any openly available code that replicated what I was trying to do, so I’d figure I’d post my code online for anyone to use/replicate/steal.

For this project, I put the icons into three columns. In each column, icons are placed side by side until we get four icons in a row, then another row is created. You can see this in action by clicking the button several times. And all of this can be adjusted in the code with the “row_length and “column_length” variables.

To move the icons, I overlaid three icons on top of each other. When the button is clicked, each of the icons gets sent to one of the columns. The icons shrink as they reach their column. After this transition is finished, three more icons are placed on the DOM. And then the whole process starts over.

A bunch of math is used to determine where exactly on the DOM each icon needs to go. Also, we have to keep track of how many icons are on the DOM already, so we can break the icons into new rows if need be.

Data is required to make D3 run so my data is a simple array of [0,1,2]. The array has three values because we have three columns on the page. We do some calculations on the values themselves and then use SVG’s transform attribute to properly place the icons on the DOM. We use D3’s transition function to make it appear like the icons are moving into the icons, not just being placed there.

Hopefully this helps others facing similar problems in D3. If you have any questions, just leave them in the comments.

Written by csessig

May 2, 2016 at 12:30 pm

Save your work because you never know when it will disappear

with 2 comments

We are a few weeks into the new year, but I wanted to look back at the biggest project I worked on in 2015: The redesign of KCRG.com.

While most of my blog posts are full of links, I can’t link to that site. Why? Because it’s gone.

What?

In a series of very unfortunate events, the site we spent many, many months planning and developing is already gone.

The timeline: We started preparing for the redesign, which was BADLY needed, in early 2015. We then built it over the course of several months. Finally, it was launched in July. Then, in a move that surprised every one, KCRG was bought by another company in September.

At the time, I was optimistic that the code could be ported over to their CMS. And the site wouldn’t die.

My optimism was short lived. Gray has a standard website template for all its news sites, and they wanted that template on KCRG.

So in December, the website we built disappeared for good.

 

The KCRG website you see now is the one used and maintained by Gray.

Obviously, this was a big shock for our team. Even worse, the code we wrote was proprietary and requires Newscycle Solutions to parse and display. So even if I wanted to put it on Github, it wouldn’t do anyone any good.

I’m not used to the impermanence of web. When I had my first reporting job in Galesburg, I saved all the newspapers where my stories appeared. And unless my parents’ house catches on fire, those newspapers will last for a long time. They are permanent.

Not so online. Websites disappear all the time. And those who build them have barely any record of their existence.

Projects like PastPages and the Wayback Machine keep screenshots of old websites, which is better than nothing. But their archives are far cries from the living, breathing websites we build. A screenshot can’t show you nifty Javascript.

It’s an eery feeling. What happens in five years? Ten years? Twenty years? Will any of our projects still be online? Even worse: Will technology have changed so much that these projects won’t even be capable of being viewed online? Will online even exist?

Think about websites from 1996. They are long gone. Hell, many sites from two years ago have vanished.

I don’t have good answers. Jacob Harris has mulled this topic and offered some good tips for making your projects last.

But it’s worth pondering when you’re done with projects. What can I do to save my work for the future? I have a directory of all of my projects from my Courier days on an external hard drive. I have an in-process directory for The Gazette as well.

I hold onto them like I did my old newspaper clippings. Although, I’m confident those clippings will last a lot longer than my web projects.

Written by csessig

January 21, 2016 at 11:52 am

New adventures

leave a comment »

Some exciting personal news: I’ll be joining the Cedar Rapids Gazette and KCRG as their Interactive News Developer at the end of the month. My last day at the Waterloo-Cedar Falls Courier is Friday.

The new job will let me develop full time, meaning I will be coding more than I have. And I will continue to work in a newsroom, which is awesome. Needless to say, I’m very excited about the move.

I spent the last three years The Courier, and as this blog shows, I learned a ton in that time span. I will greatly miss working there. And more importantly, I’ve very, very proud of all the awesome work we did while I was there.

Here’s to new adventures!

Written by csessig

May 15, 2014 at 2:37 pm

How laziness led me to produce news apps quicker

with one comment

Note: This is cross-posted from Lee’s data journalism blog, which you can read over there by clicking here.

ssAn important trait for programmers to adopt, especially for those who are writing code on deadline, is laziness. While that sounds counter-productive, it can be incredibly beneficial.

When writing code, you  do a lot of repetitive tasks. If you’re using a Javascript library for mapping, for instance, you’ll need to copy and paste the Javascript and CSS files into a new directory every time you start a new project. If you want to use plugins with that map, you will need to include their files in the directory as well.

The more complicated your project, the more libraries you will likely use, which means a lot of tedious work putting all those files in the new directory.

And that’s before you even start to code. If you want your projects to have a similar look to them (which we do at the Courier), you will need to create code that you can to replicate in the future. And then every time you want to use it, you will need to rewrite the same code or, at the very least, copy code from your old projects and paste it into your new project.

I did this for a long time. Whenever I was creating  a new app, I’d open a bunch of projects I’ve already completed, go into their CSS and JS files, copy some of it and paste it into the new files. It was maddening because I knew with every new project, I would waste hours basically replicating work I had already done.

This is where laziness pays off. If you are too lazy to do the same things over and over again, you’ll start coming up with solutions that automatically does these repetitive tasks.

That’s why many news apps teams have created templates for their applications. When they create new projects, they start with this template. This allows their apps to have a similar look. It also means they don’t have to do the same things over and over again every time they are working with a project.

App templating is something I’ve dabbled with before. But those attempts were minor. For the most part, I did a lot of the same mind-numbing tasks every time I wanted to create a map, for instance.

Fortunately I found a good solution a few months ago: Yeoman, which is dubbed “the web’s scaffolding tool for modern web apps,” has helped me eliminate much of that boring, repetitive work.

Yeoman is built in Node. Before I stumbled upon it a few months ago, I knew very little about Node. I still don’t know a ton but this video helped me grasp the basics. One advantage of Node is it allows you to run Javascript on the command line. Pretty cool.

Yeoman is built on three parts. The first is yo, which helps create new applications and all the files that go with it. It also uses Grunt, which I have come to love. Grunt is used for running tasks, like minifying code, starting a local server so you test your code, copy files into new directories and a whole bunch of other things that makes coding quicker, easier and less repetitive.

Finally, Yeoman uses Bower, which is a directory of all the useful packages you use with applications, from CSS frameworks to Javascript mapping libraries. With a combination of Bower and Grunt, you can download the files you need for your projects and put them in the appropriate directories using a Grunt task. Tasks are written in a Gruntfile and run on the command line. The packages you want to download from Bower and use in your project are stored in bower.json file.

In Yeoman, generators are created. A generator makes it easy to create new apps using a template that’s already been created. You can either create your own generator or use one of the 400+ community generators. The generators themselves are directories of files, like  images, JS files, CSS stylesheets, etc. The advantage of using generators is you can create a stylesheet, for instance, save it in the generator directory and use that same stylesheet with every new project you create. If you decide to edit it, those changes will be reflected in every new project.

When you use a generator, those files are copied into a new directory, along with Grunt tasks, which are great for when you are testing and deploying your apps. Yeoman also has a built-in prompt system that asks you a series of questions whenever you create an app. This helps copy over the appropriate files and code, depending on what you are trying to make.

I decided I would take a stab at creating a generator. The code is here, along with a lot more explanation on how to use it. Basically, I created it so it would be easier to produce simple apps. Whenever I create a new Leaflet map, for instance, I can run this and a sample map with sample data will be outputted into a new directory. I also have a few options depending on how I want to style the map. I also have options for different data types: GeoJSON, JSON with latitude and longitude coordinates and Tabletop. What files are scaffolded out depends on how you responded to the prompts that are displayed when you first fire up the generator.

There is also an option for non-maps. Finally, you can use Handlebars for templating your data, if that’s your thing.

My hope is to expand upon this generator and create more options in the future. But for now, this is easy way to avoid a bunch of repetitive tasks. And thus, laziness pays off.

If you are interested in creating your own generator, Yeoman has a great tutorial that you should check out.

Written by csessig

February 19, 2014 at 12:08 pm

How we used a Google spreadsheet to power our election app

with one comment

Election 2013 app screen shot

Last Tuesday was election night in the Cedar Valley. While we didn’t have a huge number of contested races on the ballot, we did have a mayor’s race in Waterloo and Waverly, Iowa, as well as city council races in several area towns.

As NPR’s Jeremy Bowers proudly proclaimed, election nights are exciting times for news nerds. This election, we decided to do a little bit of experimenting.

Before the election, we posted biographical information for all the candidates running in a contested race in Waterloo, Cedar Falls and Waverly. We also promoted other races in smaller towns. This gave our readers a good overview of the races on the ballot and information on the candidates. And best of all, it was all in one place.

Three reporters were responsible for getting the biographical information for each candidate and entering it online. We used a Google spreadsheet to store the information. This allowed the reporters to enter the information online themselves. I could then go into the spreadsheet, edit the text and make sure it was formatted correctly. We then used Tabletop.js to import the data into our app and Handlebars.js to template it.

The basic setup for the app is available on my Github page. This is a very similar setup mentioned in my last blog post.

Before the app went live, I exported all the data in the Google spreadsheet into JSON format using a method mentioned here. I did this for two reasons: 1) It spend up the load time of the app because browsers didn’t have to connect to the spreadsheet, download the data, format the data into JSON (which is what Tabletop.js does with the data in this app) and then display it online using templates rendered in Handlebars.js. Instead, it’s already downloaded, formatted into JSON and ready to be templated. And: 2) Tabletop.js has a bug that may cause some readers not to see any of the data at all. I wanted to avoid this problem.

The night of the election, we wanted to update the election results live. And we wanted to use the same app to display the results. Fortunately, the process was easy to do: I added new columns in our spreadsheet for vote totals and precincts reported. The new data was then pulled into our app and displayed with simple bar charts.

To display the results live, we had to ditch the exported JSON data and use the Google spreadsheet to power the app. This allowed us to update the spreadsheet and have the results display live on our website. We had one reporter at the county courthouse who punched in numbers for our Waterloo and Cedar Falls races. We had another reporter in Waverly who punched in results for races in that town. And we had another reporter in the newsroom who was monitoring the races in rural towns and giving me updated election results to enter into the spreadsheet.

The workflow worked great. While most news outlets were waiting for the county websites to update with election results, we were able to display the results right away. Unfortunately, the Black Hawk County website never wound up working on election, making our app the only place readers could go to get election results.

Our effort paid off: The app received about 11,500 pageviews, with about 10,000 of those pageviews coming the night of election (about 7,000 pageviews) and the following day. The app was more popular than any single story on our website for the month of October and November.

Reporters at the party headquarters said many of the candidates first tuned into the local television station to get election results but quickly went to our webpage when they realized we were the only ones with updated results. In fact, Waterloo’s mayor found out he won his race by looking at our website. Here’s a photo of him checking out our website on election night.

Which leads me to my last point: You’ll notice how our mayor is checking out the results on a smartphone. Our app (like all of our apps) was responsively designed, which means it looks great on mobile phones. It’s critical that news producers make sure their apps work on mobile. It’s pretty much mandatory. Because as our mayor shows, people love checking the news on their phones.

Written by csessig

November 10, 2013 at 10:19 pm

New code: Using a Google spreadsheet to power a mobile site

with 3 comments

I’ve posted a new template on Github that I hope others will find useful. It involves using Tabletop.js to transfer data from a Google spreadsheet onto a mobile website. I’ve tried to create it so you have to do little tweaking of the code that grabs and formats the data so you can focus on collecting and cleaning the data. The template will render the same regardless of what data is in the Google spreadsheet.

It’s initial purpose was to power our guides to weekend festivals, since we have quite a few in the summer. We used it, for instance, for our guide to the National Cattle Congress fair, which was last weekend. And it was designed for mobile users, in particular. You will notice how the website kind of looks like an app you would download on your smartphone with navigation buttons at the button. But it was responsively designed so it will work on all computer sizes.

You can use it for whatever you like. You can also change the code to have the data displayed as you see fit. The whole project is open-sourced.

The code can be found here. If you use it at all, please e-mail a link to you work so I can share it with the world! My e-mail is chris.essig@wcfcourier.com.

Written by csessig

September 18, 2013 at 10:47 am