Automatically Creating Github Issues Using Webtask.io

I am the co-organizer of the BeerJS meetings for TahoeJS. BeerJS is a casual meetup for JavaScript developers that has chapters all over the world. Our meetup is largely organized using a Github repo.

As co-organizer, one of my monthly tasks is to create a Github issue for the upcoming meeting. The issue announces the time, date and location of the meeting. If a speaker is organized, that can be added in also.

This weekend, I decided to automate that task using a serverless endpoint on Webtask.io.

What is a serverless endpoint?

A serverless endpoint is a bit of a misnomer. A server is still involved; it’s just that the server will likely be torn down between executions of the function. This is unlike a traditional server, where the instance generally continues to run between executions.

A serverless endpoint allows you to run a piece of code. This code can pretty much do anything that typical server-side code can do (access databases, render HTML, etc.) but since the server dies at the end of execution, serverless endpoints are meant to focus on performing one discrete task.

Serverless endpoints may be accessed through a URL or they can be scheduled to run like a cron job.

What is Webtask.io?

Webtask.io is a service from Auth0 that allows you to create and execute serverless endpoints. It has free and paid plans with the free plans being pretty generous.

Theoretically, you could host an entire low traffic site on Webtask.io. Their serverless endpoints are capable of running ExpressJS, accessing databases, etc. However, once I saw that you could schedule the endpoints, I knew it’d be perfect for automating the creation of Github issues.

One of the primary competitors for serverless endpoints is Amazon’s AWS Lambda service.

How do I get started with serverless endpoints?

I found the easiest way to get started was to develop a small Node.js program that accomplishes my task and then convert it to a webtask.

Creating the empty project

I started off by creating a new project from the command line:

$ mkdir new-project
$ cd new-project
$ git init
$ npm init
$ touch index.js
$ touch webtask.js

The git init is probably optional but I always put my projects on Github. The npm init is required since we’ll be installing some external packages. npm init will ask you a bunch of questions but you can just press Enter for most them to accept the defaults.

The two touch commands create a couple of empty files for you.

Installing libraries

Once the project was created, I knew I’d be doing some HTTP access, some date calculations and passing command-line arguments, so I found some appropriate libraries to help:

$ npm install --save minimist moment request request-promise-native

minimist makes it easier for your program to receive command-line arguments. moment is a common library for performing date calculations. request and request-promise-native work together to make promise-based HTTP requests easier in Node.js.

Creating an issue template

Github allows you to create an issue template for your project. This is simply a Markdown file called ISSUE_TEMPLATE.md in the root of your repository. Every time someone creates a new issue for your repo, this file is used to pre-populate the text field.

I want my webtask-created issues to have the same text so I created the template file and my serverless endpoint will fetch this file to use as the body of the issue.

Creating a Github token

Github doesn’t just let any random server access its API to create issues. That’d be a terrible security threat. But they do make access easy.

Simply head over to the Personal Access Tokens page in your Github Settings. Once you’re there, click the Generate New Token button.

On the next screen, give your token an easy to recognize name and select the repo permissions.

Once you generate the new token, Github will show it to you once and only once so be sure to record it somewhere secure. If you lose it, you’ll have to re-generate a new token.

Note, do NOT put your token in code that will be committed to Github. Anything committed to Github can be searched and found by other users. A common way to steal access tokens is to search Github projects. Webtask.io gives us an easy way to securely save secret information like this Github access token.

Organizing the code

You’ll note that I created two JavaScript files when I was setting up my project. index.js will give me ability to test and run my program from the command line. webtask.js contains the code that will be executed by Webtask and will be called by index.js.

It is possible to split your Webtask code into multiple files. That requires using Webtask.io’s bundle command. Since my program wasn’t very long, I didn’t really look into that command very deeply.

Writing the command line interface

My index.js file is really simple.

It imports the webtask function from webtask.js.

Then it uses minimist to parse out the command-line arguments and restructures those arguments into the form used by Webtask.io.

After that, it simply calls the webtask function with the new data structure and a callback function that outputs any message returned by the webtask.

Writing the webtask

Below is my webtask. I’ve stripped out a bunch of the code so that you can see the most important parts. See my repo for the full code.

const rp = require('request-promise-native');
// ...and other libs
 
// create some constants
 
module.exports = function (context, cb) {
  // pull out the variables from `context`
 
  // check for missing variables and return any errors
 
  const defaultOptions = {
    qs: {
      access_token: github_token
    },
    headers: {
      'User-Agent': 'Github Issue Webtask'
    },
    json: true
  };
 
  const templateOptions = {
    method: 'GET',
    uri: `${GITHUB_RAW_URL}${owner}/${repo}/master/ISSUE_TEMPLATE.md`,
  };
  rp(Object.assign({}, defaultOptions, templateOptions))
    .then((issueTemplate) => {
      // create the issue title
 
      const issueOptions = {
        method: 'POST',
        uri: `${API_URL}repos/${owner}/${repo}/issues`,
        body: {
          title,
          body: issueTemplate,
          labels: ['MEETING']
        },
      };
      rp(Object.assign({}, defaultOptions, issueOptions))
        .then((response) => /* return success */)
        .catch((error) => /* return error */);
    })
    .catch(function (error) {
      // return error
    });
};

Reading from the top, you can see:

• rp is my promise-based HTTP request library
• The module exports one function. This function takes a context parameter and a callback function, cb. context will contain all of the data passed to my webtask. cb will be used to communicate back to the calling program
• After creating some option variables and merging them using Object.assign, the code makes a request to get the template file
• Then it creates the title of the issue and creates a new option variable so that it can create the Github issue using a POST call

And that’s pretty much it for the actual webtask. The rest of the code in the webtask is used for generating the title with the date of the next event.

Uploading and Scheduling the Webtask

Now that you’ve seen how to create a webtask function, it’s time to upload the function to Webtask.io and schedule the cron job.

Webtask.io has a command line interface (CLI) that makes this really easy. First, you’ll want to go to the CLI install page, log in to the site and follow the directions there.

Once you’ve logged in, you’ll be doing the following steps in your terminal (if you followed the installation directions, skip the first 2 steps):

$ npm install wt-cli -g
$ wt init EMAIL_ADDRESS@USED_ON_WEBTASK.io
$ wt cron schedule -n beerjs_scheduler -s github_token=XXXX "0 0 22 * *" ./webtask.js
Name:        beerjs_scheduler
State:       active
Container:   wt-1234-0
Schedule:    0 0 22 * *
Next run:    9/22/2017, 12:00:00 AM
Meta.wt-node-dependencies: {"minimist":"1.2.0","moment":"2.18.1","request":"2.81.0","request-promise-native":"1.0.4"}

The first two steps install the CLI tool and link your Webtask.io account to your computer.

The third command does a bunch of things:

• wt cron schedule – tells the CLI that you’d like to schedule a new or updated webtask
• -n beerjs_scheduler – gives the task a friendly name
• -s github_token=XXXX – tells Webtask.io to securely save a secret variable. In this case, it’s the Github access token we saved earlier. This code will be made available to your webtask function in context.secrets.github_token. Secret variables are particularly handy for things you don’t want to make public in a URL
• "0 0 22 * *" – provides the cron schedule for the task. The format of the numbers is "HOUR MIN DAY_OF_MONTH MONTH YEAR". The asterisks indicate wildcards. You can also provide ranges and other values. This value schedules the task to run on the 22nd day of every month of every year at 00:00 (midnight). I chose that schedule because it occurs every month and is always after the monthly meeting but it could be whatever works for you.
• ./webtask.js – the path to the code

After the command, you can see the output created by the CLI. It gives you the name, state of the task, the container that it’s running on on the Webtask.io servers, the schedule, the next run time and the dependencies that the CLI found in package.json.

Once the upload is complete, we still need to do a little final setup using the Webtask.io online editor. To get there, run the following command:

$ wt edit beerjs_scheduler

The edit command opens a browser window to the Webtask.io online editor. You’ll give the edit command either the friendly name of your webtask (if you gave it one) or the name of the file that was uploaded.

Once the online editor is open, you’ll see this screen. We need to provide the rest of the parameters that our webtask is expecting. If we were not creating a cron task, we would simply provide these parameters using URL query string parameters (these are the ones that come after the ? at the end of a URL).

On the right side of the screen, expand the Runner. Github requires us to use POST to create an issue, so change that. Then use the URL Params fields to provide the rest of the info needed by the function. These parameters will be provided to the function under context.data.

While you have the editor open, you can also run the task and see a history of all of the runs of your function. The editor also has logs that can be useful for debugging your function if there are problems (try adding console.log statements to your function).

What are the limitations of Webtask.io?

While working on this project, I ran into a couple limitations with Webtask.io:

The first limitation was on the size of the file. Your webtask file can’t be larger than 100KB. That’s pretty big if you’re writing all of the code yourself, but not very big if you need to bundle some external libraries.

To help you get around that issue, the Webtask.io CLI tool looks for a package.json file that is in the same directory as your webtask file. This is why I created the webtask.js file in the root directory.

Any npm dependencies in package.json are automatically loaded and made available to your code once it’s on the Webtask.io servers. This should be fine for most things but I’ve heard some people have had issues with private repos.

The second limitation that I ran into was in the way that you have to load external libraries.

Moment.js has an ecosystem of plugins that people have written. These plugins attach themselves to the main Moment library and provide additional functionality. Typically, you see them imported into a project like this:

const moment = require('moment');
require('moment-recur');

That code works fine in Node.js. The functions for the Moment Recur library would then be available using moment().recur().

However, it won’t work on Webtask.io. I filed an issue with Webtask.io and they wrote back to tell me that this is a known limitation.

To get around this issue, I ended up having to write my own function to find the next scheduled meeting. If you’re interested in that function, see the full code in the repo.

Conclusion

Overall, using Webtask.io was pretty easy and enjoyable. I’ve been interested in trying serverless functions for a while but I’d heard that Amazon’s AWS Lambdas were hard to set up. The Webtask.io CLI made this pretty easy.

The documentation for Webtask.io is pretty good. There were definitely times where I could have used some more examples but I was able to figure it out with some trial and error. Mostly this was around the use of the bundling command and npm dependencies.

I did run into a couple limitations. The file size wouldn’t be too bad unless you had big internal libraries. The limitation on requiring external modules presented a larger technical challenge for this project but I don’t see a lot of libraries that use that convention.