The solution to this problem isn’t actually very complicated at all but it took me several hours of Googling and experimentation before I came up with a pretty simple and straight forward solution. Obvious as it is in hindsight, it wasn’t in foresight so this one goes out to posterity.

Above: The room where NPM stores all the packages. Photo by Samuel Zeller.

The Problem

I’ve been working on two Node.js projects in tandem. Because one of them is a direct dependency of the other, I use npm link to directly link the dependency project to the main project, and incorporate changes to the dependency without having to re-publish experimental code to NPM and then run npm update within my main package.

Of course, in Travis CI, this doesn’t work since Travis rebuilds your entire environment from scratch each build. This affects a lot of people with a development set-up like mine, and I read many creative ways to handle it such as adding multiple package files, using Docker containers, etc.

At work though, we simply clone the repository of our dependency into the software at build time, and compile it there (Python/C++/Cmake/Jenkins). I wanted to achieve this with Travis CI and Node.js.

Travis provides some pretty extensive documentation on how to do this for private repositories. There was also a lot of information about pushing to a repository from Travis. But nowhere could I find a tutorial on how to clone a public repository and use it in a build. So, I’m writing one now.

Enabling Travis CI to Clone Repositories

Initially, I tried just adding typical git clone commands to my before_install section of my .travis.yml file. I didn’t think about authentication because I was cloning a public repository. Nonetheless, we still need it.

Creating a GitHub Token

The key to solving this involves giving Travis a way to access GitHub. To do this, log in to GitHub and go to the Personal access tokens page.

You’ll see a form with a field for the name of your token and what permissions you want to grant access to. Fill in a name such as “Travis CI Pull Repo” and select the “repo” section. None of the others are necessary, so no need to give access to them. At the bottom of the page is a green button “Generate token.” You know what to do.

You’ll now see your newly minted access token. Go ahead and copy this down somewhere for the next step. Beware that if you leave the page, you won’t be able to copy the token a second time; you’ll need to generate a fresh one!

Giving Travis CI GitHub Access

Back in your code editor, create a .travis.yml file in the root directory of your project if you don’t have one already. We will be using a RubyGem for the next step, so if you don’t have Ruby gems installed, you’ll need to download it. You can check if you have it installed by running gem -v in the terminal.

If you do, run the following in the terminal to install the Travis RubyGem:

gem install travis

Next, in the terminal, make sure you’re working in your project’s root directory, and use the Travis gem to add the access token to your .travis.yml file:

travis encrypt GH_TOKEN="token-from-github-goes-here" --add

If you were successful, your .travis.yml file should have a bunch of random text an encrypted token saved:

env:
  global:
    secure: "lots-of-seemingly-random-characters"

That’s it! Travis should now be able to pull (and push if that’s what you’re into) to your GitHub repository.

Configuring the NPM Link

Obviously your .travis.yml file can vary greatly from mine, but the crux of this post comes down to replicating the same process that you might use on your local machine.

At its most basic, I set up .travis.yml like this:

language: node_js
node_js:
- '6'
cache:
  directories:
    - node_modules
install:
- npm install
script:
- npm run lint
- npm run test
# Etc.
env:
  global:
    secure: "lots-of-seemingly-random-characters"

To add the cloning and linking of the dependency, add a before_install section with the following commands:

before_install:
- git config credential.helper "store --file=.git/credentials"
- echo "https://${GH_TOKEN}:@github.com" > .git/credentials
- cd ..
- git clone https://github.com/my-name/my-dependency.git my-dependency
- cd my-dependency
- npm install
- npm link
- cd ../my-main-project

What is this actually doing?

1. We configure Git to use our saved access token.
2. We go one directory up and clone the repository into a new folder with the same name as the repository.
3. We go into the repository and install its dependencies.
4. We create a global NPM link.
5. Finally, we return to the main project (the one we are running Travis on). Note that this name must match the repository’s name on GitHub as that’s the name Travis will use.

In addition, we’ll need to actually use the link created above, so in the install section add the following line:

install:
- npm install
- npm link my-dependency

Make sure you put npm link after npm install as by default npm install will obliterate any links (a very annoying bug for those of us who use npm link).

Conclusion

That’s it! Give it a shot by committing your project with its shiny new .travis.yml file!

Hopefully this saves you an afternoon of Googling (but then again that’s probably how you stumbled across this blog post in the first place).