What I wish I knew before building a Gatsby source plugin

On this site there's a page where you can see my collection of board games. The data comes from my BoardGameGeek profile using gatsby-source-bgg.

gatsby-source-bgg is a Gatsby source plugin I built and open sourced following Gatsby's official documentation on creating a source plugin.

Here are two things I wish I didn't have to find out after hours of debugging: how to write the plugin entirely in Typescript, and how to fix an Invariant Violation error.

I got off the ground very quickly, and I thought that's it, I'm done, v1.0.0 is ready to go out.

I built a few helper functions to download data with from the external API, and I set up the plugin to pull images from the remote location so they could be optimized with the gatsby-plugin-image, rather than hotlinked.

And here started my troubles: every time I tried to build the site without clearing the cache, the process would fail with an Invariant Violation error.

To make sense of this error, I first rewrote every remaining bit of JavaScript in Typescript, and later figured out I was missing an important parameter when calling createRemoteFileNode from gatsby-source-filesystem.

How to write the entire plugin in Typescript

Gatsby in my experience is not the most Typescript friendly framework, and the guide to build a source plugin follows suit and doesn't mention Typescript. It is possible though to build Gatsby plugins in Typescript. I just wish I knew it sooner.

In Gatsby source plugins, it's important that your package contains a gatsby-node.js file in the root directory.

I initially wrote my gatsby-node.js in JavaScript, importing my functions from the build directory after transpiling all Typescript files.

I wasn't happy with this structure as functions in gatsby-node.js are not well documented and I don't like mindlessly copy-pasting stuff. In the end I managed to port everything to gatsby-node.ts.

Here's how I set up the file structure:

gatsby-source-bgg
├── dist/
│   ├── gatsby-node.js
│   └── index.js
├── gatsby-node.js
├── src/
│   ├── gatsby-node.ts
│   └── index.ts
├── package.json
├── tsconfig.json
└── yarn.lock

As you can see, I still have gatsby-node.js at the root of the package, but it simply imports and exports again the required functions.

const { sourceNodes, onCreateNode, createSchemaCustomization } = require("./dist/gatsby-node");
exports.sourceNodes = sourceNodes;
exports.onCreateNode = onCreateNode;
exports.createSchemaCustomization = createSchemaCustomization;

You can find the full source in the repo.

Invariant Violation

I wish I could find the solution to the next error on Google, maybe now it'll show up.

I was happy with my plugin. I published it to the npm registry. I added it to my site. I built once, smooth sailing. I built twice and encoutered this error:

Missing onError handler for invocation 'building-schema', error was 'Invariant Violation: Encountered an error trying to infer a GraphQL type for: coverImage___NODE. There is no corresponding node with the id field matching: "[...list of IDs]".'.

It would occur every time I tried to build my site without clearing Gatsby's .cache folder first.

The solution is pretty simple and is actually correctly implemented in Gatsby's docs (which I now realize I should've read more carefully).

When creating a node that contains images, you'll want to pull a local copy of all remote images instead of simply storing the URL. This allows you to perform transformations and optimizations at build time using gatsby-plugin-image. To do so you'll want to use createRemoteFileNode from gatsby-source-filesystem.

As of right now in gatsby-source-filesystem v3.7.1, these are the arguments you can pass to createRemoteFileNode:

// From gatsby-source-filesystem's source code https://github.com/gatsbyjs/gatsby/blob/54d4721462b9303fed723fdcb15ac5d72e103778/packages/gatsby-source-filesystem/index.d.ts
export interface CreateRemoteFileNodeArgs {
  url: string;
  store: Store;
  // TODO: use GatsbyCache type (requires [email protected])
  cache: NodePluginArgs["cache"];
  createNode: Function;
  createNodeId: Function;
  parentNodeId?: string;
  auth?: {
    htaccess_user: string;
    htaccess_pass: string;
  };
  httpHeaders?: object;
  ext?: string;
  name?: string;
  reporter: object;
}

I had mistakenly left out parentNodeId, which led to the error above as between builds the cached images would lose their relationship to their parent node.

I was once again bailed out by Typescript, and now the package seems to be free of bugs. If you check it out, let me know!