Type Regression Testing to Detect Breaking Changes in Node.js Libraries (Artifact)

This artifact provides an implementation of a novel technique, type regression testing, to automatically determine whether an update of a npm library implementation affects the types of its public interface, according to how the library is being used by other npm packages. Type regression testing is implemented in the tool NoRegrets. A run of NoRegrets is parameterized with a pre-update and post-update version of the library, and it consists of three fully automatic phases. First, NoRegrets fetches a list of clients that depend upon the pre-update library, and that have a test suite that succeeds on the pre-update version. Second, NoRegrets uses an ECMAScript 6 proxy instrumentation to generate the API model of both the pre-update and post-update libraries, based on observations of how the client test suites interact with the library. Third, the two models are compared, and inconsistencies are reported as type regressions. This artifact contains the source code and an installation of NoRegrets, with a guide for how to use the tool and reproduce the experimental results presented in the paper.