Why is Testing important

Humans make mistakes all the time. When contributing to a large scale open source project, you are going to be working with other developers in the community. You need a way to test that the code you are going to contribute to the project, will not disrupt the existing functionality of the project. Software testing is necessary because it provides a way to check anything and everything we produce as things can always go wrong. It improves consistency and performance hence it is a crucial component of software product development. It also helps developers and testers to compare actual and expected results to improve quality. Testing is essential in providing reliable software that makes it safe to use in real-world applications.

What is Jest

Jest is a JavaScript testing framework designed to ensure correctness of any JavaScript codebase. It allows you to write tests with an approachable, familiar and feature-rich API that gives you results quickly. Jest is well-documented, requires little configuration and can be extended to match your requirements. Jest makes testing delightful. - Jest Core Team

Jest Architecture

• Jest project is split up into multiple packages

• jest-cli: most people install the jest package which forwards to jest-cli. It pulls together all the other packages to run tests.

• When you run $ jest my-test it goes to jest-cli. Jest-cli talks to jest-config.

• Here’s how Jest works:

(STEP-1)

• jest-config: It figures out what should happen, what the person wants to do on a cli environment. It takes the config from some configuration file (finds this file automatically) & it also takes the argument ( — flag arguments) that were made with the call and merges them together & figures out what we actually want to do. There is a function normalize in this package that normalizes the data that we get for this invocation and the it creates 2 separate configuration objects:

• Global config: For how many workers to use, which test framework to use, which timer system to use, test reporters to run, etc.

• Project config: There might be more than one of these configs if your repository has many projects with different configurations.

• This helps jest to use on global config and many project configs to run many tests from different projects in a single test run.

• There is a type folder in jest repo so that you can figure out what each configuration object is there for.

• Now we send back the configs to the jest-cli.

  

(STEP-2)

Now that we know what the configuration is, we need to figure out what is the codebase that we operate on. Jest does a lot of static analysis to figure out how to run tests quickly also how to do dependency resolution to track specific changes from files to which tests are affected by it. For this we have a package jest-haste-map

(STEP-3)

• jest-haste-map: Jest calls this and asks what are all the files in the project and what are their dependencies between them.

• It does that by looking at import/require calls, extracts them from each file and build a map that has every file in it with their dependencies.

• Haste is a module system used at Facebook.

• It also has something called HasteContext and in that it has HastFS (Haste file system). How it works is that at Facebook there is a single folder with all html/js files and each file has a name associated with it. So you can require every module by name instead of long relative paths.

• HastFS is just a list of the files in your system and all the dependencies that are associated with that. It is a map data structure where key is the PATH & values are the MetaData.

FileData = Map<Config.Path, FileMetaData>;
FileMetaData = [ 
    id: string,  
    mtime: number,  
    size: number,  
    visited: 0 | 1,  
    dependencies: string,  
    sha1: string | null | undefined, 
];

• So it has to get a list of all the files, for this it invokes watchman (fb open source project) which aims to notify you of certain file system permissions, it asks what are all the files that exist in this folder and then it gives you list of all of those files. So when you run tests and run jest again, it will ask watchman what has changed since the last time it gave a list of files. Then watchman will only give you list of those changes. Hence, jest-haste-map has a cache.

• If you don’t have watchman it will use node.js crawler (uses the find command every single time) which is slow.

• Now it talks to jest-worker where we give it the files we have and ask for metadata i.e. dependencies between files.

(STEP-4)

jest-worker: How to read so many large files? It uses all the cores available (helps the jest-haste-map to find files). We give jest-worker all the files and ask to do the work asap. Jest-worker talks to the HasteContext and gives the information back to the jest-cli via jest-haste-map.

(STEP-5)

• Now we just has a simple data type: Context = {HasteContext, GlobalConfig, ProjectConfig}

• Context = {HasteContext, GlobalConfig, ProjectConfig}

• There is a class SearchSource (from runJest(watch?)) gets all the data from Context and says based on this data, go figure out which tests we want to run. It finds which tests to run by using the patter of writing test files in jest. SearchSource gives you an Array of tests: Array<Test> , Test = { Path, duration:number, Context }

• There is a type folder in Jest repo where you can look up at the types. But this is unordered, hence it doesn’t tell you how to run these tests.

(STEP-6)

TestSequencer class : Receives the array of tests and looks through all of the tests, contexts, project configs, and determines what is the order in which to run the tests.

It checks:

• Has this test failed in the past, if yes it will run that test first.

• Has this test run, if it took longer to run and it will be run first (to utilise the cpu)

• The above points are made use by the help of cache, if cache is not there then it will look at the file size.

• Area for improvement: Can figure out the TestPriority of tests when we change some files.

(STEP-7) Run Tests

TestScheduler:TestSequencer calls this. This is the actual system that figures out based on the sequencing how do we run them optimally. It looks at do we need to run this in band (run on the same process that jest itself runs) or do we want to create worker processes, run tests on them and then get results back. Figures out whether tests should be run serially ( — runInBand) or parallel. This is also responsible for creating the reporters. Logics from this class have been extracted to jest-runner package. (you can provide specific runner to each project)

(STEP-8)

• jest-runner: It gets the list of tests. It doesn’t make sense to spawn processes for small amount of tests because it will make it slow. So it will run those tests in the same process and give the results. This will now call to the package jest-worker. It asks jest-worker to create however many processes that we need, setup stuff, run those tests. This parallelises stuff and processes.

• Calls runTests() which can then call runTest() (single or multiple times to run in parallel) using jest-worker (see diagram).

• runTest() will spin up environments like jest-runtime, jest-circus/jasmine, jest-environment.

(STEP-9)

• It will now go to jest-circus/jest-jasmine (this has test() describe() ) to run the tests (because it uses jasmine under the hood by default).

• Here it also injects expect that gives you access to a number of “matchers” that let you validate different things. - This is helpful when you need to check that the values meet certain conditions.

• jest-circus provides apis to run tests. It uses flux architecture to build up your flat tree which determines which tests to run, etc.

• This creates the apis and uses the module called jest-runtime.

(STEP-10)

jest-runtime: When you run multiple tests, they run in isolated environments (using jest-environments). So even if the tests are running in same process, jest-runtime will make them run in different VM context (this is a module provided by node.js, used to create context for running javascript in isolation, you can specify your own global scope. It also has a require implementation for modules). Also if you have used mocking system in jest where you can actually swap out modules at runtime you can call just let it call some module and then require it later, all of that happens in here.

(STEP-11)

jest-environments: this is where the runtime creates the environments. And this is passed to jest-runtime which runs all the tests.

So jest-runtime:

• does resolution for modules

• does mocking

• transforms, when using typescript, etc

• injects require functions in the vm context and then when you require a module (experimental support for ESM), jest goes outside of this sandboxed context where your tests run back into jest-runtime where it looks at what module is that test requiring and then looks at where this module has to be transformed using babel or typescript. This happens synchronously. Because function require is synchronous and we do transformation just-in-time.

• Type: TestResult (when results come from circus or jasmine)

• We serialise this json data and give it back to jest-runner & TestScheduler collects the data from TestResult

• TestScheduler also has some callbacks that when the test is finished it will invoke those callbacks. Here we have an AggregatedTestResult type. This is similar to TestResult, has properties like has the entire test passed, etc.

• All these tests results are passed to reporters which can print certain messages.

• Then it passes everything to the jest-cli which return exit code for the same, or it keeps listening for changes if watch mode is on.

References:

• https://jestjs.io

• https://www.youtube.com/watch?v=3YDiloj8_d0

• https://www.youtube.com/watch?v=3oBrDZi43R8