![jq logo](/images/logos/jq.png)
jq can be used to query NPM package data
If you follow IT security or even general IT news, you've likely read about the recent supply chain attacks against multiple widely used NPM packages. First, we saw a compromise of UAParser.js (ua-parser-js). That was followed by back-to-back hijacks of the Command-Option-Argument (coa) and rc packages. The affected versions of each package are well documented:
- Command-Option-Argument: 2.0.3, 2.0.4, 2.1.1, 2.1.3, 3.0.1, 3.1.3
- rc: 1.2.9, 1.3.9, 2.3.9
- UAParser.js: 0.7.29, 0.8.0, 1.0.0
However, a developer might have dozens of projects on their system with each project having hundreds or even thousands of dependencies once you include child dependencies. How can you determine what NPM packages and versions are installed across an entire system to determine whether your system is at risk?
Every NPM package includes a package.json file containing data such as the package name, version, license, and dependencies. It's easy to search for all package.json files to find the installed NPM packages, but parsing the data can be a bit trickier. That's where jq, a command-line JSON processor, comes to the rescue.
Start by installing jq. On FreeBSD, it's as simple as running pkg install jq
. Once you have jq installed, run the following command and wait patiently (it takes about 15 minutes on my system with over 28,000 packages):
sudo find / -type f -name 'package\.json' -exec \
jq '. | {
name: .name,
version: .version,
deprecated: .deprecated,
license: (if (.license | type) == "array" then (.license | map(try .type // .) | join(", ")) else (try .license.type // .license) end),
description: .description,
packageFile: "{}"
}' {} \; \
> npm-packages.json 2>/dev/null
What is this command doing? We start by using find
to search for files named package.json. The -exec
parameter passes each result to jq
, which accepts a string telling it how to process the JSON in the file. The example above converts it into a new JSON object with the following properties: name, version, deprecated, license, description, and packageFile. The value of each property is set by referencing the properties in package.json using dot notation: .name, .version, etc. Since .license can be one or an array of objects/strings, there is special handling to include all licenses and to use .license.type when present or just the .license string otherwise. Note the one outlier is packageFile whose value is set to "{}": this is simply the path to the file as provided by the find
command enclosed in quotes. Finally, we redirect stdout to a file named npm-packages.json and stderr to /dev/null. The npm-packages.json file will contain one of our JSON objects for every package.json file that was found.
Now that you have a list of all your locally installed NPM packages in a single JSON file, you can use jq to quickly run any queries you want. Use the following query to convert that file to a more human-friendly CSV format:
jq -s -r '["Name", "Version", "Deprecated", "License", "Description", "Package File"],
(. | sort_by(.name, .version, .packageFile)[] | map(.))
| @csv' \
npm-packages.json > npm-packages.csv
The -s
and -r
parameters tell jq to "slurp" the file as one big input stream converted to an array. The first array defines what will be the column headings in our CSV file. Then the input is piped through sort_by()
to make the file easier to browse. Finally, we redirect the output to a file named npm-packages.csv that can be opened with LibreOffice Calc, Excel, or any other tool that supports CSV files. Note that bundled submodules don't always have a name or version, so there could be a LOT of rows at the top of the CSV file with empty values.
Couldn't you do all this with a single command using another pipe? Certainly, but this approach provides the option to perform additional extremely fast queries against the JSON file or to process the CSV using other tools depending on your needs. At this point, you could open npm-packages.csv in a spreadsheet editor and skim for the compromised package versions (easy since the CSV file is sorted by name and then version). Let's play with some example queries instead.
Find all installed versions (not just vulnerable versions) of the compromised packages sorted by name, version, and path to see if you use them at all:
jq -s -r '. |
map(select(.name == "coa", .name == "rc", .name == "ua-parser-js")) |
map({ name: .name, version: .version, packageFile: .packageFile }) |
sort_by(.name, .version, .packageFile)' npm-packages.json
Check for a compromised version of each of the 3 packages:
jq -s -r '. |
map(select(.name == "coa" and
(.version == "2.0.3", .version == "2.0.4", .version == "2.1.1", .version == "2.1.3", .version == "3.0.1", .version == "3.1.3"))) |
sort_by(.name, .version, .packageFile)' npm-packages.json
jq -s -r '. |
map(select(.name == "rc" and (.version == "1.2.9", .version == "1.3.9", .version == "2.3.9"))) |
sort_by(.name, .version, .packageFile)' npm-packages.json
jq -s -r '. |
map(select(.name == "ua-parser-js" and (.version == "0.7.29", .version == "0.8.0", .version == "1.0.0"))) |
sort_by(.name, .version, .packageFile)' npm-packages.json
You want to see an empty array returned for each of those commands—anything else means you have a problem on your hands. To be sure the query is running correctly, copy an installed version from the output of the example above and paste it into the command that checks for vulnerable versions. When doing that, you should see the installed version in the results.
It's worth pointing out that an empty array doesn't necessary mean you were never compromised: if you upgraded packages after installing a compromised version, you could have been impacted but no longer have it installed. In that case, refer to the security advisories linked above or the project issues on GitHub for help. These instructions are meant to help users check their systems at the time of a vulnerability disclosure and before upgrading packages.
These are just 3 of the latest popular packages to be compromised, and we can expect more in the future. When it happens, generate a new JSON file and adjust the queries above to check for the vulnerable versions.