Skip to content

Latest commit

 

History

History
 
 

analyzer

README.md

Omega Analyzer

The Omega Analyzer is a self-contained container image that has a broad set of security tools preinstalled, along with an orchestration script to run those tools against a target and aggregate the results.

While it can be used interactively, it's primary purpose is to be run from the host, with output send to a mapped directory.

Building

To build the container image, just run build.ps1 or docker build with your chosen tag, from this directory. The build script contains cache burst parameter, pass in -Force to build.ps1 to re-build all layers.

This will take a long time. As part of the build, we download CodeQL and pre-compile all queries (to make later analysis faster). This can take up to a few hours on typical hardware.

We're exploring making the pre-built image available.

Alternatively, can use the following command to build from docker

docker build -t openssf/omega-toolshed:$(grep -E '^LABEL version.*' Dockerfile | cut -d= -f2 | tr -d '"') . -f Dockerfile

# The command `$(grep -E '^LABEL version.*' Dockerfile | cut -d= -f2 | tr -d '"')` is responsible for searching for the version number on the Dockerfile and using that as the tag on Docker

Troubleshooting steps

MacOS M1 Chip

If using a Mac OSX with the latest Docker Desktop (4.15 as of writing), docker build build.ps1 will shoot out several error messages.

Make sure to create /etc/apt/ with sudo user

Download and install

  • wget with brew install wget
  • dkpg with brew install dpkg
  • .NET core with brew install mono-libgdiplus

There is a known issue with M1 Apple chip on MacOS, which would produce the error when running

qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

The following two options are available to work around this issue:

  1. Set the DOCKER_DEFAULT_PLATFORM environment variable to linux/amd64

export DOCKER_DEFAULT_PLATFORM=linux/amd64 2. In the FROM section of the Dockerfile, line 1, modify to the following

FROM --platform=linux/amd64 mcr.microsoft.com/mirror/docker/library/ubuntu:22.04

Running

To run the image, navigate to the worker directory and run the run-analysis-complete.ps1 script with relevant parameters:

run-analysis-complete.ps1 -PackageUrl "pkg:npm/[email protected]"
                 -PreviousVersion "1.2.0"
                 -OutputDirectoryName "output"

The result will be a directory containing all output files from the analysis placed into a subdirectory within output and if the results were "clean", a security review placed in security-reviews.

You can also run the image directly (which will not include reproducibility or a security review):

docker run --rm -it --mount type=bind,source=/tmp/output_dir,target=/opt/export openssf/omega-toolshed:latest pkg:npm/[email protected] 1.2.0

Standalone

To run this as a standalone from a built image, run the following:

# Template of command
docker run --rm -v <LOCAL_COMPUTER_DIR>:/opt/export/<PKG_DIR> --env-file .env openssf/omega-toolshed:latest pkg:<PKG_FORMAT>
# Example of command
docker run --rm -v ./npm/left-pad/:/opt/export/npm/left-pad/1.3.0 --env-file .env openssf/omega-toolshed:latest pkg:npm/[email protected]

The result will be a directory containing all output files from the analysis placed into a directory on your local machine (not the container) in ./npm/left-pad.

An example of the .env should contain librariesIO api key to get the packages from the net. Simply create an account on libraries IO to get the API key.

Local Usage

There are currently 2 ways of getting the tar file to run the analyzer locally, through libraries.io or github. Here is a working example of one might have to wget to get it on the container. Then you can use the ?local=true from ./worker/tools/runtools.sh to check for packages. Naturally, as virtue of running it locally, you lose the ability to use @latest as the <version_number>in the package format (pkg:npm/left-pad@<version_number>) and need to explicitly write the verison number

Here's an wget example using chalk from npm (as of Aug 2023):

Source (wget) Pattern
libraries.io https://registry.npmjs.org/chalk/-/chalk-5.3.0.tgz
Github https://github.com/chalk/chalk/archive/refs/tags/v5.3.0.tar.gz

Usage with the Omega Triage Portal

  • If you are running it locally:
docker run --env-file <.env file> --net="host" --rm -it openssf/omega-toolshed -u <username> -p <password> -t <triage portal endpoint> "pkg:<pkg_name>"
  • Need the extra --net="host" so that docker can speak to the portal running on the host machine

License

The Omega Analyzer scripts and all content that resides within this repository are licensed under the Apache license. However, the Dockerfile downloads and installs other tools that are provided under separate licenses; for example:

Please refer to the Dockerfile for more information about the varying licenses of the included tools and any restrictions that may apply to their use. In particular, ensure you understand the restrictions for CodeQL.