Skip to content
This repository has been archived by the owner on Feb 9, 2020. It is now read-only.
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: pbogdan/ngx-top
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.1.1.0
Choose a base ref
...
head repository: pbogdan/ngx-top
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Feb 16, 2017

  1. Copy the full SHA
    d91857f View commit details
  2. Copy the full SHA
    3070f79 View commit details
  3. Update screenshot.

    pbogdan committed Feb 16, 2017
    Copy the full SHA
    5721db0 View commit details
  4. Bump version.

    pbogdan committed Feb 16, 2017
    Copy the full SHA
    4dc040b View commit details
  5. Add release upload script.

    pbogdan committed Feb 16, 2017
    Copy the full SHA
    16eef77 View commit details

Commits on Apr 20, 2017

  1. Remove upper bound on hspec.

    pbogdan committed Apr 20, 2017
    Copy the full SHA
    2c307d2 View commit details
  2. Copy the full SHA
    4615ca1 View commit details
  3. Copy the full SHA
    8165bf0 View commit details
  4. Copy the full SHA
    bed9498 View commit details
  5. Copy the full SHA
    5064404 View commit details

Commits on Apr 21, 2017

  1. Nix integration.

    pbogdan committed Apr 21, 2017
    Copy the full SHA
    64bfb5c View commit details
  2. Update README.

    pbogdan committed Apr 21, 2017
    Copy the full SHA
    71b0019 View commit details
  3. Bump version.

    pbogdan committed Apr 21, 2017
    Copy the full SHA
    7428ae0 View commit details
  4. Make upload.sh more portable.

    pbogdan committed Apr 21, 2017
    Copy the full SHA
    05be18d View commit details

Commits on Mar 24, 2018

  1. Update to the latest LTS.

    pbogdan committed Mar 24, 2018
    Copy the full SHA
    cc5c05b View commit details
  2. Fix build.

    pbogdan committed Mar 24, 2018
    Copy the full SHA
    f932812 View commit details
  3. Merge pull request #4 from pbogdan/lts-update

    LTS update
    pbogdan authored Mar 24, 2018
    Copy the full SHA
    4b975ae View commit details

Commits on Mar 25, 2018

  1. Copy the full SHA
    3d387c1 View commit details
  2. Merge pull request #5 from pbogdan/multiple-files

    Support monitoring multiple files.
    pbogdan authored Mar 25, 2018
    Copy the full SHA
    8e451e9 View commit details
  3. Fix static builds.

    pbogdan committed Mar 25, 2018
    Copy the full SHA
    a7974c4 View commit details
  4. Bump version.

    pbogdan committed Mar 25, 2018
    Copy the full SHA
    c02f1e6 View commit details
  5. Use "set -e" in bash scripts.

    pbogdan committed Mar 25, 2018
    Copy the full SHA
    1bfd5ed View commit details

Commits on Dec 7, 2019

  1. Relax version bounds.

    pbogdan committed Dec 7, 2019
    Copy the full SHA
    122a14d View commit details
  2. Copy the full SHA
    edb9e8a View commit details
  3. Add TODO to gitignore

    pbogdan committed Dec 7, 2019
    Copy the full SHA
    20e0720 View commit details
  4. Copy the full SHA
    08cd9d8 View commit details
  5. Unbork stack builds

    pbogdan committed Dec 7, 2019
    Copy the full SHA
    37639c2 View commit details
  6. Copy the full SHA
    44247f2 View commit details

Commits on Dec 9, 2019

  1. Copy the full SHA
    7e6628e View commit details
Showing with 482 additions and 812 deletions.
  1. +5 −2 .gitignore
  2. BIN {ngx-top → }/GeoLite2-Country.mmdb
  3. 0 {ngx-top → }/LICENSE
  4. +30 −2 README.md
  5. 0 {log-parser → }/Setup.hs
  6. 0 {ngx-top → }/app/Generator.hs
  7. +8 −0 app/Main.hs
  8. +3 −0 build.sh
  9. +7 −4 docker-build.sh → build/docker-build-static.sh
  10. +24 −0 build/docker-build-ubuntu.sh
  11. 0 build/static/.gitkeep
  12. 0 build/ubuntu-12.04/.gitkeep
  13. 0 build/ubuntu-14.04/.gitkeep
  14. 0 build/ubuntu-16.04/.gitkeep
  15. +48 −0 default.nix
  16. +12 −4 docker-compose.yml
  17. +0 −30 log-parser/LICENSE
  18. +0 −3 log-parser/README.md
  19. +0 −26 log-parser/bench/Main.hs
  20. +0 −78 log-parser/log-parser.cabal
  21. +0 −61 log-parser/src/Log/Nginx/Combined.hs
  22. +0 −119 log-parser/src/Log/Nginx/Common.hs
  23. +0 −23 log-parser/src/Log/Nginx/Detect.hs
  24. +0 −79 log-parser/src/Log/Nginx/Gateway.hs
  25. +0 −33 log-parser/src/Log/Nginx/Pretty.hs
  26. +0 −46 log-parser/src/Log/Nginx/Types.hs
  27. +0 −99 log-parser/test/Nginx/CombinedSpec.hs
  28. +0 −109 log-parser/test/Nginx/GatewaySpec.hs
  29. +0 −1 log-parser/test/Spec.hs
  30. +25 −25 {ngx-top → }/ngx-top.cabal
  31. +26 −0 ngx-top.nix
  32. +0 −3 ngx-top/README.md
  33. +0 −2 ngx-top/Setup.hs
  34. +0 −13 ngx-top/app/Main.hs
  35. +38 −0 nix/sources.json
  36. +124 −0 nix/sources.nix
  37. +2 −0 shell.nix
  38. 0 {ngx-top → }/src/GeoIP.hs
  39. +47 −16 {ngx-top → }/src/NgxTop.hs
  40. +2 −1 {ngx-top → }/src/NgxTop/Bots.hs
  41. +7 −3 {ngx-top → }/src/NgxTop/Generator.hs
  42. +32 −23 {ngx-top → }/src/NgxTop/UI.hs
  43. 0 {ngx-top → }/src/Types.hs
  44. +13 −7 stack.yaml
  45. 0 {ngx-top → }/test/Spec.hs
  46. +29 −0 upload.sh
7 changes: 5 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -23,5 +23,8 @@ dist-*
.virthualenv
TAGS
*.ps
/build/ngx-top
/build/ngx-top-gen
ngx-top
ngx-top-gen
/result
/TODO.md
stack*.yaml.lock
File renamed without changes.
File renamed without changes.
32 changes: 30 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@

Monitor nginx access logs in real time.

![screenshot](https://raw.githubusercontent.com/pbogdan/ngx-top/master/screenshots/screenshot.png)
![screenshot](https://cloud.githubusercontent.com/assets/157610/22991025/eda61298-f3b2-11e6-8a17-b03d87dbb056.gif)

![Build status](https://travis-ci.org/pbogdan/ngx-top.svg?branch=master)

@@ -29,7 +29,7 @@ Statically and dynamically linked binaries are now available via Releases sectio
## Usage

```
$ ngx-top logfile
$ ngx-top logfile [logfile...]
```

Currently ngx-top supports log files in the default combined format and a custom gateway format. At this time only the custom format contains all the required information to enable full current feature set.
@@ -47,12 +47,40 @@ This will generate approximately 100 lines per second and write them to `logfile

And in another terminal window:


```
stack exec -- ngx-top logfile
```

`stack exec` invocations above can be replaced with usage of binaries in the build folder if docker-compose build method was used.

### Usage as haskell library

`ngx-top` can also be used as a Haskell library which allows supplying custom parsers without the need to making changes to `ngx-top` itself. The simplest example which replicates the default behaviour would look as follows:

```haskell
import NgxTop

main = defaultMain defaultSettings
```

If you wish to inject custom parsers you would also need to depend on the [log-parser](https://github.com/pbogdan/log-parsessr) package. The first step is to write a parser function of the following type:

```haskell
import Log.Nginx.Types

myParser :: ByteString -> Either Text AccessLogEntry
```

which is expected to process a single line of input and return either an error or `AccessLogEntry` record. The next and final step is to supply it to `defaultMain` function:

```haskell

import qualified Data.List.NonEmpty as NE

main = defaultMain Settings { parsers = NE.fromList [myParser] }
```

## Performance

Using Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz ngx-top can currently process approximately 17000 log lines per second. Memory usage will vary depending on the log file, no extensive testing has been performed yet to determine accurate figures.
File renamed without changes.
File renamed without changes.
8 changes: 8 additions & 0 deletions app/Main.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
module Main where

import Protolude

import NgxTop

main :: IO ()
main = defaultMain defaultSettings
3 changes: 3 additions & 0 deletions build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env bash

docker-compose up --no-recreate
11 changes: 7 additions & 4 deletions docker-build.sh → build/docker-build-static.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/bin/bash
set -e
cp -r /src/ .
rm -r **/*/.stack-work
apk update
@@ -11,11 +12,13 @@ apk add alpine-sdk \
cabal stack \
ncurses-dev \
ncurses-static
stack upgrade
export PATH=$HOME/.local/bin:$PATH
[ ! -f /usr/local/bin/upx ] && wget https://github.com/lalyos/docker-upx/releases/download/v3.91/upx \
-O /usr/local/bin/upx && \
chmod +x /usr/local/bin/upx
cd /tmp/src/ngx-top && \
stack install --only-dependencies --system-ghc && \
stack --local-bin-path /build/ install --test --system-ghc \
--ghc-options '-optl-static' && \
upx -q /build/ngx-top
stack install --only-dependencies --system-ghc --skip-ghc-check && \
stack --local-bin-path /build/static/ install --test --skip-ghc-check \
--system-ghc --ghc-options '-optl-static' && \
upx -q /build/static/ngx-top
24 changes: 24 additions & 0 deletions build/docker-build-ubuntu.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash
set -e
cp -r /src/ .
rm -r **/*/.stack-work
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get -y install software-properties-common
add-apt-repository ppa:hvr/ghc
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get -y install git \
ca-certificates \
zlib1g-dev \
libtinfo-dev \
wget \
ghc-8.6.5 \
cabal-install-2.4
[ ! -f /usr/local/bin/stack ] && wget -qO- https://get.haskellstack.org/ | sh
[ ! -f /usr/local/bin/upx ] && wget https://github.com/lalyos/docker-upx/releases/download/v3.91/upx \
-O /usr/local/bin/upx && \
chmod +x /usr/local/bin/upx
export PATH=/opt/ghc/bin:$PATH
cd /tmp/src/ngx-top && \
stack install --only-dependencies --system-ghc && \
stack --local-bin-path /build/ubuntu-$1 install --test --system-ghc && \
upx -q /build/ubuntu-$1/ngx-top
Empty file added build/static/.gitkeep
Empty file.
Empty file added build/ubuntu-12.04/.gitkeep
Empty file.
Empty file added build/ubuntu-14.04/.gitkeep
Empty file.
Empty file added build/ubuntu-16.04/.gitkeep
Empty file.
48 changes: 48 additions & 0 deletions default.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
let
fix = f: let x = f x; in x;
in
{ sources ? fix (
self:
(import ./nix/sources.nix) // (if static then { nixpkgs = self.nixpkgs-haskell-static; } else {})
)
, compiler ? "ghc865"
, static ? false
}:
let
overlay = self: super: {
haskell = super.haskell // {
packages = super.haskell.packages // {
"${compiler}" = super.haskell.packages.${compiler}.override {
overrides = hself: hsuper: {
log-parser = import sources.log-parser {
inherit sources compiler;
};
};
};
};
};
};
pkgs = (
import sources.nixpkgs {
overlays = [
overlay
];
}
).${if static then "pkgsMusl" else "pkgs"};
ngx-top = pkgs.haskell.packages.${compiler}.callPackage ./ngx-top.nix {};
in
pkgs.haskell.lib.overrideCabal ngx-top (
drv:
if static then {
configureFlags = drv.configureFlags or [] ++ [
"--ghc-option=-optl=-static"
"--extra-lib-dirs=${pkgs.gmp6.override { withStatic = true; }}/lib"
"--extra-lib-dirs=${pkgs.ncurses.override { enableStatic = true; }}/lib"
"--extra-lib-dirs=${pkgs.zlib.static}/lib"
"--extra-lib-dirs=${pkgs.libffi.overrideAttrs (old: { dontDisableStatic = true; })}/lib"
];
enableSharedExecutables = false;
enableSharedLibraries = false;
}
else {}
)
16 changes: 12 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
ngx-top-build:
image: mitchty/alpine-ghc:latest
container_name: ngx-top-build-container # to have fixed name
ngx-top-build-ubuntu-16.04:
image: ubuntu:16.04
container_name: ngx-top-build-container-ubuntu-16.04 # to have fixed name
volumes:
- ./build:/build
- .:/src/ngx-top
working_dir: /tmp
command: sh /src/ngx-top/docker-build.sh
command: sh /src/ngx-top/build/docker-build-ubuntu.sh 16.04
ngx-top-build-ubuntu-18.04:
image: ubuntu:18.04
container_name: ngx-top-build-container-ubuntu-18.04 # to have fixed name
volumes:
- ./build:/build
- .:/src/ngx-top
working_dir: /tmp
command: sh /src/ngx-top/build/docker-build-ubuntu.sh 18.04
30 changes: 0 additions & 30 deletions log-parser/LICENSE

This file was deleted.

3 changes: 0 additions & 3 deletions log-parser/README.md

This file was deleted.

26 changes: 0 additions & 26 deletions log-parser/bench/Main.hs

This file was deleted.

78 changes: 0 additions & 78 deletions log-parser/log-parser.cabal

This file was deleted.

Loading