Skip to content

Commit

Permalink
23 - testing onnx (#27)
Browse files Browse the repository at this point in the history
<!-- Replace {issue_number} with the issue that will be closed after
merging this PR -->
Fixes #23 

## Type of change

<!-- Write an `x` in all the boxes that apply -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] This change requires a documentation update
  • Loading branch information
mxochicale authored Nov 10, 2023
1 parent 14724a3 commit cfb59f7
Show file tree
Hide file tree
Showing 11 changed files with 1,264 additions and 17 deletions.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
## ignores temporal files
**/*.swp

## notebooks
## python related scripts
.ipynb_checkpoints/
__pycache__/

## site
.jekyll-cache/
Expand Down
59 changes: 58 additions & 1 deletion data/polyp-dataset/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,66 @@ rm kvasir-seg.zip
## Converting images to video
```
cd data/polyp-dataset/Kvasir-SEG/images
ffmpeg -framerate 1 -pattern_type glob -i 'Kvasir-SEG/images/ck*.jpg' -c:v libx264 out.mp4
ffmpeg -framerate 1 -pattern_type glob -i 'Kvasir-SEG/images/ck*.jpg' -c:v libx264 out720x576.mp4
```



## Using `convert_video_to_gxf_entities.py` (Graph Execution Framework, GXF)
```
mamba activate cagxVE
#ffmpeg -i out.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python convert_video_to_gxf_entities.py --width 1920 --height 1080 --channels 3 --framerate 1 --basename out
#ffmpeg -i out.mp4 -pix_fmt yuvj422p -f rawvideo pipe:1 | python convert_video_to_gxf_entities.py --width 600 --height 500 --channels 3 --framerate 1 --basename out
ffmpeg -i out720x576.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python ../../nvidia-clara-agx/holoscan-sdk-scripts/convert_video_to_gxf_entities.py --width 720 --height 576 --channels 3 --framerate 1 --basename out720x576
```
https://github.com/nvidia-holoscan/holoscan-sdk/tree/main/scripts#convert_video_to_gxf_entitiespy
https://docs.nvidia.com/holoscan/sdk-user-guide/gxf/gxf_by_example.html
https://docs.nvidia.com/clara-holoscan/archive/clara-holoscan-0.3.0/gxf/index.html


## Coping files to clara-agx
```
cd /workspace/holohub/data/colonoscopy_segmentation
cp ../../cmicHACKS2/data/polyp-dataset/out720x576.mp4 .
cp ../../cmicHACKS2/data/polyp-dataset/out720x576.gxf_* .
```




## Testing `raw.mp4` with `convert_video_to_gxf_entities.py`
```
mamba activate cagxVE
wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/team/clara-holoscan/holoscan_endoscopy_sample_data/20221121/files?redirect=true&path=video/raw.mp4' -O raw.mp4
wget --content-disposition 'https://api.ngc.nvidia.com/v2/resources/org/nvidia/team/clara-holoscan/holoscan_colonoscopy_sample_data/20230222/files?redirect=true&path=colon_exam_720x576.mp4' -O colon_exam_720x576.mp4
ffmpeg -i raw.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python scripts/convert_video_to_gxf_entities.py --width 1920 --height 1080 --channels 3 --framerate 30 --basename raw
ffmpeg -i colon_exam_720x576.mp4 -pix_fmt rgb24 -f rawvideo pipe:1 | python scripts/convert_video_to_gxf_entities.py --width 720 --height 576 --channels 3 --framerate 30 --basename colon_exam_720x576
```

errors

```
av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:1: Broken pipeime=00:00:00.04 bitrate=248832.0kbits/s speed=1.83x
frame= 1 fps=0.0 q=-0.0 Lsize= 1215kB time=00:00:00.04 bitrate=248832.0kbits/s speed=1.22x
video:1215kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Conversion failed!
```

https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara-holoscan/resources/holoscan_endoscopy_sample_data/files?version=20221121



## References
https://stackoverflow.com/questions/22965569/convert-from-jpg-to-mp4-by-ffmpeg
https://docs.nvidia.com/holoscan/sdk-user-guide/gxf/gxf_by_example.html



Binary file added data/polyp-dataset/out720x576.gxf_entities
Binary file not shown.
Binary file added data/polyp-dataset/out720x576.gxf_index
Binary file not shown.
File renamed without changes.
27 changes: 19 additions & 8 deletions nvidia-clara-agx/colonoscopy_segmentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,27 @@

* Run container
```
cd holohub
cd ~/holohub
./dev_container launch --ssh_x11
```

* Copy data
```
cd /workspace/holohub/data/colonoscopy_segmentation
cp ../../cmicHACKS2/data/models/ColonSegNet-07112023-2359.onnx .
cp ../../cmicHACKS2/data/polyp-dataset/out720x576.mp4 .
cp ../../cmicHACKS2/data/polyp-dataset/out720x576.gxf_* .
```


* Running in our own repository
```
cd /workspace/holohub/cmicHACKS2/nvidia-clara-agx/colonoscopy_segmentation
python colonoscopy_segmentation.py --data /workspace/holohub/data/colonoscopy_segmentation
```



* Build application (mainly download data)
```
./run build colonoscopy_segmentation
Expand All @@ -26,11 +43,7 @@ cd /workspace/holohub/applications/colonoscopy_segmentation/
python colonoscopy_segmentation.py --data /workspace/holohub/data/colonoscopy_segmentation
```

in our own repository
```
cd /workspace/holohub/nvdia-clara-agx/examples/colonoscopy_segmentation
python colonoscopy_segmentation.py --data /workspace/holohub/data/colonoscopy_segmentation
```



* log
Expand Down Expand Up @@ -174,8 +187,6 @@ NOTE: This could take a couple of minutes depending on your model size and GPU!
```

![fig](../../docs/figures/colonoscopy_segmentation.png)

## Optmise model

```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
---
replayer: # VideoStreamReplayer
#basename: "colon_exam_720x576"
basename: "out"
basename: "out720x576"
frame_rate: 0 # as specified in timestamps
repeat: true # default: false
realtime: true # default: true
Expand Down
11 changes: 11 additions & 0 deletions nvidia-clara-agx/holoscan-sdk-scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Holoscan utility scripts

We have few examples on how to use `convert_video_to_gxf_entities.py` [here](../../data/polyp-dataset/README.md).
For further details see references.

## References
https://github.com/nvidia-holoscan/holoscan-sdk/tree/main/scripts#convert_video_to_gxf_entitiespy
https://docs.nvidia.com/holoscan/sdk-user-guide/gxf/gxf_by_example.html
https://docs.nvidia.com/clara-holoscan/archive/clara-holoscan-0.3.0/gxf/index.html


Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import argparse
import sys

import numpy as np
from gxf_entity_codec import EntityWriter


def iter_input_frames(f, width, height, channels):
frame_byte_count = width * height * channels
while True:
frame_bytes = f.read(frame_byte_count)
if len(frame_bytes) != frame_byte_count:
break
array = np.frombuffer(frame_bytes, np.uint8)
frame = array.reshape(height, width, channels)
yield frame


def main():
parser = argparse.ArgumentParser(
description=(
"Command line utility for writing raw video frames in GXF Tensors for "
"stream playback."
)
)
parser.add_argument("--width", required=True, type=int, help="Input width in pixels")
parser.add_argument("--height", required=True, type=int, help="Input height in pixels")
parser.add_argument("--channels", default=3, type=int, help="Channel count")
parser.add_argument("--framerate", default=30, type=int, help="Output frame rate")
parser.add_argument("--basename", default="tensor", help="Basename for gxf entities to write")
parser.add_argument("--directory", default="./", help="Directory for gxf entities to write")
args = parser.parse_args()

with EntityWriter(
directory=args.directory, basename=args.basename, framerate=args.framerate
) as recorder:
for frame in iter_input_frames(
sys.stdin.buffer,
width=args.width,
height=args.height,
channels=args.channels,
):
recorder.add(frame)


if __name__ == "__main__":
main()
Loading

0 comments on commit cfb59f7

Please sign in to comment.