where is the SAM model service you call deployed？

[Tongzhenguo]

where is the SAM model service you call deployed？

[wkkuniquegmail]

in this file meta-sam-demo/src/components/helpers/modelAPI.tsx,
const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model";

when i changed it to my server, like
const API_ENDPOINT = "http://localhost:8000/";

it can work, but the performance is not as good as the official one.

so "https://model-zoo.metademolab.com/predictions/segment_everything_box_model" use this https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth or not?

[Stinosko]

From what i understand the model used in this project is a quantized ONNX model like the explain in the segment-anything github?

You use a server like confidential ONNX inference server to host the ONNX file?

Does the ONNX server run the model for you or does it serve the ONNX model file and runs it on your local computer?

[MiscellaneousStuff]

The quantized ONNX files included in this repository are used for decoding the masks from the image embedding. As correctly explained above, const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model"; is the API endpoint which computes the Image Embedding using the SAM model which is being hosted by Meta.

[Stinosko]

I'm new to the ONNX files, can you point me to some documentation how to serve the provided .onnx files? 🙂

[MiscellaneousStuff]

The two ONNX files which are included with this repository are included inside of the repository and are served using a WASM (Web Assembly) engine on your browser. This is done through onnxruntime-web. So in other words, you don't have to do anything, it will automatically just work when you run it :)

[onefish51]

in this file meta-sam-demo/src/components/helpers/modelAPI.tsx, const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model";

when i changed it to my server, like const API_ENDPOINT = "http://localhost:8000/";

it can work, but the performance is not as good as the official one.

so "https://model-zoo.metademolab.com/predictions/segment_everything_box_model" use this https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth or not?

@wkkuniquegmail @MiscellaneousStuff I got the same result when I changed the image embedding server from const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model" to const API_ENDPOINT = "http://localhost:8000/" (https://github.com/kevmo314/magic-copy/tree/main/server-example)

• input image:
  
• const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model":
  
• const API_ENDPOINT = "http://localhost:8000/":
  

I also run the GitHub JS demo(https://github.com/facebookresearch/segment-anything/tree/main/demo)，
embedding server use http://localhost:8000, onnx model not "./interactive_module_quantized_592547_2023_03_19_sam6_long_uncertain.onnx" because it not work , so I used sam_onnx_quantized_example.onnx

point_coords: Coordinates of sparse input prompts, corresponding to both point inputs and box inputs. Boxes are encoded using two points, one for the top-left corner and one for the bottom-right corner. Coordinates must already be transformed to long-side 1024.

In my opinion , input image resize before embedding and onnx model const MODEL_DIR = "./interactive_module_quantized_592547_2023_03_19_sam6_long_uncertain.onnx" post-processing may bring it！

[sun631998316]

in this file meta-sam-demo/src/components/helpers/modelAPI.tsx, const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model";
when i changed it to my server, like const API_ENDPOINT = "http://localhost:8000/";
it can work, but the performance is not as good as the official one.
so "https://model-zoo.metademolab.com/predictions/segment_everything_box_model" use this https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth or not?

@wkkuniquegmail @MiscellaneousStuff I got the same result when I changed the image embedding server from const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model" to const API_ENDPOINT = "http://localhost:8000/" (https://github.com/kevmo314/magic-copy/tree/main/server-example)

• input image:
  
• const API_ENDPOINT = "https://model-zoo.metademolab.com/predictions/segment_everything_box_model":
  
• const API_ENDPOINT = "http://localhost:8000/":
  

I also run the GitHub JS demo(https://github.com/facebookresearch/segment-anything/tree/main/demo)，%EF%BC%8C) embedding server use http://localhost:8000, onnx model not "./interactive_module_quantized_592547_2023_03_19_sam6_long_uncertain.onnx" because it not work , so I used sam_onnx_quantized_example.onnx

point_coords: Coordinates of sparse input prompts, corresponding to both point inputs and box inputs. Boxes are encoded using two points, one for the top-left corner and one for the bottom-right corner. Coordinates must already be transformed to long-side 1024.

In my opinion , input image resize before embedding and onnx model const MODEL_DIR = "./interactive_module_quantized_592547_2023_03_19_sam6_long_uncertain.onnx" post-processing may bring it！

Have you found a solution? I use the vit_h model deployed by myself to extract the image embedding. The segmentation accuracy is very low.

[MiscellaneousStuff]

I haven't tried this repo with a self deployed model, but from what I can tell resizing the input image to the 1024x1024 expected by the model might help performance? Sorry this isn't very helpful but I'm not exactly sure what pre-processing the model zoo endpoint uses for computing the Image Embedding.

[onefish51]

@sun631998316 I rewrited the code base https://github.com/facebookresearch/segment-anything/tree/main/demo , it work well !

And you don't need to do anything before image embedding , it will transform the input image to long-side 1024 in set_image function

[haibingtown]

I have personally exported an ONNX model and conducted a successful test using the image mentioned above。

This project (https://github.com/haibingtown/segment-matting) provides a complete demo.