Deprecate Coral TPU suggestion from documentation as upstream dependencies are abandoned #10056
Comments
|
Could you give some alternatives at a similar price point? |
|
@apalrd Does this apply to the USB based TPU also ? |
|
I would appreciate any links you can provide for these claims. A significant number of Frigate users are using a Coral device, so we will support it for as long as possible. This includes running older versions of tflite-runtime as necessary. While I do see the gasket driver issues and find the lack of updates concerning, I have yet to see any official statement that the devices have been abandoned. If you have seen this, please provide a link or source for that claim. Also, the gasket driver issues do not apply to the USB Coral which does not depend on the Gasket driver. Is there any public discussion to document "Google developers have not kept it up to date or made any of the changes required to merge it into the kernel"? From my vantage point, it looks like in the future many devices will have support for accelerated AI detection via the processor or an onboard NPU. This project will evolve and adapt based on what happens. |
|
I'm not suggesting we remove support, only suggesting we stop recommending new users to buy the hardware, because of the evidence that the hardware vendor has abandoned their drivers for some time. I've been trying to get in contact with anyone at coral.ai to get the community-produced Gasket fixes into their Deb repo (which Frigate relies on for pycoral, but users of PCIe devices also rely on for the DKMS module). It looks like the users who used to respond to Github issues in the edgetpu repos no longer work there (their Google emails bounce with 'Address not found'). The InRelease file on the deb repo was last updated on 2022/03/01 (yes, 2 years ago). This file contains the signatures of lower-level files and is GPG signed when any update is made to the repo, so no packages have been pushed in at least 2 years. This repository is in the official coral.ai instructions. We can find the versions of packages distributed in this repo in the Packages file. Frigate uses python3-tflite-runtime and python3-pycoral directly from this repo, so we are relying on these libraries for edgetpu but also for the CPU detector. The current tflite version in that repo is 2.5.0, while the current version in PyPi is 2.14.0. 2.5.0 was released in PyPi on 2021/10/11. So Frigate is currently using a version of tflite that is pinned at October 2021, including for the CPU detector. To make it even worse, this package depends on Python << 3.10, and no wheels have been built + they do not compile for more recent Python versions, so this sticks Frigate to Python 3.9. Python 3.9 has already reached end of full support, but is still receiving security updates for 18 months. So we have 18 months to figure out what to do with edgetpu. While it should be possible to build pycoral from source to get around the Python 3.9 / ancient tflite issue, the code has not been updated and this results in a segmentation fault. @feranick was able to fork all of the repos involved and update them, to support more recent versions of tflite and therefore Python - see google-coral/edgetpu#812 for details on that. He has an open PR to the Google repo, but it's gone unanswered for some time now, and it appears that they have never merged a PR to any of their google-coral Github repositories ever (as far as I can see). They have been pinging the previous maintainers with no luck in getting a response. The TensorFlow team treated the issue seriously, but it looks like they've also been unable to get in contact with the google-coral repository owners in the past month (despite being the 'same' company). Also due to their bad stewardship of the repos, they created a Python package that isn't in PyPi (pycoral), distributed it via deb packages, and there is now a 'real' pycoral in PyPi which is completely different. Oops. As for the Gasket driver, it was originally in kernel staging, but in 2021 was pushed out by Greg KH due to his view of it being 'abandoned', as Google never pushed any of the fixes required to get it to merge and it stopped compiling in staging. After that, Google maintained it on github as an out of tree module, which has received only minimal updates since 2021. Notably, two contributors (not Google affiliated) were able to resolve build issues on Linux >=6.4 and specific backported kernel compile errors, and these PRs were merged, but the resulting code was never released as a deb package update. It has been 6 months since the fix to Linux >= 6.4 was merged by the community and the deb packages have not been updated, so all of the installation instructions on Coral's site no longer work. Now that Ubuntu has shipped kernel 6.5, this has gotten a lot more attention, and the gasket / edgetpu Github repos are flooded with 'Gasket fails to compile' issues from users. Proxmox tracks Ubuntu kernels, so their users are affected as well. I wrote a Proxmox+Frigate tutorial including the m.2 TPU and I'm getting emails about it not working as well. Again, I'm not suggesting we drop code to support edgetpu, only remove it's recommendation from the documentation. We are slowly getting more alternatives (like OpenVino and TensorRT) to the Coral TPU, so new users have options. |
|
Thank you for the mention. Just wanted to provide some additional info
that might be helpful, even if in limited capacity.
1. While, of course, official support for the various libraries is
highly preferred, as I rely on the edgetpu myself for my research I took
the matter in my own hands. I created two repos (for the libedgetpu
<https://github.com/feranick/libedgetpu> and tflite_runtime
<https://github.com/feranick/TFlite-builds> libraries) with binaries for
all recent and supported python versions (3.9, 3.10, 3.11), for TF
v.2.15.0, for Linux/debian and MacOS. I don't use Windows, so no support
there, although, I tried to update libraries as needed there as well, so
one should be able to compile/build them themselves. For those using the
M.2 edgetpu, in needs of updated gasket-dkms for Linux kernels > 6.5, I
made an updated deb package in here
<https://github.com/feranick/gasket-driver/releases/tag/v1.0-18.1>.
These three libraries have been tested and work as expected for models
created in TF proper and converted to TFlite, which is my main
workstream. I plan to keep them in sync with future TF releases (I have
pre-releases versions of tflite_runtime v2.16.0), and python.
2. For libcoral/pycoral the situation is much more complicated. While I
have forked the repo, as of now, I am still battling with what basically
account as incompatible code on both ends (TF and libcoral). I
personally don't use it, but would like to have it operational. An
incompatibility bug was just fixed in 2.16 that will make life easier in
compiling libcoral (which is the backend needed by pycoral) with TF.
3. I have been complaining with the sales team (which apparently is
operated by ASUS) and with significant reluctance, they admitted to
technical support challenges in a ever changing team at Google. Yet, I
have been told that support is supposedly coming back, although not
immediately (2-3 months). Take it as you like, i.e. don't bet on it.
Once again, what I made available suits my needs perfectly because I
don't use pycoral (and never found a edgetpu-specific library to be
particularly useful, especially when mindful of google's mercurial
attitude towards product support, and therefore I relied on TF proper
for model training, conversion etc).
If you find any issue with my repos/binaries please file issues there.
…On 2/27/24 9:29 AM, Andrew wrote:
I'm not suggesting we remove support, only suggesting we stop
recommending new users to buy the hardware, because of the evidence
that the hardware vendor has abandoned their drivers for some time.
I've been trying to get in contact with anyone at coral.ai to get the
community-produced Gasket fixes into their Deb repo (which Frigate
relies on for pycoral, but users of PCIe devices also rely on for the
DKMS module). It looks like the users who used to respond to Github
issues in the edgetpu repos no longer work there (their Google emails
bounce with 'Address not found').
The InRelease file on the deb repo
<https://packages.cloud.google.com/apt/dists/coral-edgetpu-stable/InRelease>
was last updated on 2022/03/01 (yes, 2 years ago). This file contains
the signatures of lower-level files and is GPG signed when any update
is made to the repo, so no packages have been pushed in at least 2
years. This repository is in the official coral.ai instructions.
We can find the versions of packages distributed in this repo in the
Packages file
<https://packages.cloud.google.com/apt/dists/coral-edgetpu-stable/main/binary-amd64/Packages>.
Frigate uses python3-tflite-runtime and python3-pycoral directly from
this repo, so we are relying on these libraries for edgetpu but also
for the CPU detector. The current tflite version in that repo is
2.5.0, while the current version in PyPi
<https://pypi.org/project/tflite-runtime/>is 2.14.0. 2.5.0 was
released in PyPi on 2021/10/11. So Frigate is currently using a
version of tflite that is pinned at October 2021, including for the
CPU detector. To make it even worse, this package depends on Python <<
3.10, and no wheels have been built + they do not compile for more
recent Python versions, so this sticks Frigate to Python 3.9. Python
3.9 has already reached end of full support, but is still receiving
security updates for 18 months. So we have 18 months to figure out
what to do with edgetpu.
While it should be possible to build pycoral from source to get around
the Python 3.9 / ancient tflite issue, the code has not been updated
and this results in a segmentation fault. @feranick
<https://github.com/feranick> was able to fork all of the repos
involved and update them, to support more recent versions of tflite
and therefore Python - see google-coral/edgetpu#812
<google-coral/edgetpu#812> for details on
that. He has an open PR to the Google repo, but it's gone unanswered
for some time now, and it appears that they have never merged a PR to
any of their google-coral Github repositories ever (as far as I can
see). They have been pinging the previous maintainers with no luck in
getting a response. The TensorFlow team treated the issue seriously
<tensorflow/tensorflow#62371>, but it looks
like they've also been unable to get in contact with the google-coral
repository owners in the past month (despite being the 'same' company).
Also due to their bad stewardship of the repos, they created a Python
package that isn't in PyPi (pycoral), distributed it via deb packages,
and there is now a 'real' pycoral <https://pypi.org/project/pycoral/>
in PyPi which is completely different. Oops.
As for the Gasket driver, it was originally in kernel staging, but in
2021 was pushed out by Greg KH
<https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-5.15.y&id=918ce05bbe52df43849a803010b4d2bcd31ea69c>
due to his view of it being 'abandoned', as Google never pushed any of
the fixes required to get it to merge and it stopped compiling in
staging. After that, Google maintained it on github as an out of tree
module, which has received only minimal updates since 2021. Notably,
two contributors (not Google affiliated) were able to resolve build
issues on Linux >=6.4 and specific backported kernel compile errors,
and these PRs were merged, but the resulting code was never released
as a deb package update. It has been 6 months since the fix to Linux
>= 6.4 was merged by the community and the deb packages have not been
updated, so all of the installation instructions on Coral's site no
longer work. Now that Ubuntu has shipped kernel 6.5, this has gotten a
lot more attention, and the gasket / edgetpu Github repos are flooded
with 'Gasket fails to compile' issues from users. Proxmox tracks
Ubuntu kernels, so their users are affected as well. I wrote a
Proxmox+Frigate tutorial including the m.2 TPU and I'm getting emails
about it not working as well.
Again, I'm not suggesting we drop code to support edgetpu, only remove
it's recommendation from the documentation. We are slowly getting more
alternatives (like OpenVino and TensorRT) to the Coral TPU, so new
users have options.
—
Reply to this email directly, view it on GitHub
<#10056 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAIY76X4F6DVH4CGZN7ZSLYVXUTBAVCNFSM6AAAAABD2PK5LSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWGY3TMNJZGI>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--------------TYE0Kh7wvtsJ99ka6troMJEd
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
Thank you for the mention. Just wanted to provide some additional
info that might be helpful, even if in limited capacity. <br>
<br>
1. While, of course, official support for the various libraries is
highly preferred, as I rely on the edgetpu myself for my research I
took the matter in my own hands. I created two repos (for the <a href="https://github.com/feranick/libedgetpu">libedgetpu</a> and <a href="https://github.com/feranick/TFlite-builds">tflite_runtime</a>
libraries) with binaries for all recent and supported python
versions (3.9, 3.10, 3.11), for TF v.2.15.0, for Linux/debian and
MacOS. I don't use Windows, so no support there, although, I tried
to update libraries as needed there as well, so one should be able
to compile/build them themselves. For those using the M.2 edgetpu,
in needs of updated gasket-dkms for Linux kernels > 6.5, I made
an <a href="https://github.com/feranick/gasket-driver/releases/tag/v1.0-18.1">updated
deb package in here</a>. These three libraries have been tested
and work as expected for models created in TF proper and converted
to TFlite, which is my main workstream. I plan to keep them in sync
with future TF releases (I have pre-releases versions of
tflite_runtime v2.16.0), and python. <br>
<br>
2. For libcoral/pycoral the situation is much more complicated.
While I have forked the repo, as of now, I am still battling with
what basically account as incompatible code on both ends (TF and
libcoral). I personally don't use it, but would like to have it
operational. An incompatibility bug was just fixed in 2.16 that will
make life easier in compiling libcoral (which is the backend needed
by pycoral) with TF. <br>
<br>
3. I have been complaining with the sales team (which apparently is
operated by ASUS) and with significant reluctance, they admitted to
technical support challenges in a ever changing team at Google. Yet,
I have been told that support is supposedly coming back, although
not immediately (2-3 months). Take it as you like, i.e. don't bet on
it. <br>
<br>
Once again, what I made available suits my needs perfectly because I
don't use pycoral (and never found a edgetpu-specific library to be
particularly useful, especially when mindful of google's mercurial
attitude towards product support, and therefore I relied on TF
proper for model training, conversion etc). <br>
<br>
If you find any issue with my repos/binaries please file issues
there. <br>
<br>
<div class="moz-cite-prefix">On 2/27/24 9:29 AM, Andrew wrote:<br>
</div>
<blockquote type="cite" ***@***.***">
<p dir="auto">I'm not suggesting we remove support, only
suggesting we stop recommending new users to buy the hardware,
because of the evidence that the hardware vendor has abandoned
their drivers for some time.</p>
<p dir="auto">I've been trying to get in contact with anyone at
coral.ai to get the community-produced Gasket fixes into their
Deb repo (which Frigate relies on for pycoral, but users of PCIe
devices also rely on for the DKMS module). It looks like the
users who used to respond to Github issues in the edgetpu repos
no longer work there (their Google emails bounce with 'Address
not found').</p>
<p dir="auto">The <a href="https://packages.cloud.google.com/apt/dists/coral-edgetpu-stable/InRelease" rel="nofollow" moz-do-not-send="true">InRelease file on the
deb repo</a> was last updated on 2022/03/01 (yes, 2 years
ago). This file contains the signatures of lower-level files and
is GPG signed when any update is made to the repo, so no
packages have been pushed in at least 2 years. This repository
is in the official coral.ai instructions.</p>
<p dir="auto">We can find the versions of packages distributed in
this repo in the <a href="https://packages.cloud.google.com/apt/dists/coral-edgetpu-stable/main/binary-amd64/Packages" rel="nofollow" moz-do-not-send="true">Packages file</a>.
Frigate uses python3-tflite-runtime and python3-pycoral directly
from this repo, so we are relying on these libraries for edgetpu
but also for the CPU detector. The current tflite version in
that repo is 2.5.0, while the <a href="https://pypi.org/project/tflite-runtime/" rel="nofollow" moz-do-not-send="true">current version in PyPi </a>is 2.14.0.
2.5.0 was released in PyPi on 2021/10/11. So Frigate is
currently using a version of tflite that is pinned at October
2021, including for the CPU detector. To make it even worse,
this package depends on Python << 3.10, and no wheels have
been built + they do not compile for more recent Python
versions, so this sticks Frigate to Python 3.9. Python 3.9 has
already reached end of full support, but is still receiving
security updates for 18 months. So we have 18 months to figure
out what to do with edgetpu.</p>
<p dir="auto">While it should be possible to build pycoral from
source to get around the Python 3.9 / ancient tflite issue, the
code has not been updated and this results in a segmentation
fault. <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/feranick/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/feranick" ***@***.***</a>
was able to fork all of the repos involved and update them, to
support more recent versions of tflite and therefore Python -
see <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2033320107" data-permission-text="Title is private" data-url="google-coral/edgetpu#812" data-hovercard-type="issue" data-hovercard-url="/google-coral/edgetpu/issues/812/hovercard" href="google-coral/edgetpu#812" moz-do-not-send="true">google-coral/edgetpu#812</a> for
details on that. He has an open PR to the Google repo, but it's
gone unanswered for some time now, and it appears that they have
never merged a PR to any of their google-coral Github
repositories ever (as far as I can see). They have been pinging
the previous maintainers with no luck in getting a response. The
<a href="tensorflow/tensorflow#62371" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/62371/hovercard" moz-do-not-send="true">TensorFlow team treated the issue
seriously</a>, but it looks like they've also been unable to
get in contact with the google-coral repository owners in the
past month (despite being the 'same' company).</p>
<p dir="auto">Also due to their bad stewardship of the repos, they
created a Python package that isn't in PyPi (pycoral),
distributed it via deb packages, and there is <a href="https://pypi.org/project/pycoral/" rel="nofollow" moz-do-not-send="true">now a 'real' pycoral</a> in PyPi which
is completely different. Oops.</p>
<p dir="auto">As for the Gasket driver, it was originally in
kernel staging, but <a href="https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-5.15.y&id=918ce05bbe52df43849a803010b4d2bcd31ea69c" rel="nofollow" moz-do-not-send="true">in 2021 was pushed out
by Greg KH</a> due to his view of it being 'abandoned', as
Google never pushed any of the fixes required to get it to merge
and it stopped compiling in staging. After that, Google
maintained it on github as an out of tree module, which has
received only minimal updates since 2021. Notably, two
contributors (not Google affiliated) were able to resolve build
issues on Linux >=6.4 and specific backported kernel compile
errors, and these PRs were merged, but the resulting code was
never released as a deb package update. It has been 6 months
since the fix to Linux >= 6.4 was merged by the community and
the deb packages have not been updated, so all of the
installation instructions on Coral's site no longer work. Now
that Ubuntu has shipped kernel 6.5, this has gotten a lot more
attention, and the gasket / edgetpu Github repos are flooded
with 'Gasket fails to compile' issues from users. Proxmox tracks
Ubuntu kernels, so their users are affected as well. I wrote a
Proxmox+Frigate tutorial including the m.2 TPU and I'm getting
emails about it not working as well.</p>
<p dir="auto">Again, I'm not suggesting we drop code to support
edgetpu, only remove it's recommendation from the documentation.
We are slowly getting more alternatives (like OpenVino and
TensorRT) to the Coral TPU, so new users have options.</p>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br>
Reply to this email directly, <a href="#10056 (comment)" moz-do-not-send="true">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AAAIY76X4F6DVH4CGZN7ZSLYVXUTBAVCNFSM6AAAAABD2PK5LSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWGY3TMNJZGI" moz-do-not-send="true">unsubscribe</a>.<br>
You are receiving this because you were mentioned.<img src="https://github.com/notifications/beacon/AAAIY76STFSQJAFX5MGQYW3YVXUTBA5CNFSM6AAAAABD2PK5LSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTVHENHA.gif" alt="" moz-do-not-send="true" width="1" height="1"><span style="color: transparent; font-size: 0; display: none; visibility: hidden; overflow: hidden; opacity: 0; width: 0; height: 0; max-width: 0; max-height: 0; mso-hide: all">Message
ID: <span><blakeblackshear/frigate/issues/10056/1966676592</span><span>@</span><span>github</span><span>.</span><span>com></span></span></p>
<script type="application/ld+json">[
{
***@***.***": "http://schema.org",
***@***.***": "EmailMessage",
"potentialAction": {
***@***.***": "ViewAction",
"target": "#10056 (comment)",
"url": "#10056 (comment)",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
***@***.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>
</blockquote>
<br>
</body>
</html>
--------------TYE0Kh7wvtsJ99ka6troMJEd--
|
|
I think everyone using frigate wants to see Google fix coral.ai support @pkgoogle |
|
Looks like Google is getting back on top of things aka coral support which is nice |
|
Yes, good to see this got the attention it deserves. In general I think more official statements from Google or specific evidence of Google dropping support (rather than neglect) is needed to consider the google coral deprecated and remove the recommendation from the docs. For now it seems that the original devs weren't on the project and new devs needed to be given notice. Hopefully this continues and things are kept up to date. This issue can be reopened in the future if things change. |
|
For what is worth, updated Details here: |
As Coral.ai has been abandoned by Google, stop recommending people purchase the hardware in the documentation.
Specifically:
The text was updated successfully, but these errors were encountered: