clickhouse datasource plugin.unavailable

[xiaoyuan2019]

• grafana logs：
  logger=context userId=2 orgId=1 uname=wpsadmin t=2024-07-16T06:52:53.367321067Z level=error msg="Request Completed" method=GET path=/api/datasources/uid/cdrwjw6m3vda8a/health status=500 remote_addr=10.1.1.1 time_ms=1 duration=1.148295ms size=95 referer=https://x.x.x.x/grafana/connections/datasources/edit/cdrwjw6m3vda8a handler=/api/datasources/uid/:uid/health status_source=server
• clickhouse datasource
  
  
• F12 error：
  

[SpencerTorres]

Hello, thanks for submitting an issue.

Do you have any more logs from the grafana process? I see you pasted the HTTP request log, but there should be more details about the error on a different line.

It seems like the plugin process may be failing to start. How are you running Grafana? Which version of Grafana are you running?

[xiaoyuan2019]

@SpencerTorres Grafana is started using docker-compose, with version v10.4.1. Additionally, when editing the ClickHouse data source in Grafana, there are no more logs available, only that one error log.

[SpencerTorres]

Perhaps you could increase the log level on the grafana deployment to debug? Are you able to update to grafana v11?

[xiaoyuan2019]

@SpencerTorres grafana debug logs：
logger=accesscontrol.evaluator t=2024-07-18T07:22:52.176179059Z level=debug msg="Matched scope" userscope=datasources:* targetscope=datasources:uid:eds3rfholibk0a
logger=datasources t=2024-07-18T07:22:52.176326787Z level=debug msg="Received command to update data source" url=
logger=datasource t=2024-07-18T07:22:52.176346042Z level=debug msg="Applying default URL parsing for this data source type" type=grafana-clickhouse-datasource url=
logger=datasource t=2024-07-18T07:22:52.17635266Z level=debug msg="Data source URL doesn't specify protocol, so prepending it with http:// in order to make it unambiguous" type=grafana-clickhouse-datasource url=
logger=sqlstore.session t=2024-07-18T07:22:52.177069138Z level=debug msg="reusing existing session" transaction=true
logger=sqlstore.session t=2024-07-18T07:22:52.177672094Z level=debug msg="reusing existing session" transaction=true
logger=secrets.kvstore t=2024-07-18T07:22:52.185152866Z level=debug msg="got secret value" orgId=1 type=datasource namespace=grafana-clickhouse-datasource
logger=sqlstore.session t=2024-07-18T07:22:52.185194628Z level=debug msg="reusing existing session" transaction=true
logger=sqlstore.session t=2024-07-18T07:22:53.438429622Z level=debug msg="reusing existing session" transaction=true
logger=secrets.kvstore t=2024-07-18T07:22:53.439235618Z level=debug msg="secret value updated" orgId=1 type=datasource namespace=grafana-clickhouse-datasource
logger=sqlstore.transactions t=2024-07-18T07:22:53.43927048Z level=debug msg="skip committing the transaction because it belongs to a session created in the outer scope"
logger=sqlstore.transactions t=2024-07-18T07:22:53.439287744Z level=debug msg="skip committing the transaction because it belongs to a session created in the outer scope"
logger=secrets.kvstore t=2024-07-18T07:22:53.516275566Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=grafana-clickhouse-datasource
logger=oauth_token.sync t=2024-07-18T07:22:53.533036664Z level=debug msg="Singleflight request for OAuth token sync" key=user:2
logger=oauthtoken t=2024-07-18T07:22:53.533081266Z level=debug msg="Expiration check has been cached, no need to refresh" userID=2
logger=accesscontrol.service t=2024-07-18T07:22:53.533119396Z level=debug msg="Using cached permissions" key=rbac-permissions-1-user-2
logger=oauth_token.sync t=2024-07-18T07:22:53.718698151Z level=debug msg="Singleflight request for OAuth token sync" key=user:2
logger=oauthtoken t=2024-07-18T07:22:53.718736605Z level=debug msg="Expiration check has been cached, no need to refresh" userID=2
logger=accesscontrol.service t=2024-07-18T07:22:53.718763223Z level=debug msg="Using cached permissions" key=rbac-permissions-1-user-2
logger=accesscontrol.evaluator t=2024-07-18T07:22:53.718952402Z level=debug msg="Matched scope" userscope=datasources:* targetscope=datasources:uid:eds3rfholibk0a
logger=secrets.kvstore t=2024-07-18T07:22:53.719686488Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=grafana-clickhouse-datasource
logger=oauth_token.sync t=2024-07-18T07:22:53.75323206Z level=debug msg="Singleflight request for OAuth token sync" key=user:2
logger=oauthtoken t=2024-07-18T07:22:53.75329547Z level=debug msg="Expiration check has been cached, no need to refresh" userID=2
logger=accesscontrol.service t=2024-07-18T07:22:53.753331696Z level=debug msg="Using cached permissions" key=rbac-permissions-1-user-2
logger=datasources t=2024-07-18T07:22:53.753509997Z level=debug msg="Querying for data source via SQL store" uid=eds3rfholibk0a orgId=1
logger=secrets.kvstore t=2024-07-18T07:22:53.754222911Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=grafana-clickhouse-datasource
logger=context userId=2 orgId=1 uname=wpsadmin t=2024-07-18T07:22:53.754448183Z level=error msg="Internal server error" error="[plugin.unavailable] plugin unavailable" remote_addr=10.226.11.24 traceID=
logger=context userId=2 orgId=1 uname=wpsadmin t=2024-07-18T07:22:53.754502787Z level=error msg="Request Completed" method=GET path=/api/datasources/uid/eds3rfholibk0a/health status=500 remote_addr=10.226.11.24 time_ms=2 duration=2.076036ms size=95 referer=https://10.226.48.76:8081/grafana/connections/datasources/edit/eds3rfholibk0a handler=/api/datasources/uid/:uid/health status_source=server
logger=secrets t=2024-07-18T07:22:55.05038341Z level=debug msg="Removing expired data keys from cache..."
logger=secrets t=2024-07-18T07:22:55.050429706Z level=debug msg="Removing expired data keys from cache finished successfully"

[xiaoyuan2019]

@SpencerTorres grafana upgrade to v11.1.0，clickhouse datasource is working，but why？

[SpencerTorres]

I'm unsure why the plugin would be unavailable on older versions, the logs don't show any details about why this would be happening. I'll reach out to the other maintainers to see if they know. Thanks for the extra info.

[bossinc]

I'm not able to reproduce this issue using v10.4.1 of Grafana. Maybe restarting Grafana when you switched to v11 fixed this issue and not v11 itself.

[SpencerTorres]

It seems like this could've been a one-off issue related to your specific deployment since we're unable to reproduce the error. Perhaps we should close this and re-open if others have a similar issue

[aramperes]

I ran into this when building on non-amd64 platform i.e. macOS arm64. The Go binary is built with a different name and is not executed on startup.

[SpencerTorres]

I ran into this when building on non-amd64 platform i.e. macOS arm64. The Go binary is built with a different name and is not executed on startup.

The binary has to have a specific name. It must match the configured prefix (gpx_clickhouse, as defined in plugin.json) and the suffix must match the host machine's architecture.

The official plugin build should include binaries for common architectures. I just noticed this issue doesn't include architecture or whether or not this is an official build. Why did it work when upgrading to Grafana v11?

@xiaoyuan2019 please provide these details if this issue is still relevant, otherwise we can close.

[aangelisc]

Hi @xiaoyuan2019,

I'll be closing this issue as we're unable to reproduce. If the issue reoccurs please reopen the issue 😊