Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Enable HDBSCAN to build knn graph using NN Descent #5939

Open
wants to merge 35 commits into
base: branch-24.10
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f0b1ac7
enable nn descent in hdbscan
jinsolp Jun 14, 2024
107b636
Merge branch 'rapidsai:branch-24.08' into hdbscan-nndescent
jinsolp Jun 14, 2024
8f036a9
change epilogue functor
jinsolp Jun 16, 2024
e158a74
cleanup
jinsolp Jun 16, 2024
ac03040
fix test + add param for compute_core_dist
jinsolp Jun 17, 2024
e16a619
Merge branch 'rapidsai:branch-24.08' into hdbscan-nndescent
jinsolp Jun 17, 2024
f2c3c92
remove and add comments
jinsolp Jun 17, 2024
b461d31
refine distances due to precision issues
jinsolp Jun 18, 2024
5b15ce5
add return_distances for cdef
jinsolp Jun 18, 2024
437406f
Add to param names
jinsolp Jun 22, 2024
7683d36
add documentation
jinsolp Jun 23, 2024
70b8835
return distances param in python
jinsolp Jul 12, 2024
152a8a3
change test
jinsolp Jul 12, 2024
f0cdd3c
add copy kernel
jinsolp Jul 12, 2024
9545f79
Merge branch 'rapidsai:branch-24.08' into hdbscan-nndescent
jinsolp Jul 12, 2024
be5c167
remove build_algo in test
jinsolp Jul 12, 2024
0a27614
Merge branch 'branch-24.08' into hdbscan-nndescent
jinsolp Jul 18, 2024
e2735f5
auto option as default for build_algo
jinsolp Jul 24, 2024
47ec894
Merge branch 'rapidsai:branch-24.08' into hdbscan-nndescent
jinsolp Jul 24, 2024
4ebf734
Merge branch 'rapidsai:branch-24.10' into hdbscan-nndescent
jinsolp Aug 19, 2024
746ac33
use slice kernels
jinsolp Aug 19, 2024
2b46746
tests
jinsolp Aug 20, 2024
6bb6357
make data view depending on host/dev
jinsolp Aug 21, 2024
6da003f
Merge branch 'rapidsai:branch-24.10' into hdbscan-nndescent
jinsolp Aug 21, 2024
3ec57c7
adding arg
jinsolp Aug 21, 2024
d93aee6
for building + CI check
jinsolp Aug 21, 2024
7dbe38d
ann types fix
jinsolp Aug 22, 2024
337aa88
type fix
jinsolp Aug 22, 2024
ce96893
Merge branch 'rapidsai:branch-24.10' into hdbscan-nndescent
jinsolp Aug 22, 2024
154bbda
change if to raft_expects
jinsolp Aug 23, 2024
6083812
revert fork and pinned tag
jinsolp Aug 23, 2024
732a06e
Trigger CI
jinsolp Aug 23, 2024
517f7ab
Merge branch 'rapidsai:branch-24.10' into hdbscan-nndescent
jinsolp Aug 23, 2024
fea3dc8
Merge branch 'rapidsai:branch-24.10' into hdbscan-nndescent
jinsolp Sep 21, 2024
904ab1b
change to switch
jinsolp Sep 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 18 additions & 7 deletions cpp/include/cuml/cluster/hdbscan.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

#include <raft/core/handle.hpp>
#include <raft/distance/distance_types.hpp>
#include <raft/neighbors/nn_descent_types.hpp>

#include <rmm/device_uvector.hpp>

Expand All @@ -27,6 +28,8 @@ namespace ML {
namespace HDBSCAN {
namespace Common {

using nn_index_params = raft::neighbors::experimental::nn_descent::index_params;

/**
* The Condensed hierarchicy is represented by an edge list with
* parents as the source vertices, children as the destination,
Expand Down Expand Up @@ -134,6 +137,7 @@ class CondensedHierarchy {
};

enum CLUSTER_SELECTION_METHOD { EOM = 0, LEAF = 1 };
enum GRAPH_BUILD_ALGO { BRUTE_FORCE_KNN = 0, NN_DESCENT = 1 };

class RobustSingleLinkageParams {
public:
Expand All @@ -151,6 +155,8 @@ class RobustSingleLinkageParams {
class HDBSCANParams : public RobustSingleLinkageParams {
public:
CLUSTER_SELECTION_METHOD cluster_selection_method = CLUSTER_SELECTION_METHOD::EOM;
GRAPH_BUILD_ALGO build_algo = GRAPH_BUILD_ALGO::BRUTE_FORCE_KNN;
nn_index_params nn_descent_params = {};
};

/**
Expand Down Expand Up @@ -495,14 +501,19 @@ namespace HDBSCAN::HELPER {
* @param n number of columns in X
* @param metric distance metric to use
* @param min_samples minimum number of samples to use for computing core distances
* @param build_algo build algo for building the knn graph (default: brute_force_knn)
* @param build_params build parameters for build_algo
*/
void compute_core_dists(const raft::handle_t& handle,
const float* X,
float* core_dists,
size_t m,
size_t n,
raft::distance::DistanceType metric,
int min_samples);
void compute_core_dists(
const raft::handle_t& handle,
const float* X,
float* core_dists,
size_t m,
size_t n,
raft::distance::DistanceType metric,
int min_samples,
HDBSCAN::Common::GRAPH_BUILD_ALGO build_algo = HDBSCAN::Common::GRAPH_BUILD_ALGO::BRUTE_FORCE_KNN,
HDBSCAN::Common::nn_index_params build_params = Common::nn_index_params{});

/**
* @brief Compute the map from final, normalize labels to the labels in the CondensedHierarchy
Expand Down
Loading
Loading