-
Notifications
You must be signed in to change notification settings - Fork 704
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding Missing filters to CLIENT LIST and Dedup Parsing #1401
Adding Missing filters to CLIENT LIST and Dedup Parsing #1401
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #1401 +/- ##
============================================
- Coverage 70.94% 70.93% -0.01%
============================================
Files 120 120
Lines 65044 65078 +34
============================================
+ Hits 46143 46161 +18
- Misses 18901 18917 +16
|
54adfa6
to
7630eba
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks.
I just took a quick look and I can see some things that appear to be broken:
- CLIENT LIST with multiple IDs, for example
CLIENT LIST ID 3 4 5
. (It means id is 3 or 4 or 5.) We a test case for this. - The default for "skipme" for CLIENT KILL should be YES by default (when filters are used), but for CLIENT LIST, skipme should be NO by default. Otherwise, we have a breaking change. Please add some tests without SKIPME but with other filters matching the calling client to cover the default skipme value for LIST and KILL.
We need some test cases combining multiple filter rules at the same time.
Thanks @zuiderkwast for the comments! I am working on them |
049f2d4
to
e3c1c5a
Compare
@zuiderkwast thank you for your comments. Adressed most of them. Working on adding tests meanwhile. Trying to see how can I have tests with multiple clients. Sharing some responses from local testing Client Kill Command
Client List Command
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Trying to see how can I have tests with multiple clients.
You can create multiple clients.
set c1 [valkey_client]
set c2 [valkey_client]
set c3 [valkey_client]
...
$c1 close
$c2 close
$c3 close
fd743fe
to
6681e47
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The refactoring looks good. I have just a few small comments and there is some merge conflict.
6681e47
to
52fd611
Compare
52fd611
to
ac3f5e5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Just two nits.
@valkey-io/core-team Please approve the extended syntax for CLIENT LIST as described in the top comment. The MAXAGE argument for CLIENT LIST is a little strange, because it's the max age of clients to exclude, for symmetry with CLIENT KILL. An alternative is to forbid MAXAGE for CLIENT LIST. |
3b789c2
to
a944e0e
Compare
2120e61
to
301993f
Compare
Signed-off-by: Sarthak Aggarwal <[email protected]>
3f171c8
to
110c7d3
Compare
Core team meeting. Can we run a quick performance test to validate that it's not significantly regressing. Let's run |
Current suggestion:
Let’s capture data with modifying client parameter for both steps. If there are any other suggestion @hwware / @zuiderkwast, feel free to add. We'll capture that. |
For the benchmark, we should run |
I would like to suggest as following:
|
@hwware @zuiderkwast @hpatro Thank you for the suggestions, really helpful. I had just one query, since some of the filters were not supported in |
I was able to extract the numbers for 10000 clients as well. Seeing negligible variance across the latency / throughput parameters for with and without filters. Sharing Command Info Stats for 10000 Clients. Please review. Unstable:Command:
|
My idea is sending read/write operations through 10000 or more clients, and at the same time, simulating operator runs CLIENT LIST filter in another valkey-bench, and you can watch if there are qps drop in the read/write side. Because the client list with filter will execute a slow operation, the server qps could drop. If there is no significant drop, we can merge this PR, thanks |
I can see in the benchmarks that this PR actually makes CLIENT LIST faster than before with 1000 clients connected. It seems that there is some initial cost to set up the filter struct (so slightly dropped latency when there are very few clients connected) but once the filter struct is set up, it is faster to filter each client. Because of this, I would assume that QPS for the read/write clients will be affected less by CLIENT LIST with this PR compared to unstable. Testing is of course better, but it looks pretty safe to me. |
Thanks folks for the suggestions and inputs. Based on @hwware suggestions, I performed the SET/GET operations while running Sharing the results: Notes:
|
@sarthakaggarwal97 Thanks for sharing the results and is inline with our expectation. I'm good with the stats. The above benchmarking is performed with one client running the |
It looks good to me. Approved |
I see atleast 4 core-team member have approved this either through voting or PR approval. @valkey-io/core-team Do you want to review further or is it good to be merged? |
Let's merge it. @sarthakaggarwal97 do you want to update the docs too? The |
Thank you everyone for helping in this one. @zuiderkwast let me raise the doc PR as well. Thank you! |
Adds filter options to CLIENT LIST: * USER <username> Return clients authenticated by <username>. * ADDR <ip:port> Return clients connected from the specified address. * LADDR <ip:port> Return clients connected to the specified local address. * SKIPME (YES|NO) Exclude the current client from the list (default: no). * MAXAGE <maxage> Only list connections older than the specified age. Modifies the ID filter to CLIENT KILL to allow multiple IDs * ID <client-id> [<client-id>...] Kill connections by client ids. This makes CLIENT LIST and CLIENT KILL accept the same options. For backward compatibility, the default value for SKIPME is NO for CLIENT LIST and YES for CLIENT KILL. The MAXAGE comes from CLIENT KILL, where it *keeps* clients with the given max age and kills the older ones. This logic becomes weird for CLIENT LIST, but is kept for similary with CLIENT KILL, for the use case of first testing manually using CLIENT LIST, and then running CLIENT KILL with the same filters. The `ID client-id [client-id ...]` no longer needs to be the last filter. The parsing logic determines if an argument is an ID or not based on whether it can be parsed as an integer or not. Partly addresses: valkey-io#668 --------- Signed-off-by: Sarthak Aggarwal <[email protected]> Signed-off-by: proost <[email protected]>
Documentation for valkey-io/valkey#1401 --------- Signed-off-by: Sarthak Aggarwal <[email protected]>
Adds filter options to CLIENT LIST: * USER <username> Return clients authenticated by <username>. * ADDR <ip:port> Return clients connected from the specified address. * LADDR <ip:port> Return clients connected to the specified local address. * SKIPME (YES|NO) Exclude the current client from the list (default: no). * MAXAGE <maxage> Only list connections older than the specified age. Modifies the ID filter to CLIENT KILL to allow multiple IDs * ID <client-id> [<client-id>...] Kill connections by client ids. This makes CLIENT LIST and CLIENT KILL accept the same options. For backward compatibility, the default value for SKIPME is NO for CLIENT LIST and YES for CLIENT KILL. The MAXAGE comes from CLIENT KILL, where it *keeps* clients with the given max age and kills the older ones. This logic becomes weird for CLIENT LIST, but is kept for similary with CLIENT KILL, for the use case of first testing manually using CLIENT LIST, and then running CLIENT KILL with the same filters. The `ID client-id [client-id ...]` no longer needs to be the last filter. The parsing logic determines if an argument is an ID or not based on whether it can be parsed as an integer or not. Partly addresses: valkey-io#668 --------- Signed-off-by: Sarthak Aggarwal <[email protected]>
Adds filter options to CLIENT LIST:
Modifies the ID filter to CLIENT KILL to allow multiple IDs
This makes CLIENT LIST and CLIENT KILL accept the same options.
For backward compatibility, the default value for SKIPME is NO for CLIENT LIST and YES for CLIENT KILL.
The MAXAGE comes from CLIENT KILL, where it keeps clients with the given max age and kills the older ones. This logic becomes weird for CLIENT LIST, but is kept for similary with CLIENT KILL, for the use case of first testing manually using CLIENT LIST, and then running CLIENT KILL with the same filters.
The
ID client-id [client-id ...]
no longer needs to be the last filter. The parsing logic determines if an argument is an ID or not based on whether it can be parsed as an integer or not.Partly addresses: #668