Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add get_or_insert function to cache layer #673

Merged
merged 13 commits into from
Aug 6, 2024
77 changes: 48 additions & 29 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,51 @@
## Running Tests

Before running tests make sure that:

[ ] redis is running
[ ] starters/saas frontend package is built:

```
$ cd starters/saas/frontend
$ npm i -g pnpm
$ pnpm i && pnpm build
```

Running all tests should be done with:

```
$ cargo xtask test
```


## Publishing a new version

**Test your changes**

* [ ] Ensure you have the necessary local resources, such as `DB`/`Redis`, by executing the command `cargo loco doctor --environment test`. In case you don't have them, refer to the relevant documentation section for guidance.
* [ ] run `cargo test` on the root to test Loco itself
* [ ] cd `examples/demo` and run `cargo test` to test our "driver app" which exercises the framework in various ways
* [ ] push your changes to Github to get the CI running and testing in various additional configurations that you don't have
* [ ] CI should pass. Take note that all `starters-*` CI are using a **fixed version** of Loco and are not seeing your changes yet


**Actually bump version + test and align starters**

* [ ] in project root, run `cargo xtask bump-version` and give it the next version. Versions are without `v` prefix. Example: `0.1.3`.
* [ ] Did the xtask testing workflow fail?
* [ ] YES: fix errors, and re-run `cargo xtask bump-version` **with the same version as before**.
* [ ] NO: great, move to publishing
* [ ] Your repo may be dirty with fixes. Now that tests are passing locally commit the changes. Then run `cargo publish` to publish the next Loco version (remember: the starters at this point are pointing to the **next version already**, so we don't want to push until publish finished)
* [ ] When publish finished successfully, push your changes to github
* [ ] Wait for CI to finish. You want to be focusing more at the starters CI, because they will now pull the new version.
* [ ] Did CI fail?
* [ ] YES: This means you had a circumstance that's not predictable (e.g. some operating system issue). Fix the issue and **repeat the bumping process, advance a new version**.
* [ ] NO: all good! you're done.

**Book keeping**

* [ ] Update changelog: (1) move vnext to be that new version of yours, (2) create a blank vnext
* [ ] Think about if any of the items in the new version needs new documentation or update to the documentation -- and do it
## Errors

Errors are done with `thiserror`. We adopt a minimalistic approach to errors.
Expand Down Expand Up @@ -45,32 +93,3 @@ In this case, we duplicate the YAML error type, leave one of those for auto conv
Some files contain a special `CONTRIBUTORS` comment. This comment should
contain context, special notes for that module, and a checklist if needed, so please make sure to follow it.


## Publishing a new version

**Test your changes**

* [ ] Ensure you have the necessary local resources, such as `DB`/`Redis`, by executing the command `cargo loco doctor --environment test`. In case you don't have them, refer to the relevant documentation section for guidance.
* [ ] run `cargo test` on the root to test Loco itself
* [ ] cd `examples/demo` and run `cargo test` to test our "driver app" which exercises the framework in various ways
* [ ] push your changes to Github to get the CI running and testing in various additional configurations that you don't have
* [ ] CI should pass. Take note that all `starters-*` CI are using a **fixed version** of Loco and are not seeing your changes yet


**Actually bump version + test and align starters**

* [ ] in project root, run `cargo xtask bump-version` and give it the next version. Versions are without `v` prefix. Example: `0.1.3`.
* [ ] Did the xtask testing workflow fail?
* [ ] YES: fix errors, and re-run `cargo xtask bump-version` **with the same version as before**.
* [ ] NO: great, move to publishing
* [ ] Your repo may be dirty with fixes. Now that tests are passing locally commit the changes. Then run `cargo publish` to publish the next Loco version (remember: the starters at this point are pointing to the **next version already**, so we don't want to push until publish finished)
* [ ] When publish finished successfully, push your changes to github
* [ ] Wait for CI to finish. You want to be focusing more at the starters CI, because they will now pull the new version.
* [ ] Did CI fail?
* [ ] YES: This means you had a circumstance that's not predictable (e.g. some operating system issue). Fix the issue and **repeat the bumping process, advance a new version**.
* [ ] NO: all good! you're done.

**Book keeping**

* [ ] Update changelog: (1) move vnext to be that new version of yours, (2) create a blank vnext
* [ ] Think about if any of the items in the new version needs new documentation or update to the documentation -- and do it
7 changes: 4 additions & 3 deletions examples/demo/src/app.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,9 @@ use migration::Migrator;
use sea_orm::DatabaseConnection;

use crate::{
controllers,
controllers::middlewares,
controllers::{self, middlewares},
initializers,
models::_entities::{notes, users},
models::_entities::{notes, roles, users, users_roles},
tasks,
workers::downloader::DownloadWorker,
};
Expand Down Expand Up @@ -109,6 +108,8 @@ impl Hooks for App {
}

async fn truncate(db: &DatabaseConnection) -> Result<()> {
truncate_table(db, users_roles::Entity).await?;
truncate_table(db, roles::Entity).await?;
truncate_table(db, users::Entity).await?;
truncate_table(db, notes::Entity).await?;
Ok(())
Expand Down
19 changes: 18 additions & 1 deletion examples/demo/src/controllers/cache.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::models::users;
use loco_rs::prelude::*;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
pub struct CacheResponse {
value: Option<String>,
Expand All @@ -11,14 +11,31 @@ async fn get_cache(State(ctx): State<AppContext>) -> Result<Response> {
value: ctx.cache.get("value").await.unwrap(),
})
}

async fn insert(State(ctx): State<AppContext>) -> Result<Response> {
ctx.cache.insert("value", "loco cache value").await.unwrap();
format::empty()
}

async fn get_or_insert(State(ctx): State<AppContext>) -> Result<Response> {
let res = ctx
.cache
.get_or_insert("user", async {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something still seems off to me w/errors.

i believe, async block should just throw the regular Result from Loco (i.e. CacheError does not belong in that async block)

the block runs like any other "linear" code flow, only that in our case, the result of the block is cached. So the block can return a result or fail, like any other code flow. The failure should be for any original reason the block itself has failed.

a good way to look at it:

if you remove the cache completely, the code block should mostly stay the same

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got you, changed for supporting it

let user = users::Model::find_by_email(&ctx.db, "[email protected]").await?;
Ok(user.name)
})
.await;

match res {
Ok(username) => format::text(&username),
Err(_e) => format::text("not found"),
}
}

pub fn routes() -> Routes {
Routes::new()
.prefix("cache")
.add("/", get(get_cache))
.add("/insert", post(insert))
.add("/get_or_insert", get(get_or_insert))
}
1 change: 1 addition & 0 deletions examples/demo/tests/cmd/cli.trycmd
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ $ blo-cli routes --environment test
[POST] /auth/reset
[POST] /auth/verify
[GET] /cache
[GET] /cache/get_or_insert
[POST] /cache/insert
[GET] /mylayer/admin
[GET] /mylayer/echo
Expand Down
19 changes: 1 addition & 18 deletions examples/demo/tests/models/roles.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,10 @@ use blo::{
app::App,
models::{roles, sea_orm_active_enums, users, users::RegisterParams, users_roles},
};
use loco_rs::{db::truncate_table, prelude::*, testing};
use loco_rs::{prelude::*, testing};
use sea_orm::DatabaseConnection;
use serial_test::serial;

async fn truncate_this(db: &DatabaseConnection) -> Result<(), ModelError> {
truncate_table(db, roles::Entity).await?;
truncate_table(db, users::Entity).await?;
truncate_table(db, users_roles::Entity).await?;
Ok(()).map_err(|_: ModelError| ModelError::EntityNotFound)
}

macro_rules! configure_insta {
($($expr:expr),*) => {
let mut settings = insta::Settings::clone_current();
Expand All @@ -28,8 +21,6 @@ async fn can_add_user_to_admin() {
configure_insta!();

let boot = testing::boot_test::<App>().await.unwrap();
testing::seed::<App>(&boot.app_context.db).await.unwrap();
let _t = truncate_this(&boot.app_context.db).await;
let new_user: Result<users::Model, ModelError> = users::Model::create_with_password(
&boot.app_context.db,
&RegisterParams {
Expand All @@ -52,8 +43,6 @@ async fn can_add_user_to_user() {
configure_insta!();

let boot = testing::boot_test::<App>().await.unwrap();
testing::seed::<App>(&boot.app_context.db).await.unwrap();
let _t = truncate_this(&boot.app_context.db).await;
let new_user: Result<users::Model, ModelError> = users::Model::create_with_password(
&boot.app_context.db,
&RegisterParams {
Expand All @@ -76,8 +65,6 @@ async fn can_convert_between_user_and_admin() {
configure_insta!();

let boot = testing::boot_test::<App>().await.unwrap();
testing::seed::<App>(&boot.app_context.db).await.unwrap();
let _t = truncate_this(&boot.app_context.db).await;
let new_user: Result<users::Model, ModelError> = users::Model::create_with_password(
&boot.app_context.db,
&RegisterParams {
Expand Down Expand Up @@ -108,8 +95,6 @@ async fn can_find_user_roles() {
configure_insta!();

let boot = testing::boot_test::<App>().await.unwrap();
testing::seed::<App>(&boot.app_context.db).await.unwrap();
let _t = truncate_this(&boot.app_context.db).await;
let new_user: Result<users::Model, ModelError> = users::Model::create_with_password(
&boot.app_context.db,
&RegisterParams {
Expand Down Expand Up @@ -147,8 +132,6 @@ async fn cannot_find_user_before_conversation() {
configure_insta!();

let boot = testing::boot_test::<App>().await.unwrap();
testing::seed::<App>(&boot.app_context.db).await.unwrap();
let _t = truncate_this(&boot.app_context.db).await;
let new_user: Result<users::Model, ModelError> = users::Model::create_with_password(
&boot.app_context.db,
&RegisterParams {
Expand Down
12 changes: 1 addition & 11 deletions examples/demo/tests/models/users_roles.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,9 @@ use blo::{
app::App,
models::{roles, sea_orm_active_enums, users, users::RegisterParams, users_roles},
};
use loco_rs::{db::truncate_table, prelude::*, testing};
use loco_rs::{prelude::*, testing};
use sea_orm::{ColumnTrait, DatabaseConnection};
use serial_test::serial;
async fn truncate_this(db: &DatabaseConnection) -> Result<(), ModelError> {
truncate_table(db, users_roles::Entity).await?;
truncate_table(db, users::Entity).await?;
truncate_table(db, roles::Entity).await?;
Ok(()).map_err(|_: ModelError| ModelError::EntityNotFound)
}
macro_rules! configure_insta {
($($expr:expr),*) => {
let mut settings = insta::Settings::clone_current();
Expand All @@ -26,8 +20,6 @@ async fn can_connect_user_to_user_role() {
configure_insta!();

let boot = testing::boot_test::<App>().await.unwrap();
testing::seed::<App>(&boot.app_context.db).await.unwrap();
let _t = truncate_this(&boot.app_context.db).await;
let new_user: Result<users::Model, ModelError> = users::Model::create_with_password(
&boot.app_context.db,
&RegisterParams {
Expand Down Expand Up @@ -70,8 +62,6 @@ async fn can_connect_user_to_admin_role() {
configure_insta!();

let boot = testing::boot_test::<App>().await.unwrap();
testing::seed::<App>(&boot.app_context.db).await.unwrap();
let _t = truncate_this(&boot.app_context.db).await;
let new_user: Result<users::Model, ModelError> = users::Model::create_with_password(
&boot.app_context.db,
&RegisterParams {
Expand Down
25 changes: 24 additions & 1 deletion examples/demo/tests/requests/cache.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
use blo::app::App;
use blo::{app::App, models::users};
use insta::assert_debug_snapshot;
use loco_rs::testing;
use sea_orm::ModelTrait;
use serial_test::serial;

// TODO: see how to dedup / extract this to app-local test utils
// not to framework, because that would require a runtime dep on insta
Expand All @@ -14,6 +16,7 @@ macro_rules! configure_insta {
}

#[tokio::test]
#[serial]
async fn ping() {
configure_insta!();

Expand All @@ -27,3 +30,23 @@ async fn ping() {
})
.await;
}

#[tokio::test]
#[serial]
async fn can_get_or_insert() {
configure_insta!();

testing::request::<App, _, _>(|request, ctx| async move {
testing::seed::<App>(&ctx.db).await.unwrap();
let response = request.get("/cache/get_or_insert").await;
assert_eq!(response.text(), "user1");

let user = users::Model::find_by_email(&ctx.db, "[email protected]")
.await
.unwrap();
user.delete(&ctx.db).await.unwrap();
let response = request.get("/cache/get_or_insert").await;
assert_eq!(response.text(), "user1");
})
.await;
}
2 changes: 1 addition & 1 deletion examples/demo/tests/requests/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
mod auth;
mod cache;
pub mod mylayer;
mod mylayer;
mod notes;
mod ping;
mod prepare_data;
Expand Down
2 changes: 1 addition & 1 deletion examples/demo/tests/requests/prepare_data.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ pub async fn init_user_login(request: &TestServer, ctx: &AppContext) -> LoggedIn
});

//Creating a new user
request.post("/auth/register").json(&register_payload).await;
let _res = request.post("/auth/register").json(&register_payload).await;
let user = users::Model::find_by_email(&ctx.db, USER_EMAIL)
.await
.unwrap();
Expand Down
63 changes: 63 additions & 0 deletions src/cache/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@
//! This module provides a generic cache interface for various cache drivers.
pub mod drivers;

use std::future::Future;

use self::drivers::CacheDriver;
use crate::Result as LocoResult;

/// Errors related to cache operations
#[derive(thiserror::Error, Debug)]
Expand Down Expand Up @@ -84,6 +87,40 @@ impl Cache {
self.driver.insert(key, value).await
}

/// Retrieves the value associated with the given key from the cache,
/// or inserts it if it does not exist, using the provided closure to
/// generate the value.
///
/// # Example
/// ```
/// use loco_rs::{app::AppContext};
/// use loco_rs::tests_cfg::app::*;
///
/// pub async fn get_or_insert(){
/// let app_ctx = get_app_context().await;
/// let res = app_ctx.cache.get_or_insert("key", async {
/// Ok("value".to_string())
/// }).await.unwrap();
/// assert_eq!(res, "value");
/// }
/// ```
///
/// # Errors
///
/// A [`LocoResult`] indicating the success of the operation.
pub async fn get_or_insert<F>(&self, key: &str, f: F) -> LocoResult<String>
where
F: Future<Output = LocoResult<String>> + Send,
{
if let Some(value) = self.driver.get(key).await? {
Ok(value)
} else {
let value = f.await?;
self.driver.insert(key, &value).await?;
Ok(value)
}
}

/// Removes a key-value pair from the cache.
///
/// # Example
Expand Down Expand Up @@ -122,3 +159,29 @@ impl Cache {
self.driver.clear().await
}
}

#[cfg(test)]
mod tests {

use crate::tests_cfg;

#[tokio::test]
async fn can_get_or_insert() {
let app_ctx = tests_cfg::app::get_app_context().await;
let get_key = "loco";

assert_eq!(app_ctx.cache.get(get_key).await.unwrap(), None);

let result = app_ctx
.cache
.get_or_insert(get_key, async { Ok("loco-cache-value".to_string()) })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you try a test where the async block captures something from app_ctx? like try a real DB call for example

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added endpoint usage example + adding tests

.await
.unwrap();

assert_eq!(result, "loco-cache-value".to_string());
assert_eq!(
app_ctx.cache.get(get_key).await.unwrap(),
Some("loco-cache-value".to_string())
);
}
}
Loading
Loading