Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More cli support #18

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
65e1ce3
add an option to funclist to generate function from paths
outscale-mgo Nov 8, 2024
651460a
remove argument-list.json
outscale-mgo Nov 8, 2024
d6bad35
add debug function
outscale-mgo Nov 14, 2024
85afe61
add some debug
outscale-mgo Nov 14, 2024
2416cfe
enable to choose prefix for Function
outscale-mgo Nov 14, 2024
8bcaa35
funclist now support costum function suffix
outscale-mgo Nov 15, 2024
92945b2
Document --function-suffix=suffix
outscale-mgo Nov 15, 2024
9e45172
path is on thw way
outscale-mgo Nov 19, 2024
0927c7a
add more helper functions
outscale-mgo Nov 21, 2024
56e4c39
add more helper to handle path type
outscale-mgo Nov 25, 2024
e4ddaa5
descripttion for path mode
outscale-mgo Nov 29, 2024
7377d15
remove debug
outscale-mgo Dec 2, 2024
311f5b4
fix name for path calls
outscale-mgo Dec 2, 2024
21eddf5
somewhat generate arguments corectly with path
outscale-mgo Dec 3, 2024
11ecece
fix header, and handle query maybe
outscale-mgo Dec 3, 2024
65b5cd3
handle path post data mix with other path arguments
outscale-mgo Dec 9, 2024
a358f07
bin: add missing get_path_type
outscale-mgo Dec 10, 2024
8a5cb45
add new C helpers documentation
outscale-mgo Dec 10, 2024
855bff1
fix from thiery feedback
outscale-mgo Dec 10, 2024
45fcbfc
default endpoint is now retrive from osc-api.json
outscale-mgo Dec 13, 2024
18c75ac
enable to set user agent
outscale-mgo Dec 13, 2024
3d9786e
support ssl_verify in config.json
outscale-mgo Dec 13, 2024
f7e951c
rename sdk-user-agent to sdk-name
outscale-mgo Dec 16, 2024
a68744d
improve doc
outscale-mgo Dec 16, 2024
b6195ee
work with guru API
outscale-mgo Dec 17, 2024
20bee6c
add documentation
outscale-mgo Dec 17, 2024
c51d577
improve readme
outscale-mgo Dec 19, 2024
7f9efba
improve Doc
outscale-mgo Dec 20, 2024
d6d23e3
improve configure --help
outscale-mgo Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions HACKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

*Essential Information for Hacking COGNAC*

## Enabling debug

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enabling Debug Mode


To enable debug, use `--debug-mode` in `./configure`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To enable debug mode, use the --debug-mode option with the ./configure command


then the `debug` commande in the shell script will print some information.
without this option debug will print nothing

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once enabled, the debug command in the shell script will print debug information. Without this option, the debug command will not output any information.


## Unix Philosophy and COGNAC Paradigms.

Most people have heard the phrase "Do One Thing and Do It Well" and associate it with the Unix philosophy. It’s said that this is how Unix operates and how it should be.
Expand Down Expand Up @@ -35,6 +42,28 @@ A relatively easy way to add a feature in Cognac is to start by modifying the ge

For exemple, if you modify a function like `parse_thatarg()` in "osc_sdk.c", once the changes are working, you can then search (using a tool like grep) through the generator code to find where `parse_thatarg` is generated. From there, you can add modifications to the generator to make the change permanent and automated.

## Example to add a new API, that doesn't work at first.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example: Adding a New API That Initially Doesn't Work


Let's say you ahve configure cognac like this

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's say you have configured Cognac like this:


```sh
./configure --sdk-name=myapi-sdk --cli-name=guru --api-script='cat myapi-api.json > osc-api.json' --debug-mode --from-path
```

doing that you will use try to generate the api from path, with debug enable.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By doing that, you will attempt to generate the API from the path with debug enabled.


now let's say you have this output while calling `make`:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, let's say you get this output while running make

```
____complex_struct_func_parser____
nothing found____cli_parser____
nothing founderror in <stdin> jsonnothing founderror in <stdin> jsonnothing founderror in <stdin> jsonnothing founderror in <stdin> jsonnothing founderror in <stdin> jsonnothing founderror in <stdin> jsonnothing founderror in <stdin> json./cognac_gen.sh lib.h osc_sdk.h c
debug mode is on
```

This mean that durring `____complex_struct_func_parser____` `____cli_parser____`, the scripts fails to pasre some part of osc-api json.
To debug it, you should look at what happen durring `____complex_struct_func_parser____` and `____cli_parser____` inside `cognac_gen.sh`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that during 'complex_struct_func_parser' and 'cli_parser', the script fails to parse some parts of the osc-api JSON.



## helpers.sh

Most functions in helper.sh are documented, so read through it if you want to understand some of the most commonly used functions in cognac_gen.
53 changes: 39 additions & 14 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ help:

include oapi-cli.mk

BIN_DEPENDANCIES=bin/path_to_snakecase bin/path_to_camelcase bin/line_check bin/get_argument_list bin/funclist bin/get_path_type bin/get_path_description bin/arg_placement bin/construct_path bin/construct_endpoint

osc-api.json::
./bin/osc-api-seems-valid.sh osc-api.json "need_remove"

Expand All @@ -33,38 +35,61 @@ make_cli: $(CLI_NAME) $(CLI_NAME)-completion.bash
bin/funclist: bin/funclist.c
$(CC) -O3 bin/funclist.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/funclist

bin/construct_path: bin/construct_path.c
$(CC) -O3 bin/construct_path.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/construct_path

bin/construct_endpoint: bin/construct_endpoint.c
$(CC) -O3 bin/construct_endpoint.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/construct_endpoint

bin/path_to_snakecase: bin/path_to_snakecase.c
$(CC) -O3 bin/path_to_snakecase.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/path_to_snakecase

bin/arg_placement: bin/arg_placement.c
$(CC) -O3 bin/arg_placement.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/arg_placement

bin/path_to_camelcase: bin/path_to_camelcase.c
$(CC) -O3 bin/path_to_camelcase.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/path_to_camelcase

bin/line_check: bin/line_check.c
$(CC) -O3 bin/line_check.c -o bin/line_check

main.c: bin/line_check osc-api.json call_list arguments-list.json config.sh main_tpl.c cognac_gen.sh mk_args.c.sh
bin/get_argument_list: bin/get_argument_list.c
$(CC) -O3 -g bin/get_argument_list.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/get_argument_list

bin/get_path_type: bin/get_path_type.c
$(CC) -O3 -g bin/get_path_type.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/get_path_type

bin/get_path_description: bin/get_path_description.c
$(CC) -O3 -g bin/get_path_description.c $(JSON_C_LDFLAGS) $(JSON_C_CFLAGS) -o bin/get_path_description

main.c: $(BIN_DEPENDANCIES) osc-api.json call_list config.sh main_tpl.c cognac_gen.sh mk_args.c.sh
./cognac_gen.sh main_tpl.c main.c c

osc_sdk.c: bin/line_check osc-api.json call_list arguments-list.json config.sh lib.c cognac_gen.sh construct_data.c.sh mk_args.c.sh
osc_sdk.c: $(BIN_DEPENDANCIES) osc-api.json call_list config.sh lib.c cognac_gen.sh construct_data.c.sh mk_args.c.sh
./cognac_gen.sh lib.c osc_sdk.c c

osc_sdk.h: bin/line_check osc-api.json call_list arguments-list.json config.sh lib.h cognac_gen.sh mk_args.c.sh
osc_sdk.h: $(BIN_DEPENDANCIES) osc-api.json call_list config.sh lib.h cognac_gen.sh mk_args.c.sh
./cognac_gen.sh lib.h osc_sdk.h c

$(CLI_NAME)-completion.bash: bin/line_check osc-api.json call_list arguments-list.json config.sh oapi-cli-completion-tpl.bash cognac_gen.sh
$(CLI_NAME)-completion.bash: $(BIN_DEPENDANCIES) osc-api.json call_list config.sh oapi-cli-completion-tpl.bash cognac_gen.sh
./cognac_gen.sh oapi-cli-completion-tpl.bash $(CLI_NAME)-completion.bash bash

config.sh:
config.sh: configure config.mk
echo "alias json-search=$(JSON_SEARCH)" > config.sh
echo $(SED_ALIAS) >> config.sh
echo FUNCTION_SUFFIX=$(FUNCTION_SUFFIX) >> config.sh
echo "export CLI_NAME=$(CLI_NAME)" >> config.sh

arguments-list.json: osc-api.json
$(JSON_SEARCH) -s Request osc-api.json | $(JSON_SEARCH) -K properties \
| sed 's/]/ /g' \
| tr -d "\n[],\"" | sed -r 's/ +/ \n/g' \
| sort | uniq | tr -d "\n" > arguments-list.json
echo "export SDK_NAME=$(SDK_NAME)" >> config.sh
echo "export FROM_PATH=$(FROM_PATH)" >> config.sh
echo -e "debug()\n{" >> config.sh
echo -e '\tif [[ "$$DEBUG_MODE" == "1" ]] ; then echo "$$@" >&2 ; fi\n}' >> config.sh
echo export DEBUG_MODE=$(DEBUG_MODE) >> config.sh

call_list: osc-api.json bin/funclist
bin/funclist osc-api.json > call_list

bin/funclist osc-api.json $(FUNCLIST_ARG) > call_list

clean:
rm -vf osc-api.json call_list osc_sdk.c arguments-list.json osc_sdk.h main.c $(CLI_NAME) config.sh $(CLI_NAME)-completion.bash bin/line_check bin/funclist
rm -vf osc-api.json call_list osc_sdk.c osc_sdk.h main.c $(CLI_NAME) config.sh $(CLI_NAME)-completion.bash bin/line_check bin/funclist bin/path_to_snakecase

.PHONY: clean list_api_version help make_cli

41 changes: 35 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,20 @@ It takes three arguments: a source file, a destination file, and a language.
The language argument is crucial as certain keywords might be interpreted differently depending on the target language.

The script uses a file called osc-api.json, which represents the OpenAPI specification in JSON format.
For the Outscale API, the YAML source is converted to JSON using yq.
For the Outscale API, the YAML source is converted to JSON using `yq`: (https://kislyuk.github.io/yq/ or https://github.com/mikefarah/yq)

When generating API calls, COGNAC assumes that the OpenAPI file contains components named CallRequest.
When generating API calls, COGNAC by default assumes that the OpenAPI file contains components named CallRequest.
For example, if the API has a call named `CreatePony`, the corresponding component should be located at `#/components/schemas/CreatePonyRequest`.

You can modify the suffix of functions by using `--function-suffix=FUNCTION_SUFFIX` option with `./configure` command.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can modify the function suffix by using the --function-suffix=FUNCTION_SUFFIX option with the ./configure command.


Or you can generate functions using what is inside `paths`, using `./configure --from-path`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, you can generate functions based on the contents of paths by using the ./configure --from-path option.


*Note: There are two versions of yq: one written in Python and one in Go. The default version depends on your distribution. On Arch-based distributions, the Python version is typically the default, whereas on Debian-based distributions, the Go version is default. COGNAC supports both, but to use the Go version, you need to pass `--yq-go` to `./configure`.*

### Example: Generating a CLI for a New API
### Examples: Generating a CLI for a New API

#### Configure cognac for an Api using elements in schema as entry point for API Calls

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configure Cognac for an API using elements in the schema as entry points for API calls.


Let’s say you have an API that is not the Outscale API, and you want to generate a CLI for it.
You have a URL to a YAML file, such as `https://ponyapi.yolo/`, and the API request components are named `XInput` instead of `XRequest`.
Expand All @@ -44,22 +50,45 @@ To configure the Makefile to generate the CLI with the name `pony-cli`, and adju

Run the following command:
```bash
./configure --cli-name=pony-cli --api-script='curl -s https://ponyapi.yolo | yq $(YQ_ARG) | sed "s/Input/Request/" > osc-api.json'
./configure --cli-name=pony-cli --function-suffix Input --api-script='curl -s https://ponyapi.yolo | yq $(YQ_ARG)" > osc-api.json'
```

`-cli-name=pony-cli` set the generated binary name to `pony-cli`

```bash
--api-script='curl -s https://ponyapi.yolo | yq $(YQ_ARG) | sed "s/Input/Request/" > osc-api.json'
--function-suffix Input
```

Search for functions named XInput instead of XRequest.
```bash
--api-script='curl -s https://ponyapi.yolo | yq $(YQ_ARG) > osc-api.json'
```


This script is used to fetch the API file.


Here’s what the script does:

1. Retrieves the API in YAML format using curl -s `https://ponyapi.yolo/.`
2. Converts the YAML to JSON using yq `$(YQ_ARG)`. *Note the usage of `$(YQ_ARG)`, so ./configure can handle go version of yq*
3. Renames all components named `XInput` to `XRequest`.


#### Configure cognac for an Api using elements in path as entry point for API Calls

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configure Cognac for an API using elements in the path as entry points for API calls.


For this example we will use [guru](https://apis.guru/api-doc/)

Run the following commande

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run the following command:

```
./configure --sdk-name=guru-sdk --cli-name=guru --from-path --api-script='curl -s https://api.apis.guru/v2/openapi.yaml | yq $(YQ_ARG)" > osc-api.json'
```

`--cli-name=guru` set the generated binary name to `guru`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--cli-name=guru: Sets the generated binary name to guru.

`--sdk-name=guru-sdk` set the generate sdk name and UserAgent to `guru-sdk`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--sdk-name=guru-sdk: Sets the generated SDK name and the UserAgent to guru-sdk.

`--from-path` generate API calls from elements in `paths` instead of `components.schemas`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--from-path: Generates API calls from elements in paths instead of components.schemas.



#### Generte the code

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generate the Code


Once this setup is complete, you can now use the Makefile. It's also a good idea to run ./configure --help, as it contains several useful options.
- `--wget-json-search`: Helps with downloading `json-search`, which can be tricky to install, **If unsure, we recommend using this by default**
Expand Down
109 changes: 109 additions & 0 deletions bin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,112 @@ This is also much faster than multiple calls to grep.
# osc-api-seems-valid.sh

This utility is used to check if osc-api.json has been generated correctly. It verifies that the file is a valid JSON and contains at least one API function.

# path_to_snakecase
snakecasise an object name.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converts an object name to snake_case.

example `ReadVms` become `read_vms`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Examples:

  • ReadVms becomes read_vms
  • /my/{NAME}/getId becomes my_name_get_id

`/my/{NAME}/getId` become `my_name_get_id`

usage:
```
./path_to_snakecase STRING
```

# path_to_camelcase
camelcasise an object name.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converts an object name to camelCase.

example `ReadVms` become `read_vms`
`/my/{NAME}/getId` become `my_name_get_id`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Examples:

  • read_vms becomes readVms
  • /my/{name}/get_id becomes myNameGetId


usage:
```
./path_to_camelcase STRING
```

# get_argument_list

Take an object componant in `paths` or in `components.schema`, and give a list of all it arguments.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retrieves an object component from paths or components.schema and returns a list of all its arguments.


usage:
```
./get_argument_list file.json componant_name
```

# get_path_type

usage:
```
./bin/get_path_type osc-api.json PATH ARGUMENT_NAME
```

example:

```
$ ./bin/get_path_type osc-api.json /projects id
int
```

assuming id is of type int

or

usage:
```
./bin/get_path_type JSON_ELEM ARGUMENT_NAME
```

example:
```
$ ./bin/get_path_type '{"post": {"parameters": [ {"name": "a", "type": "string"} ]}}' "a"
string

```

# get_path_description

Just return "" so far, as the api I test this with, had no description in the path

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returns an empty string "" for now, as the API I tested this with had no description in the path.


usage:
```
./get_path_description JSON_ELEM ARG
```

example:
```
./get_path_description "{...}" argument_named_titi
```

# arg_placement

return where the argument is used.
`path`, if the argument is use in the path
`header` if it is an http header
`query` if it is in a query string
`data` if it is use in post data

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return where the argument is used:

  • path: If the argument is used in the URL path.
  • header: If the argument is used as an HTTP header.
  • query: If the argument is in a query string.
  • data: If the argument is used in POST data.



usage:
```
./arg_placement osc-api.json PATH ARGUMENT_NAME
```

example:
```
$ ./arg_placement osc-api.json /projects id
query
```

# construct_path

generate the C code to create a osc_str that the path of the call.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generate the C code to create an osc_str for the path of the call.


example:

```
$ ./construct_path /project/{id}/get

osc_str_append_string(&end_call, "/project/");
osc_str_append_string(&end_call, args->id);
osc_str_append_string(&end_call, "/get");
```

The code generated is not exhaustive, but you get the idea.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated code is not exhaustive, but it conveys the general idea.

44 changes: 44 additions & 0 deletions bin/arg_placement.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include "json.h"
#define UNFOUND "data"
#include "helper.h"

int main(int ac, char **av)
{
struct json_object *j_file = json_object_from_file(av[1]);
char *componant_name = av[2];
char *arg_name = av[3];
int ret;
struct json_object *path;
struct json_object *post_or_get;
struct json_object *parameters;

path = get_path_from_file(j_file, componant_name);
post_or_get = get_or_post_from_path(path);
if (!post_or_get)
goto err;

OBJ_GET(post_or_get, "parameters", &parameters);
int len = json_object_array_length(parameters);
for (int i = 0; i < len; ++i) {
struct json_object *param = json_object_array_get_idx(parameters, i);
struct json_object *name_obj;
struct json_object *in;

OBJ_GET(param, "name", &name_obj);
const char *name = json_object_get_string(name_obj);
struct json_object *type;
struct json_object *schema;

if (strcmp(name, arg_name))
continue;
OBJ_GET(param, "in", &in);
puts(json_object_get_string(in));
goto out;
}
out:
err:
json_object_put(j_file);
}
26 changes: 26 additions & 0 deletions bin/construct_endpoint.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#include <stdio.h>
#include <string.h>


int main(int ac, char **av)
{
int buf_i;
char *componant_name = av[1];

again:
char *brk = strchr(componant_name, '{');
if (!brk) {
printf("\tosc_str_append_string(&e->endpoint, \"%s\");\n", componant_name);
return 0;
}

*brk = 0;
printf("\tosc_str_append_string(&e->endpoint, \"%s\");\n", componant_name);

printf("\tosc_str_append_string(&e->endpoint, e->");
for (++brk; *brk != '}'; ++brk)
putchar(*brk);
puts(");");
componant_name = brk + 1;
goto again;
}
Loading