Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extend the client to use ESI SDK #52

Merged
merged 1 commit into from
May 9, 2024
Merged

Extend the client to use ESI SDK #52

merged 1 commit into from
May 9, 2024

Conversation

DanNiESh
Copy link
Contributor

@DanNiESh DanNiESh commented Apr 26, 2024 •
edited
Loading

Change the logic to connect to ESI SDK instead of make calls to esi-leap directly.
Related issue: CCI-MOC/esi#474

@DanNiESh DanNiESh requested a review from tzumainn April 26, 2024 16:54
@tzumainn
Copy link
Contributor

I've put an initial version of esisdk on pypi (https://pypi.org/project/esisdk/). Can you update requirements.txt here?

@DanNiESh
Copy link
Contributor Author

I've put an initial version of esisdk on pypi (https://pypi.org/project/esisdk/). Can you update requirements.txt here?

Done!

tzumainn
tzumainn reviewed Apr 30, 2024
View reviewed changes
Copy link
Contributor

@tzumainn tzumainn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question - is there a reason to have a v2? It would make sense to me if esi-leap had a different REST API version - then you would need both to ensure backwards compatibility - but this change really only shuffles some frontend logic around.

@DanNiESh
Copy link
Contributor Author

Quick question - is there a reason to have a v2? It would make sense to me if esi-leap had a different REST API version - then you would need both to ensure backwards compatibility - but this change really only shuffles some frontend logic around.

Actually I was not sure when I implemented the code. I was thinking there might be a chance that we want to make calls to esi-leap directly, then we can switch to v1. But it doesn't hurt to use the sdk as the only interface for cli.

@DanNiESh DanNiESh force-pushed the sdk branch 2 times, most recently from ea20e04 to 457ac29 Compare April 30, 2024 17:52
@tzumainn
Copy link
Contributor

Quick question - is there a reason to have a v2? It would make sense to me if esi-leap had a different REST API version - then you would need both to ensure backwards compatibility - but this change really only shuffles some frontend logic around.

Actually I was not sure when I implemented the code. I was thinking there might be a chance that we want to make calls to esi-leap directly, then we can switch to v1. But it doesn't hurt to use the sdk as the only interface for cli.

If that's the case, I don't see why we need two versions at all - may as well simplify the code base and only keep one.

@DanNiESh
Copy link
Contributor Author

DanNiESh commented May 1, 2024

Quick question - is there a reason to have a v2? It would make sense to me if esi-leap had a different REST API version - then you would need both to ensure backwards compatibility - but this change really only shuffles some frontend logic around.

Actually I was not sure when I implemented the code. I was thinking there might be a chance that we want to make calls to esi-leap directly, then we can switch to v1. But it doesn't hurt to use the sdk as the only interface for cli.

If that's the case, I don't see why we need two versions at all - may as well simplify the code base and only keep one.

Okay!

@DanNiESh DanNiESh force-pushed the sdk branch 2 times, most recently from 653bcf4 to a7a0557 Compare May 7, 2024 14:37
@DanNiESh
Copy link
Contributor Author

DanNiESh commented May 7, 2024

I've updated the code to have v1 only.

tzumainn
tzumainn requested changes May 7, 2024
View reviewed changes
Copy link
Contributor

@tzumainn tzumainn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good - however I'm seeing some strange behavior. When I run, say, esi lease show --debug, I get the following:

START with options: esi lease show ff3a59fa-4798-4831-b262-9d79f54c47e9 --debug
options: Namespace(access_token='***', access_token_endpoint='', access_token_type='', application_credential_id='', application_credential_name='', application_credential_s)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_co}
defaults: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'auth_type': 'password', 'baremetal_status_code_retries': 5, 'baremetal_introspect}
cloud cfg: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retrie}
compute API version 2.latest, cmd group openstack.compute.v2
identity API version 3, cmd group openstack.identity.v3
/home/tzumainn/development/python-esileapclient/.venv/lib64/python3.6/site-packages/openstack/image/image_signer.py:14: CryptographyDeprecationWarning: Python 3.6 is no long.
  from cryptography.hazmat.backends import default_backend
image API version 2, cmd group openstack.image.v2
network API version 2, cmd group openstack.network.v2
object_store API version 1, cmd group openstack.object_store.v1
volume API version 3, cmd group openstack.volume.v3
lease API version 1, cmd group openstack.lease.v1
baremetal API version 1.72, cmd group openstack.baremetal.v1
command: esi lease show -> esileapclient.osc.v1.lease.ShowLease (auth=True)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_co}
Using auth plugin: password
Using parameters {'auth_url': 'https://esi.massopen.cloud:13000', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'admin', 'user_domain_name': 'Defaul}
Get auth_ref
REQ: curl -g -i -X GET https://esi.massopen.cloud:13000 -H "Accept: application/json" -H "User-Agent: openstacksdk/1.2.0 keystoneauth1/5.1.2 python-requests/2.27.1 CPython/3"
Starting new HTTPS connection (1): esi.massopen.cloud:13000
https://esi.massopen.cloud:13000 "GET / HTTP/1.1" 300 275
RESP: [300] Content-Length: 275 Content-Type: application/json Date: Tue, 07 May 2024 18:40:54 GMT Location: https://esi.massopen.cloud:13000/v3/ Server: Apache Vary: X-Auth2
RESP BODY: {"versions": {"values": [{"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", "href": "https://esi.massopen.cloud:1300}
GET call to https://esi.massopen.cloud:13000/ used request id req-5fbb1d80-6b9b-4561-abd9-fd5c0e14f8d2
Making authentication request to https://esi.massopen.cloud:13000/v3/auth/tokens
https://esi.massopen.cloud:13000 "POST /v3/auth/tokens HTTP/1.1" 201 7474
{"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "a2eb20a1bf94440e807c903113c71d12", "name": "admin", "password_expires_at"}
run(Namespace(columns=[], fit_width=False, formatter='table', max_width=0, noindent=False, prefix='', print_empty=False, uuid='ff3a59fa-4798-4831-b262-9d79f54c47e9', variabl)
REQ: curl -g -i -X GET http://129.10.5.142:7777 -H "Accept: application/json" -H "User-Agent: openstacksdk/1.2.0 keystoneauth1/5.1.2 python-requests/2.27.1 CPython/3.6.8"
Starting new HTTP connection (1): 129.10.5.142:7777
http://129.10.5.142:7777 "GET / HTTP/1.1" 401 114
RESP: [401] Connection: keep-alive Content-Length: 114 Content-Type: application/json Date: Tue, 07 May 2024 18:40:56 GMT Www-Authenticate: Keystone uri="https://esi.massope"
RESP BODY: {"error": {"code": 401, "title": "Unauthorized", "message": "The request you have made requires authentication."}}
Request returned failure status: 401
REQ: curl -g -i -X GET http://129.10.5.142:7777 -H "Accept: application/json" -H "User-Agent: openstacksdk/1.2.0 keystoneauth1/5.1.2 python-requests/2.27.1 CPython/3.6.8" -H"
http://129.10.5.142:7777 "GET / HTTP/1.1" 300 118
RESP: [300] Connection: keep-alive Content-Length: 118 Content-Type: application/json Date: Tue, 07 May 2024 18:40:59 GMT
RESP BODY: {"versions": [{"id": "v1.0", "status": "CURRENT", "links": [{"href": "http://129.10.5.142:7777/v1", "rel": "self"}]}]}
REQ: curl -g -i -X GET http://129.10.5.142:7777/v1/leases/ff3a59fa-4798-4831-b262-9d79f54c47e9 -H "User-Agent: openstacksdk/1.2.0 keystoneauth1/5.1.2 python-requests/2.27.1 "
http://129.10.5.142:7777 "GET /v1/leases/ff3a59fa-4798-4831-b262-9d79f54c47e9 HTTP/1.1" 200 839
RESP: [200] Connection: keep-alive Content-Length: 839 Content-Type: application/json Date: Tue, 07 May 2024 18:41:08 GMT
RESP BODY: {"name": null, "uuid": "ff3a59fa-4798-4831-b262-9d79f54c47e9", "project_id": "f9f5654ee2be438ba76538cc4f944993", "project": "storage-research", "owner_id": "ce548}
clean_up ShowLease: 
END return value: 0

Note the unauthorized response error in the middle, right after Starting new HTTP connection. With the current esileapclient, the same command has the following:

START with options: esi lease show ff3a59fa-4798-4831-b262-9d79f54c47e9 --debug
options: Namespace(access_token='***', access_token_endpoint='', access_token_type='', application_credential_id='', application_credential_name='', application_credential_secret='***', auth_methods='', auth_type='password', auth_url='https://esi.massopen.cloud:13000', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', insecure=None, interface='', key='', log_file=None, oauth2_client_id='', oauth2_client_secret='***', oauth2_endpoint='', openid_scope='', os_baremetal_api_version='1.72', os_beta_command=False, os_compute_api_version='2.latest', os_dns_api_version='2', os_esiclient_api_version='1', os_identity_api_version='3', os_image_api_version='2', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='3', passcode='', password='***', profile='', project_domain_id='', project_domain_name='Default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='regionOne', remote_project_domain_id='', remote_project_domain_name='', remote_project_id='', remote_project_name='', service_provider='', system_scope='', timing=False, token='***', trust_id='', user_domain_id='', user_domain_name='Default', user_id='', username='admin', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'region_name': 'regionOne', 'default_domain': 'default', 'timing': False, 'auth_url': 'https://esi.massopen.cloud:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'identity_api_version': '3', 'image_api_version': '2', 'volume_api_version': '3', 'baremetal_api_version': '1.72', 'esiclient_api_version': '1', 'dns_api_version': '2', 'auth_type': 'password', ': []}
defaults: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'auth_type': 'password', 'baremetal_status_code_retries': 5, 'baremetal_introspection_status_code_retries': 5, 'image_status_code_retries': 5, 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active'}
cloud cfg: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'region_name': 'regionOne', 'default_domain': 'default', 'timing': False, 'auth_url': 'https://esi.massopen.cloud:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'identity_api_version': '3', 'image_api_version': '2', 'volume_api_version': '3', 'baremetal_api_version': '1.72', 'esiclient_api_version': '1', 'dns_api_version': '2', 'auth_type': 'password', ': []}
compute API version 2.latest, cmd group openstack.compute.v2
identity API version 3, cmd group openstack.identity.v3
/home/tzumainn/development/moc-esi/.venv/lib64/python3.6/site-packages/openstack/image/image_signer.py:14: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography (40.0) will be the last to support Python 3.6.
  from cryptography.hazmat.backends import default_backend
image API version 2, cmd group openstack.image.v2
network API version 2, cmd group openstack.network.v2
object_store API version 1, cmd group openstack.object_store.v1
volume API version 3, cmd group openstack.volume.v3
baremetal API version 1.72, cmd group openstack.baremetal.v1
esiclient API version 1, cmd group openstack.esiclient.v1
lease API version 1, cmd group openstack.lease.v1
neutronclient API version 2, cmd group openstack.neutronclient.v2
dns API version 2, cmd group openstack.dns.v2
command: esi lease show -> esileapclient.osc.v1.lease.ShowLease (auth=True)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('osc-lib', '2.7.0')], 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'region_name': 'regionOne', 'default_domain': 'default', 'timing': False, 'auth_url': 'https://esi.massopen.cloud:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'identity_api_version': '3', 'image_api_version': '2', 'volume_api_version': '3', 'baremetal_api_version': '1.72', 'esiclient_api_version': '1', 'dns_api_version': '2', 'auth_type': 'password', ': []}
Using auth plugin: password
Using parameters {'auth_url': 'https://esi.massopen.cloud:13000', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'admin', 'user_domain_name': 'Default', 'password': '***'}
Get auth_ref
REQ: curl -g -i -X GET https://esi.massopen.cloud:13000 -H "Accept: application/json" -H "User-Agent: openstacksdk/1.0.1 keystoneauth1/5.1.2 python-requests/2.27.1 CPython/3.6.8"
Starting new HTTPS connection (1): esi.massopen.cloud:13000
https://esi.massopen.cloud:13000 "GET / HTTP/1.1" 300 275
RESP: [300] Content-Length: 275 Content-Type: application/json Date: Tue, 07 May 2024 18:40:06 GMT Location: https://esi.massopen.cloud:13000/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id: req-e9683407-c8aa-419d-b9ab-59d759e572cc
RESP BODY: {"versions": {"values": [{"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", "href": "https://esi.massopen.cloud:13000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}]}}
GET call to https://esi.massopen.cloud:13000/ used request id req-e9683407-c8aa-419d-b9ab-59d759e572cc
Making authentication request to https://esi.massopen.cloud:13000/v3/auth/tokens
https://esi.massopen.cloud:13000 "POST /v3/auth/tokens HTTP/1.1" 201 7474
{"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "a2eb20a1bf94440e807c903113c71d12", "name": "admin", "password_expires_at": null}, "audit_ids": ["W7yDh1ofRNOCMfh9b8yJ5Q"], "expires_at": "2024-05-07T19:40:06.000000Z", "issued_at": "2024-05-07T18:40:06.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "ce54871194264a1dafd508058c0b8614", "name": "admin"}, "is_domain": false, "roles": [{"id": "87bf286e240143dab8907a5d6f212707", "name": "admin"}, {"id": "c2f2d543f1ef45d49bfd427a7d65b909", "name": "reader"}, {"id": "82a5aebeb7c14b33b30e2cf3a1d2f2bc", "name": "member"}], "catalog": [{"endpoints": [{"id": "123ad6c4d5bb451998ff522c1f1c3758", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:9696", "region": "regionOne"}, {"id": "197fc3e771274b5eb8bc18962c13ee83", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:9696", "region": "regionOne"}, {"id": "656472038c4d44d59db4dcd3a264c623", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13696", "region": "regionOne"}], "id": "00b06db0c2bd4e05ab7fdb695ebc7fc1", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "731ea36e729b478292c9c95de279cb0e", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:9292", "region": "regionOne"}, {"id": "ce8b1957648f4b7a9312a79ab6ad4924", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13292", "region": "regionOne"}, {"id": "f250db80b9ad47a3806705d8cc10c267", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:9292", "region": "regionOne"}], "id": "179ef34d2fcd494c937df3da5eb219a4", "type": "image", "name": "glance"}, {"endpoints": [{"id": "4225997025554878b5be2211e6911879", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8080/v1/AUTH_ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "61be568541ce402abb4a8740d061dbf9", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8080", "region": "regionOne"}, {"id": "c7818643df564a6f956b06d12cc59f36", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13808/v1/AUTH_ce54871194264a1dafd508058c0b8614", "region": "regionOne"}], "id": "25223a4fc116444e939e1c53aa2253a1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0fd1ff5b35774c58927dcfe09a26e12a", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8776/v3/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "14f1e63a290d417ab509a11f5e151912", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8776/v3/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "37b37fb8c6e2454492b29a131052ab36", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13776/v3/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}], "id": "30f7862d96a141d5b37f7e62a0e9cf23", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "cda7bcba710c48919e65bd53836939aa", "interface": "public", "region_id": "regionOne", "url": "http://129.10.5.142:7777", "region": "regionOne"}], "id": "5e148afa4931463aa041ef0ef8586f9a", "type": "lease", "name": "esi-leap"}, {"endpoints": [{"id": "31c3a5bf9ccf42739af93003d1d7522f", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:35357", "region": "regionOne"}, {"id": "5d48132a56f14876ac18094479dc976e", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13000", "region": "regionOne"}, {"id": "80b5e915cb164d188c4578f2b3e0b790", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:5000", "region": "regionOne"}], "id": "62241e4e534d43418615219ccd07e56d", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "0aed3aff1b0a422782c5a5baabbe32ac", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8004/v1/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "0ebb0c7110af4b368ca48d64686b6943", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13004/v1/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "4a0b43f8e4ef40aa99e34840f4d7d962", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8004/v1/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}], "id": "94a58465244a4bb29d0132a00557e0ce", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "251399db6c034af286ca21fd4ab1e625", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:5050", "region": "regionOne"}, {"id": "33bbb5006ffb4898badffe1ed75b607a", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:5050", "region": "regionOne"}, {"id": "ada4204c606049cea8744bb2189e4b79", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13050", "region": "regionOne"}], "id": "9feac3e4e0e24034ac09075630a5fa10", "type": "baremetal-introspection", "name": "ironic-inspector"}, {"endpoints": [{"id": "a4f4d892b44d47bebf347166c8b0f8d6", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13385", "region": "regionOne"}, {"id": "d3dd6f904a184f1891b88c7fb9c50097", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:6385", "region": "regionOne"}, {"id": "d4b9de3d21684ae687e6be2bfaaa0754", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:6385", "region": "regionOne"}], "id": "caceb6a76bb944baaba752554644f3b2", "type": "baremetal", "name": "ironic"}, {"endpoints": [{"id": "1621aa7cd88f403587689023907e21bc", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8778/placement", "region": "regionOne"}, {"id": "3c1e24205f1245339d101e10052c906d", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8778/placement", "region": "regionOne"}, {"id": "5bb170b48108402698a2189ec6d2bbc3", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13778/placement", "region": "regionOne"}], "id": "cdda5e2ca70a4a4eabe230e02c7e92be", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "2a0731c3f56541f8a0d035f71feb774a", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8000/v1", "region": "regionOne"}, {"id": "ab622a2ca3e34bd89ecf3f6bcfb446ad", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13005/v1", "region": "regionOne"}, {"id": "b04832cf905c422baa91259a070346a3", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8000/v1", "region": "regionOne"}], "id": "df16589e88bb40b08f4789375c0843cc", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "2958818e85b748418e3b66ae828c3028", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8774/v2.1", "region": "regionOne"}, {"id": "5a514ad30d1044d2ae4d119c1812942c", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13774/v2.1", "region": "regionOne"}, {"id": "b413b7d5982d4a4a8fd8faeb9dd30df8", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8774/v2.1", "region": "regionOne"}], "id": "f0497db9b47f4cddbd7cbcc12e8388f7", "type": "compute", "name": "nova"}]}}
run(Namespace(columns=[], fit_width=False, formatter='table', max_width=0, noindent=False, prefix='', print_empty=False, uuid='ff3a59fa-4798-4831-b262-9d79f54c47e9', variables=[]))
REQ: curl -g -i -X GET http://129.10.5.142:7777/v1/leases/ff3a59fa-4798-4831-b262-9d79f54c47e9 -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-esileapclient" -H "X-Auth-Token: {SHA256}34e967f502c9939b86bee1e19bbf148a3be5c1834d8ec7accba566fd72c3fa21" -H "X-OpenStack-ESI-Leap-API-Version: 1"
Starting new HTTP connection (1): 129.10.5.142:7777
http://129.10.5.142:7777 "GET /v1/leases/ff3a59fa-4798-4831-b262-9d79f54c47e9 HTTP/1.1" 200 839
RESP: [200] Connection: keep-alive Content-Length: 839 Content-Type: application/json Date: Tue, 07 May 2024 18:40:19 GMT
RESP BODY: {"name": null, "uuid": "ff3a59fa-4798-4831-b262-9d79f54c47e9", "project_id": "f9f5654ee2be438ba76538cc4f944993", "project": "storage-research", "owner_id": "ce54871194264a1dafd508058c0b8614", "owner": "admin", "resource_type": "ironic_node", "resource_uuid": "4a4d57c6-d90e-40bf-807c-0c0e78e412fc", "resource_class": "fc430", "resource_properties": {"vendor": "dell inc", "local_gb": "185", "cpus": "40", "cpu_arch": "x86_64", "memory_mb": "131072", "cpu_model_name": "Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz", "cpu_frequency": "3400.0000", "traits": []}, "resource": "MOC-R4PAC08U33-S3A", "start_time": "2024-05-03T17:55:58.964213", "fulfill_time": "2024-05-03T17:56:42.563589", "expire_time": null, "end_time": "2024-05-31T00:00:00", "status": "active", "properties": {}, "purpose": null, "offer_uuid": null, "parent_lease_uuid": null}

There's no error message there. The difference seems to result in the updated code taking ~2 seconds longer. Is this error something you could eliminate?

@DanNiESh
Copy link
Contributor Author

DanNiESh commented May 8, 2024

I think the reason is that esisdk attempt to discover the lease endpoint at 7777 at first. And once the endpoint is discovered, it makes calls to es-leap lease show API. I'm trying to see if we can skip the endpoint discovery step.

@DanNiESh
Copy link
Contributor Author

DanNiESh commented May 8, 2024

I think the reason is that esisdk attempt to discover the lease endpoint at 7777 at first. And once the endpoint is discovered, it makes calls to es-leap lease show API. I'm trying to see if we can skip the endpoint discovery step.

Update: new PR in esisdk to skip discovery: CCI-MOC/esisdk#3. openstack esi lease show --debug doesn't have 404 error now because it doesn't call http://129.10.5.142:7777 to discover the api endpoint.

@tzumainn
Copy link
Contributor

tzumainn commented May 9, 2024

I've packaged an updated esisdk with your endpoint change; can you update the requirements to 0.2 and verify that the endpoint discovery is gone?

@DanNiESh
Copy link
Contributor Author

DanNiESh commented May 9, 2024

Updated esisdk version, this is the debug info when I ran openstack esi offer show 0a5ed50a-5919-4036-bb7a-dc81d3eca6d4 --debug

START with options: esi offer show 0a5ed50a-5919-4036-bb7a-dc81d3eca6d4 --debug
options: Namespace(access_token='***', access_token_endpoint='', access_token_type='', application_credential_id='', application_credential_name='', application_credential_secret='***', auth_methods='', auth_type='password', auth_url='https://esi.massopen.cloud:13000', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', insecure=None, interface='', key='', log_file=None, oauth2_client_id='', oauth2_client_secret='***', oauth2_endpoint='', openid_scope='', os_baremetal_api_version='1.78', os_beta_command=False, os_compute_api_version='2.latest', os_esileap_api_version='1', os_identity_api_version='3', os_image_api_version='2', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='3', passcode='', password='***', project_domain_id='', project_domain_name='Default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='regionOne', remote_project_domain_id='', remote_project_domain_name='', remote_project_id='', remote_project_name='', service_provider='', system_scope='', timing=False, token='***', trust_id='', user_domain_id='', user_domain_name='Default', user_id='', username='admin', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'region_name': 'regionOne', 'default_domain': 'default', 'timing': False, 'auth_url': 'https://esi.massopen.cloud:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'identity_api_version': '3', 'image_api_version': '2', 'volume_api_version': '3', 'baremetal_api_version': '1.78', 'esileap_api_version': '1', 'auth_type': 'password', ': []}
defaults: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'auth_type': 'password', 'baremetal_status_code_retries': 5, 'baremetal_introspection_status_code_retries': 5, 'image_status_code_retries': 5, 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active'}
cloud cfg: {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'region_name': 'regionOne', 'default_domain': 'default', 'timing': False, 'auth_url': 'https://esi.massopen.cloud:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'identity_api_version': '3', 'image_api_version': '2', 'volume_api_version': '3', 'baremetal_api_version': '1.78', 'esileap_api_version': '1', 'auth_type': 'password', ': []}
compute API version 2.latest, cmd group openstack.compute.v2
identity API version 3, cmd group openstack.identity.v3
image API version 2, cmd group openstack.image.v2
network API version 2, cmd group openstack.network.v2
object_store API version 1, cmd group openstack.object_store.v1
volume API version 3, cmd group openstack.volume.v3
baremetal API version 1.78, cmd group openstack.baremetal.v1
lease API version 1, cmd group openstack.lease.v1
command: esi offer show -> esileapclient.osc.v1.offer.ShowOffer (auth=True)
Auth plugin password selected
auth_config_hook(): {'api_timeout': None, 'verify': True, 'cacert': None, 'cert': None, 'key': None, 'baremetal_status_code_retries': '5', 'baremetal_introspection_status_code_retries': '5', 'image_status_code_retries': '5', 'disable_vendor_agent': {}, 'interface': 'public', 'floating_ip_source': 'neutron', 'image_api_use_tasks': False, 'image_format': 'qcow2', 'message': '', 'network_api_version': '2', 'object_store_api_version': '1', 'secgroup_source': 'neutron', 'status': 'active', 'auth': {'user_domain_name': 'Default', 'project_domain_name': 'Default', 'project_name': 'admin'}, 'additional_user_agent': [('osc-lib', '2.9.0')], 'verbose_level': 3, 'deferred_help': False, 'debug': True, 'region_name': 'regionOne', 'default_domain': 'default', 'timing': False, 'auth_url': 'https://esi.massopen.cloud:13000', 'username': 'admin', 'password': '***', 'beta_command': False, 'compute_api_version': '2.latest', 'identity_api_version': '3', 'image_api_version': '2', 'volume_api_version': '3', 'baremetal_api_version': '1.78', 'esileap_api_version': '1', 'auth_type': 'password', ': []}
Using auth plugin: password
Using parameters {'auth_url': 'https://esi.massopen.cloud:13000', 'project_name': 'admin', 'project_domain_name': 'Default', 'username': 'admin', 'user_domain_name': 'Default', 'password': '***'}
Get auth_ref
REQ: curl -g -i -X GET https://esi.massopen.cloud:13000 -H "Accept: application/json" -H "User-Agent: openstacksdk/1.2.0 keystoneauth1/5.1.2 python-requests/2.31.0 CPython/3.7.17"
Starting new HTTPS connection (1): esi.massopen.cloud:13000
https://esi.massopen.cloud:13000 "GET / HTTP/1.1" 300 275
RESP: [300] Content-Length: 275 Content-Type: application/json Date: Thu, 09 May 2024 16:12:43 GMT Location: https://esi.massopen.cloud:13000/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id: req-c7ed22b9-d421-4e49-9168-65a16e059c1b
RESP BODY: {"versions": {"values": [{"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", "href": "https://esi.massopen.cloud:13000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}]}}
GET call to https://esi.massopen.cloud:13000/ used request id req-c7ed22b9-d421-4e49-9168-65a16e059c1b
Making authentication request to https://esi.massopen.cloud:13000/v3/auth/tokens
https://esi.massopen.cloud:13000 "POST /v3/auth/tokens HTTP/1.1" 201 7474
{"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "a2eb20a1bf94440e807c903113c71d12", "name": "admin", "password_expires_at": null}, "audit_ids": ["6u26OIUMS1-I5Jte7QvDNg"], "expires_at": "2024-05-09T17:12:44.000000Z", "issued_at": "2024-05-09T16:12:44.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "ce54871194264a1dafd508058c0b8614", "name": "admin"}, "is_domain": false, "roles": [{"id": "c2f2d543f1ef45d49bfd427a7d65b909", "name": "reader"}, {"id": "87bf286e240143dab8907a5d6f212707", "name": "admin"}, {"id": "82a5aebeb7c14b33b30e2cf3a1d2f2bc", "name": "member"}], "catalog": [{"endpoints": [{"id": "123ad6c4d5bb451998ff522c1f1c3758", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:9696", "region": "regionOne"}, {"id": "197fc3e771274b5eb8bc18962c13ee83", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:9696", "region": "regionOne"}, {"id": "656472038c4d44d59db4dcd3a264c623", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13696", "region": "regionOne"}], "id": "00b06db0c2bd4e05ab7fdb695ebc7fc1", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "731ea36e729b478292c9c95de279cb0e", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:9292", "region": "regionOne"}, {"id": "ce8b1957648f4b7a9312a79ab6ad4924", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13292", "region": "regionOne"}, {"id": "f250db80b9ad47a3806705d8cc10c267", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:9292", "region": "regionOne"}], "id": "179ef34d2fcd494c937df3da5eb219a4", "type": "image", "name": "glance"}, {"endpoints": [{"id": "4225997025554878b5be2211e6911879", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8080/v1/AUTH_ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "61be568541ce402abb4a8740d061dbf9", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8080", "region": "regionOne"}, {"id": "c7818643df564a6f956b06d12cc59f36", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13808/v1/AUTH_ce54871194264a1dafd508058c0b8614", "region": "regionOne"}], "id": "25223a4fc116444e939e1c53aa2253a1", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "0fd1ff5b35774c58927dcfe09a26e12a", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8776/v3/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "14f1e63a290d417ab509a11f5e151912", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8776/v3/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "37b37fb8c6e2454492b29a131052ab36", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13776/v3/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}], "id": "30f7862d96a141d5b37f7e62a0e9cf23", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "cda7bcba710c48919e65bd53836939aa", "interface": "public", "region_id": "regionOne", "url": "http://129.10.5.142:7777", "region": "regionOne"}], "id": "5e148afa4931463aa041ef0ef8586f9a", "type": "lease", "name": "esi-leap"}, {"endpoints": [{"id": "31c3a5bf9ccf42739af93003d1d7522f", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:35357", "region": "regionOne"}, {"id": "5d48132a56f14876ac18094479dc976e", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13000", "region": "regionOne"}, {"id": "80b5e915cb164d188c4578f2b3e0b790", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:5000", "region": "regionOne"}], "id": "62241e4e534d43418615219ccd07e56d", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "0aed3aff1b0a422782c5a5baabbe32ac", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8004/v1/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "0ebb0c7110af4b368ca48d64686b6943", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13004/v1/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}, {"id": "4a0b43f8e4ef40aa99e34840f4d7d962", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8004/v1/ce54871194264a1dafd508058c0b8614", "region": "regionOne"}], "id": "94a58465244a4bb29d0132a00557e0ce", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "251399db6c034af286ca21fd4ab1e625", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:5050", "region": "regionOne"}, {"id": "33bbb5006ffb4898badffe1ed75b607a", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:5050", "region": "regionOne"}, {"id": "ada4204c606049cea8744bb2189e4b79", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13050", "region": "regionOne"}], "id": "9feac3e4e0e24034ac09075630a5fa10", "type": "baremetal-introspection", "name": "ironic-inspector"}, {"endpoints": [{"id": "a4f4d892b44d47bebf347166c8b0f8d6", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13385", "region": "regionOne"}, {"id": "d3dd6f904a184f1891b88c7fb9c50097", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:6385", "region": "regionOne"}, {"id": "d4b9de3d21684ae687e6be2bfaaa0754", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:6385", "region": "regionOne"}], "id": "caceb6a76bb944baaba752554644f3b2", "type": "baremetal", "name": "ironic"}, {"endpoints": [{"id": "1621aa7cd88f403587689023907e21bc", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8778/placement", "region": "regionOne"}, {"id": "3c1e24205f1245339d101e10052c906d", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8778/placement", "region": "regionOne"}, {"id": "5bb170b48108402698a2189ec6d2bbc3", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13778/placement", "region": "regionOne"}], "id": "cdda5e2ca70a4a4eabe230e02c7e92be", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "2a0731c3f56541f8a0d035f71feb774a", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8000/v1", "region": "regionOne"}, {"id": "ab622a2ca3e34bd89ecf3f6bcfb446ad", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13005/v1", "region": "regionOne"}, {"id": "b04832cf905c422baa91259a070346a3", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8000/v1", "region": "regionOne"}], "id": "df16589e88bb40b08f4789375c0843cc", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "2958818e85b748418e3b66ae828c3028", "interface": "internal", "region_id": "regionOne", "url": "http://192.168.24.17:8774/v2.1", "region": "regionOne"}, {"id": "5a514ad30d1044d2ae4d119c1812942c", "interface": "public", "region_id": "regionOne", "url": "https://esi.massopen.cloud:13774/v2.1", "region": "regionOne"}, {"id": "b413b7d5982d4a4a8fd8faeb9dd30df8", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.17:8774/v2.1", "region": "regionOne"}], "id": "f0497db9b47f4cddbd7cbcc12e8388f7", "type": "compute", "name": "nova"}]}}
run(Namespace(columns=[], fit_width=False, formatter='table', max_width=0, noindent=False, prefix='', print_empty=False, uuid='0a5ed50a-5919-4036-bb7a-dc81d3eca6d4', variables=[]))
REQ: curl -g -i -X GET http://129.10.5.142:7777/v1/offers/0a5ed50a-5919-4036-bb7a-dc81d3eca6d4 -H "User-Agent: openstacksdk/1.2.0 keystoneauth1/5.1.2 python-requests/2.31.0 CPython/3.7.17" -H "X-Auth-Token: {SHA256}f0244d49f2aa915ebe4260bfdef047c119a90b34427ad70adab54b351d7cc8bb"
Starting new HTTP connection (1): 129.10.5.142:7777
http://129.10.5.142:7777 "GET /v1/offers/0a5ed50a-5919-4036-bb7a-dc81d3eca6d4 HTTP/1.1" 200 800
RESP: [200] Connection: keep-alive Content-Length: 800 Content-Type: application/json Date: Thu, 09 May 2024 16:12:57 GMT
RESP BODY: {"name": null, "uuid": "0a5ed50a-5919-4036-bb7a-dc81d3eca6d4", "project_id": "aa224e1b59c44f1fb1cd6824a7654baf", "project": "cloudlab_esi_demo", "lessee_id": "d4ece291ef5f497eb3c13a78e9beb83f", "lessee": "cloudlab", "resource_type": "ironic_node", "resource_uuid": "5a4061c1-5630-4093-8121-c043f7d37c68", "resource": "oct4-cha1-1a", "resource_class": "fc430", "resource_properties": {"root_device": {"size": ">= 50"}, "vendor": "dell inc", "local_gb": "185", "cpus": "32", "cpu_arch": "x86_64", "memory_mb": "98304", "cpu_model_name": "Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz", "cpu_frequency": "3400.0000", "traits": []}, "start_time": "2024-02-08T23:32:46.674299", "end_time": "9999-12-31T23:59:59.999999", "status": "available", "properties": {}, "availabilities": [], "parent_lease_uuid": null}

@tzumainn
Copy link
Contributor

tzumainn commented May 9, 2024

Awesome - looks good! Thanks for sticking with this.

@tzumainn tzumainn merged commit 56b3da5 into CCI-MOC:master May 9, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tzumainn tzumainn tzumainn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@DanNiESh @tzumainn